Remove Temporary Tables from Apache SQL Spark

Question

I have registertemptable in Apache Spark using Zeppelin below:

val hvacText = sc.textFile("...")

case class Hvac(date: String, time: String, targettemp: Integer, actualtemp: Integer, buildingID: String)

val hvac = hvacText.map(s => s.split(",")).filter(s => s(0) != "Date").map(
    s => Hvac(s(0),
            s(1),
            s(2).toInt,
            s(3).toInt,
            s(6))).toDF()

hvac.registerTempTable("hvac")

After I have done with my queries with this temp table, how do I remove it?

I checked all docs and it seems I am getting nowhere.

Any guidance?

Answer 1

Spark 2.x

For temporary views you can use Catalog.dropTempView:

spark.catalog.dropTempView("df")

For global views you can use Catalog.dropGlobalTempView:

spark.catalog.dropGlobalTempView("df")

Just note that if view doesn't exist both the methods are safe to call and, since Spark 2.1, return boolean indicating if the operation succeeded.

Spark 1.x

You can use SQLContext.dropTempTable:

scala.util.Try(sqlContext.dropTempTable("df"))

It can be still used in Spark 2.0, but delegates processing to Catalog.dropTempView and is safe to use if table doesn't exist.