How to save DataFrame directly to Hive?

Question

Is it possible to save DataFrame in spark directly to Hive.

I have tried with converting DataFrame to Rdd and then saving as text file and then loading in hive. But i am wondering if i can directly save dataframe to hive

Answer 1

One way is to create an in-memory temporary table and store them in hive table using sqlContext.

Lets set a data frame as myDf. You can create one temporary table using:

myDf.createOrReplaceTempView("mytempTable") 

Then, simply use a hive statement to create table and from your temp table dump the data into it:

sqlContext.sql("create table mytable as select * from mytempTable");

If you want to know more about Spark, then do check out this awesome video tutorial: