How to export data from Spark SQL to CSV

Question

Using Spark SQL I'm getting an error with an org.apache.spark.sql.hive.HiveQl stack trace:

java.lang.RuntimeException: Unsupported language features in query:
    insert overwrite directory '/data/home.csv' select * from testtable

Please guide me to write export to CSV feature in Spark SQL.

Answer 1

You can use below statement to write the contents of dataframe in CSV format

df.write.csv("/data/home/csv")

If you need to write the whole dataframe into a single CSV file, then use

df.coalesce(1).write.csv("/data/home/sample.csv")