0 votes
1 view
in Big Data Hadoop & Spark by (11.5k points)

Using Spark SQL I'm getting an error with an org.apache.spark.sql.hive.HiveQl stack trace:

java.lang.RuntimeException: Unsupported language features in query:
    insert overwrite directory '/data/home.csv' select * from testtable


Please guide me to write export to CSV feature in Spark SQL.

1 Answer

0 votes
by (31.4k points)

You can use below statement to write the contents of dataframe in CSV format

df.write.csv("/data/home/csv")


If you need to write the whole dataframe into a single CSV file, then use

df.coalesce(1).write.csv("/data/home/sample.csv")

Welcome to Intellipaat Community. Get your technical queries answered by top developers !


Categories

...