Back

Explore Courses Blog Tutorials Interview Questions
0 votes
1 view
in Big Data Hadoop & Spark by (55.5k points)

Can anyone tell me how many ways to create DataFrame in spark?

1 Answer

0 votes
by (119k points)

Some of the ways to create a DataFrame in Spark:

Create Spark DataFrame from RDD

val dfFromRDD1 = rdd.toDF()

dfFromRDD1.printSchema()

Create Spark DataFrame from List and Seq

val dfFromData1 = data.toDF()

Creating Spark DataFrame from CSV

val df2 = spark.read.csv("/src/resources/file.csv")

Creating from text (TXT) file

val df2 = spark.read.text("/src/resources/file.txt")

Creating from JSON file

val df2 = spark.read.json("/src/resources/file.json")

If you wish to learn Spark then register for this Spark Training course by Intellipaat that offers instructor-led training, hands-on projects, and certification.

Also, watch this video on Spark DataFrames:

Browse Categories

...