Explore Courses Blog Tutorials Interview Questions
0 votes
in Big Data Hadoop & Spark by (11.4k points)

If I wanted to create a StructType (i.e. a DataFrame.schema) out of a case class, is there a way to do it without creating a DataFrame? I can easily do:

case class TestCase(id: Long)
val schema = Seq[TestCase]().toDF.schema

But it seems overkill to actually create a DataFrame when all I want is the schema.

1 Answer

0 votes
by (32.3k points)

You can do it the same way SQLContext.createDataFrame does it:

import org.apache.spark.sql.catalyst.ScalaReflection

val schema = ScalaReflection.schemaFor[TestCase].dataType.asInstanceOf[StructType]

Browse Categories