0 votes
1 view
in Big Data Hadoop & Spark by (11.5k points)

I have 2 DataFrames as followed :

Source data

I need union like this:

enter image description here

The unionAll function doesn't work because the number and the name of columns are different.

How can I do this?

1 Answer

0 votes
by (32.5k points)
edited by

Using Scala, you just have to append all missing columns as nulls, as given below:


Now, the unionALl function will work:


If you want to know more about Spark, then do check out this awesome video tutorial:

Related questions

Welcome to Intellipaat Community. Get your technical queries answered by top developers !