Explore Courses Blog Tutorials Interview Questions
0 votes
1 view
in Big Data Hadoop & Spark by (11.4k points)

I have 2 DataFrames as followed :

Source data

I need union like this:

enter image description here

The unionAll function doesn't work because the number and the name of columns are different.

How can I do this?

1 Answer

0 votes
by (32.3k points)
edited by

Using Scala, you just have to append all missing columns as nulls, as given below:


Now, the unionALl function will work:


If you want to know more about Spark, then do check out this awesome video tutorial:

Related questions

0 votes
1 answer
0 votes
1 answer
0 votes
1 answer
Welcome to Intellipaat Community. Get your technical queries answered by top developers!

28.4k questions

29.7k answers


94.1k users

Browse Categories