0 votes
1 view
in Big Data Hadoop & Spark by (11.5k points)

I have 2 DataFrames as followed :

Source data

I need union like this:

enter image description here

The unionAll function doesn't work because the number and the name of columns are different.

How can I do this?

1 Answer

0 votes
by (31.4k points)
edited by

Using Scala, you just have to append all missing columns as nulls, as given below:

image

Now, the unionALl function will work:

image

If you want to know more about Spark, then do check out this awesome video tutorial:

Related questions

0 votes
1 answer
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer
Welcome to Intellipaat Community. Get your technical queries answered by top developers !


Categories

...