How to perform union on two DataFrames with different amounts of columns in spark?

Question

I have 2 DataFrames as followed :

I need union like this:

The unionAll function doesn't work because the number and the name of columns are different.

How can I do this?

Answer 1

Using Scala, you just have to append all missing columns as nulls, as given below:

Now, the unionALl function will work:

If you want to know more about Spark, then do check out this awesome video tutorial: