Flattening Rows in Spark

Question

I am doing some testing for spark using scala. We usually read json files which needs to be manipulated like the following example:

test.json:

{"a":1,"b":[2,3]}
 

val test = sqlContext.read.json("test.json")

How can I convert it to the following format:

 

{"a":1,"b":2}
{"a":1,"b":3}

Answer 1

To convert it into your desired format I would suggest you to use explode function.

explode() takes in an array (or a map) as an input and outputs the elements of the array (map) as separate rows. 

Follow the approach given below: