0 votes
1 view
in Big Data Hadoop & Spark by (11.5k points)

I am doing some testing for spark using scala. We usually read json files which needs to be manipulated like the following example:

test.json:

{"a":1,"b":[2,3]}
 

val test = sqlContext.read.json("test.json")

How can I convert it to the following format:

 

{"a":1,"b":2}
{"a":1,"b":3
}

1 Answer

0 votes
by (31.4k points)

To convert it into your desired format I would suggest you to use explode function.

explode() takes in an array (or a map) as an input and outputs the elements of the array (map) as separate rows. 

Follow the approach given below:

Welcome to Intellipaat Community. Get your technical queries answered by top developers !


Categories

...