Back

Explore Courses Blog Tutorials Interview Questions
0 votes
1 view
in Big Data Hadoop & Spark by (11.4k points)

I am trying to take my input data:

A    B       C
--------------
4    blah    2
2            3
56   foo     3


And add a column to the end based on whether B is empty or not:

A    B       C     D
--------------------
4    blah    2     1
2            3     0
56   foo     3     1


I can do this easily by registering the input dataframe as a temp table, then typing up a SQL query.

But I'd really like to know how to do this with just Scala methods and not having to type out a SQL query within Scala.

I've tried .withColumn, but I can't get that to do what I want.

1 Answer

0 votes
by (32.3k points)

I tried withColumn approach and it worked. I would suggest you to go through my code given below:

image

image

In above code I added the (100, null, 5) row for testing the isNull case.

image

Welcome to Intellipaat Community. Get your technical queries answered by top developers!

28.4k questions

29.7k answers

500 comments

94.1k users

Browse Categories

...