spark dataframe filter

Question

I am filtering the Spark DataFrame using the filter:

var notFollowingList=List(9.8,7,6,3,1)

df.filter(col("uid”).isin(notFollowingList))

But I get an error saying:

Unsupported literal type classscala.collection.immutable.$colon$colon 

Can anyone help me in resolving the error?

Answer 1

Use the function as following piece of code to filter:

var notFollowingList=List(9.8,7,6,3,1)

df.filter(col("uid").isin(notFollowingList:_*))