Back

Explore Courses Blog Tutorials Interview Questions
0 votes
1 view
in Big Data Hadoop & Spark by (11.4k points)

I have a data frame with four fields. One of the field name is Status and I am trying to use a OR condition in .filter for a dataframe . I tried below queries but no luck.

df2 = df1.filter(("Status=2") || ("Status =3"))

df2 = df1.filter("Status=2" || "Status =3")


Has anyone used this before. Please help me out!

1 Answer

0 votes
by (32.3k points)

Instead of:

df2 = df1.filter("Status=2" || "Status =3")

Simply try:

df2 = df1.filter($"Status" === 2 || $"Status" === 3)

Learn Spark with this Spark Certification Course by Intellipaat.

Welcome to Intellipaat Community. Get your technical queries answered by top developers!

28.4k questions

29.7k answers

500 comments

94k users

Browse Categories

...