dropping rows from dataframe based on a “not in” condition

Question

I want to drop rows from a pandas dataframe when the value of the date column is in a list of dates. The following code doesn't work:

a=['2015-01-01' , '2015-02-01']

df=df[df.datecolumn not in a]

I get the following error:

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Answer 1

Using pandas.Dataframe.isin which will return boolean values that will depend on whether each element is inside the list a or not.

Then you can invert this with the ~ to convert True to False and vice versa.

import pandas as pd

a = ['2015-01-01' , '2015-02-01']

df = pd.DataFrame(data={'date':['2015-01-01' , '2015-02-01', '2015-03-01' , '2015-04-01', '2015-05-01' , '2015-06-01']})

print(df)

#         date

#0  2015-01-01

#1  2015-02-01

#2  2015-03-01

#3  2015-04-01

#4  2015-05-01

#5  2015-06-01

df = df[~df['date'].isin(a)]

print(df)

#         date

#2  2015-03-01

#3  2015-04-01

#4  2015-05-01

#5  2015-06-01

If you want to learn Python proogramming language for Data Science then you can watch this complete video tutorial: