Python pandas Filtering out nan from a data selection of a column of strings

Question

Without using groupby how would I filter out data without NaN?

Let say I have a matrix where customers will fill in 'N/A', 'n/a' or any of its variations and others leave it blank:

import pandas as pd

import numpy as np

df = pd.DataFrame({'movie': ['thg', 'thg', 'mol', 'mol', 'lob', 'lob'],

                  'rating': [3., 4., 5., np.nan, np.nan, np.nan],

                  'name': ['John', np.nan, 'N/A', 'Graham', np.nan, np.nan]})

nbs = df['name'].str.extract('^(N/A|NA|na|n/a)')

nms=df[(df['name'] != nbs) ]

output:

 >>> nms

  movie    name rating

0   thg   John     3

1   thg     NaN   4

3   mol Graham     NaN

4   lob     NaN NaN

5   lob     NaN NaN

How would I filter out NaN values so I can get results to work with like this:

 movie    name rating

0   thg   John     3

3   mol Graham     NaN

I am guessing I need something like ~np.isnan but the Tilda does not work with strings.

Answer 1

Just apply this:

filtered_df = df[df['name'].notnull()]

Thus, it helps in filtering out only rows that don't have NaN values in the 'name' column.

If you are interested to learn Pandas visit this Python Pandas Tutorial.

For more information, kindly refer to our Python course.