Explore Courses Blog Tutorials Interview Questions
0 votes
in Data Science by (17.6k points)

My numpy arrays use np.nan to designate missing values. As I iterate over the data set, I need to detect such missing values and handle them in special ways.

Naively I used numpy.isnan(val), which works well unless val isn't among the subset of types supported by numpy.isnan(). For example, missing data can occur in string fields, in which case I get:

>>> np.isnan('some_string')

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

TypeError: Not implemented for this type

Other than writing an expensive wrapper that catches the exception and returns False, is there a way to handle this elegantly and efficiently?

1 Answer

0 votes
by (41.4k points)

pandas.isnull() is used for checking the missing values in numeric and string/object arrays.


import pandas as pd

import numpy as np

s = pd.Series(['hello’', np.nan, 'world'])



0    False

1     True

2    False

dtype: bool

Here, numpy.nan is used for representing the missing values.

Browse Categories