python - check if nan float in dictionary

Question

import numpy as np
import pandas as pd
import tia.bbg.datamgr as dm
mgr = dm.BbgDataManager()

bb_yearb4 = "2016-12-30"
bb_today = "2017-09-22"

indices = [list of indices]
sids_index = mgr[indices]
df_idx = sids_index.get_historical('PX_LAST', bb_yearb4, bb_today)

nan = np.nan

price_test = {}
for index in indices:
    price_test["{0}".format(index)] = df_idx.loc[bb_today][index]

The output shows multiple nan float values:

In [1]: price_test.values()
Out[1]: [nan, nan, nan, 47913.199999999997, nan, 1210.3299999999999, nan]

However, testing for nan shows false:

In [2]: nan in price_test.values()
Out[2]: False

What is the correct way to test this?

Macklin

np.nan in price_test.values() – Bobbyebobbysocks

@Wen: They're already doing that, and it doesn't work. – Roobbie

Answer 1

NaN is weird, because NaN != NaN. There's a good reason for that, but it still breaks in checks and everything else that assumes normal == behavior.

Check for NaN with NaN-specific checks, like numpy.isnan:

any(np.isnan(val) for val in d.values())

or in a non-NumPy context,

any(math.isnan(val) for val in d.values())

Roobbie

Maybe math.isnan instead of val != val ? – Palsgrave

@JonClements: Oh, huh, that's a thing. I forgot the math module actually has an isnan. – Roobbie

@piRSquared: The NaN in their dict isn't the same object as np.nan. Python does an is check before == here, which is throwing you off. – Roobbie

Answer 2

This dictionary comprehension helps extract a dict with NaN's (unlike np.isnan(v), which raises an error):

{k:v for k,v in my_dict.items() if v is np.nan}

Then you can check for len() of the resulting dict being positive if boolean info is sufficient.

Palomo