Create a mask both for nan and inf values in an array

Question

I have to remove both nan and inf values from two arrays. I found this post useful https://mcmap.net/q/989185/-pearson-correlation-and-nan-values for removing nan. Is there any similar solution when I can create a mask to remove both nan and inf values?

The example below is just illustrative, I have arrays of large dimensions (400 elements)

import numpy as np
from numpy import nan, inf

a = np.asarray([0.5, 6.2, np.nan, 4.5, np.inf])
b = np.asarray([np.inf, np.inf, 0.3, np.nan, 0.5])

bad = ~np.logical_or(np.isnan(a), np.isnan(b))

X = np.compress(bad, a)  
Y = np.compress(bad, b) 

BIAS = np.nanmean(X - Y)
RMSE = np.sqrt(np.nanmean((X - Y)**2))
CORR = np.corrcoef(X, Y)

I need this in order to get both the statistics and plots correctly

Mendicant

Use np.isinf and np.isnan? – Anastigmat

no, those arrays are quite short just as an example. It's a huge dataset. I need to remove a whole lot of them from each array of 400 elements – Mendicant

instead of (or in addition to) linking to another question, post the code you're using. – Carrier

Seems like an XY problem to me. – Anastigmat

Answer 1

np.isfinite

Test element-wise for finiteness (not infinity or not Not a Number).

Straightforward

Answer 2

You can use np.isfinite(). It will return a boolean mask with True wherever the values are neither infinite nor NAN.

You can get the finite values this way:

a = np.asarray(a)
a = a[np.isfinite(a)]

Or for both arrays together:

mask = np.isfinite(a) | np.isfinite(b)
a = a[mask]
b = b[mask]

Voluminous

OK but how do you then obtain the actual array without the values and not the boolean? – Mendicant

thank you! Just one further clarification: how can you take into consideration the elements that are good in both arrays, a and b? So to get them both with the same dimension? – Mendicant

Answer 3

It works fine to me.

I created it with this to solve the problem in merge NaN and masked arrays:

masked_red_diff = masked_red_diff[np.isfinite(masked_red_diff)]
masked_red_diff.mean()

Delladelle