sklearn's PLSRegression: "ValueError: array must not contain infs or NaNs"

Question

When using sklearn.cross_decomposition.PLSRegression:

import numpy as np
import sklearn.cross_decomposition

pls2 = sklearn.cross_decomposition.PLSRegression()
xx = np.random.random((5,5))
yy = np.zeros((5,5) ) 

yy[0,:] = [0,1,0,0,0]
yy[1,:] = [0,0,0,1,0]
yy[2,:] = [0,0,0,0,1]
#yy[3,:] = [1,0,0,0,0] # Uncommenting this line solves the issue

pls2.fit(xx, yy)

I get:

C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:44: RuntimeWarning: invalid value encountered in divide
  x_weights = np.dot(X.T, y_score) / np.dot(y_score.T, y_score)
C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:64: RuntimeWarning: invalid value encountered in less
  if np.dot(x_weights_diff.T, x_weights_diff) < tol or Y.shape[1] == 1:
C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:67: UserWarning: Maximum number of iterations reached
  warnings.warn('Maximum number of iterations reached')
C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:297: RuntimeWarning: invalid value encountered in less
  if np.dot(x_scores.T, x_scores) < np.finfo(np.double).eps:
C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py:275: RuntimeWarning: invalid value encountered in less
  if np.all(np.dot(Yk.T, Yk) < np.finfo(np.double).eps):
Traceback (most recent call last):
  File "C:\svn\hw4\code\test_plsr2.py", line 8, in <module>
    pls2.fit(xx, yy)
  File "C:\Anaconda\lib\site-packages\sklearn\cross_decomposition\pls_.py", line 335, in fit
    linalg.pinv(np.dot(self.x_loadings_.T, self.x_weights_)))
  File "C:\Anaconda\lib\site-packages\scipy\linalg\basic.py", line 889, in pinv
    a = _asarray_validated(a, check_finite=check_finite)
  File "C:\Anaconda\lib\site-packages\scipy\_lib\_util.py", line 135, in _asarray_validated
    a = np.asarray_chkfinite(a)
  File "C:\Anaconda\lib\site-packages\numpy\lib\function_base.py", line 613, in asarray_chkfinite
    "array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs

What could be the issue?

I am aware of scikit-learn GitHub issue #2089, but since I use scikit-learn 0.16.1 (with Python 2.7.10 x64) this problem should be solved (the code snippets mentioned in the GitHub issue work fine).

Haden

Is this an example you are trying to reproduce or did you define this data? It may be the case that you have an intractable problem, hence you get nans or inf values for some weights, which fail at some later point in the calculation – Leta

@Leta My original data were larger (dropbox.com/s/zjrz6upfeln07ga/SE-sklearn-PLSR.zip?dl=0), I tried to narrow down the issue. – Haden

Answer 1

The issue is caused by a bug in scikit-learn. I reported it on GitHub: https://github.com/scikit-learn/scikit-learn/issues/2089#issuecomment-152753095

Haden

Answer 2

Please check if any of your values being passed in are NaN or inf:

np.isnan(xx).any()
np.isnan(yy).any()

np.isinf(xx).any()
np.isinf(yy).any()

If any of those yields true. Remove the nan entries or inf entries. E.g. you can set them to 0 with:

xx = np.nan_to_num(xx)
yy = np.nan_to_num(yy)

It's also possible for numpy to be fed such large positive and negative and zeroed values, that the equations deep down in the library are producing zeros, Nan's or Inf's. One workaround, oddly enough, is to send in smaller numbers (say representative numbers between -1 and 1. One way to do this is by standardization, see: https://mcmap.net/q/777704/-scipy-optimize-curvefit-array-must-not-contain-infs-or-nans

If none of that solves the problem, then you may be dealing with a low level bug in the library your using, or some sort of singularity in your data. Create an sscce and post it to stackoverflow or create a new bug report on the library maintaining your software.

Dudgeon

Thanks. Both evaluate False: np.isnan(xx).any(): False; np.isnan(yy).any(): False – Haden

I've narrowed down the issue: github.com/scikit-learn/scikit-learn/issues/… – Haden

Interesting. Your github issue points toward some sort of singularity in the data. These should be handled gracefully by scikit-learn, so the way it is acting now is definitely buggy. – Dudgeon

nan and inf check can be combined in np.isfinite() – Dun

Answer 3

I found a tricky little solution that worked for me.

I was doing time series featurization through cesium with this code:

timeInput = np.array(timeData)
valueInput = np.array(data)

#Featurizing Data
featurizedData = featurize.featurize_time_series(times=timeInput,
                                                     values=valueInput,
                                                     errors=None,
                                                     features_to_use=featuresToUse)

which was resulting in this error:

ValueError: array must not contain infs or NaNs

for laughs, I checked the lengths and types of the data:

data:
70
<class 'numpy.int32'>

timeData: 
70
<class 'numpy.float64'>

I decided I'd try to convert data types with this one line of code:

valueInput = valueInput.astype(float)

and it worked, resulting in this code:

timeInput = np.array(timeData)
valueInput = np.array(data)
valueInput = valueInput.astype(float)

#Featurizing Data
try:
    featurizedData = featurize.featurize_time_series(times=timeInput,
                                                     values=valueInput,
                                                     errors=None,
                                                     features_to_use=featuresToUse)

if you're getting an error like this, give matching datatypes a shot

Turbulent

Answer 4

I can reproduce the same bug, I silenced this bug by filtering all 0s away

threshold_for_bug = 0.00000001 # could be any value, ex numpy.min
xx[xx < threshold_for_bug] = threshold_for_bug

This silences the bug (i never check the precision difference)

My system info:

numpy-1.11.2
python-3.5
macOS Sierra

Litha

Answer 5

You may want to check your weights for negative values, since this error will also be triggered with negative weights.

Moll

Answer 6

I had a similar problem with PRINCE library, for MCA study. My solution was to use "object" dtype, instead of "category". Very frustrating because I spend many hours to find an solution.

Brennabrennan