sklearn DeprecationWarning truth value of an array

Question

Running a rasa_core example from the docs with

› python3 -m rasa_core.run -d models/dialogue -u models/nlu/default/current

and get this error output after each message in the dialog:

.../sklearn/...: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.

It's an issue with numpy that has been fixed but not been published in the latest release: https://github.com/scikit-learn/scikit-learn/issues/10449

The following has not worked to temporarily silence the warning:

1. Adding -W ignore

python3 -W ignore -m rasa_core.run -d models/dialogue -u models/nlu/default/current

2. warnings.simplefilter

python3

>>> warnings.simplefilter('ignore', DeprecationWarning)
>>> exit()

python3 -m rasa_core.run -d models/dialogue -u models/nlu/default/current

Pretender

I was facing the same problem while trying the default example in an iPython notebook, and this link helped suppress the warnings in my case. Might help you too in case you haven't tried this one! – Masticate

Answer 1

This warning is caused by numpy which deprecated the truth value check on empty array

Rationale for this change is

It is impossible to take advantage of the fact that empty arrays are False, because an array can be False for other reasons.

Check following example:

>>> import numpy as np
>>> bool(np.array([]))
False
>>> # but this is not a good way to test for emptiness, because...
>>> bool(np.array([0]))
False

Solution

As per issue 10449 on scikit-learn library, this has been fixed in master branch of library. However that will be available around August 2018 so one possible alternate is to use a lesser version of numpy library which does not have this issue i.e. 1.13.3 since scikit-library by default would refer latest version of numpy(which is 1.14.2 at the time of writing this answer)

sudo pip install numpy==1.13.3

or with pip3 as follows

sudo pip3 install numpy==1.13.3

Ignoring the warning(s)

In case we want to use the latest version of library(numpy in this case) which is giving the deprecation warning and just want to silence the deprecation warning then we can achieve it by using filterwarnings method of python's Warnings module

Following example below would produce the deprecation warning mentioned in question above:

from sklearn import preprocessing

if __name__ == '__main__':
    le = preprocessing.LabelEncoder()
    le.fit([1, 2, 2, 6])
    le.transform([1, 1, 2, 6])
    le.inverse_transform([0, 0, 1, 2])

produces

/usr/local/lib/python2.7/dist-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use array.size > 0 to check that an array is not empty.

And to take care of it, add filterwarnings for DeprecationWarning

from sklearn import preprocessing
import warnings

if __name__ == '__main__':
    warnings.filterwarnings(action='ignore', category=DeprecationWarning)
    le = preprocessing.LabelEncoder()
    le.fit([1, 2, 2, 6])
    le.transform([1, 1, 2, 6])
    le.inverse_transform([0, 0, 1, 2])

In case there are multiple modules which are giving warning and we want to selectively silent warning then use module attribute. e.g. to silent deprecation warning from scikit learn module

warnings.filterwarnings(module='sklearn*', action='ignore', category=DeprecationWarning)

Deese

+1 but not accepted as answer because it doesn't address silencing the error without changing versions of numpy – Pretender

@Pretender I have edited the answer to show how to ignore warnings. We can use filterwarnings method to ignore any category of warning. – Deese

@Deese for some reason this does not work for me. It keeps printing the warnings :( – Ette

@CarlosA.JimenezHolmquist, Can you share a reproducible example where you are seeing the issue? – Deese

Answer 2

I too had the same problem. The following solution worked for me. Go to the path file and line number mentioned in the warning that is if you use anaconda go to Anaconda\envs\py2\Lib\site-packages\sklearn\preprocessing\label.py Line number:151.

Change the following code

if diff:
    raise ValueError("y contains new labels: %s" % str(diff))

to the following

if diff.size>0:
    raise ValueError("y contains new labels: %s" % str(diff))

Teets