How do I get Pylint to recognize NumPy members?

Question

I am running Pylint on a Python project. Pylint makes many complaints about being unable to find NumPy members. How can I avoid this while avoiding skipping membership checks?

From the code:

import numpy as np

print np.zeros([1, 4])

Which, when ran, I get the expected:

[[ 0. 0. 0. 0.]]

However, Pylint gives me this error:

E: 3, 6: Module 'numpy' has no 'zeros' member (no-member)

For versions, I am using Pylint 1.0.0 (astroid 1.0.1, common 0.60.0) and trying to work with NumPy 1.8.0.

Solorio

Answer 1

If using Visual Studio Code with Don Jayamanne's excellent Python extension, add a user setting to whitelist NumPy:

{
    // Whitelist NumPy to remove lint errors
    "python.linting.pylintArgs": [
        "--extension-pkg-whitelist=numpy"
    ]
}

Corticosterone

Hi there. I am using the same extension. Just a question: adding such setting will also whitelist all the errors in numpy? (btw: it doesn't work). – King

This has worked effectively for me. It is specifically for Visual Studio Code. Are you familiar with user settings? Menu item is Code > Preferences > User Settings. – Corticosterone

Hello, I am using vs code 1.9 on Ubuntu 16.04. Tried this solution, but pylint still reports same error 'E1101:Module 'numpy.random' has no 'shuffle'member. – Hurtle

This helped! on VSCode 1.12.2 confirmed it works on WIndows 10 x64. – Struble

I needed more: "python.linting.pylintArgs": [ "--ignored-modules=numpy", "--ignored-classes=numpy", "--extension-pkg-whitelist=numpy" ] – Dentistry

@Dentistry 's solution working in Windows 7 x64 and Visual Studio Code 1.15.1!! – Chateau

This answer doesn't really add anything to this answer. It is only describing how to pass command-line arguments to pylint. – Dishwater

@ZevSpitz I don't disagree. It's a very specific use case for developers who are using Visual Studio Code – Corticosterone

@Chateau Peter's post does not solve the problem, it ignores it. If I could downvote this comment I would.. – Aphyllous

This isn't working for me anymore with pylint 2.3.0. – Rader

Can someone please clarify where exactly I can add David's answer in vscode? – Bimestrial

That has changed a little - it used to open as a json text file, now it has its own settings editor. Check dev.to/thegeoffstevens/… for more info. – Corticosterone

It just turns numpy suggestions of, it don't solve the issue. – Jamal

In the new settings, find Python › Linting: Pylint Args, and then add the line --extension-pkg-whitelist=numpy – Skidmore

@Skidmore which "Python" do I click on? There is a Python text in the lower left corner and the lower right corner. I'm guessing lower right, but there is no "Linting" in the drop down menu. – Channel

Answer 2

I had the same issue here, even with the latest versions of all related packages (astroid 1.3.2, logilab_common 0.63.2, pylon 1.4.0).

The following solution worked like a charm: I added numpy to the list of ignored modules by modifying my pylintrc file, in the [TYPECHECK] section:

[TYPECHECK]

ignored-modules = numpy

Depending on the error, you might also need to add the following line (still in the [TYPECHECK] section):

ignored-classes = numpy

Premise

On linux with pylint 1.4.4, astroid 1.3.8 and Python 3.4.3, this worked, but I had to put the extension-pkg-whitelist=numpy line under the [MASTER] heading of the .pylintrc file. pylint seems to be fairly brittle software and requires an expert's touch to keep it working for basic tasks. – Introductory

This is not a good solution. All it does is completely disable pylint checking for the existence of any members. Ideally you'd want it to recognise them properly, which is what the other solutions do. – Puree

@Puree It's a security measure. At runtime, module definitions can change dynamically; but enabling this in pylint would involve running arbitrary code. Nevertheless I would still expect some sort of note in the answer about --extension-pkg-whitelist, which actually carries out the import for the specified module. – Dishwater

Answer 3

I was getting the same error for a small NumPy project I was working on and decided that ignoring the NumPy modules would do just fine. I created a .pylintrc file with:

$ pylint --generate-rcfile > ~/.pylintrc

And following paduwan's and j_houg's advice I modified the following sectors:

[MASTER]

# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=numpy

and

[TYPECHECK]

# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=numpy

# List of classes names for which member attributes should not be checked
# (useful for classes with attributes dynamically set). This supports can work
# with qualified names.
ignored-classes=numpy

and it "fixed" my issue.

Nalda

Are you sure you had to add it to the two ignored-* entrances as well? For me, just adding a module to the extension whitelist works perfectly. – Puree

Answer 4

In recent versions of Pylint you can add --extension-pkg-whitelist=numpy to your Pylint command.

They had fixed this problem in an earlier version in an unsafe way. Now if you want them to look more carefully at a package outside of the standard library, you must explicitly whitelist it. See here.

Perrine

Seems like it works for modules and packages but not class names. – Cullum

Answer 5

For ignoring all the errors generated by numpy.core‘s attributes, we can now use:

$ pylint a.py --generated-members=numpy.*

As another solution, add this option to ~/.pylintrc or /etc/pylintrc file:

[TYPECHECK]

# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=numpy.*

This feature was introduced in PyLint 1.6.0. It should be noted that code snippet from original question passed linting with this version even without any additional settings. However, this is useful in more complex cases.

Milagro

I had the same problem when using patsy.dmatrices. Adding generated-members=patsy.dmatrices solved my problem. – Artemis

What do you mean by "For mentioned in question code by now" (seems incomprehensible)? – Golliner

updated note about PyLint version @Dentistry Mortensen – Milagro

Answer 6

Since this is the top result in Google Search and it gave me the impression that you have to ignore those warnings in all files:

The problem has actually been fixed in the sources of Pylint/astroid last month https://bitbucket.org/logilab/astroid/commits/83d78af4866be5818f193360c78185e1008fd29e but are not yet in the Ubuntu packages.

To get the sources, just

hg clone https://bitbucket.org/logilab/pylint/
hg clone https://bitbucket.org/logilab/astroid
mkdir logilab && touch logilab/__init__.py
hg clone http://hg.logilab.org/logilab/common logilab/common
cd pylint && python setup.py install

whereby the last step will most likely require a sudo and of course you need Mercurial to clone.

Perigynous

I would say that you do not need to clone the new logilab/common but that you do need to install the new logilab/astroid. By re-installing logilab/astroid and logilab/pylint, it solves the bug for me. – Intrados

I've managed to solve the bug purely installing the new version of astroid without reinstalling pylint. As the commit is only in astroid I don't think you need to update pylint – Marsipobranch

@Elliot is most likely right since the bug is in astroid. I just prefer to have a coherent version of a package, i.e. either development version or tested version to avoid incompatibilities. – Perigynous

Which versions are you guys using? I'm on astroid 1.3.2 and pylint 1.4.0 and I still get the problem with this code from numpy import ceil results in E: 1, 0: No name 'ceil' in module 'numpy' (no-name-in-module) I checked the commit referenced above and it appears that those changes are in the version of astroid I have. – Lallans

Did exactly as suggested on Xubuntu 14.04 and this resulted in a non-working pylint: py2.7.egg/pylint/lint.py", line 866, in check_astroid_module astroid.close() AttributeError: 'Module' object has no attribute 'close' – Sublime

Also got a non-working pylint with an error File "/usr/local/lib/python2.7/dist-packages/pylint-1.4.0-py2.7.egg/pylint/utils.py", line 137, in tokenize_module with module.stream() as stream: AttributeError: 'Module' object has no attribute 'stream' – Rosenfeld

I'm getting this error as well, despite using pylint 1.4.1, astroid 1.3.4, and common 0.63.2. It happens with CPython 2.7 and CPython 3.4. I am not enabling absolute imports in 2.7. – Karalynn

Perhaps this is a regression - there does appear to have been a release intended to fix the problem. Either way, I've opened a new issue about it at bitbucket.org/logilab/pylint/issue/453/… – Karalynn

Apparently this is still not fixed in pylint 1.4.2, astroid 1.3.4: Module 'numpy' has no 'zeros' member (no-member) – Mymya

Installing with above gave me other errors using linuxmint , python 2.7 – Zoellick

The link is broken ("That link has no power here"). – Golliner

Answer 7

If you don't want to add more configuration, please add this code to your configuration file, instead of 'whitelist'.

{
    "python.linting.pylintArgs": ["--generate-members"],
}

Antennule

You should mention that this applies very specifically to VS Code. – Muscadel

It will output pylint: error: no such option: --generate-members – Waterfall

What is the name of the configuration file? – Golliner

@Spaceship222, should append =["numpy"] – Mezoff

Answer 8

There have been many different bugs reported about this over the past few years i.e. https://bitbucket.org/logilab/pylint/issue/58/false-positive-no-member-on-numpy-imports

I'd suggest disabling for the lines where the complaints occur.

# pylint: disable=E1103
print np.zeros([1, 4])
# pylint: enable=E1103

Pasadis

I use numpy so much, that I might as well just disable the no-member check in the entire file, however, I want to avoid doing that. – Solorio

-1 Just because @bijancn's answer should now supercede this one. – Stearne

@Stearne it doesn't though. Problem still exists in 1.4.2. paduwan's solution is better in that it doesn't require adding hacky cruft to your code. – Sepulveda

now I followed the advice in dev.to/ldsands/the-best-linter-for-black-in-vs-code-54a0 . Levi Sands means, it is the best to use the linter flake8 and to add lines in settings – Nicosia

The link is broken ("That link has no power here"). – Golliner

Answer 9

Probably, it's confused with NumPy's abstruse method of methods import. Namely, zeros is in fact numpy.core.multiarray.zeros, imported in NumPy with the statement

from .core import *

in turn imported with

from .numeric import *

and in numeric you'll find

zeros = multiarray.zeros

I guess I would be confused in place of Pylint!

See this bug for the Pylint side of view.

Fourcycle

I wish I could just import single methods like that, but I use far too many functions and it would make the imports be a huge mess. – Solorio

@Solorio It would be a huge mess even to find all of them. See suggestion in the link in the end of my answer. – Fourcycle

That SO link makes PyLint ignore importing some modules. I am not so sure that it would make it suppress no-member errors for those files. I also would like to avoid patching my PyLint if at all possible. – Solorio

@Solorio I guess you should wait for a patch to PyLint then. – Fourcycle

Answer 10

I had the same problem with a different module (kivy.properties) which is a wrapped C module like NumPy.

Using Visual Studio Code V1.38.0, the accepted solution stopped all linting for the project. So, while it did indeed remove the false-positive no-name-in-module, it didn't really improve the situation.

The best workaround for me was to use the --ignored-modules argument on the offending module. The trouble is, passing any argument via python.linting.pylintArgs wipes out the default Visual Studio Code settings, so you need to reset those also. That left me with the following settings.json file:

{
    "python.pythonPath": "C:\\Python\\Python37\\python.exe",
    "python.linting.pylintEnabled": true,
    "python.linting.enabled": true,
    "python.linting.pylintArgs": [
        "--ignored-modules=kivy.properties",
        "--disable=all",
        "--enable=F,E,unreachable,duplicate-key,unnecessary-semicolon,global-variable-not-assigned,unused-variable,binary-op-exception,bad-format-string,anomalous-backslash-in-string,bad-open-mode"
    ]
}

Bewhiskered

"python.linting.pylintArgs": [ "--generate-members=kivy.properties" ] – Choriocarcinoma

This was it! Thank you for sharing. Yes, the only way to get rid of the errors in VSCode is using --ignored-modules. But I could not figure out why I was getting info errors about my classes not having docstrings - turns out it was indeed because the default VSCode arguments passed to pylint were wiped out and replaced with only --ignored-modules. @Choriocarcinoma No, that causes pylint to throw an error and exit, resulting in zero linting. --generate-members is not a valid pylint flag. I've seen that flag in way too many answers/comments... – Tees

Correct usage: --generated-members – Arreola

Answer 11

This has finally been resolved in Pylint 1.8.2. It works out of the box, and pylintrc tweaks aren't needed!

Foliolate

Answer 12

I had to add this at the top of any file where I use NumPy a lot.

# To ignore numpy errors:
#     pylint: disable=E1101

Just in case someone in eclipse is having trouble with Pydev and pylint...

Aftermath

Answer 13

In extension to j_hougs answer, you can now add the modules in question to this line in .pylintrc, which is already prepared empty on generation:

extension-pkg-whitelist=numpy

You can generate a sample .pylintrc by doing:

pylint --generate-rcfile > .pylintrc

And then edit the mentioned line.

Celt

Answer 14

This is the pseudo-solution I have come up with for this problem.

#pylint: disable=no-name-in-module
from numpy import array as np_array, transpose as np_transpose, \
      linspace as np_linspace, zeros as np_zeros
from numpy.random import uniform as random_uniform
#pylint: enable=no-name-in-module

Then, in your code, instead of calling NumPy functions as np.array and np.zeros and so on, you would write np_array, np_zeros, etc. Advantages of this approach vs. other approaches suggested in other answers:

• The Pylint disable/enable is restricted to a small region of your code
• That means that you don't have to surround every single line that has an invocation of a NumPy function with a Pylint directive.
• You are not doing Pylint disable of the error for your whole file, which might mask other issues with your code.

The clear disadvantage is that you have to explicitly import every NumPy function you use. The approach could be elaborated on further. You could define your own module, call it say, numpy_importer as follows

""" module: numpy_importer.py
       explicitely import numpy functions while avoiding Pylint errors
"""
#pylint: disable=unused-import
#pylint: disable=no-name-in-module
from numpy import array, transpose, zeros  #add all things you need
from numpy.random import uniform as random_uniform
#pylint: enable=no-name-in-module

Then, your application code could import this module only (instead of NumPy) as

import numpy_importer as np

and use the names as usual: np.zeros, np.array etc.

The advantage of this is that you will have a single module in which all NumPy related imports are done once and for all, and then you import it with that single line, wherever you want. Still you have to be careful that numpy_importer does not import names that don’t exist in NumPy as those errors won't be caught by Pylint.

Mothering

Re "pseudo-solution": Do you mean "pseudocode-solution"? – Golliner

@PeterMortensen: No I mean "pseudo-solution". There is no pseudocode anywhere. Pseudo is a generic prefix that you can attach to many things. – Mothering

Answer 15

I had this problem with NumPy, SciPy, sklearn, nipy, etc., and I solved it by wrapping epylint like so:

File epylint.py

#!/usr/bin/python

"""
Synopsis: epylint wrapper that filters a bunch of false-positive warnings and errors
Author: DOHMATOB Elvis Dopgima <[email protected]> <[email protected]>

"""

import os
import sys
import re
from subprocess import Popen, STDOUT, PIPE

NUMPY_HAS_NO_MEMBER = re.compile("Module 'numpy(?:\..+)?' has no '.+' member")
SCIPY_HAS_NO_MEMBER = re.compile("Module 'scipy(?:\..+)?' has no '.+' member")
SCIPY_HAS_NO_MEMBER2 = re.compile("No name '.+' in module 'scipy(?:\..+)?'")
NIPY_HAS_NO_MEMBER = re.compile("Module 'nipy(?:\..+)?' has no '.+' member")
SK_ATTR_DEFINED_OUTSIDE_INIT = re.compile("Attribute '.+_' defined outside __init__")
REL_IMPORT_SHOULD_BE = re.compile("Relative import '.+', should be '.+")
REDEFINING_NAME_FROM_OUTER_SCOPE = re.compile("Redefining name '.+' from outer scope")

if __name__ == "__main__":
    basename = os.path.basename(sys.argv[1])
    for line in Popen(['epylint', sys.argv[1], '--disable=C,R,I'  # filter thesew arnings
                       ], stdout=PIPE, stderr=STDOUT, universal_newlines=True).stdout:
        if line.startswith("***********"):
            continue
        elif line.startswith("No config file found,"):
            continue
        elif "anomalous-backslash-in-string," in line:
            continue
        if NUMPY_HAS_NO_MEMBER.search(line):
            continue
        if SCIPY_HAS_NO_MEMBER.search(line):
            continue
        if SCIPY_HAS_NO_MEMBER2.search(line):
            continue
        if "Used * or ** magic" in line:
            continue
        if "No module named" in line and "_flymake" in line:
            continue
        if SK_ATTR_DEFINED_OUTSIDE_INIT.search(line):
            continue
        if "Access to a protected member" in line:
            continue
        if REL_IMPORT_SHOULD_BE.search(line):
            continue
        if REDEFINING_NAME_FROM_OUTER_SCOPE.search(line):
            continue
        if NIPY_HAS_NO_MEMBER.search(line):
            continue
        # XXX extend by adding more handles for false-positives here
        else:
            print line,

This script simply runs epylint, and then scrapes its output to filter out false-positive warnings and errors. You can extend it by added more elif cases.

N.B.: If this applies to you, then you'll want to modify your pychechers.sh so it likes like this

#!/bin/bash

epylint.py "$1" 2>/dev/null
pyflakes "$1"
pep8 --ignore=E221,E701,E202 --repeat "$1"
true

(Of course, you have to make epylint.py executable first.)

Here is a link to my .emacs https://github.com/dohmatob/mydotemacs.

Octastyle

Answer 16

This seems to work in at least Pylint 1.1.0:

[TYPECHECK]

ignored-classes=numpy

Foliolate

Answer 17

This solution worked for me.

Basically, go to select the gear icon from the bottom left → Setting → Workspace Setting → Extension → Python Configuration → click on any Settings.json → add this in the file "python.linting.pylintArgs" : [ "--extension-pkg-whitelist=numpy" ]

I am using Visual Studio Code 1.27.2.

Gibbie

Answer 18

A little bit of copy paste from the previous answer to summarize what is working (at least for me: Debian 8 (Jessie))

1. In some older version of Pylint there was a problem preventing it working with NumPy (and other similar packages).

2. Now that problem has been solved, but external C packages (Python interfaces to C code -like NumPy-) are disabled by default for security reasons.

3. You can create a white list, to allow Pylint to use them in the file ~/.pylintrc.

Basic command to run:

# ONLY if you do not already have a .pylintrc file in your home
$ pylint --generate-rcfile > .pylintrc

Then open the file and add the packages you want after extension-pkg-whitelist= separated by comma. You can have the same behavior using the option --extension-pkg-whitelist=numpy from the command line.

If you ignore some packages in the [TYPECHECK] section that means that Pylint will never show errors related to those packages. In practice, Pylint will not tell you anything about those packages.

Behah

Answer 19

I've been working on a patch to Pylint to solve the issue with dynamic members in libraries such as NumPy.

It adds a "dynamic-modules" option which forces to check if members exist during runtime by making a real import of the module. See Issue #413 in logilab/pylint. There is also a pull request; see link in one of the comments.

Kebab

This is how pydev solves it (a special list of modules to load-inspect). How's that work going? – Jacobian

The link is broken ("That link has no power here"). – Golliner

The issue #413 is here. "Add "dynamic-modules" option to allow for checking member attributes dynamically (patch attached) #413" github.com/PyCQA/pylint/issues/413 – Stellarator

It was closed with this message dated 18 Feb 2016. It's reasonable pylint won't have it soon. Closing this since it's not going to happen. Apart of extension-pkg-whitelist, no other feature which involves dynamic code execution will be involved in pylint, since it breaks the promise of a static analysis tool. The idea is that pylint 2.0 / astroid 2.0 will become capable enough so that a combination of extension-pkg-whitelist and advanced inference should be enough for not having false positives as the ones caused by not importing the module in the first place. – Stellarator

Answer 20

A quick answer: update Pylint to 1.7.1 (use conda-forge provided Pylint 1.7.1 if you use Conda to manage packages).

I found a similar issue in Pylint GitHub here and someone replied everything getting OK after updating to 1.7.1.

Fetid

Answer 21

I'm not sure if this is a solution, but in Visual Studio Code once I wrote explicitly in my user settings to enable Pylint, all modules were recognized.

{
    "python.linting.pep8Enabled": true,
    "python.linting.pylintEnabled": true
}

Paradis

Answer 22

Lately (since something changed in Spyder or Pylint or ?), I have been getting E1101 errors ("no member") from Spyder's static code analysis on astropy.constants symbols. I don't have any idea why.

My simplistic solution for all users on a Linux or Unix system (Mac is probably similar) is to create an /etc/pylintrc file as follows:

[TYPECHECK]
ignored-modules=astropy.constants

Of course, this could, instead, be put in a personal $HOME/.pylintrc file. And, I could have updated an existing file.

Chardin

Answer 23

It's an old bug with generated-members. If you use another name than numpy, like np or foo, then you must add it in the Pylint configuration (space or comma-separated - no matter), because Pylint doesn't recognize that. So it should look like this:

[pylint]
generated-members=numpy.*,np.*,foo.*

It even throws a full path in no-member error, but the expression needs to be exact. It's an old bug and seems to be fixed soon. Check Issue #2498 in logilab/pylint.

Carloscarlota