Importing a library from (or near) a script with the same name raises "AttributeError: module has no attribute" or an ImportError or NameError
Asked Answered
U

4

96

I have a script named requests.py that needs to use the third-party requests package. The script either can't import the package, or can't access its functionality.

Why isn't this working, and how do I fix it?

Trying a plain import and then using the functionality results in an AttributeError:

import requests

res = requests.get('http://www.google.ca')
print(res)
Traceback (most recent call last):
  File "/Users/me/dev/rough/requests.py", line 1, in <module>
    import requests
  File "/Users/me/dev/rough/requests.py", line 3, in <module>
    requests.get('http://www.google.ca')
AttributeError: module 'requests' has no attribute 'get'

In more recent versions of Python, the error message instead reads AttributeError: partially initialized module 'requests' has no attribute 'get' (most likely due to a circular import).

Using from-import of a specific name results in an ImportError:

from requests import get

res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
  File "requests.py", line 1, in <module>
    from requests import get
  File "/Users/me/dev/rough/requests.py", line 1, in <module>
    from requests import get
ImportError: cannot import name 'get'

In more recent versions of Python, the error message instead reads ImportError: cannot import name 'get' from partially initialized module 'requests' (most likely due to a circular import) (/Users/me/dev/rough/requests.py).

Using from-import for a module inside the package results in a different ImportError:

from requests.auth import AuthBase
Traceback (most recent call last):
  File "requests.py", line 1, in <module>
    from requests.auth import AuthBase
  File "/Users/me/dev/rough/requests.py", line 1, in <module>
    from requests.auth import AuthBase
ImportError: No module named 'requests.auth'; 'requests' is not a package

Using a star-import and then using the functionality raises a NameError:

from requests import *

res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
  File "requests.py", line 1, in <module>
    from requests import *
  File "/Users/me/dev/rough/requests.py", line 3, in <module>
    res = get('http://www.google.ca')
NameError: name 'get' is not defined

For cases where you name your module the same as an existing one on purpose and want to handle that situation, see How can I import from the standard library, when my project has a module with the same name? (How can I control where Python looks for modules?)

Unman answered 27/3, 2016 at 17:27 Comment(1)
Please note that this is intended as a canonical answer to a common question ... please don't close as duplicate unless you're sure the other question has a better, more complete answer.Lowspirited
U
90

This happens because your local module named requests.py shadows the installed requests module you are trying to use. The current directory is prepended to sys.path, so the local name takes precedence over the installed name.

An extra debugging tip when this comes up is to look at the Traceback carefully, and realize that the name of your script in question is matching the module you are trying to import:

Notice the name you used in your script:

File "/Users/me/dev/rough/requests.py", line 1, in <module>

The module you are trying to import: requests

Rename your module to something else to avoid the name collision.

Python may generate a requests.pyc file next to your requests.py file (in the __pycache__ directory in Python 3). Remove that as well after your rename, as the interpreter will still reference that file, re-producing the error. However, the pyc file in __pycache__ should not affect your code if the py file has been removed.

In the example, renaming the file to my_requests.py, removing requests.pyc, and running again successfully prints <Response [200]>.


Note: This doesn't only happen when naming your file as the module you are trying to import. This can also happen if you name your file the same as a module imported by a module you import directly. For example, having a file called copy.py and trying to import pandas from there, will give

ImportError: cannot import name 'copy' from 'copy'

That is because pandas imports copy. There is no magic solution here as you can't know all the modules' names in the world, but a rule of thumb is to try to make names of modules as unique as possible and try to change the name whenever you get such error.

Unman answered 27/3, 2016 at 17:27 Comment(0)
W
28

The error occurs because a user-created script has a name-clash with a library filename. Note, however, that the problem can be caused indirectly. It might take a little detective work to figure out which file is causing the problem.

For example: suppose that you have a script mydecimal.py that includes import decimal, intending to use the standard library decimal library for accurate floating-point calculations with decimal numbers. That doesn't cause a problem, because there is no standard library mydecimal. However, it so happens that decimal imports numbers (another standard library module) for internal use, so a script called numbers.py in your project would cause the problem.

If you still encounter problems like this after tracking own and renaming or removing the appropriate .py files in your project, also check for .pyc files that Python uses to cache bytecode compilation when importing modules. In 3.x, these will be stored in folders with the special name __pycache__; it is safe to delete such folders and files, and possible to suppress them (but you normally won't want to).

Wieland answered 11/9, 2018 at 8:16 Comment(0)
N
2

Summary

Problems such as this occur when a Python source file in the project has the same name as some external library module (either in the standard library or an installed third-party package). When attempting to import from the external library (which requires using an absolute import), the project's own module is found instead, causing various errors depending on the exact details.

The easiest way to fix the problem, in ordinary cases, is to rename the affected file. It may also be necessary to locate and delete any corresponding .pyc files. (This is perfectly safe; the files are just a cache of the work that was previously done to translate Python source code into bytecode for the Python virtual machine, analogous to .class files in Java.)

When the project's own module is preferred, this is because of how Python searches for module source code for absolute imports. Depending on exactly how Python is started, the module search path defined in sys.path will usually start with a path that is within the current project. Python will look there first, before looking in any standard library or third-party library folders. This search process does not care about the folder where the importing module is located; any relative paths are relative to the process' current working directory, not the importing module. (However, the standard library paths will normally be absolute paths, and near the end of sys.path, anyway.)

AttributeError

An AttributeError occurs because the project's own module simply doesn't define the function, class etc. that the calling code wants to use from the external library module. As a result, the 'module' object that represents the project's module has no attribute with the specified name, so that's exactly what the error message claims.

In rare, unfortunate cases, the project's own module might happen to define something with the same name such that it does something different. This can cause any kind of exception or other logical error. For example, if the first example from the question is modified:

import requests

def get():
    pass

res = requests.get('http://www.google.ca')
print(res)

now a TypeError will be raised instead, because the code will attempt to call the locally-defined get function with the wrong number of arguments.

ImportError

Using a specific from-import causes the error to be reported differently because the problem is now detected during the importing process itself. In the example in the question, from requests import get means, instead of creating a new global name requests which names the module, there should be a new global name get which names the function from that module (the one which would be referred to as requests.get after a plain import). This requires looking up the get attribute from that module; but since the wrong module was loaded, that attribute lookup fails. More recent versions of Python report this as a probable circular import.

Other import attempts can cause different ImportErrors which complain that the imported module "is not a package". This is self-explanatory: in the example in the question, requests is an ordinary module (because it is defined by a source code file, requests.py), but the desired requests - defined by the third-party library - is a package (defined by several files inside a requests folder elsewhere, including an __init__.py which defines some top-level package contents that are not modules, like the get function).

NameError

Using a star-import, like from requests import *, will not fail directly - it creates global names for all the module contents. However, since the module in the question example doesn't contain anything named get, its attempt to star-import itself will not define that name either. Thus, a NameError occurs - the normal error for trying to use a global name that doesn't exist.

Aggravating factors

  1. Python's default import system is based on names, not file paths, and Python does not distinguish between "(driver) script files" and "library (module) files". Any Python source code can be imported. Although Python caches module imports, the main script will not generally be in this cache, which means it is perfectly capable of attempting to import itself. This is what happens in the example in the question: since requests was not already imported, there is nothing in the cache, so the attempt in the driver script (named requests.py) to import requests will search for something to import, and find the driver script (i.e., its own source file).

    This only causes a problem at import time, if the module tries to use that import in its own initialization, e.g. by doing a from-import. Otherwise, the problem will be deferred, resulting in (typically) AttributeError or NameError when trying to use the functionality. See also:

  2. Whenever a module is loaded (i.e., imported from its source, rather than using the cache), its own top-level import statements run, causing more imports, recursively. Any of these indirect imports could potentially find the local code. The Python standard library is not a package, and mostly uses absolute import to refer to its other components. For example, as pointed out in Dave Rove's answer, attempting to import the standard library decimal module could fail when tried from a source file named numbers.py, or within a project that has such a source file.

    In one especially pernicious case, having a file named token.py in a project (or the current working directory, when starting up Python in interactive mode) causes the interactive help to break:

    $ touch token.py
    $ python
    Python 3.8.10 (default, Nov 14 2022, 12:59:47) 
    [GCC 9.4.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> help
    Type help() for interactive help, or help(object) for help about object.
    >>> help()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.8/_sitebuiltins.py", line 102, in __call__
        import pydoc
      File "/usr/lib/python3.8/pydoc.py", line 66, in <module>
        import inspect
      File "/usr/lib/python3.8/inspect.py", line 40, in <module>
        import linecache
      File "/usr/lib/python3.8/linecache.py", line 11, in <module>
        import tokenize
      File "/usr/lib/python3.8/tokenize.py", line 35, in <module>
        from token import EXACT_TOKEN_TYPES
    ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/current/working directory/token.py)
    

    The traceback tells us all we need to know: calling help triggers a deferred import of the standard library pydoc, which indirectly attempts to import the standard library token, but finds our token.py which doesn't contain the appropriate name. In older versions of Python, it was even worse: tokenize would do a star-import from token, and then its top-level code would try to use a name defined there, resulting in NameError - and a stack trace not mentioning the file name token.py.

  3. Python can import from cached bytecode files (.pyc) even if the corresponding source file is renamed or deleted. This is intended to speed up imports, but it creates an extra step to keep code directories "clean". After renaming any .py file to resolve a problem like this, make sure to check the bytecode cache as well.

Solving the problem

Of course, the standard advice is simple and straightforward: just rename the .py files that were mistakenly imported, and delete any cached bytecode .pyc files for those modules. By default, in Python 3.x, the .pyc files for any .py files within a folder will be put into a subfolder with the special name __pycache__; in 2.x, they simply appeared alongside the corresponding .py files.

However, while this approach quickly solves the immediate problem for most people encountering it, the advice does not scale very well. There are a lot of potentially problematic names; while most projects will not be interested in including a source file named e.g. os.py, some other names might be more desirable, or harder to work around. So, here are some other useful techniques:

Controlling sys.path

Of course, since the problem is caused by sys.path specifying that absolute imports should look in the current project first, it can be avoided by just changing the sys.path.

The problematic path is described in the documentation as a "potentially unsafe path". When using the interactive Python prompt, this will be an empty string (a relative path equivalent to '.') - i.e., the current working directory for the Python process, reflecting any changes made by using e.g. os.chdir. For driver scripts started normally (python driver.py), it will be the directory where the script is located, as an absolute path (not necessarily the current working directory, since a path could be specified on the command line, like python path/to/driver.py). For modules run using the -m command-line flag, it will be the initial current working directory (not necessarily where the module is located, but also an absolute path that will not be affected by os.chdir).

To avoid having this path in sys.path, do one of the following:

  • In Python 3.11 and up, set the PYTHONSAFEPATH environment variable, or use the -P command-line option to Python.

  • In Python 3.4 and up, use the -I command-line option to start in isolated mode. This, however, has several other effects: it will ignore environment variables like PYTHONPATH and PYTHONHOME, and it will also skip adding the user-specific site-packages directory to the path (therefore the code will not have access to third-party libraries that were installed using the --user option for Pip).

  • If all else fails, consider manually manipulating sys.path. This is messy and error-prone, but sys.path is just an ordinary list of strings with file paths, and modifying its contents will affect future imports. (Do not try to replace the list object; this will not work, because Python does not look for it by the name sys.path, but uses a hard-coded internal reference. It is not possible to destroy or replace that object from Python; sys.path is just a name that is initialized to refer to it.)

Keep in mind that removing the "unsafe" path will prevent intentional absolute imports from working within the package. This is inconvenient for some small projects, but also a good reason to learn proper package organization. Speaking of which:

Using relative imports within packages

A well-organized Python project will typically consist of one or more (usually just one) packages, stored in subfolders of the main project folder, plus one or more driver scripts placed outside of the package subfolders. The driver scripts will typically use a single absolute import to access the package functionality, while implementing some simple wrapper logic (e.g. to parse command-line arguments, format and report otherwise-uncaught exceptions, etc.). The package, meanwhile, will use relative imports throughout for its own content, and absolute import only where necessary to access other packages (the standard library and third-party dependencies).

For example, a project might be organized like:

project
├── src
│   └── my_package
│       └── x.py
│       └── y.py
│       └── z.py
└── driver.py

Code in driver.py will use absolute import into the package, like:

from my_package.x import entry_point

if __name__ == '__main__':
    entry_point()

(If logic is needed to parse the command-line arguments, it should normally go in the driver rather than in the package's code.)

Then, code inside the package will use relative imports - so x.py might contain something like:

from .y import first_thing
from .z import second_thing

def entry_point():
    first_thing()
    second_thing()

This gets the best of both worlds: the initial absolute import sets up the top-level package so that relative imports will work, and the relative imports will avoid depending on the sys.path configuration. Even without taking steps to configure sys.path, it will typically include the folder with the driver scripts, but not any package folders; thus, this also automatically avoids import path conflicts. (An absolute import won't find the package contents unless it specifies the corresponding package path; but typically when it does, importing from the current package was intentional.)

This also avoids setting traps for the next project which has this one as a dependency. Say for example that we implement and publish an API package, and someone else writes Client which has API as a dependency. Since API code will use relative import for other API functionality (say, from . import functionality), the Client project's own functionality.py won't cause a problem.

For the initial absolute import to work, of course, the top-level package folder needs to be mentioned in sys.path. However, this is normally accomplished by installing the package, so it does not cause a problem.

Disabling bytecode caching

If a file needs to be renamed, avoiding problems with .pyc files will be easier if they simply don't exist in the first place. They are not necessary, after all; they are, again, simply intended to speed up imports on subsequent runs of the program.

To suppress .pyc file generation, use the -B command-line option or set the PYTHONDONTWRITEBYTECODE environment variable. (There is no built-in solution to delete all existing .pyc files, but this is easy enough to implement by hand using e.g. the shutil standard library module.)

Linting to avoid problematic names

Consider using third-party tools (such as IDE plugins) to warn about filenames used by the standard library or by third-party libraries in the project. In Python 3.10 and up, the full list of standard library module names is also available as sys.stdlib_module_names. This includes names that might not be present in the current Python installation (e.g. OS-specific components or things that are sometimes disabled or omitted, such as Tkinter).

Nittygritty answered 25/4, 2023 at 16:41 Comment(0)
S
1

just put down the import which causing error to the last

ex:
import requests
import flask
import numpy

if import flask is causing error then move that to bottom of the imports

solution:-
import requests
import numpy
import flask
Susa answered 7/11, 2023 at 12:18 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.