Prologue: some terminology
Absolute import is when Python searches folders in what I will call the system module path (SMP), one at a time, until it finds one that contains the module.
The SMP is created at startup, based on the PYTHONPATH
environment variable and some other things. It is represented within Python as a list of strings. After importing the sys
standard library module, it is available as sys.path
, but it exists whether that module is imported or not. (Usually, Python programmers just call this list "sys.path". However, since we are discussing the import mechanism in deep technical detail, and using modules named sys
in examples, it seems appropriate to establish a separate term.)
On the other hand, relative import directly specifies where the module's code should be, relative to the current module, within the context of the current package.
Summary
How can we import from the standard library instead of the current package?
If the standard library module in question is not implemented as a builtin, it may be necessary to set the PYTHONSAFEPATH
environment variable before starting Python. This prevents Python from putting the main script's directory (when started like python script.py
) or the current working directory (otherwise) at the start of the SMP, as it does by default.
In 3.11, the -P option can be used instead of setting PYTHONSAFEPATH
. In 3.4 and up, -I can also be used, but this will also ignore other Python-specific environment variables, and skip adding the per-user site-packages directory to the SMP. (It is also possible, of course, to modify sys.path
programmatically to fix the SMP. Unless, of course, the problem is with trying to import sys
.)
Once that is taken care of, just use an absolute import.
In 3.x:
import sys # or
from sys import version # or
from sys import * # but beware namespace pollution
In 2.5 through 2.7, a __future__
import is required first:
from __future__ import absolute_import
# proceed as above
In 2.4 and below, a hack is necessary to prevent implicit relative import:
sys = __import__("sys", {})
This should also work in other versions, but is far uglier and more complex than necessary.
Alternately, try hooking into the import machinery to change the lookup behaviour, as described in Omnifarious' answer, Boaz Yaniv's answer or casey's answer. All of these approaches work by emulating (part of) the internal algorithm for searching for modules, but skipping the relative import step and also skipping the first element of sys.path
. (The premise is that sys.path
is assumed to be "unsafe" - per the above discussion of PYTHONSAFEPATH
- and thus using that path would look within the project.)
How can we prevent the standard library from importing from the current package, and make sure it imports from itself?
The standard library generally uses absolute imports. If this causes a problem due to shadowing by the current package, the standard advice is to rename the modules in the current package. Failing that, the same environment variable and command-line flag tricks should work.
How can we import from the current package instead of the standard library?
In 2.4 and earlier, this will happen by default.
In 2.5 onward, use a relative import, after ensuring that the package is set up correctly.
For the most part, this means making sure that the root of the package is on the SMP. The most robust way to do this is to activate a virtual environment and install the package in that virtual environment. Otherwise, assuming the program starts from some "driver" script, make sure it is in the same folder as the package root. Alternately, run the package (or a subpackage, with the appropriate dotted-path name) as a module, using the -m
flag, from the directory that contains the package root.
Assuming the package is set up properly, relative imports look like:
from . import sys # sys.py in the same folder as this source file
from .sys import something # from our source, not the standard library
from .sys import * # again, beware namespace pollution
from .child import sys # child/sys.py
from .. import sys # ../sys.py, IF .. is still within the package root
from ..sibling import sys # ../sibling/sys.py
# Use more .'s to go up more levels first.
# This will not work beyond the package root.
Note that all of these use the from
syntax. As explained in the PEP:
Relative imports must always use from <> import
; import <>
is always absolute.... because after import XXX.YYY.ZZZ
... XXX.YYY.ZZZ
is usable in an expression. But .moduleY
is not usable in an expression.
As an absolute last resort, the SMP can be deliberately modified via sys.path
, after determining the path to the current file and thus computing the path to the package root, so that absolute imports will work. Please note that this shouldn't ever be necessary in ordinary circumstances. Many popular, heavy-duty Python libraries manage to span hundreds of thousands of lines of code, absolutely none of which mention sys.path
in any way.
How can we entice the standard library to import from the current package, rather than importing from itself like it was designed to?
I'm not aware offhand of any places where the Python standard library uses relative imports internally, and I also can't fathom a good reason to attempt this. That said, it's probably possible by hacking the import machinery. It doesn't sound easy or fun and I'm not about to attempt it here.
How can we import from elsewhere?
See How can I import a module dynamically given the full path?. Brandon Squizzato's answer also summarizes the usual technique.
Where Python looks for the module
2.4 and before
Originally, there was no special syntax for relative imports. Python would try a relative import first, and then an absolute import if that failed. So for example, code like import sys
would import a sys.py
from the same folder as the current module - if it existed - rather than the standard library module.
My understanding is that such relative imports would be relative to the package root, not to the current module; but I cannot easily verify this at the moment.
It was apparently possible to work around this, by supplying an empty dict for the globals
context to __import__
. Quoting:
sys = __import__("sys", {})
The import
statement uses the global namespace to determine which
package it is called on; if you pass an empty namespace, it cannot infer
package information.
So, by using the __import__
function directly, it was possible to skip the attempted relative import.
2.5 through 2.7
PEP 328 introduced a proposal to make Python imports absolute by default, and use relative imports only with explicit syntax. (It also introduced the terms "absolute import" and "relative import" to the Python ecosystem.) Relative imports using this syntax are relative to the module doing the importing; additional leading .
s can be used to specify parent packages.
Starting with the very first release of Python 2.5, the new behaviour became available by using a __future__
import:
from __future__ import absolute_import
import math # checks the SMP
On the other hand,
from __future__ import absolute_import
from . import math # imports from the same directory as this source file
or
from __future__ import absolute_import
from .. import math # imports from the immediate parent directory,
# IF it is still within the package root
3.x onward
The behaviour described in PEP 328 became the default. 2.4's implicit relative import is no longer available, and the code explicitly specifies either absolute or relative import - no __future__
import required.
Just choosing absolute or relative import is not enough
Some hiccups that come up at this point
- Apparently, many people find relative imports tricky to use because of the subtlety of the package system. The key point is that the
.
s used in relative import syntax do not refer to levels in a directory tree, but to levels in a package tree. (After all, there are ways to load modules that don't involve .py
files on disk at all!)
- Even when an absolute import is used, code from the current project could still unexpectedly shadow the standard library. This is because of how the SMP is configured. In most cases, either the current working directory of the Python process, or the directory of the "driver" script that was launched from the command line, will be first on the SMP. Thus,
import this
might not find the standard library this
even though it's doing an absolute import. (On the other hand, import sys
would find the standard library; see the final section.)
- Meanwhile, the standard library doesn't take advantage of packaging very well. It doesn't have its own root package; and for the most part, when standard library modules depend upon each other, they use absolute import.
The last two points can interact in surprising ways:
$ touch token.py
$ python
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/_sitebuiltins.py", line 102, in __call__
import pydoc
File "/usr/lib/python3.8/pydoc.py", line 66, in <module>
import inspect
File "/usr/lib/python3.8/inspect.py", line 40, in <module>
import linecache
File "/usr/lib/python3.8/linecache.py", line 11, in <module>
import tokenize
File "/usr/lib/python3.8/tokenize.py", line 35, in <module>
from token import EXACT_TOKEN_TYPES
ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/current/working/directory/token.py)
This problem can be avoided by fixing the SMP with any of the techniques from the summary:
$ touch token.py
$ python -I
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()
Welcome to Python 3.8's help utility!
If this is your first time using Python, you should definitely check out
the tutorial on the Internet at https://docs.python.org/3.8/tutorial/.
Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".
To get a list of available modules, keywords, symbols, or topics, type
"modules", "keywords", "symbols", or "topics". Each module also comes
with a one-line summary of what it does; to list the modules whose name
or summary contain a given string such as "spam", type "modules spam".
help>
This still doesn't tell the whole story
The module loading system is actually much more complicated than described above. It's deliberately filled with hooks to allow for modifying the behaviour. In particular: aside from the SMP, there is also a "metapath" (made available as sys.meta_path
) that contains module-loaders that are actually used to load the modules.
The documentation gives a recipe approximating the process. (Of course, the real thing does not have to solve the bootstrapping problem of importing sys
and importlib.util
in order to implement the import system!) However, it doesn't really show what each finder in sys.meta_path
is doing.
Roughly, by default, for absolute imports:
First, _frozen_importlib.BuiltinImporter
checks if the module name matches a module that is built in to Python, so that it can be imported directly. Part of the standard library is implemented this way, and part is not; and that portion has changed over time.
Then, _frozen_importlib.FrozenImporter
tries to load a frozen module with the specified name.
Finally, if both of those fail, _frozen_importlib_external.PathFinder
searches the SMP.
Because of this, user code might shadow absolute imports of some standard library modules while not shadowing others. Suppose we have this test script import_test.py
:
def test_import(name):
module = __import__(name)
return any(attr for attr in dir(module) if not attr.startswith('__'))
if __name__ == '__main__':
print('imported this from standard library?', test_import('this'))
print('imported sys from standard library?', test_import('sys'))
Let's see what happens if empty Python files with those names are added to the CWD first:
$ touch this.py
$ touch sys.py
$ python import_test.py
imported this from standard library? False
imported sys from standard library? True
Since at least 2.0, sys.builtin_module_names
lists the names of modules that are built in to Python. Also, since 3.10, sys.stdlib_module_names
lists all possible module names in the standard library (even ones that were deliberately excluded when Python was compiled, or which are not available for the current operating system).
from __future__ import absolute_import
That way when Iimport module_name
I got the library but when I didimport mypackage.module_name
I got the local module. – Stadium