How can I import from the standard library, when my project has a module with the same name? (How can I control where Python looks for modules?)
Asked Answered
S

6

151

There is a module in my project folder called calendar. Elsewhere in the code, I would like to use the standard library Calendar class. But when I try to import this class, using from calendar import Calendar, this imports from my own module instead, causing errors later.

How can I avoid this? Do I have to rename the module?

Sabbatical answered 17/5, 2011 at 13:34 Comment(8)
It is a best practice not to name modules to hide builtin modules.Mandiemandingo
The solution is "pick a different name". Your approach of not renaming is a bad idea. Why can't you rename your module? What's wrong with renaming?Noodlehead
Indeed. It is because there is no good answer to this question that shadowing stdlib modules is so strongly discouraged.Bevbevan
I avoided using the same module name as the solutions seemed more trouble than it's worth. Thanks!Sabbatical
@Mandiemandingo This advice doesn’t scale, pure and simple. PEP328 readily acknowledges this.Greenery
See my answer here: #12011827Williamsen
I had a similar issue and landed on this question. I solved it with from __future__ import absolute_import That way when I import module_name I got the library but when I did import mypackage.module_name I got the local module.Stadium
Arguably related: Importing installed package from script with the same name raises "AttributeError: module has no attribute" or an ImportError or NameErrorMadel
P
8

In Python 3.5 and up, use the standard library importlib module to import directly from a specified path, bypassing import's lookup mechanism:

import importlib.util
import sys

# For illustrative purposes.
import tokenize
file_path = tokenize.__file__  # returns "/path/to/tokenize.py"
module_name = tokenize.__name__  # returns "tokenize"

spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)

In actual code, file_path can be set to any path to a .py file to import; module_name should be the name of the module that will be imported (the name that the import system uses to look up the module when further import statements are attempted). Subsequent code will use module as the name of the module; change the variable name module to use a different name.

To load a package instead of a single file, file_path should be the path to the package's root __init__.py.

Pouf answered 8/1, 2020 at 17:36 Comment(9)
Works like a charm... Used this to test while developing a library, so that my tests were always using the developing version and not the published (and installed) one. In windows 10 i had to write the path to my module like this: file_path=r"C:\Users\My User\My Path\Module File.py". Then i called module_name just like the released module so that i had full working script that, stripped off this snippet, coud be used on other pcsNigercongo
Where are the default python modules located? How do you know it will be the same on every system? Is this solutions portable?Eupheemia
The first sentence should be removed... or it would let people think that this solution contains a now-deprecated approach.Spire
I also want to ask whether the argument "module_name" is important or not, since it seems to be ok even if I pass an empty string into it.Spire
But how does one import a module from the standard library?Merilee
How's this even an accepted answer? The question is to import from the standard lib, not user code.Tetracaine
Good points. My reply here was pointing out that the methods of importing a custom path in the previously accepted answer were now deprecated, but my reply is not showing the complete answer to the question. Presumably it will need to be a combination of the logic Boaz Yaniv gave (first find the std lib package's init.py) and then the code here to actually import that file.Pouf
@Tetracaine because this shows a technique from importing directly from an explicitly specified full path to a file. To import from the standard library, this technique only requires first determining the path to where the standard library modules are stored.Madel
On this other hand, this does not completely solve the problem of importing directly from the standard library, because some standard library modules are built-in and not represented by any file on disk. It won't be possible to load them dynamically given a file path (this technique), because there is no path to use.Madel
B
146

It's not necessary to rename the module. Instead, in Python 2.5 and above, use absolute_import to change the importing behavior.

For example, to import the standard library socket module, even if there is a socket.py in the project:

from __future__ import absolute_import
import socket

In Python 3.x, this behaviour is the default. Pylint will complain about the code, but it's perfectly valid.

Breathe answered 26/11, 2011 at 18:40 Comment(9)
This seems the correct answer to me. See the 2.5 changelog or PEP328 for more.Schargel
This is the correct solution. Unfortunately, it doesn’t work when code from within the package is launched because then the package isn’t recognised as such, and the local path is prepended to PYTHONPATH. Another question shows how to solve that.Greenery
This is the solution. I checked for Python 2.7.6 and this is required, it's still not the default.Sumerian
Indeed: The first python version where this behaviour is the default was 3.0, according to docs.python.org/2/library/__future__.htmlRotogravure
If I have a file named gzip.py, and I call import gzip there, it always ends up as being a reference to __main__ module (i.e. - it has a gzip.gzip field, but no gzip.open).Demonetize
Then do not name your main module as one that clashes with a builtin module.Cachou
@TomaszGandor, you could use the python convention for nameclashes: add a trailing underline. gzip -> gzip_Dace
The link in this question just shows me a PGP signature, not any code.Madel
This does not address the problem. Absolute import means that Python will search the sys.path directly, rather than looking for a path relative to the current source code. However, it is possible and common for certain paths that are near the current source code, to be at the front of the sys.path. Please see my answer for details.Madel
A
39

Actually, solving this is rather easy, but the implementation will always be a bit fragile, because it depends python import mechanism's internals and they are subject to change in future versions.

(the following code shows how to load both local and non-local modules and how they may coexist)

def import_non_local(name, custom_name=None):
    import imp, sys

    custom_name = custom_name or name

    f, pathname, desc = imp.find_module(name, sys.path[1:])
    module = imp.load_module(custom_name, f, pathname, desc)
    f.close()

    return module

# Import non-local module, use a custom name to differentiate it from local
# This name is only used internally for identifying the module. We decide
# the name in the local scope by assigning it to the variable calendar.
calendar = import_non_local('calendar','std_calendar')

# import local module normally, as calendar_local
import calendar as calendar_local

print calendar.Calendar
print calendar_local

The best solution, if possible, is to avoid naming your modules with the same name as standard-library or built-in module names.

Applicant answered 17/5, 2011 at 14:6 Comment(12)
How will this interact with sys.modules and subsequent attempts to load the local module?Gingili
@Omnifarious: It will add the module to sys.modules with its name, which will prevent loading the local module. You can always use a custom name to avoid that.Applicant
@Boaz Yaniv: You should be using a custom name for the local calendar, not the standard one. Other Python modules might try to import the standard one. And if you do that, what you achieve with this is to basically rename the local module without having to rename the file.Gingili
@Omnifarious: You could do it either way. Some other code may try to load the local module and get the very same error. You'll have to make a compromise, and it's up to you to decide which module to support.Applicant
And just in case someone wants to do the reverse, as Omnifarious suggested, and use a custom name for the local module - all you need to do is to change sys.path[1:] to sys.path[:1] and import_non_local() becomes import_local().Applicant
Thanks for that Boaz! Although your snippet is shorter (and document), I think it is just easier to rename the module than have some hacky code that may confuse people (or myself) in the future.Sabbatical
My f was None, so I added an if clause to help. Thanks!Letitialetizia
Enabling absolute imports as suggested in other solution is better. from __future__ import absolute_importPropagandist
Does anybody have a version of this code snippet that works without import imp which is deprecated?Caulicle
What if the local file is named sys.py?Eupheemia
imp is deprecated how can this be done with importlib?Eupheemia
Code like this should validate the content of sys.path[0] before skipping it - it could be a valid standard library path that should be searched. See my answer for details.Madel
G
14

The only way to solve this problem is to hijack the internal import machinery yourself. This is not easy, and fraught with peril. You should avoid the grail shaped beacon at all costs because the peril is too perilous.

Rename your module instead.

If you want to learn how to hijack the internal import machinery, here is where you would go about finding out how to do this:

There are sometimes good reasons to get into this peril. The reason you give is not among them. Rename your module.

If you take the perilous path, one problem you will encounter is that when you load a module it ends up with an 'official name' so that Python can avoid ever having to parse the contents of that module ever again. A mapping of the 'official name' of a module to the module object itself can be found in sys.modules.

This means that if you import calendar in one place, whatever module is imported will be thought of as the module with the official name calendar and all other attempts to import calendar anywhere else, including in other code that's part of the main Python library, will get that calendar.

It might be possible to design a customer importer using the imputil module in Python 2.x that caused modules loaded from certain paths to look up the modules they were importing in something other than sys.modules first or something like that. But that's an extremely hairy thing to be doing, and it won't work in Python 3.x anyway.

There is an extremely ugly and horrible thing you can do that does not involve hooking the import mechanism. This is something you should probably not do, but it will likely work. It turns your calendar module into a hybrid of the system calendar module and your calendar module. Thanks to Boaz Yaniv for the skeleton of the function I use. Put this at the beginning of your calendar.py file:

import sys

def copy_in_standard_module_symbols(name, local_module):
    import imp

    for i in range(0, 100):
        random_name = 'random_name_%d' % (i,)
        if random_name not in sys.modules:
            break
        else:
            random_name = None
    if random_name is None:
        raise RuntimeError("Couldn't manufacture an unused module name.")
    f, pathname, desc = imp.find_module(name, sys.path[1:])
    module = imp.load_module(random_name, f, pathname, desc)
    f.close()
    del sys.modules[random_name]
    for key in module.__dict__:
        if not hasattr(local_module, key):
            setattr(local_module, key, getattr(module, key))

copy_in_standard_module_symbols('calendar', sys.modules[copy_in_standard_module_symbols.__module__])
Gingili answered 17/5, 2011 at 13:58 Comment(23)
imputil is considered deprecated. You should use the imp module.Applicant
Which is perfectly compatible with Python 3, by the way. And not that hairy to use at all. But you should always be aware that code that relies on python treating paths in one way or looking up modules in that order may break sooner or later.Applicant
@Boaz Yaniv: I know, but it doesn't seem like there's a replacement for the functionality until Python 3. As far as I can tell, the imp module merely provides an implementation of the standard import mechanism, not a way to globally hook the import mechanism to make it do what you want.Gingili
Right, but in such an isolated case (module name collision) hooking the import mechanism is an overkill. And since it's hairy and incompatible, it's better left alone.Applicant
@Boaz Yaniv: But in order to really get import calendar to do exactly what you're expecting in every instance, hooking the import mechanism is the only way to go. Though, I can think of an extremely ugly way to handle this in this specific case. I will outline it. It's evil though. It does use your idea.Gingili
@Boaz Yaniv: There, my answer now includes the evil, horrible, ugly mechanism that you should not use.Gingili
Ah, I see. Yes, it is quite a hack. :) But even more than the random names and messing with sys.modules, I'd worry about effectively merging the namespace of both local and standard modules.Applicant
@Boaz Yaniv: That is worrisome. And if the functions in the system calendar module put any more names into the module global namespace, there could be trouble if code outside the module expects to be able to see them.Gingili
Thank you for such a detailed response! Now that I know what is involved I'll definitely stay clear of that peril. I was hoping there would be an easy way of getting around that as "calendar" is quite a common name.Sabbatical
+1: Merging namespaces is both incredibly clever /and/ incredibly dangerous. In a nutshell, I loved it :)Darla
And, ugly hack or not, it worked beautifully. I'll soon use this very trick, believe or not, in production code! I have a project whose source package is called code, and I'm not willing to rename everything to src right now just because PyDev imports stdlib'd code.InteractiveConsoleDarla
@MestreLion: Well, I'm not certain if I should be pleased or horrified. ;-)Gingili
Take a look and decide for yourself: github.com/MestreLion/singularity/blob/master/code/__init__.pyDarla
@Omnifarious: actually, I had to remove the del statement to make it truly work. Despite the setattr to local_module, Classes were still bound to random_name module. Functions and other vars worked fine tho. Any ideas why?Darla
How could you tell that classes were still bound to random_name module?Gingili
@Gingili Now that I have enough rep points, I can comment and ask this :) Why is this an "ugly and terrible" solution? I want to use it to combine socket functions, in order to interpose on logging. This merging could be done one time upon installation. I have posed a question on this, and one of the suggestions was to use monkey-hijaking. Is this ugly and terrible as well? Does it help that it will just be for logging messages?Metaplasia
@Darla Have you had any issues with this solution in production?Metaplasia
@Metaplasia nope, so far so good, but collision would only occur when using PyDev's debbuger, not in regular use. And make sure you check the latest code (the URL in github), since it changed a bit from the above answerDarla
@Darla was the issue of collision the main consideration? I've received some advice that says merging could affect backwards compatibility as well, though it doesn't sound like it's been an issue in your case.Metaplasia
@jspacek: It's a game, not a library, so in my case backward compatibility is not a concern at all. And the namespace collision occurs only when using running via PyDev IDE (which uses Python's code std module), meaning only a fraction of developers could ever have any issues with this "merging hack". Users would not be affected at all.Darla
@Darla thanks for taking the time to clarify the context, really appreciated. I'm going to explore patching which was suggested to help with the compatibility issue as users & devs could be affected in our case.Metaplasia
@Metaplasia - It's an ugly hack because it basically monkey patches your module with a whole tons of names from some system module. Monkey patching in a very targeted way is a highly questionable practice, and doing it en-masse this way seems quite disturbing.Gingili
Code like this should validate the content of sys.path[0] before skipping it - it could be a valid standard library path that should be searched. See my answer for details.Madel
P
8

In Python 3.5 and up, use the standard library importlib module to import directly from a specified path, bypassing import's lookup mechanism:

import importlib.util
import sys

# For illustrative purposes.
import tokenize
file_path = tokenize.__file__  # returns "/path/to/tokenize.py"
module_name = tokenize.__name__  # returns "tokenize"

spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)

In actual code, file_path can be set to any path to a .py file to import; module_name should be the name of the module that will be imported (the name that the import system uses to look up the module when further import statements are attempted). Subsequent code will use module as the name of the module; change the variable name module to use a different name.

To load a package instead of a single file, file_path should be the path to the package's root __init__.py.

Pouf answered 8/1, 2020 at 17:36 Comment(9)
Works like a charm... Used this to test while developing a library, so that my tests were always using the developing version and not the published (and installed) one. In windows 10 i had to write the path to my module like this: file_path=r"C:\Users\My User\My Path\Module File.py". Then i called module_name just like the released module so that i had full working script that, stripped off this snippet, coud be used on other pcsNigercongo
Where are the default python modules located? How do you know it will be the same on every system? Is this solutions portable?Eupheemia
The first sentence should be removed... or it would let people think that this solution contains a now-deprecated approach.Spire
I also want to ask whether the argument "module_name" is important or not, since it seems to be ok even if I pass an empty string into it.Spire
But how does one import a module from the standard library?Merilee
How's this even an accepted answer? The question is to import from the standard lib, not user code.Tetracaine
Good points. My reply here was pointing out that the methods of importing a custom path in the previously accepted answer were now deprecated, but my reply is not showing the complete answer to the question. Presumably it will need to be a combination of the logic Boaz Yaniv gave (first find the std lib package's init.py) and then the code here to actually import that file.Pouf
@Tetracaine because this shows a technique from importing directly from an explicitly specified full path to a file. To import from the standard library, this technique only requires first determining the path to where the standard library modules are stored.Madel
On this other hand, this does not completely solve the problem of importing directly from the standard library, because some standard library modules are built-in and not represented by any file on disk. It won't be possible to load them dynamically given a file path (this technique), because there is no path to use.Madel
M
3

Prologue: some terminology

Absolute import is when Python searches folders in what I will call the system module path (SMP), one at a time, until it finds one that contains the module.

The SMP is created at startup, based on the PYTHONPATH environment variable and some other things. It is represented within Python as a list of strings. After importing the sys standard library module, it is available as sys.path, but it exists whether that module is imported or not. (Usually, Python programmers just call this list "sys.path". However, since we are discussing the import mechanism in deep technical detail, and using modules named sys in examples, it seems appropriate to establish a separate term.)

On the other hand, relative import directly specifies where the module's code should be, relative to the current module, within the context of the current package.


Summary

How can we import from the standard library instead of the current package?

If the standard library module in question is not implemented as a builtin, it may be necessary to set the PYTHONSAFEPATH environment variable before starting Python. This prevents Python from putting the main script's directory (when started like python script.py) or the current working directory (otherwise) at the start of the SMP, as it does by default.

In 3.11, the -P option can be used instead of setting PYTHONSAFEPATH. In 3.4 and up, -I can also be used, but this will also ignore other Python-specific environment variables, and skip adding the per-user site-packages directory to the SMP. (It is also possible, of course, to modify sys.path programmatically to fix the SMP. Unless, of course, the problem is with trying to import sys.)

Once that is taken care of, just use an absolute import.

In 3.x:

import sys # or
from sys import version # or
from sys import * # but beware namespace pollution

In 2.5 through 2.7, a __future__ import is required first:

from __future__ import absolute_import
# proceed as above

In 2.4 and below, a hack is necessary to prevent implicit relative import:

sys = __import__("sys", {})

This should also work in other versions, but is far uglier and more complex than necessary.

Alternately, try hooking into the import machinery to change the lookup behaviour, as described in Omnifarious' answer, Boaz Yaniv's answer or casey's answer. All of these approaches work by emulating (part of) the internal algorithm for searching for modules, but skipping the relative import step and also skipping the first element of sys.path. (The premise is that sys.path is assumed to be "unsafe" - per the above discussion of PYTHONSAFEPATH - and thus using that path would look within the project.)

How can we prevent the standard library from importing from the current package, and make sure it imports from itself?

The standard library generally uses absolute imports. If this causes a problem due to shadowing by the current package, the standard advice is to rename the modules in the current package. Failing that, the same environment variable and command-line flag tricks should work.

How can we import from the current package instead of the standard library?

In 2.4 and earlier, this will happen by default.

In 2.5 onward, use a relative import, after ensuring that the package is set up correctly.

For the most part, this means making sure that the root of the package is on the SMP. The most robust way to do this is to activate a virtual environment and install the package in that virtual environment. Otherwise, assuming the program starts from some "driver" script, make sure it is in the same folder as the package root. Alternately, run the package (or a subpackage, with the appropriate dotted-path name) as a module, using the -m flag, from the directory that contains the package root.

Assuming the package is set up properly, relative imports look like:

from . import sys # sys.py in the same folder as this source file
from .sys import something # from our source, not the standard library
from .sys import * # again, beware namespace pollution
from .child import sys # child/sys.py
from .. import sys # ../sys.py, IF .. is still within the package root
from ..sibling import sys # ../sibling/sys.py
# Use more .'s to go up more levels first.
# This will not work beyond the package root.

Note that all of these use the from syntax. As explained in the PEP:

Relative imports must always use from <> import; import <> is always absolute.... because after import XXX.YYY.ZZZ... XXX.YYY.ZZZ is usable in an expression. But .moduleY is not usable in an expression.

As an absolute last resort, the SMP can be deliberately modified via sys.path, after determining the path to the current file and thus computing the path to the package root, so that absolute imports will work. Please note that this shouldn't ever be necessary in ordinary circumstances. Many popular, heavy-duty Python libraries manage to span hundreds of thousands of lines of code, absolutely none of which mention sys.path in any way.

How can we entice the standard library to import from the current package, rather than importing from itself like it was designed to?

I'm not aware offhand of any places where the Python standard library uses relative imports internally, and I also can't fathom a good reason to attempt this. That said, it's probably possible by hacking the import machinery. It doesn't sound easy or fun and I'm not about to attempt it here.

How can we import from elsewhere?

See How can I import a module dynamically given the full path?. Brandon Squizzato's answer also summarizes the usual technique.


Where Python looks for the module

2.4 and before

Originally, there was no special syntax for relative imports. Python would try a relative import first, and then an absolute import if that failed. So for example, code like import sys would import a sys.py from the same folder as the current module - if it existed - rather than the standard library module.

My understanding is that such relative imports would be relative to the package root, not to the current module; but I cannot easily verify this at the moment.

It was apparently possible to work around this, by supplying an empty dict for the globals context to __import__. Quoting:

sys = __import__("sys", {})

The import statement uses the global namespace to determine which package it is called on; if you pass an empty namespace, it cannot infer package information.

So, by using the __import__ function directly, it was possible to skip the attempted relative import.

2.5 through 2.7

PEP 328 introduced a proposal to make Python imports absolute by default, and use relative imports only with explicit syntax. (It also introduced the terms "absolute import" and "relative import" to the Python ecosystem.) Relative imports using this syntax are relative to the module doing the importing; additional leading .s can be used to specify parent packages.

Starting with the very first release of Python 2.5, the new behaviour became available by using a __future__ import:

from __future__ import absolute_import
import math # checks the SMP

On the other hand,

from __future__ import absolute_import
from . import math # imports from the same directory as this source file

or

from __future__ import absolute_import
from .. import math # imports from the immediate parent directory,
                    # IF it is still within the package root

3.x onward

The behaviour described in PEP 328 became the default. 2.4's implicit relative import is no longer available, and the code explicitly specifies either absolute or relative import - no __future__ import required.


Just choosing absolute or relative import is not enough

Some hiccups that come up at this point

  1. Apparently, many people find relative imports tricky to use because of the subtlety of the package system. The key point is that the .s used in relative import syntax do not refer to levels in a directory tree, but to levels in a package tree. (After all, there are ways to load modules that don't involve .py files on disk at all!)
  2. Even when an absolute import is used, code from the current project could still unexpectedly shadow the standard library. This is because of how the SMP is configured. In most cases, either the current working directory of the Python process, or the directory of the "driver" script that was launched from the command line, will be first on the SMP. Thus, import this might not find the standard library this even though it's doing an absolute import. (On the other hand, import sys would find the standard library; see the final section.)
  3. Meanwhile, the standard library doesn't take advantage of packaging very well. It doesn't have its own root package; and for the most part, when standard library modules depend upon each other, they use absolute import.

The last two points can interact in surprising ways:

$ touch token.py
$ python
Python 3.8.10 (default, Nov 14 2022, 12:59:47) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/_sitebuiltins.py", line 102, in __call__
    import pydoc
  File "/usr/lib/python3.8/pydoc.py", line 66, in <module>
    import inspect
  File "/usr/lib/python3.8/inspect.py", line 40, in <module>
    import linecache
  File "/usr/lib/python3.8/linecache.py", line 11, in <module>
    import tokenize
  File "/usr/lib/python3.8/tokenize.py", line 35, in <module>
    from token import EXACT_TOKEN_TYPES
ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/current/working/directory/token.py)

This problem can be avoided by fixing the SMP with any of the techniques from the summary:

$ touch token.py
$ python -I
Python 3.8.10 (default, Nov 14 2022, 12:59:47) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()

Welcome to Python 3.8's help utility!

If this is your first time using Python, you should definitely check out
the tutorial on the Internet at https://docs.python.org/3.8/tutorial/.

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules.  To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, symbols, or topics, type
"modules", "keywords", "symbols", or "topics".  Each module also comes
with a one-line summary of what it does; to list the modules whose name
or summary contain a given string such as "spam", type "modules spam".

help>

This still doesn't tell the whole story

The module loading system is actually much more complicated than described above. It's deliberately filled with hooks to allow for modifying the behaviour. In particular: aside from the SMP, there is also a "metapath" (made available as sys.meta_path) that contains module-loaders that are actually used to load the modules.

The documentation gives a recipe approximating the process. (Of course, the real thing does not have to solve the bootstrapping problem of importing sys and importlib.util in order to implement the import system!) However, it doesn't really show what each finder in sys.meta_path is doing.

Roughly, by default, for absolute imports:

  • First, _frozen_importlib.BuiltinImporter checks if the module name matches a module that is built in to Python, so that it can be imported directly. Part of the standard library is implemented this way, and part is not; and that portion has changed over time.

  • Then, _frozen_importlib.FrozenImporter tries to load a frozen module with the specified name.

  • Finally, if both of those fail, _frozen_importlib_external.PathFinder searches the SMP.

Because of this, user code might shadow absolute imports of some standard library modules while not shadowing others. Suppose we have this test script import_test.py:

def test_import(name):
    module = __import__(name)
    return any(attr for attr in dir(module) if not attr.startswith('__'))

if __name__ == '__main__':
    print('imported this from standard library?', test_import('this'))
    print('imported sys from standard library?', test_import('sys'))

Let's see what happens if empty Python files with those names are added to the CWD first:

$ touch this.py
$ touch sys.py
$ python import_test.py 
imported this from standard library? False
imported sys from standard library? True

Since at least 2.0, sys.builtin_module_names lists the names of modules that are built in to Python. Also, since 3.10, sys.stdlib_module_names lists all possible module names in the standard library (even ones that were deliberately excluded when Python was compiled, or which are not available for the current operating system).

Madel answered 10/1, 2023 at 10:34 Comment(0)
B
0

I'd like to offer my version, which is a combination of Boaz Yaniv's and Omnifarious's solution. It will import the system version of a module, with two main differences from the previous answers:

  • Supports the 'dot' notation, eg. package.module
  • Is a drop-in replacement for the import statement on system modules, meaning you just have to replace that one line and if there are already calls being made to the module they will work as-is

Put this somewhere accessible so you can call it (I have mine in my __init__.py file):

class SysModule(object):
    pass

def import_non_local(name, local_module=None, path=None, full_name=None, accessor=SysModule()):
    import imp, sys, os

    path = path or sys.path[1:]
    if isinstance(path, basestring):
        path = [path]

    if '.' in name:
        package_name = name.split('.')[0]
        f, pathname, desc = imp.find_module(package_name, path)
        if pathname not in __path__:
            __path__.insert(0, pathname)
        imp.load_module(package_name, f, pathname, desc)
        v = import_non_local('.'.join(name.split('.')[1:]), None, pathname, name, SysModule())
        setattr(accessor, package_name, v)
        if local_module:
            for key in accessor.__dict__.keys():
                setattr(local_module, key, getattr(accessor, key))
        return accessor
    try:
        f, pathname, desc = imp.find_module(name, path)
        if pathname not in __path__:
            __path__.insert(0, pathname)
        module = imp.load_module(name, f, pathname, desc)
        setattr(accessor, name, module)
        if local_module:
            for key in accessor.__dict__.keys():
                setattr(local_module, key, getattr(accessor, key))
            return module
        return accessor
    finally:
        try:
            if f:
                f.close()
        except:
            pass

Example

I wanted to import mysql.connection, but I had a local package already called mysql (the official mysql utilities). So to get the connector from the system mysql package, I replaced this:

import mysql.connector

With this:

import sys
from mysql.utilities import import_non_local         # where I put the above function (mysql/utilities/__init__.py)
import_non_local('mysql.connector', sys.modules[__name__])

Result

# This unmodified line further down in the file now works just fine because mysql.connector has actually become part of the namespace
self.db_conn = mysql.connector.connect(**parameters)
Bigot answered 28/8, 2013 at 20:10 Comment(1)
Code like this should validate the content of sys.path[0] before skipping it - it could be a valid standard library path that should be searched. See my answer for details.Madel

© 2022 - 2024 — McMap. All rights reserved.