How to use the __import__ function to import a name from a submodule?

Question

I'm trying to replicate from foo.bar import object using the __import__ function and I seem to have hit a wall.

A simpler case from glob import glob is easy: glob = __import__("glob").glob

The problem I'm having is that I am importing a name from a subpackage (i.e. from foo.bar):

So what I'd like is something like

string_to_import = "bar"
object = __import__("foo." + string_to_import).object

But this just imported the top-level foo package, not the foo.bar subpackage:

__import__("foo.bar")
<module 'foo' from 'foo/__init__.pyc'>

Selmner

Answer 1

The __import__ function will return the top level module of a package, unless you pass a nonempty fromlist argument:

_temp = __import__('foo.bar', fromlist=['object']) 
object = _temp.object

See the Python docs on the __import__ function.

Aperture

or object = __import__('foo.bar.object', fromlist=[None]) to get it directly. – Aspersion

Answer 2

How to use python's __import__() function properly?

There are two kinds of uses:

• direct importing
• a hook to alter import behavior

For the most part, you don't really need to do either.

For user-space importing

Best practice is to use importlib instead. But if you insist:

Trivial usage:

>>> sys = __import__('sys')
>>> sys
<module 'sys' (built-in)>

Complicated:

>>> os = __import__('os.path')
>>> os
<module 'os' from '/home/myuser/anaconda3/lib/python3.6/os.py'>
>>> os.path
<module 'posixpath' from '/home/myuser/anaconda3/lib/python3.6/posixpath.py'>

If you want the rightmost child module in the name, pass a nonempty list, e.g. [None], to fromlist:

>>> path = __import__('os.path', fromlist=[None])
>>> path
<module 'posixpath' from '/home/myuser/anaconda3/lib/python3.6/posixpath.py'>

Or, as the documentation declares, use importlib.import_module:

>>> importlib = __import__('importlib')
>>> futures = importlib.import_module('concurrent.futures')
>>> futures
<module 'concurrent.futures' from '/home/myuser/anaconda3/lib/python3.6/concurrent/futures/__init__.py'>

Documentation

The docs for __import__ are the most confusing of the builtin functions.

__import__(...)
    __import__(name, globals=None, locals=None, fromlist=(), level=0) -> module

    Import a module. Because this function is meant for use by the Python
    interpreter and not for general use it is better to use
    importlib.import_module() to programmatically import a module.

    The globals argument is only used to determine the context;
    they are not modified.  The locals argument is unused.  The fromlist
    should be a list of names to emulate ``from name import ...'', or an
    empty list to emulate ``import name''.
    When importing a module from a package, note that __import__('A.B', ...)
    returns package A when fromlist is empty, but its submodule B when
    fromlist is not empty.  Level is used to determine whether to perform 
    absolute or relative imports. 0 is absolute while a positive number
    is the number of parent directories to search relative to the current module.

If you read it carefully, you get the sense that the API was originally intended to allow for lazy-loading of functions from modules. However, this is not how CPython works, and I am unaware if any other implementations of Python have managed to do this.

Instead, CPython executes all of the code in the module's namespace on its first import, after which the module is cached in sys.modules.

__import__ can still be useful. But understanding what it does based on the documentation is rather hard.

Full Usage of __import__

To adapt the full functionality to demonstrate the current __import__ API, here is a wrapper function with a cleaner, better documented, API.

def importer(name, root_package=False, relative_globals=None, level=0):
    """ We only import modules, functions can be looked up on the module.
    Usage: 

    from foo.bar import baz
    >>> baz = importer('foo.bar.baz')

    import foo.bar.baz
    >>> foo = importer('foo.bar.baz', root_package=True)
    >>> foo.bar.baz

    from .. import baz (level = number of dots)
    >>> baz = importer('baz', relative_globals=globals(), level=2)
    """
    return __import__(name, locals=None, # locals has no use
                      globals=relative_globals, 
                      fromlist=[] if root_package else [None],
                      level=level)

To demonstrate, e.g. from a sister package to baz:

baz = importer('foo.bar.baz')    
foo = importer('foo.bar.baz', root_package=True)
baz2 = importer('bar.baz', relative_globals=globals(), level=2)

assert foo.bar.baz is baz is baz2

Dynamic access of names in the module

To dynamically access globals by name from the baz module, use getattr. For example:

for name in dir(baz):
    print(getattr(baz, name))

Hook to alter import behavior

You can use __import__ to alter or intercept importing behavior. In this case, let's just print the arguments it gets to demonstrate we're intercepting it:

old_import = __import__

def noisy_importer(name, locals, globals, fromlist, level):
    print(f'name: {name!r}')
    print(f'fromlist: {fromlist}')
    print(f'level: {level}')
    return old_import(name, locals, globals, fromlist, level)

import builtins
builtins.__import__ = noisy_importer

And now when you import you can see these important arguments.

>>> from os.path import join as opj
name: 'os.path'
fromlist: ('join',)
level: 0
>>> opj
<function join at 0x7fd08d882618>

Perhaps in this context getting the globals or locals could be useful, but no specific uses for this immediately come to mind.

Stalagmite

I don't understand why this is not the best answer. – Guadalajara

just been wondering what's up with fromlist=[None] and noticed that CPython (current, i.e. 3.12, back to at least 3.7) only checks whether it's truthy. hence passing fromlist=True or fromlist=1 would work. if you want to be obscure fromlist=... would even work as the Ellipsis object is truthy – Rattlepate

Answer 3

You should use importlib.import_module, __import__ is not advised outside the interpreter.

In __import__'s docstring:

Import a module. Because this function is meant for use by the Python interpreter and not for general use it is better to use importlib.import_module() to programmatically import a module.

It also supports relative imports.

Tractable

Documentation for __import__ states: "Programmatic importing of modules should use import_module() instead of this function." Per docs.python.org/3/library/importlib.html – Kotta

This is out of the scope of the question. It's clear he want to use/investigate the dunder import. – Niobium

Answer 4

Rather than use the __import__ function I would use the getattr function:

model = getattr(module, model_s)

where module is the module to look in and and model_s is your model string. The __import__ function is not meant to be used loosely, where as this function will get you what you want.

Aperture

Actually, I've just realized that this only works after I've already imported using the normal method – Selmner

Hmm, you should be able to import foo (if you know it's value already and don't need to import it dynamically as a string value) with the normal import statement. Once the module is imported you can import anything within its directory as a string using getattr. import foo bar = getattr(foo, 'bar') object=bar.object – Aperture

But bar isn't an attribute, it's a script within the folder foo. – Selmner

Ok I see. import sys. bar = sys.modules['foo.bar'] object=bar.object or if 'object also needs to imported from string, object=getattr(bar, 'object') – Aperture

I get KeyError if I list sys.modules I get 'maps': <module 'foo' from 'foo/__init__.py'> ` – Selmner

This is the problem, until I import foo.bar, all I see is foo.init – Selmner

Don't know what to tell you. I believe you should be able to access whatever folders and modules from sys.modules as you would from a normal import statement. Your list suggests 'maps' is the only module on your path. Try looking at sys.path and make sure .../foo... is on your path. – Aperture

Please set this up yourself, add a folder called foo with bar.py that contains anything; even a sting, and the init. cd ../foo, run python and try. I just need a sanity check! – Selmner

let us continue this discussion in chat – Aperture

since getattr has nothing to do with importing, may I suggest this answer be either deleted or updated? – Stalagmite

Answer 5

In addition to these excellent answers, I use __import__ for convenience, to call an one-liner on the fly. Examples like the following can also be saved as auto-triggered snippets at your IDE.

• Plant an ipdb break-point (triggered with "ipdb")

__import__("ipdb").set_trace(context=9)

• Print prettily (triggered with "pp")

__import__("pprint").pprint(<cursor-position>)

This way, you get a temporary object, that is not referenced by anything, and call an attribute on the spot. Also, you can easily comment, uncomment or delete a single line.

Valonia

Answer 6

>>> module_name = 'math'
>>> func_name = 'sqrt'
>>> math = __import__(module_name)
>>> math
<module 'math' (built-in)>
>>> sqrt = getattr(math, func_name)
>>> sqrt(9)
3.0

But you can directly do this:

sqrt = getattr(__import__(module_name), func_name)

In your case:

string_to_import = "bar"
object = getattr(__import__("foo"), string_to_import)

Antetype