How to import a submodule in Python? (without `exec`)

Question

I would like to import a submodule without knowing its name beforehand,

>>> __import__("os.path")
<module 'os' from '/usr/lib/python3.3/os.py'>

Doesn't work as you might expect, returning os, not os.path.

I came up with this solution.

def import_submodule(mod, submod):
    ns = {}
    exec_str = "from %s import %s as submod" % (mod, submod)
    exec(exec_str, ns, ns)
    return ns["submod"]

This gives the result:

>>> import_submodule("os", "path")
<module 'posixpath' from '/usr/lib/python3.3/posixpath.py'>

However I would rather not use exec() because its rather bad practice and seems unnecessary when Pythons import mechanisms are available already through __import__, imp and importlib modules.

Is there a way in Python3.x to do this kind of import though a function call, rather then using exec() ?

Garget

possible duplicate of How to dynamically load a Python class – Smelly

Note the name.split('.'), then loop to use getattr() to retrieve the 'subobject'; .path in your case. – Smelly

Basing your examples on os.path might lead you on a wild goose chase, since it doesn't necessarily work like other modules – Geri

@Martijn Pieters, this is not a duplicate of "How to dynamically load a Python class'. And using getattr() in a loop does not work, a module wont always import its submodules, take packages with a totally empty __init__.py file as an example (which is fairly common). The while point of my question is to have this work exactly like from A import B as C. – Garget

@ideasman42: __import__('parentmodule.submodule') does import submodule but returns parentmodule. – Smelly

@Martijn Pieters, right, but then you need to get the module some other way afterwards. Would you suggest this as best practice? my_import = lambda mod: (__import__(mod), __import__('sys').modules[mod])[1] – Garget

@ideasman42: See the linked question; split then name on . and use everything but the first element in a loop with getattr(). – Smelly

I've submitted an answer, notice that this works for a submodule and not other data (classes in modules for eg), which is an example of how this question is different. – Garget

@MartijnPieters I believe this shouldn't be considered a duplicate since the OP is explicitly targetting python3.3 for which a better answer exist(i.e. importlib) which isn't cited in the answers to the other question. Marking this as a duplicate might mean that people will still use other home-made solutions instead of the new import machinery. On current versions of python you almost never need to call __import__ directly. – Publish

@Bakuriu: fair enough; I conveniently forgot about importlib.import_module() here. – Smelly

Answer 1

Use importlib.import_module:

>>> import importlib
>>> importlib.import_module('os.path')
<module 'posixpath' from '/usr/lib/python2.7/posixpath.pyc'>

This should work in python2.7+ and 3.1+.

Publish

Answer 2

Note that if you want do: from A import B as C as a function call, importlib.import_module won't always work, since B may not be a module.

Heres a function which uses importlib and getattr.

def my_import_from(mod_name, var_name):
    import importlib
    mod = importlib.import_module(mod_name)
    var = getattr(mod, var_name)
    return var

So this:

from os.path import dirname as var

Can be replaced with this:

var = my_import_from("os.path", "dirname")

Which avoids exec and allows both submodules and any variables defined in the module.

Since my question explicitly says importing a submodule, the answer from @Bakuriu is correct, however including this for completeness and it may help others who run into the same problem.

Garget

Answer 3

Import both parts

As @Marius pointed out in the comment section, what appears as a submodule is not always a submodule. This is the case with os.path and I encountered the same with lxml.etree. The trick is to import both parts, i.e. both os and os.path. Here is what worked for me for sets of modules:

# PyPI imports
import pkg_resources, subprocess, sys

modules   = {'lxml.etree', 'pandas', 'screeninfo'}
required  = {m.split('.')[0] for m in modules}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing   = required - installed

if missing:
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', '--upgrade', 'pip'])
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', *missing])

for module in set.union(required, modules):
    globals()[module] = __import__(module)

Tests:

print(pandas.__version__)
print(lxml.etree.LXML_VERSION)

Toothache