How to dynamically load a Python class

Question

Given a string of a Python class, e.g. my_package.my_module.MyClass, what is the best possible way to load it?

In other words I am looking for a equivalent Class.forName() in Java, function in Python. It needs to work on Google App Engine.

Preferably this would be a function that accepts the FQN of the class as a string, and returns a reference to the class:

my_class = load_class('my_package.my_module.MyClass')
my_instance = my_class()

Cogency

I need to be able to assign the class reference to a variable as well. – Cogency

This appears to be a duplicate of: #453469 – Hawkes

You are right it is a duplicate, thanks for finding it – Cogency

@JohnTyree How does loading classes dynamically mean a program isn't interesting? Can you give an example so that this criticism can be more useful to SO members? – Mclean

Interesting "enough." It's just a tongue-in-cheek way of saying that doing weird things is sometimes necessary because of factors that you can't control and are hard to predict. The previous comment basically said, "Why don't you just import the normal way?" and I'm saying that they too will someday have a weird corner case that requires doing something ugly. – Guinea

Answer 1

From the python documentation, here's the function you want:

def my_import(name):
    components = name.split('.')
    mod = __import__(components[0])
    for comp in components[1:]:
        mod = getattr(mod, comp)
    return mod

The reason a simple __import__ won't work is because any import of anything past the first dot in a package string is an attribute of the module you're importing. Thus, something like this won't work:

__import__('foo.bar.baz.qux')

You'd have to call the above function like so:

my_import('foo.bar.baz.qux')

Or in the case of your example:

klass = my_import('my_package.my_module.my_class')
some_object = klass()

EDIT: I was a bit off on this. What you're basically wanting to do is this:

from my_package.my_module import my_class

The above function is only necessary if you have a empty fromlist. Thus, the appropriate call would be like this:

mod = __import__('my_package.my_module', fromlist=['my_class'])
klass = getattr(mod, 'my_class')

Begorra

I tried my_import('my_package.my_module.my_class') but get no module found my_class, which makes sense since it is a class not a module. Howver if I can use gettattr to get the class after the call to my_import – Cogency

That's odd. Everything past the first dot is called using getattr. There shouldn't be any difference. – Begorra

Thanks I think this is the best way. Now I only need the best way to split the string 'my_pakcage.my_module.my_class' into mod_name, klass_name but I guess I can figure that out :) – Cogency

The my_import() code is not needed, as you can use dots in an __import__() call just fine. They all have to refer to module names however. A symbol from the module needs to be fetched with getattr(). – Embay

If you use mod = __import__(name.split('.')[0]) and then components=name.split('.')[1:] (or equivalent) it works also for classnames. E.g. my_import("datetime.date") – Trichomoniasis

Your edit solution works without the fromlist option (in python 2) i.e. mod = __import__('my_package.my_module'); klass = getattr(mod, 'my_class') is fine. I can't for the life of me work out the effect of fromlist, the documentation is not clear to me. – Surcharge

python documentation (on the code) of import says to use importlib. so should checkout answer by Adam Spence – Pyramidal

Link @Pyramidal is referring to: docs.python.org/3/library/importlib.html#importlib.__import__ – Essie

Answer 2

import importlib

module = importlib.import_module('my_package.my_module')
my_class = getattr(module, 'MyClass')
my_instance = my_class()

Yvetteyvon

once you've imported the module dynamically you have access to the class via the module – Yvetteyvon

Great! It's even part of standard library from 2.7 and up. – Shaunteshave

This is the correct way to access a module / class. The docs state this here: docs.python.org/3/library/importlib.html#importlib.__import__ – Essie

Answer 3

If you don't want to roll your own, there is a function available in the pydoc module that does exactly this:

from pydoc import locate
my_class = locate('my_package.my_module.MyClass')

The advantage of this approach over the others listed here is that locate will find any python object at the provided dotted path, not just an object directly within a module. e.g. my_package.my_module.MyClass.attr.

If you're curious what their recipe is, here's the function:

def locate(path, forceload=0):
    """Locate an object by name or dotted path, importing as necessary."""
    parts = [part for part in split(path, '.') if part]
    module, n = None, 0
    while n < len(parts):
        nextmodule = safeimport(join(parts[:n+1], '.'), forceload)
        if nextmodule: module, n = nextmodule, n + 1
        else: break
    if module:
        object = module
    else:
        object = __builtin__
    for part in parts[n:]:
        try:
            object = getattr(object, part)
        except AttributeError:
            return None
    return object

It relies on pydoc.safeimport function. Here are the docs for that:

"""Import a module; handle errors; return None if the module isn't found.

If the module *is* found but an exception occurs, it's wrapped in an
ErrorDuringImport exception and reraised.  Unlike __import__, if a
package path is specified, the module at the end of the path is returned,
not the package at the beginning.  If the optional 'forceload' argument
is 1, we reload the module from disk (unless it's a dynamic extension)."""

Wan

I upvoted this answer. BTW, here is the code that also have safeimport as it seems odd to import pydoc just for this: github.com/python/cpython/blob/… – Salade

I upvoted this answer too, this is the best of all relevant answers. – Tutto

This seems to be able to handle qualname (object not at top of module namespace) correctly as well. – Oily

yes, you can do locate('my_package.my_module.MyClass.attr.method.etc') – Wan

actually the best answer – Ashtoreth

Slightly off-topic, but it seems like this should be in a different package than in pydoc. – Bauer

Upvoted. But it is odd that this function is in module pydoc and still un-documented. – Gossipmonger

Answer 4

If you're using Django you can use import_string.

Yes i'm aware OP did not ask for django, but i ran across this question looking for a Django solution, didn't find one, and put it here for the next boy/gal that looks for it.

# It's available for v1.7+
# https://github.com/django/django/blob/stable/1.7.x/django/utils/module_loading.py
from django.utils.module_loading import import_string

Klass = import_string('path.to.module.Klass')
func = import_string('path.to.module.func')
var = import_string('path.to.module.var')

Keep in mind, if you want to import something that doesn't have a ., like re or argparse use:

re = __import__('re')

Rainband

Answer 5

def import_class(cl):
    d = cl.rfind(".")
    classname = cl[d+1:len(cl)]
    m = __import__(cl[0:d], globals(), locals(), [classname])
    return getattr(m, classname)

Klagenfurt

This is the clean solution! You could consider using: (modulename, classname) = cl.rsplit('.', 2) – Embay

It's great) I had created putils package with different utils, also class importing there. If you want, you can use it from that package. – Klagenfurt

@Embay rsplit('.', 1) ? – Savannahsavant

I managed to pass {} instead of globals/locals and it still works fine – Insomuch

Answer 6

Here is to share something I found on __import__ and importlib while trying to solve this problem.

I am using Python 3.7.3.

When I try to get to the class d in module a.b.c,

mod = __import__('a.b.c')

The mod variable refer to the top namespace a.

So to get to the class d, I need to

mod = getattr(mod, 'b') #mod is now module b
mod = getattr(mod, 'c') #mod is now module c
mod = getattr(mod, 'd') #mod is now class d

If we try to do

mod = __import__('a.b.c')
d = getattr(mod, 'd')

we are actually trying to look for a.d.

When using importlib, I suppose the library has done the recursive getattr for us. So, when we use importlib.import_module, we actually get a handle on the deepest module.

mod = importlib.import_module('a.b.c') #mod is module c
d = getattr(mod, 'd') #this is a.b.c.d

Shaylynn

Answer 7

def my_import(name):
    components = name.split('.')
    mod = __import__(".".join(components[:-1]))
    for comp in components[1:]:
        mod = getattr(mod, comp)
    return mod

Fix error import on python 3.11

Katakana

Hello, please don't post code only and add an explantation as to why you think that this is the optimal solution. People are supposed to learn from your answer, which might not occur if they just copy paste code without knowing why it should be used. – Morey

Answer 8

OK, for me that is the way it worked (I am using Python 2.7):

a = __import__('file_to_import', globals(), locals(), ['*'], -1)
b = a.MyClass()

Then, b is an instance of class 'MyClass'

Unclose

Answer 9

If you happen to already have an instance of your desired class, you can use the 'type' function to extract its class type and use this to construct a new instance:

class Something(object):
    def __init__(self, name):
        self.name = name
    def display(self):
        print(self.name)

one = Something("one")
one.display()
cls = type(one)
two = cls("two")
two.display()

Aver

Answer 10

Python has an inbuilt library importlib to get the job done. :, How to access module method and class method dynamically bypassing package name as a param. An example is given below.

Module 1:

def get_scenario_data():
    return "module1 scenario data"


class Module1:

    def module1_function1(self):
        return "module1_function"

    def module1_function2(self):
        return "module2_function"

Module 2:

def get_scenario_data():
    return "module2 scenario data"



class Module2:

    def module2_function1(self):
        return "module2_function1"

    def module2_function2(self):
        return "module2_function2"

ModuleTest:

1. Will access the module methods dynamically based on the package name as param
2. Will access the class methods dynamically based on the package name as param.

ModuleTest

import importlib

module = importlib.import_module('pack1.nestedpack1.module1')
print(module.get_scenario_data())
modul1_cls_obj = getattr(module, 'Module1')()
print(modul1_cls_obj.module1_function1())
print(modul1_cls_obj.module1_function2())

module = importlib.import_module('pack1.nestedpack1.module2')
modul2_cls_obj = getattr(module, 'Module2')()
print(modul2_cls_obj.module2_function1())
print(modul2_cls_obj.module2_function2())
print(module.get_scenario_data())

Results

module1 scenario data
module1_function
module2_function
module2_function1
module2_function2
module2 scenario data

Fulks

Answer 11

PyPI module autoloader & import

# PyPI imports
import pkg_resources, subprocess, sys

modules   = {'lxml.etree', 'pandas', 'screeninfo'}
required  = {m.split('.')[0] for m in modules}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing   = required - installed

if missing:
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', '--upgrade', 'pip'])
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', *missing])

for module in set.union(required, modules):
    globals()[module] = __import__(module)

Tests:

print(pandas.__version__)
print(lxml.etree.LXML_VERSION)

Atomic

Answer 12

Adding a bit of sophistication to the existing answers....

Depending on the use case, it may be somewhat inconvenient to have to explicitly specify the full path (E.g. package.subpackage.module...) of the class/method you want to import. On top of importlib, we can leverage __init__.py to make things even cleaner.

Let's say I have a python package, like so:

├── modes
│   ├── __init__.py
│   ├── bar.py
│   ├── foo.py
│   ├── modes.py

foo.py, say, have some class/functions we'd like to use somewhere else in our program:

from modes.modes import Mode

class Foo(Mode):
    def __init__(self, *arg, **kwargs):
        super(Foo, self).__init__(*arg, **kwargs)
        
    def run(self):
        self.LOG.info(f"This is FOO!")

With a command line argument, I can pass an argument that corresponds to a mode that I want to run. I'd like to be able to so something like this:

def set_mode(mode):
    """  """
    import importlib
    module = importlib.import_module('modes.foo')
    getattr(module, mode)().run()

which outputs:

>> set_mode("Foo")
>> engine_logger:INFO - This is FOO!

That works fine, however what we'd REALLY want to get at is this:

def set_mode(mode):
    """  """
    import importlib
    module = importlib.import_module('modes')  # only import the package, not modules explicitely
    getattr(module, mode)().run()

Which raises an error:

>> set_mode("Foo")
>> AttributeError: module 'modes' has no attribute 'Foo'

However, we can add the following to /modes/__init__.py:

from .foo import Foo
from .bar import Bar

Then, we can do:

>> set_mode("Foo")
>> engine_logger:INFO - This is FOO!

>> set_mode("Bar")
>> engine_logger:INFO - This is BAR!

In other worlds, all sub modules/functions/classes we import in init.py will be found directly with importlib.import_module(...), without having to specify the full path from outside.

Input

Answer 13

I have created this method

import importlib
import sys
import os


def import_classes(root_directory: str):
    imported_classes, import_errors = [], []
    for root, dirs, files in os.walk(root_directory):

    if os.path.isabs(root):
        """
            When an absolute path is specified: 'C:\\Users\\admin\\PycharmProjects\\my_project\\_dev\\plugin_discovery\\scenarios'
            we will add the parent directory: 'C:\\Users\\admin\\PycharmProjects\\my_project\\_dev\\plugin_discovery' into "sys.path" variable
            and we will start to create the import path starting from the last parent directory: "scenarios"
        """
        parent_dir, last_parent_directory = os.path.dirname(root), os.path.basename(root)

    else:
        # When a relative path is specified; just add the current working directory into "sys.path" variable
        parent_dir, last_parent_directory = os.getcwd(), root

    """
    Python searches for modules starting from each path specified in "sys.path" list.
        Example:
            Current working directory: "C:\\Users\\admin\\PycharmProjects\\my_project\\_dev\\plugin_discovery"
            last_parent_directory:  "scenarios"

            If current working directory in sys.path, python will try to import the module from:
                "C:\\Users\\admin\\PycharmProjects\\my_project\\_dev\\plugin_discovery\\scenarios"
    
    This is why we add the parent directory into "sys.path" variable
    """
    if parent_dir not in sys.path:
        sys.path.insert(0, parent_dir)

    # Import classes from Python Files
    for filename in files:
        if filename.endswith('.py'):
            module_name = os.path.splitext(filename)[0]  # Remove the file extension to get the module name
            last_parent_directory = last_parent_directory.replace("..\\", "").replace("../", "").replace(".\\", "").replace("./", "")  # Remove relative path prefix
            last_parent_directory = last_parent_directory.replace("\\", ".").replace("/", ".")  # Replace path separators with dots

            module_import_path = f"{last_parent_directory}.{module_name}"  # Build module import path

            try:
                # Force the module to be reimported if it has already been imported
                if module_import_path in sys.modules:
                    del sys.modules[module_import_path]
                
                module_object = importlib.import_module(module_import_path)
                
                # Iterate over items in the module_object
                for attribute_name in dir(module_object):
                    # Get the attributes from the module_object
                    attribute = getattr(module_object, attribute_name)

                    # Check if it's a class and append to list
                    if isinstance(attribute, type):
                        imported_classes.append(attribute)

            except Exception as import_error:
                # In case of import errors; save the import arguments and the error and continue with other files
                import_errors.append((parent_dir, module_import_path, import_error))

return imported_classes, import_errors

Now, lets suppose we have the following folder structure and classes:

scenarios_root (directory)
  scenarios (directory)
    scenario1.py
      -> Scenario1 (class)
      -> Scenario2 (class)
      -> Scenario3 (class)
                
    scenario2.py
      -> ScenarioA (class)
      -> ScenarioB (class)
      -> ScenarioC (class)

If we execute the import function:

if __name__ == '__main__':
  imported_clas, import_err = import_classes(r"..\_dev\2023_08_30_plugin_discovery\scenarios_root")
  print(f"Number of imported classes: {len(imported_clas)}")
  print(f"Imported classes: {imported_clas}")
  print(f"Import errors: {import_err}", end="\n\n")

Output:

Number of imported classes: 6
Imported classes: [<class '_dev.2023_08_30_plugin_discovery.scenarios_root.scenarios.scenario1.Scenario1'>, <class '_dev.2023_08_30_plugin_discovery.scenarios_root.scenarios.scenario1.Scenario2'>, <class '_dev.2023_08_30_plugin_discovery.scenarios_root.scenarios.scenario1.Scenario3'>, <class '_dev.2023_08_30_plugin_discovery.scenarios_root.scenarios.scenario2.ScenarioA'>, <class '_dev.2023_08_30_plugin_discovery.scenarios_root.scenarios.scenario2.ScenarioB'>, <class '_dev.2023_08_30_plugin_discovery.scenarios_root.scenarios.scenario2.ScenarioC'>]
Import errors: []

Paget

Answer 14

One can try this:

from pydoc import locate

module = locate("path to py file"))
if module != None:
  classname = getattr(module, "class name")
  if classname != None:
       classobject = classname("arguments")

Athelstan

Answer 15

In Google App Engine there is a webapp2 function called import_string. For more info see here:https://webapp-improved.appspot.com/api/webapp2.html

So,

import webapp2
my_class = webapp2.import_string('my_package.my_module.MyClass')

For example this is used in the webapp2.Route where you can either use a handler or a string.

Igorot

Answer 16

module = __import__("my_package/my_module")
the_class = getattr(module, "MyClass")
obj = the_class()

Overlord

Note that this works because of a bug in the import function. File paths should not be used in the import function and will not work in python 2.6 and above: docs.python.org/whatsnew/2.6.html#porting-to-python-2-6 – Begorra