How do I use importlib.LazyLoader?

Question

In my module, I have a couple of functions that depend on an external module with a long startup time. How do I use LazyLoader? If I have

import veggies

or

import veggies.brussels.sprouts

or

from veggies.brussels import sprouts

how would I replace those statements to use LazyLoader such that the execution of the contents of the module are postponed until needed?

It is not immediately obvious from the documentation how to use it. There is no example, and nullege code search only comes up with the unit test included with Python itself.

Institution

Answer 1

The original issue has some code that seems to to a full import lazily:

The following files imports two modules lazily:

import sys
import importlib.util

def lazy(fullname):
  try:
    return sys.modules[fullname]
  except KeyError:
    spec = importlib.util.find_spec(fullname)
    module = importlib.util.module_from_spec(spec)
    loader = importlib.util.LazyLoader(spec.loader)
    # Make module with proper locking and get it inserted into sys.modules.
    loader.exec_module(module)
    return module

os = lazy("os")
myown = lazy("myown")

print(os.name)
myown.test()

To test, I used the following in myown.py.

print("Executed myown.")
def test():
  print("OK")

That worked nicely (Python 3.8a0).

Highspeed

Can you show an example of how to actually use LadyLoader? – Institution

The answer now contains a 100% complete working example. – Highspeed

I would suggest using loader = importlib.util.LazyLoader.factory(spec.loader) which also checks for the exec_module attribute. – Rosyrot

Doesn't look like this answers the question though. Is there a way to make import statements lazy? – Anadromous

Another example use case is in mercurial-scm.org/repo/hg/file/tip/hgdemandimport/… – Shortage

This appears to work in CPython 3.5 through 3.10b4, perhaps later. – Charlatanry

Current Python doc has a snippet - very similar! - for this which could be considered more canonical than this answer: docs.python.org/3/library/… – Patsis

There should be no need for that try block. util.find_spec already checks for fullname in sys.modules. docs.python.org/3/library/… – Supervisory

Answer 2

6y after this thread, I am testing some approaches to lazy importing so that I can improve the startup time of an app and while PEP690 is not available (Python 3.12)

While testing the LazyLoader approach, be mindful that Python doesn't keep that import and your lazy object just gets garbage collected as any other one.

Here's an example

lib.py

print('Executing lib.py')


class MyClass():
    def get_name(self):
        return self.__class__.__name__

main.py

def lazy(fullname):
    import sys
    import importlib.util
    try:
        return sys.modules[fullname]
    except KeyError:
        spec = importlib.util.find_spec(fullname)
        module = importlib.util.module_from_spec(spec)
        loader = importlib.util.LazyLoader(spec.loader)
        # Make module with proper locking and get it inserted into sys.modules.
        loader.exec_module(module)
        return module


def method1():
    lib = lazy("lib")
    my_class = lib.MyClass()
    print(my_class.get_name())


def method2():
    import lib
    my_class = lib.MyClass()
    print(my_class.get_name())


if __name__ == '__main__':
    methods = [method1, method2]
    for method in methods:
        print('Executing {}'.format(method.__name__))
        for _ in range(2):
            method()
        print('---------------------------------')

results:

Executing method1
Executing lib.py
MyClass
Executing lib.py
MyClass
---------------------------------
Executing method2
Executing lib.py
MyClass
MyClass
---------------------------------

Ido

Looks like PEP 690 was rejected. – Institution