Autoload in Python [closed]

Question

In the past I've used Perl's AUTOLOAD facility for implementing lazy loading of symbols into a namespace, and wanted the same functionality in python.

Traditionally the closest you appear to be able to get is to use a class and a __getattr__ class to achieve this sort of thing. However I've also tried rummanging around in sys.modules, and come up with this:

# mymod.py
def greet(greeting="Hello World"):
   print greeting

class Autoload(object):
    def __init__(self, __name__):
        super(Autoload, self).__init__()
        self.wrapped_name = __name__
        self.wrapped = sys.modules[__name__]
    def __getattr__(self, name):
        try:
            return getattr(self.wrapped, name)
        except AttributeError:
            def f():
                greet(name+" "+self.wrapped_name)
            return f

if __name__ != "__main__":
    import sys
    sys.modules[__name__] = autoload(__name__)

This does work the way I'd like from a user perspective:

~> python
Python 2.5.1 (r251:54863, Jan 10 2008, 18:01:57)
[GCC 4.2.1 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mymod
>>> mymod.hello()
hello mymod
>>> from mymod import Hello_World
>>> Hello_World()
Hello_World mymod

But it strikes me - is there a standard approach that people tend to use for autoloading in python?

Secondly, a question for experienced python developers really is "does this strike you as good or bad practice"? I'm a reasonably experienced python developer, and it strikes me as really useful, but it strikes me as borderline and interested in whether this can be viewed as good practice, bad practice or similar.

Gann

https://mcmap.net/q/20725/-__getattr__-on-a-module and stackoverflow.com/questions/922550/… – Wilkie

I'd delete this if I could, because the changes to the question text, and the code and the tags no longer reflect my original question. The answers below aren't particularly helpful either. I do wish that people would actually ask before changing someone's question text. For example, the reason the original code used "__name__" rather than "mod_name" is to show the direct similarity to sys.modules[__name__] and so on. This has been broken by edits. – Gann

I'm voting to close this question as opinion-based. @MichaelSparks, if you also vote to close, it can then be delete-voted and potentially eventually deleted, per your wish in the comment above (if you still feel that way). – Bui

Answer 1

To answer the question of using a class to impersonate a module:

Yes, the functionality is not accidental. It has been there since early in the 2.x series and still works in the 3.x series.

To answer the question of lazy loading:

There are several ways to do it, and each one is going to be a bit mysterious. Using a module impersonator is a fine method.

Diplococcus

Answer 2

"Lazy imports" could be built on top of the "import hooks" specified in PEP 302, and now fully implemented. PEP 369 used to cover "lazy imports" as well as post-import hooks, but has since been simplified and now only covers post-import hooks; still, you might be interested in the original draft.

A good implementation of "lazy imports" via the meta_path hook can be found in this recipe.

Cartilage

Answer 3

No, I don't think that's useful.

Why create a function on the fly for every attribute you're trying to get from the module? Seems confusing to me. It appears that new functions are being created by magic so one has to look deep into the implementation to understand what is going on. And all that for no syntactical benefits.

And even if you had a good reason to do that, why do it with a module? Why register your class instance in sys.modules? I think making things appear what they aren't is frowned upon in python.

Unless you are obfuscating code on purpose, I don't see why one would do all that.

Gulden

this is certainly useful in some cases, when you want to minimize import time for example. But that's always borderline, and it makes things more complicated (in particular debugging). Import hooks are difficult to get right for every situation. I would avoid them as much as possible fir 'core' libraries which are intended to be used outside their primary usage. – Rivalry

+1 for David's comment. Lazy imports add an additional level of complexity to any module's state. It's true that it might save a few moments writing import statements but that time will be lost in debugging. Incidentally many modern IDEs have the ability to manage imports. Check out pydev on Eclipse. – Nadler

For things like REST API wrappers, where the name of the function becomes the URI, for example, it's perfectly useful. Naming each function is silly. And having each function name be a string causes the user's code to be harder, not easier, to read. – Thetes

@ErikAronesty is right here, it makes it very easy to wrap an upstream API for convenience and leave validation etc to the other side. Also requires less manual work and upkeep to track API changes – Lilas

The original context I was thinking was based on rummaging around a network, and providing an API based on the APIs presented by devices on the network. So @ErikAronesty's point is particularly pertinent. (Being able to manually update this would be impractical) Incidentally, this allows things like: from networkdevices.local import robot robot.forward() print("Speed =", robot.speed) Which is quite nice to work with. – Gann