Callable modules

Question

Why doesn't Python allow modules to have a __call__ method? (Beyond the obvious that it wouldn't be easy to import directly.) Specifically, why doesn't using a(b) syntax find the __call__ attribute like it does for functions, classes, and objects? (Is lookup just incompatibly different for modules?)

>>> print(open("mod_call.py").read())
def __call__():
    return 42

>>> import mod_call
>>> mod_call()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'module' object is not callable
>>> mod_call.__call__()
42

Upanchor

Migrating a decorator from a package into its own sub-module. @example(...) was by far still the most common use-case, but @example.special_case(...) was a new use. I didn't want to implement it with an example class and static methods, since that was a poor fit. Not sure a callable module is a better fit, but I started investigating it and then wanted to know why it didn't work. – Upanchor

I had also thought it could simplify some modules such as datetime and decimal, by making the module.__call__ be datetime.datetime or decimal.Decimal respectively. However, then type(decimal('1')) wouldn't be the same as decimal, and possible other issues. shrug It was an idea. – Upanchor

"Beyond the obvious that it wouldn't be easy to import directly." Why do you think that? – Teratism

@Longpoke: It would be cumbersome and inconsistent to import just call. Perhaps I could've phrased that better (when I asked this over a year ago), but it still appears that way to me. – Upanchor

Not sure what you mean, like from mymodule import __call__ ? – Teratism

@Longpoke: Yes, that's what I meant by "import just call". – Upanchor

Warning: this trick will make your code not working with tools like Pylint. Everything fails. I guess typing more characters at the end save the time in this case: see libraries like tqdm whose are primarily called with from tqdm import tqdm but they didn't apply this trick. – Stipulate

Answer 1

Special methods are only guaranteed to be called implicitly when they are defined on the type, not on the instance. (__call__ is an attribute of the module instance mod_call, not of <type 'module'>.) You can't add methods to built-in types.

https://docs.python.org/reference/datamodel.html#special-lookup

Asdic

Package forbiddenfruit begs to differ about being able to add special methods onto builtin types, but I mention this only as an academic curiosity. I think there are only relatively rare cases where monkeypatching builtin types is acceptable, and it would be particularly bad for a typical imported module to do so. (Incidentally, forbiddenfruit cannot monkeypatch object, and it can cause segfaults later in execution if you try, but it can monkeypatch <type 'function'>, so I'm assuming it can probably monkeypatch <type 'module'> but I wouldn't bet my life on it.) – Wiskind

Answer 2

Python doesn't allow modules to override or add any magic method, because keeping module objects simple, regular and lightweight is just too advantageous considering how rarely strong use cases appear where you could use magic methods there.

When such use cases do appear, the solution is to make a class instance masquerade as a module. Specifically, code your mod_call.py as follows:

import sys

class mod_call:
    def __call__(self):
        return 42

sys.modules[__name__] = mod_call()

Now your code importing and calling mod_call works fine.

Indigoid

Wow. I was seconds from posting a code example based on this example of yours: #881030 – Ormazd

Note: instead of the class with a __call__ method, a normal function is also possible – Athodyd

Note: This solution does hide other functions (or other items) in the module. This can be solved by adding them to class – Athodyd

You may want to subclass types.ModuleType so that it is still considered a module object. (then you really would have a callable module) – Yazbak

I like @TadhgMcDonald-Jensen 's suggestion of subclassing ModuleType but be aware that if you do, you'll need to override __init__ to pass in the name of the module to super().__init__('moduleName') – Teletype

@MichaelScottCuthbert or equivalently replace sys.modules[__name__] = mod_call() with sys.modules[__name__] = mod_call(__name__) – Zanze

This works for import mod_call, but for from mod_call import *; mod_call() I get NameError: name 'mod_call' is not defined. Is there a way to solve this? – Langtry

@Friedrich I guess none. – Stipulate

@Friedrich Put mod_call as an attribute named mod_call on itself. mod_call.mod_call = mod_call below the end of the class mod_call definition. You also need to define __all__ = ["mod_call"] or equivalent in class mod_call. – Wiskind

@Wiskind Thx. This is basically what I suggested below in https://mcmap.net/q/261856/-callable-modules. – Langtry

@Friedrich Cheers, I didn't scroll very far in the answers. Just saw your comment here and felt like improvising a solution. Glad to see you had figured it out much sooner. – Wiskind

Answer 3

As Miles says, you need to define the call on class level. So an alternative to Alex post is to change the class of sys.modules[__name__] to a subclass of the type of sys.modules[__name__] (It should be types.ModuleType).

This has the advantage that the module is callable while keeping all other properties of the module (like accessing functions, variables, ...).

import sys

class MyModule(sys.modules[__name__].__class__):
    def __call__(self):  # module callable
        return 42

sys.modules[__name__].__class__ = MyModule

Note: Tested with python3.6.

Spurling

This only works on Python 3.5 and up, because it relies on a special case added in 3.5 that singles out modules as the only objects implemented in C that can have their __class__ reassigned. – Thrill

For later readers: don't forget to inherit sys.modules[__name__].__class__ since: TypeError: __class__ assignment only supported for heap types or ModuleType subclasses.. – Stipulate

Answer 4

Christoph Böddeker's answer seems to be the best way to create a callable module, but as a comment says, it only works in Python 3.5 and up.

The benefit is that you can write your module like normal, and just add the class reassignment at the very end, i.e.

# coolmodule.py
import stuff

var = 33
class MyClass:
   ...
def function(x, y):
   ...

class CoolModule(types.ModuleType):
    def __call__(self):
        return 42
sys.modules[__name__].__class__ = CoolModule

and everything works, including all expected module attributes like __file__ being defined. (This is because you're actually not changing the module object resulting from the import at all, just "casting" it to a subclass with a __call__ method, which is exactly what we want.)

To get this to work similarly in Python versions below 3.5, you can adapt Alex Martelli's answer to make your new class a subclass of ModuleType, and copy all the module's attributes into your new module instance:

#(all your module stuff here)

class CoolModule(types.ModuleType):
    def __init__(self):
        types.ModuleType.__init__(self, __name__)
        # or super().__init__(__name__) for Python 3
        self.__dict__.update(sys.modules[__name__].__dict__)
    def __call__(self):
        return 42

sys.modules[__name__] = CoolModule()

Now __file__, __name__ and other module attributes are defined (which aren't present if just following Alex's answer), and your imported module object still "is a" module.

Cubeb

Answer 5

All answers work only for import mod_call. To get it working simultaneously for from mod_call import *, the solution of @Alex Martelli can be enhanced as follow

import sys

class mod_call:
    def __call__(self):
        return 42
    mod_call = __call__
    __all__ = list(set(vars().keys()) - {'__qualname__'})   # for python 2 and 3

sys.modules[__name__] = mod_call()

This solution was derived with the discussion of an answer of a similar problem.

Tolson

The no-maintenance/no-duplication __all__ is a cool addition! Personally, I would filter out anything that starts with an _ from __all__, and I would also make __all__ a tuple. Something like __all__ = tuple(key for key in vars() if not key.startswith('_')). – Wiskind

Answer 6

To turn the solution into a convenient reusable function:

def set_module(cls, __name__):
    import sys

    class cls2(sys.modules[__name__].__class__, cls):
        pass

    sys.modules[__name__].__class__ = cls2

save it to, say, util.py. Then in your module,

import util

class MyModule:
    def __call__(self):  # module callable
        return 42

util.set_module(MyModule, __name__)

Hurray!

I wrote this because I need to enhance a lot of modules with this trick.

---

P.S. Few days after I wrote this answer, I removed this trick from my code, since it is so tricky for tools like Pylint to understand.

Stipulate

Answer 7

Using Python version 3.10.8, formatting my code as below allowed me to:

• Make my module callable (thanks Alex)
• Keep all properties of the module accessible (e.g. methods) (thanks Nick)
• Allow the module to work when used as from CallableModule import x, y, z, although not as from CallableModule import * (I tried to employ Friedrich's solution)

    from types import ModuleType
    
    class CallableModule(ModuleType):
        def __init__(self):
            ModuleType.__init__(self, __name__)
            self.__dict__.update(modules[__name__].__dict__)
    
        def __call__(self):
            print("You just called a module UwU")
      
        mod_call= __call__
        __all__= list(set(vars().keys()) - {'__qualname__'})
    modules[__name__]= CallableModule()

Everywhere