Neat way to define __all__?

Question

Hi I'm building my own package and I have a question on __all__.
Are there any neat way to define __all__, other than explicitly typing each and every function in the module?
I find it very tedious...

I'm trying to make some code which wraps on frequently used libraries such as numpy, pytorch, os. The problem is, the libraries I used to create my modules also gets imported when I import my package.
I want to import every function / class that I defined, but I don't want the third-party libraries that I used in the process to get imported.
I use from .submodule import * in my __init__.py so that I can access my functions inside the submodule directly. (Just like we can access functions directly from the top package like np.sum(), torch.sum() )
My submodule has a lot of functions, and I want to import all of them to __init__.py, except for the third-party packages that I used.

I see that __all__ defines what to import when from package import * is called.
For example,

utils.py

__all__ = ['a']
def a():
    pass
def b():
    pass

__init__.py

from .utils import *

and

>>> import package
>>> package.a()
None
>>> package.b()
NameError: 'package.b' is not defined

What I want is something like

__all__ = Some_neat_fancy_method()

I tried locals() and dir(), but got lost along the way. Any suggestions?

Silversmith

You can just omit __all__ if you want to include everything – Dour

using * to import is the wrong practice. I suggest yout import by name, you can use __all__ in __init__ file and define which packages you want to make avaiable – Hollishollister

I see how everyone got confused. I added more explanation. I'm building a package on top of other third-party libaries, and the problem is, they also get imported. I want to import every function / class that I defined, but I don't want the third-party libraries that I used in the process to get imported. – Silversmith

Answer 1

As others have pointed out, the whole point of __all__ is to explicitly specify what gets exposed to star-imports. By default everything is. If you really want to specify what doesn't get exposed instead, you can do a little trick and include all modules in __all__ and then remove the ones you want to exclude.

For example:

def _exclude(exclusions: list) -> list:
    import types

    # add everything as long as it's not a module and not prefixed with _
    functions = [name for name, function in globals().items()
                 if not (name.startswith('_') or isinstance(function, types.ModuleType))]

    # remove the exclusions from the functions
    for exclusion in exclusions:
        if exclusion in functions:
            functions.remove(exclusion)

    del types  # deleting types from scope, introduced from the import
    return functions


# the _ prefix is important, to not add these to the __all__
_exclusions = ["function1", "function2"]
__all__ = _exclude(_exclusions)

You can of course repurpose this to simply include everything that's not a function or prefixed with _ but it serves little use since everything is included in star-imports if you don't specify the __all__, so I thought it was better to include the exclusion idea. This way you can simply tell it to exclude specific functions.

Coachwork

Answer 2

Are there any neat way to define all, other than explicitly typing each and every function in the module?

Not built-in no. But defining __all__ by hand is basically the entire point, if you want to include everything in __all__ you can just do nothing at all:

If __all__ is not defined, the statement from sound.effects import * [...] ensures that the package sound.effects has been imported (possibly running any initialization code in __init__.py) and then imports whatever names are defined in the package.

The entire point of __all__ is to restricts what gets "exported" by star-imports. There's no real way for Python to know that except by having you tell it, for each symbol, whether it should be there or not.

Joscelin

Answer 3

One easy workaround is to alias all of your imports with a leading underscore. Anything with a leading underscore is excluded from from x import * style imports.

import numpy as _np
import pandas as _pd

def my_fn():
    ...

Amnesty

Answer 4

I would define a decorator named "export" and decorate the functions, classes, etc. that I want to add to all. The decorator adds the item to a list of "exports" and and then

all = exports

Nyhagen