Using list instead of tuple in module __all__

Question

In many large projects, even in such as Django, and in official Python documentation use list to list the "available from outside" module components in the __init__.py file:

__all__ = [foo, bar, other]

However, the record

__all__ = (foo, bar, other)

it will also work, and in theory it does not give a significant, but an increase in code performance.

Why, then use it to list?

Maybe there is some magic PEP that I don't know about?

Knowall

Be aware the performance of __all__ is practically insignificant for program runtime. It is only relevant for *-imports to begin with, which are discouraged via PEP 8. – Beaufort

@Beaufort I understand this, but still why use list specifically? – Knowall

Because lists are idiomatic for sequence-like containers of arbitrary size. You should never be using a tuple for any extremely slight performance increase. Tuples are used for "record-like" data – Permanency

Also, that is a tuple not a set. – Permanency

I have adjusted the title to match the question body (init->all, set->tuple). Be aware that a set cannot be used for __all__. – Beaufort

Because once your library gets past a certain size, you end up mangling/constructing __all__ dynamically a lot of the time. – Hereunder

Answer 1

There is no binding reason to use either list or tuple. However, list idiomatically represents a sequence of same kind of items but tuple represents a sequence of different kind of items. This is also encoded by type hints, which have some list[str | int] but positional fields inside tuple[str, int, int].

As such, list more accurately represents "arbitrary sequence of names".

PEP 8 repeatedly makes mention of __all__ being a list:

To better support introspection, modules should explicitly declare the names in their public API using the __all__ attribute. Setting __all__ to an empty list indicates that the module has no public API.

"""This is the example module.

This module does stuff.
"""
...
__all__ = ['a', 'b', 'c']

Beaufort

The difference you describe is more of a conventional/intentional nature. The main factual difference between the two is mutability. – Route

@schwobaseggl. Thanks for pointing that out. The mutability of __all__ is frequently used. I've posted an answer based on my usage experience, which boils down to the fact that a mutable __all__ is often necessary. – Hereunder

To the point that @schwobaseggl made, do you have any sources or examples to back up your point about the idiom regarding similar vs different type objects in lists vs tuples? I’ve never heard of such a distinction being made between those two collections, even as a matter of style. – Fasciation

@Michael "Tuples are immutable, and usually contain a heterogeneous sequence of elements that are accessed via unpacking (see later in this section) or indexing (or even by attribute in the case of namedtuples). Lists are mutable, and their elements are usually homogeneous and are accessed by iterating over the list." – Beaufort

Answer 2

The language reference says:

The public names defined by a module are determined by checking the module’s namespace for a variable named __all__; if defined, it must be a sequence of strings which are names defined or imported by that module.

A sequence is something that supports iteration (for x in __all__) and access using integer indices (__all__[i]). So it can be a list, or a tuple.

Selinski

Answer 3

From a practical standpoint, it's somewhat common to add elements to __all__ semi-dynamically. It happens when you want to expose functions defined deep in the package structure at the top level. This is much easier to do with a list.

A couple of examples of modules that do this are numpy and pyserial. I strongly suspect that Django does this too in places, but am not familiar enough with it to know for sure.

The idiom looks something like this in __init__.py:

__all__ = []  # or might have some initial names

from .subpackage import (name1, name2, name3)

__all__.extend(['name1', 'name2', 'name3']) # or append 1-by-1 or +=

I've even seen a slightly sloppier approach, although arguably more maintainable under certain circumstances, that looks like this:

__all__ = []

from .subpackage import *
from .subpackage import __all__ as _all

__all__.extend(_all)
del _all

Clearly this is greatly simplified by having a mutable __all__. There is no substantial benefit to turning it into a tuple after the fact or "appending" to a tuple using +=.

Another way a mutable __all__ is useful is when your API depends on optional external packages. It's much easier to enable or disable names in a list than a tuple.

Here is an example of a module that enables additional functionality if a library called optional_dependency is installed:

# Core API
__all__ = ['name', 'name2', 'name3']

from .sub1 import name1
from .sub2 import name2, name3

try:
    import optional_dependency
except ImportError:
    # Let it be, it maybe issue a warning
    pass
else:
    from .opt_support import name4, name5

    __all__ += ['name4', 'name5']

Hereunder

The first example makes no sense to me. It would be equally simple to just write __all__ = ("name1", "name2", "name3"). Why would you need to first create an empty list? For the second example, you could do __all__ = (..., *_all). Same for the third example. I see no advantage in using lists here. – Shiflett

@iuvbio. You can do it that way too. Lists are more flexible, but flexibility is not always appropriate. – Hereunder

Answer 4

Just wanted to document a little error I ran into relevant to this post: note that you need a trailing comma to create a single element tuple. So this:

__all__ = ['foo'] # I am list with one element

Is not the same as this:

__all__ = ('foo') # I am a string

Here's an example of this going wrong. In the second case, if you try to import with the wildcard*:

from mymodule import *

You get the confusing error:

AttributeError: module 'mypackage.mymodule' has no attribute 'f'

What is 'f'?!? Well, it is the first element of __all__, which is pointing to the string 'foo', not the single-element tuple ('foo',).

---

* Using from x import * is maybe what is more to blame here, as opposed to the tuple vs. list choice. But this still seems to be a relatively common pattern in __init__.py files, and makes me lean towards preferring lists.

Hydrastine

I'm not sure that I would call this a bug. You can also define tuples without parentheses: __all__ = 'foo', 'bar'. The representation of a tuple uses parentheses, but the parser still sees a single element in parentheses as a single element. Since a string is iterable, that's why you get the attribute error for the first character. – Dilemma

@Supra621 Thanks for the input. I guess by "bug" I meant a bug in my code (i.e. not doing what is expected), not a bug in Python. But I will rephrase because that is misleading. – Hydrastine