Default values for iterable unpacking

Question

I've often been frustrated by the lack of flexibility in Python's iterable unpacking.

Take the following example:

a, b = range(2)

Works fine. a contains 0 and b contains 1, just as expected. Now let's try this:

a, b = range(1)

Now, we get a ValueError:

ValueError: not enough values to unpack (expected 2, got 1)

Not ideal, when the desired result was 0 in a, and None in b.

There are a number of hacks to get around this. The most elegant I've seen is this:

a, *b = function_with_variable_number_of_return_values()
b = b[0] if b else None

Not pretty, and could be confusing to Python newcomers.

So what's the most Pythonic way to do this? Store the return value in a variable and use an if block? The *varname hack? Something else?

Carlynne

The most pythonic way is not to have the function return a variable number of things. Either return a container, or a consistent number of things, or raise an error (from the function or on unpacking) if it can't meet the requirements of its interface for some reason. – Animator

@jonrsharpe, what about functions that return lists? – Carlynne

Well that is a container, and semantically it's unusual to have specific requirements for the number of things in a list. – Animator

What’s an example of a real-world situation when you’ve needed this? It affects the right solution. – Ewen

@jonrsharpe, true, but it might be useful if you were doing something along the lines of a, b, *rest = function_returning_list() – Carlynne

Yes, maybe, but are you? As @Ryan says this requires some concrete context. There isn't a general solution; b = b[0] if b else None potentially ignores the third and above elements, quietly ignoring an unexpected occurrence, which is absolutely not Pythonic. And there should be an error in that case if you need two things but it only gives one. – Animator

@Ryan, it's mostly a hypothetical question, but here's an example of where it might be useful: You're processing command line arguments, and you want to split out the first few, but provide defaults if they're not there. In that case, _, arg1, arg2, *rest = sys.argv would sometimes work, but would often raise an error. – Carlynne

In that case I'd recommend using argparse, which will also generate useful help for you, or something like click. – Animator

@jonrsharpe, going with the list example, you could use something like b, rest = (b[0], b[1:]) if b else (None, []) – Carlynne

@jonrsharpe, again, it's mostly a hypothetical question. That was just an example. – Carlynne

You could, but I really wouldn't. This isn't an answerable question in the general case. – Animator

Like @Animator said, argparse would be better for parsing arguments, and it really does depend on the situation otherwise. You could use iter() and next() with a default, or a wrapper around that, or coroutines, or a function that fills in up to some number of values with defaults, or… – Ewen

Answer 1

As mentioned in the comments, the best way to do this is to simply have your function return a constant number of values and if your use case is actually more complicated (like argument parsing), use a library for it.

However, your question explicitly asked for a Pythonic way of handling functions that return a variable number of arguments and I believe it can be cleanly accomplished with decorators. They're not super common and most people tend to use them more than create them so here's a down-to-earth tutorial on creating decorators to learn more about them.

Below is a decorated function that does what you're looking for. The function returns an iterator with a variable number of arguments and it is padded up to a certain length to better accommodate iterator unpacking.

def variable_return(max_values, default=None):
    # This decorator is somewhat more complicated because the decorator
    # itself needs to take arguments.
    def decorator(f):
        def wrapper(*args, **kwargs):
            actual_values = f(*args, **kwargs)
            try:
                # This will fail if `actual_values` is a single value.
                # Such as a single integer or just `None`.
                actual_values = list(actual_values)
            except:
                actual_values = [actual_values]
            extra = [default] * (max_values - len(actual_values))
            actual_values.extend(extra)
            return actual_values
        return wrapper
    return decorator

@variable_return(max_values=3)
# This would be a function that actually does something.
# It should not return more values than `max_values`.
def ret_n(n):
    return list(range(n))

a, b, c = ret_n(1)
print(a, b, c)
a, b, c = ret_n(2)
print(a, b, c)
a, b, c = ret_n(3)
print(a, b, c)

Which outputs what you're looking for:

0 None None
0 1 None
0 1 2

The decorator basically takes the decorated function and returns its output along with enough extra values to fill in max_values. The caller can then assume that the function always returns exactly max_values number of arguments and can use fancy unpacking like normal.

Lafave

This is a nice idea. I'd not thought of using decorators. – Carlynne

The body of wrapper could be replaced with return islice(chain(f(*args, **kwargs), repeat(default)), max_values) (requiring from itertools import islice, chain, repeat) – Unlookedfor

Answer 2

Here's an alternative version of the decorator solution by @supersam654, using iterators rather than lists for efficiency:

def variable_return(max_values, default=None):
    def decorator(f):
        def wrapper(*args, **kwargs):
            actual_values = f(*args, **kwargs)
            try:
                for count, value in enumerate(actual_values, 1):
                    yield value
            except TypeError:
                count = 1
                yield actual_values
            yield from [default] * (max_values - count)
        return wrapper
    return decorator

It's used in the same way:

@variable_return(3)
def ret_n(n):
    return tuple(range(n))

a, b, c = ret_n(2)

This could also be used with non-user-defined functions like so:

a, b, c = variable_return(3)(range)(2)

Carlynne

The body of wrapper could be replaced with return islice(chain(f(*args, **kwargs), repeat(default)), max_values) (requiring from itertools import islice, chain, repeat) – Unlookedfor

Answer 3

Shortest known to me version (thanks to @KellyBundy in comments below):

a, b, c, d, e, *_ = *my_list_or_iterable, *[None]*5

Obviously it's possible to use other default value than None if necessary.

Also there is one nice feature in Python 3.10 which comes handy here when we know upfront possible numbers of arguments - like when unpacking sys.argv

Previous method:

import sys.argv

_, x, y, z, *_ = *sys.argv, *[None]*3

New method:

import sys


match sys.argv[1:]: #slice needed to drop first value of sys.argv
    case [x]:
        print(f'x={x}')
    case [x,y]:
        print(f'x={x}, y={y}')
    case [x,y,z]:
        print(f'x={x}, y={y}, z={z}')
    case _:
        print('No arguments')

Fricandeau

@KellyBundy - fair notice. Fixed example. – Fricandeau

Btw, my usual suggestion is a bit shorter. – Roselane

Nice! I've forgot about this notation. – Fricandeau

What you have now as "Previous method" doesn't work, as you removed your slicing. – Roselane

(facepalm) - fixed and tested – Fricandeau