Why results of map() and list comprehension are different? [duplicate]

Question

The following test fails:

#!/usr/bin/env python
def f(*args):
    """
    >>> t = 1, -1
    >>> f(*map(lambda i: lambda: i, t))
    [1, -1]
    >>> f(*(lambda: i for i in t)) # -> [-1, -1]
    [1, -1]
    >>> f(*[lambda: i for i in t]) # -> [-1, -1]
    [1, -1]
    """
    alist = [a() for a in args]
    print(alist)

if __name__ == '__main__':
    import doctest; doctest.testmod()

In other words:

>>> t = 1, -1
>>> args = []
>>> for i in t:
...   args.append(lambda: i)
...
>>> map(lambda a: a(), args)
[-1, -1]
>>> args = []
>>> for i in t:
...   args.append((lambda i: lambda: i)(i))
...
>>> map(lambda a: a(), args)
[1, -1]
>>> args = []
>>> for i in t:
...   args.append(lambda i=i: i)
...
>>> map(lambda a: a(), args)
[1, -1]

Pehlevi

For those like me read the question but don't notice any problem at first: note the [-1, -1]! Essentially lambda i: ... in a loop doesn't capture the current value of i. – Mumford

related from Python FAQ: Why do lambdas defined in a loop with different values all return the same result? – Pehlevi

Answer 1

They are different, because the value of i in both the generator expression and the list comp are evaluated lazily, i.e. when the anonymous functions are invoked in f.
By that time, i is bound to the last value if t, which is -1.

So basically, this is what the list comprehension does (likewise for the genexp):

x = []
i = 1 # 1. from t
x.append(lambda: i)
i = -1 # 2. from t
x.append(lambda: i)

Now the lambdas carry around a closure that references i, but i is bound to -1 in both cases, because that is the last value it was assigned to.

If you want to make sure that the lambda receives the current value of i, do

f(*[lambda u=i: u for i in t])

This way, you force the evaluation of i at the time the closure is created.

Edit: There is one difference between generator expressions and list comprehensions: the latter leak the loop variable into the surrounding scope.

Hospitable

Lambdas are evil because it isn't clear what what runtime context really is. – Kenleigh

@S.Lott: Ordinary functions in Python are not that different. def f(): return i You don't know what i really is regardless function or lambda is considered. – Pehlevi

In Python3, "the loop control variables are no longer leaked into the surrounding scope." – Dejection

Answer 2

The lambda captures variables, not values, hence the code

lambda : i

will always return the value i is currently bound to in the closure. By the time it gets called, this value has been set to -1.

To get what you want, you'll need to capture the actual binding at the time the lambda is created, by:

>>> f(*(lambda i=i: i for i in t)) # -> [-1, -1]
[1, -1]
>>> f(*[lambda i=i: i for i in t]) # -> [-1, -1]
[1, -1]

Tepid

Answer 3

Expression f = lambda: i is equivalent to:

def f():
    return i

Expression g = lambda i=i: i is equivalent to:

def g(i=i):
    return i

i is a free variable in the first case and it is bound to the function parameter in the second case i.e., it is a local variable in that case. Values for default parameters are evaluated at the time of function definition.

Generator expression is the nearest enclosing scope (where i is defined) for i name in the lambda expression, therefore i is resolved in that block:

f(*(lambda: i for i in (1, -1)) # -> [-1, -1]

i is a local variable of the lambda i: ... block, therefore the object it refers to is defined in that block:

f(*map(lambda i: lambda: i, (1,-1))) # -> [1, -1]

Pehlevi