Assignment inside lambda expression in Python

Question

I have a list of objects and I want to remove all objects that are empty except for one, using filter and a lambda expression.

For example if the input is:

[Object(name=""), Object(name="fake_name"), Object(name="")]

...then the output should be:

[Object(name=""), Object(name="fake_name")]

Is there a way to add an assignment to a lambda expression? For example:

flag = True 
input = [Object(name=""), Object(name="fake_name"), Object(name="")] 
output = filter(
    (lambda o: [flag or bool(o.name), flag = flag and bool(o.name)][0]),
    input
)

Licastro

Why not just pass a regular old function into filter? – Footman

I wanted to use lambda just so it would be a really compact solution. I remember in OCaml I could chain print statements before the return expression, thought this could be replicated in Python – Licastro

It is quite painful to be in the flow of developing a chained pipeilne then realize: "oh I want to create a temp var to make the flow more clear" or "i want to log this intermediate step" : and then you have to jump somewhere else to create a function to do it: and name that function and keep track of it - even though it's used in just one place . – Reprography

See also Assignment inside lambda expression in Python - Stack Overflow for the special case (++ or -- operator in C) – Aloin

Answer 1

The assignment expression operator := added in Python 3.8 supports assignment inside of lambda expressions. This operator can only appear within a parenthesized (...), bracketed [...], or braced {...} expression for syntactic reasons. For example, we will be able to write the following:

import sys
say_hello = lambda: (
    message := "Hello world",
    sys.stdout.write(message + "\n")
)[-1]
say_hello()

In Python 2, it was possible to perform local assignments as a side effect of list comprehensions.

import sys
say_hello = lambda: (
    [None for message in ["Hello world"]],
    sys.stdout.write(message + "\n")
)[-1]
say_hello()

However, it's not possible to use either of these in your example because your variable flag is in an outer scope, not the lambda's scope. This doesn't have to do with lambda, it's the general behaviour in Python 2. Python 3 lets you get around this with the nonlocal keyword inside of defs, but nonlocal can't be used inside lambdas.

There's a workaround (see below), but while we're on the topic...

---

In some cases you can use this to do everything inside of a lambda:

(lambda: [
    ['def'
        for sys in [__import__('sys')]
        for math in [__import__('math')]

        for sub in [lambda *vals: None]
        for fun in [lambda *vals: vals[-1]]

        for echo in [lambda *vals: sub(
            sys.stdout.write(u" ".join(map(unicode, vals)) + u"\n"))]

        for Cylinder in [type('Cylinder', (object,), dict(
            __init__ = lambda self, radius, height: sub(
                setattr(self, 'radius', radius),
                setattr(self, 'height', height)),

            volume = property(lambda self: fun(
                ['def' for top_area in [math.pi * self.radius ** 2]],

                self.height * top_area))))]

        for main in [lambda: sub(
            ['loop' for factor in [1, 2, 3] if sub(
                ['def'
                    for my_radius, my_height in [[10 * factor, 20 * factor]]
                    for my_cylinder in [Cylinder(my_radius, my_height)]],

                echo(u"A cylinder with a radius of %.1fcm and a height "
                     u"of %.1fcm has a volume of %.1fcm³."
                     % (my_radius, my_height, my_cylinder.volume)))])]],

    main()])()

A cylinder with a radius of 10.0cm and a height of 20.0cm has a volume of 6283.2cm³.
A cylinder with a radius of 20.0cm and a height of 40.0cm has a volume of 50265.5cm³.
A cylinder with a radius of 30.0cm and a height of 60.0cm has a volume of 169646.0cm³.

Please don't.

---

...back to your original example: though you can't perform assignments to the flag variable in the outer scope, you can use functions to modify the previously-assigned value.

For example, flag could be an object whose .value we set using setattr:

flag = Object(value=True)
input = [Object(name=''), Object(name='fake_name'), Object(name='')] 
output = filter(lambda o: [
    flag.value or bool(o.name),
    setattr(flag, 'value', flag.value and bool(o.name))
][0], input)

[Object(name=''), Object(name='fake_name')]

If we wanted to fit the above theme, we could use a list comprehension instead of setattr:

    [None for flag.value in [bool(o.name)]]

---

But really, in serious code you should always use a regular function definition instead of a lambda if you're going to be doing outer assignment.

flag = Object(value=True)
def not_empty_except_first(o):
    result = flag.value or bool(o.name)
    flag.value = flag.value and bool(o.name)
    return result
input = [Object(name=""), Object(name="fake_name"), Object(name="")] 
output = filter(not_empty_except_first, input)

Chiquia

The last example in this answer doesn't produce the same output as the example, but it looks to me like the example output is incorrect. – Chiquia

in short, this boils down to: use .setattr() and alikes (dictionaries should do as well, for instance) to hack side effects into functional code anyway, cool code by @JeremyBanks was shown :) – Tremayne

Answer 2

You cannot really maintain state in a filter/lambda expression (unless abusing the global namespace). You can however achieve something similar using the accumulated result being passed around in a reduce() expression:

>>> f = lambda a, b: (a.append(b) or a) if (b not in a) else a
>>> input = ["foo", u"", "bar", "", "", "x"]
>>> reduce(f, input, [])
['foo', u'', 'bar', 'x']
>>> 

You can, of course, tweak the condition a bit. In this case it filters out duplicates, but you can also use a.count(""), for example, to only restrict empty strings.

Needless to say, you can do this but you really shouldn't. :)

Lastly, you can do anything in pure Python lambda: http://vanderwijk.info/blog/pure-lambda-calculus-python/

Granado

Answer 3

Normal assignment (=) is not possible inside a lambda expression, although it is possible to perform various tricks with setattr and friends.

Solving your problem, however, is actually quite simple:

input = [Object(name=""), Object(name="fake_name"), Object(name="")]
output = filter(
    lambda o, _seen=set():
        not (not o and o in _seen or _seen.add(o)),
    input
    )

which will give you

[Object(Object(name=''), name='fake_name')]

As you can see, it's keeping the first blank instance instead of the last. If you need the last instead, reverse the list going in to filter, and reverse the list coming out of filter:

output = filter(
    lambda o, _seen=set():
        not (not o and o in _seen or _seen.add(o)),
    input[::-1]
    )[::-1]

which will give you

[Object(name='fake_name'), Object(name='')]

One thing to be aware of: in order for this to work with arbitrary objects, those objects must properly implement __eq__ and __hash__ as explained here.

Bundy

Answer 4

There's no need to use a lambda, when you can remove all the null ones, and put one back if the input size changes:

input = [Object(name=""), Object(name="fake_name"), Object(name="")] 
output = [x for x in input if x.name]
if(len(input) != len(output)):
    output.append(Object(name=""))

Matta

I think you have a small mistake in your code. The second line should be output = [x for x in input if x.name]. – Fanchet

Order of elements may be important. – Formant

Answer 5

UPDATE:

[o for d in [{}] for o in lst if o.name != "" or d.setdefault("", o) == o]

or using filter and lambda:

flag = {}
filter(lambda o: bool(o.name) or flag.setdefault("", o) == o, lst)

Previous Answer

OK, are you stuck on using filter and lambda?

It seems like this would be better served with a dictionary comprehension,

{o.name : o for o in input}.values()

I think the reason that Python doesn't allow assignment in a lambda is similar to why it doesn't allow assignment in a comprehension and that's got something to do with the fact that these things are evaluated on the C side and thus can give us an increase in speed. At least that's my impression after reading one of Guido's essays.

My guess is this would also go against the philosophy of having one right way of doing any one thing in Python.

Downstairs

So this isn't completely right. It won't preserve order, nor will it preserve duplicates of non-empty-stringed objects. – Polygamy

Answer 6

TL;DR: When using functional idioms it's better to write functional code

As many people have pointed out, in Python lambdas assignment is not allowed. In general when using functional idioms your better off thinking in a functional manner which means wherever possible no side effects and no assignments.

Here is functional solution which uses a lambda. I've assigned the lambda to fn for clarity (and because it got a little long-ish).

from operator import add
from itertools import ifilter, ifilterfalse
fn = lambda l, pred: add(list(ifilter(pred, iter(l))), [ifilterfalse(pred, iter(l)).next()])
objs = [Object(name=""), Object(name="fake_name"), Object(name="")]
fn(objs, lambda o: o.name != '')

You can also make this deal with iterators rather than lists by changing things around a little. You also have some different imports.

from itertools import chain, islice, ifilter, ifilterfalse
fn = lambda l, pred: chain(ifilter(pred, iter(l)), islice(ifilterfalse(pred, iter(l)), 1))

You can always reoganize the code to reduce the length of the statements.

Moorings

Answer 7

The pythonic way to track state during iteration is with generators. The itertools way is quite hard to understand IMHO and trying to hack lambdas to do this is plain silly. I'd try:

def keep_last_empty(input):
    last = None
    for item in iter(input):
        if item.name: yield item
        else: last = item
    if last is not None: yield last

output = list(keep_last_empty(input))

Overall, readability trumps compactness every time.

Ulcerate

Answer 8

If instead of flag = True we can do an import instead, then I think this meets the criteria:

>>> from itertools import count
>>> a = ['hello', '', 'world', '', '', '', 'bob']
>>> filter(lambda L, j=count(): L or not next(j), a)
['hello', '', 'world', 'bob']

Or maybe the filter is better written as:

>>> filter(lambda L, blank_count=count(1): L or next(blank_count) == 1, a)

Or, just for a simple boolean, without any imports:

filter(lambda L, use_blank=iter([True]): L or next(use_blank, False), a)

Irenairene

Answer 9

No, you cannot put an assignment inside a lambda because of its own definition. If you work using functional programming, then you must assume that your values are not mutable.

One solution would be the following code:

output = lambda l, name: [] if l==[] \
             else [ l[ 0 ] ] + output( l[1:], name ) if l[ 0 ].name == name \
             else output( l[1:], name ) if l[ 0 ].name == "" \
             else [ l[ 0 ] ] + output( l[1:], name )

Orelee

Answer 10

If you need a lambda to remember state between calls, I would recommend either a function declared in the local namespace or a class with an overloaded __call__. Now that all my cautions against what you are trying to do is out of the way, we can get to an actual answer to your query.

If you really need to have your lambda to have some memory between calls, you can define it like:

f = lambda o, ns = {"flag":True}: [ns["flag"] or o.name, ns.__setitem__("flag", ns["flag"] and o.name)][0]

Then you just need to pass f to filter(). If you really need to, you can get back the value of flag with the following:

f.__defaults__[0]["flag"]

Alternatively, you can modify the global namespace by modifying the result of globals(). Unfortunately, you cannot modify the local namespace in the same way as modifying the result of locals() doesn't affect the local namespace.

Polygamy

Or just use the original Lisp: (let ((var 42)) (lambda () (setf var 43))). – Sauer

Answer 11

You can use a bind function to use a pseudo multi-statement lambda. Then you can use a wrapper class for a Flag to enable assignment.

bind = lambda x, f=(lambda y: y): f(x)

class Flag(object):
    def __init__(self, value):
        self.value = value

    def set(self, value):
        self.value = value
        return value

input = [Object(name=""), Object(name="fake_name"), Object(name="")]
flag = Flag(True)
output = filter(
            lambda o: (
                bind(flag.value, lambda orig_flag_value:
                bind(flag.set(flag.value and bool(o.name)), lambda _:
                bind(orig_flag_value or bool(o.name))))),
            input)

Damnify

Answer 12

Kind of a messy workaround, but assignment in lambdas is illegal anyway, so it doesn't really matter. You can use the builtin exec() function to run assignment from inside the lambda, such as this example:

>>> val
Traceback (most recent call last):
  File "<pyshell#31>", line 1, in <module>
    val
NameError: name 'val' is not defined
>>> d = lambda: exec('val=True', globals())
>>> d()
>>> val
True

Fascia

Answer 13

first , you dont need to use a local assigment for your job, just check the above answer

second, its simple to use locals() and globals() to got the variables table and then change the value

check this sample code:

print [locals().__setitem__('x', 'Hillo :]'), x][-1]

if you need to change the add a global variable to your environ, try to replace locals() with globals()

python's list comp is cool but most of the triditional project dont accept this(like flask :[)

hope it could help

Necrotomy

You cannot use locals(), it explicitly says in the documentation that changing it doesn't actually change the local scope (or at least it won't always). globals() on the other hand works as expected. – Polygamy

@Polygamy just try it, dont follow the document blindly. and assignment in lambda is breaking rule already – Necrotomy

It unfortunately only works in the global namespace, in which case you really should be using globals(). pastebin.com/5Bjz1mR4 (tested in both 2.6 and 3.2) proves it. – Polygamy