Non-lazy evaluation version of map in Python3?

Question

I'm trying to use map in Python3. Here's some code I'm using:

import csv

data = [
    [1],
    [2],
    [3]
]

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    map(writer.writerow, data)

However, since map in Python3 returns an iterator, this code doesn't work in Python3 (but works fine in Python2 since that version of map always return a list)

My current solution is to add a list function call over the iterator to force the evaluation. But it seems odd (I don't care about the return value, why should I convert the iterator into a list?)

Any better solutions?

Huppert

Using map for side effects is what's odd. Python 2 map also collects the return values. The new behavior merely highlights it further. Just don't do that, use a for loop. – Huskamp

@delnan Thank you for the link, indeed I shouldn't use map for side effect. – Huppert

for Python 3, list(map(lambda x:2*x, [1,2,3])) – Emeliaemelin

I feel like "map for the side effects" is a common-enough idiom that one could justify including it as a language feature. However, python3 has decided that map shall be a lazily-evaluated function more in line with other languages. In any case, I find myself typing [*map(...)] almost all the time when I'm using Python for scientific statistics, where usually an immediate result is needed (e.g. numpy.array(...) understands lists but not generators). So the non-lazy map is the superior default for some applications. – Musicianship

FWIW I've defined the helpers lmap and amap that add the necessary boiler plate to immediately collect the results of map() as a list or np.array, respectively. It makes the code significantly cleaner, easier to read, and easier to maintain. Scattering casts to list or [*map(...)] everywhere, or using a list comprehension when map is much more succinct, just looks sloppy. – Musicianship

Answer 1

Using map for its side-effects (eg function call) when you're not interested in returned values is undesirable even in Python2.x. If the function returns None, but repeats a million times - you'd be building a list of a million Nones just to discard it. The correct way is to either use a for-loop and call:

for row in data:
    writer.writerow(row)

or as the csv module allows, use:

writer.writerows(data)

If for some reason you really, really wanted to use map, then you can use the consume recipe from itertools and generate a zero length deque, eg:

from collections import deque
deque(map(writer.writerow, data), maxlen=0)

Fugacity

Answer 2

You can set up a zero length deque to do this:

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    collections.deque(map(writer.writerow, data),0)

This is the same way that itertools.consume(iterator, None) recipe works. It functionally will exhaust the iterator without building the list.

You can also just use the consume recipe from itertools.

But a loop is more readable and Pythonic to me, but YMMV.

Jaquenette

Answer 3

list(map(lambda x: do(x),y))

will trigger an evaluation and stay in english pen-and-paper math semantics. Some like this more than others, but I've had a coder who spoke English as a second language tell me this is harder to understand than:

[ do(x) for x in y ]

¯\(ツ)/¯

Anyhow, if you want to flush the map stack, trigger it with a list type conversion or a list comprehension.

---

With context:

import csv

data = [
    [1],
    [2],
    [3]
]

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    list(map(writer.writerow, data))
    # or: [ writer.writerow(x) for x in data ]

Baculiform

Exactly what I'm doing. Glad to know that I'm not missing out too much lol – Dungeon

Answer 4

If you don't care about the return value, then map is not the best tool for the job. A simple for would be better:

for d in data:
    writer.writerow(d)

That'll work fine in Python 2.x and 3.x. Notice that map is useful when you want to create a new list, if you're traversing over an iterable just for the effect, then use a for.

Kreg

Answer 5

You can also use list comprehension, as suggested in the official FAQ:

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    [writer.writerow(elem) for elem in data]

List comprehension will force the evaluation of each element even if you don't assign the newly created list to any variable.

Note, however, that the list may still be created behind the scenes, creating a potential performance pitfall, so while relatively succinct, it should not be used if the input sequence can get very long.

Foundling

Answer 6

I would use a function to extract data from the iterable using something like this:

def rake(what, where=None):
    for i in what: 
        if where: where.append(i)

rake(map(writer.writerow, data))

If you know up front that you won't ever be collecting the output of the mapped function then you could simplify that to just:

for i in what: pass

But neither approach keeps excess data around unless you supply a list to put it in. And this approach should work equally well with map, filter, reduce, generators, range, and anything else that you could pass in to the rake function that for can iterate over.

Georginegeorglana