Getting a map() to return a list in Python 3.x [duplicate]

Question

I'm trying to map a list into hex, and then use the list elsewhere. In python 2.6, this was easy:

A: Python 2.6:

>>> map(chr, [66, 53, 0, 94])
['B', '5', '\x00', '^']

However, in Python 3.1, the above returns a map object.

B: Python 3.1:

>>> map(chr, [66, 53, 0, 94])
<map object at 0x00AF5570>

How do I retrieve the mapped list (as in A above) on Python 3.x?

Alternatively, is there a better way of doing this? My initial list object has around 45 items and id like to convert them to hex.

Matabele

It's more pythonic to use a list comprehension. map() was almost removed from the language because there's no reason to use it over a list comprehension or a for loop. – Douty

^^^ That comment should be the approved answer! – Counterforce

Answer 1

Do this:

list(map(chr,[66,53,0,94]))

In Python 3+, many processes that iterate over iterables return iterators themselves. In most cases, this ends up saving memory, and should make things go faster.

If all you're going to do is iterate over this list eventually, there's no need to even convert it to a list, because you can still iterate over the map object like so:

# Prints "ABCD"
for ch in map(chr,[65,66,67,68]):
    print(ch)

Birdwatcher

Of course, you can iterate over this, too: (chr(x) for x in [65,66,67,68]). It doesn't even need map. – Milled

@Milled The argument for using 3.1's map would be lazy evaluation when iterating on a complex function, large data sets, or streams. – Shoshana

@Andrew actually Hugh is uing a generator comprehension which would do the same thing. Note the parentheses rather than square brackets. – Birdwatcher

Alternate solution (faster for large inputs too) when the values are known to be ASCII/latin-1 is to do bulk conversions at the C layer: bytes(sequence_of_ints_in_range_0_to_256).decode('latin-1') which makes a str faster by avoiding Python function calls for each element in favor of a bulk conversion of all elements using only C level function calls. You can wrap the above in list if you really need a list of the individual characters, but since str is already an iterable of its own characters, the only reason you'd do so is if you need mutability. – Cowart

list(map(str, [1,2,3])) gives "Error in argument" for Python 3.4.3 on CentOS 7. List comprehension works. – Lidialidice

The "Error in argument" occures only in PDB debugger. See: #17290814 – Lidialidice

"In Python 3+, many processes that iterate over iterables return iterators themselves.": But in Python 2 there is imap and ifilter, so the new semantics of map and filter is not new, it is just the old semantics of imap and ifilter with a new name. So have map and filter returning a list been dropped completely? Having to add an extra call to list looks like unnecessary boilerplate to me. – Malacca

please be warned that the map object is an iterator, which means it will be come empty after consuming. #45018437 – Archducal

I don't know if this can be of any help, but as @Milled pointed out - if using that method; Would this work? Please tell me if it is anything to be careful with: list(chr(x) for x in range(65,68)) # to print same results as (chr(x) for x in [65,66,67,68])., as pointed out by H – Wagers

Also, I have a version of @Triptych (using range() as my above comment) in it: for ch in map(chr,range(65,68)): print(ch) – Wagers

@Archducal Thanks for confirming this - been going nuts all afternoon chasing emptied map objects! – Folberth

@DanielMGessel Me too, when I was new to python this had me troubled too. – Wagers

Answer 2

New and neat in Python 3.5:

[*map(chr, [66, 53, 0, 94])]

Thanks to Additional Unpacking Generalizations

UPDATE

Always seeking for shorter ways, I discovered this one also works:

*map(chr, [66, 53, 0, 94]),

Unpacking works in tuples too. Note the comma at the end. This makes it a tuple of 1 element. That is, it's equivalent to (*map(chr, [66, 53, 0, 94]),)

It's shorter by only one char from the version with the list-brackets, but, in my opinion, better to write, because you start right ahead with the asterisk - the expansion syntax, so I feel it's softer on the mind. :)

Kaylee

@Quelklef list() doesn't look as neat – Cruel

@Quelklef: Also, the unpacking approach is trivially faster thanks to not needing to look up the list constructor and invoke general function call machinery. For a long input, it won't matter; for a short one, it can make a big difference. Using the above code with the input as a tuple so it's not repeatedly reconstructed, ipython microbenchmarks show the list() wrapping approach takes about 20% longer than unpacking. Mind you, in absolute terms, we're talking about 150 ns, which is trivial, but you get the idea. – Cowart

What was wrong with the old map? Maybe with a new name (lmap?) if the new default is to return an iterator? – Malacca

*map() gives syntax error on Python 3.6: can't use starred expression here. You need to put it in a list: [ *map() ] – Curl

@ALH You missed the comma on the end of the command. Easy mistake to make! – Pilch

I've found that using list actually runs faster than this – Stokehold

Yuk. You would surely only do that if profiling indicated you have a performance problem. Otherwise this is line-noise-programming... what is wrong with a list comprehension? – Counterforce

list(map(...)) is the idiomatic way. – Unlawful

Answer 3

Why aren't you doing this:

[chr(x) for x in [66,53,0,94]]

It's called a list comprehension. You can find plenty of information on Google, but here's the link to the Python (2.6) documentation on list comprehensions. You might be more interested in the Python 3 documenation, though.

Khano

Hmmmm. Maybe there needs to be a general posting on list comprehensions, generators, map(), zip(), and a lot of other speedy iteration goodness in python. – Milled

I guess because it's more verbose, you have to write an extra variable (twice)... If the operation is more complex and you end up writing a lambda, or you need also to drop some elements, I think a comprehension is definitively better than a map+filter, but if you already have the function you want to apply, map is more succinct. – Lector

+1: Easier to read & allows you to use functions with many parameters – Prone

map(chr, [66,53,0,94]) is definitely more concise than [chr(x) for x in [66,53,0,94]]. – Malacca

way faster than the other answers – Remitter

Because I don't want to return the list, I just want to modify all its elements. Or, I'm calling a function that returns None (like dict.update()) so a comprehension just gives me [None, None, ...]. – Ploughshare

Answer 4

List-returning map function has the advantage of saving typing, especially during interactive sessions. You can define lmap function (on the analogy of python2's imap) that returns list:

lmap = lambda func, *iterable: list(map(func, *iterable))

Then calling lmap instead of map will do the job: lmap(str, x) is shorter by 5 characters (30% in this case) than list(map(str, x)) and is certainly shorter than [str(v) for v in x]. You may create similar functions for filter too.

There was a comment to the original question:

I would suggest a rename to Getting map() to return a list in Python 3.* as it applies to all Python3 versions. Is there a way to do this? – meawoppl Jan 24 at 17:58

It is possible to do that, but it is a very bad idea. Just for fun, here's how you may (but should not) do it:

__global_map = map #keep reference to the original map
lmap = lambda func, *iterable: list(__global_map(func, *iterable)) # using "map" here will cause infinite recursion
map = lmap
x = [1, 2, 3]
map(str, x) #test
map = __global_map #restore the original map and don't do that again
map(str, x) #iterator

Hullo

Answer 5

Converting my old comment for better visibility: For a "better way to do this" without map entirely, if your inputs are known to be ASCII ordinals, it's generally much faster to convert to bytes and decode, a la bytes(list_of_ordinals).decode('ascii'). That gets you a str of the values, but if you need a list for mutability or the like, you can just convert it (and it's still faster). For example, in ipython microbenchmarks converting 45 inputs:

>>> %%timeit -r5 ordinals = list(range(45))
... list(map(chr, ordinals))
...
3.91 µs ± 60.2 ns per loop (mean ± std. dev. of 5 runs, 100000 loops each)

>>> %%timeit -r5 ordinals = list(range(45))
... [*map(chr, ordinals)]
...
3.84 µs ± 219 ns per loop (mean ± std. dev. of 5 runs, 100000 loops each)

>>> %%timeit -r5 ordinals = list(range(45))
... [*bytes(ordinals).decode('ascii')]
...
1.43 µs ± 49.7 ns per loop (mean ± std. dev. of 5 runs, 1000000 loops each)

>>> %%timeit -r5 ordinals = list(range(45))
... bytes(ordinals).decode('ascii')
...
781 ns ± 15.9 ns per loop (mean ± std. dev. of 5 runs, 1000000 loops each)

If you leave it as a str, it takes ~20% of the time of the fastest map solutions; even converting back to list it's still less than 40% of the fastest map solution. Bulk convert via bytes and bytes.decode then bulk converting back to list saves a lot of work, but as noted, only works if all your inputs are ASCII ordinals (or ordinals in some one byte per character locale specific encoding, e.g. latin-1).

Cowart

Unfortunately, your code doesn't work in IPython 8.0.1: "UsageError: Line magic function %%timeit not found." Anyway I would prefer simple Python without "magic" (will write that myself). – Filipe

@YaroslavNikitenko: The magic is solely for microbenchmarking to demonstrate relative speed easily; the actual code ([*bytes(ordinals).decode('ascii')] or bytes(ordinals).decode('ascii')) is plain Python. You could always use the timeit module's timeit.repeat function to achieve the same result (it just looks a little uglier than the IPython display). Or just use it via the command line directly, e.g. python3 -mtimeit -s "ordinals = list(range(45))" "[*bytes(ordinals).decode('ascii')]". – Cowart

@YaroslavNikitenko: Hmm... According to the docs, IPython made updates to %timeit/%%timeit in the 8.0 timeframe, and it's still a documented feature so it should still exist. Makes me think your installation is broken in some way. – Cowart

(seems I can't insert your name). Strangely I got notification on only your last commit (oops, comment!). Anyway you are right, this method works for me in the "line mode" (with one percent sign, %timeit). Don't use IPython often. Thanks for the suggestion. I know the timeit module, and already used that in a script (calling timeit.timeit directly with its setup argument). – Filipe

Answer 6

list(map(chr, [66, 53, 0, 94]))

map(func, *iterables) --> map object Make an iterator that computes the function using arguments from each of the iterables. Stops when the shortest iterable is exhausted.

"Make an iterator"

means it will return an iterator.

"that computes the function using arguments from each of the iterables"

means that the next() function of the iterator will take one value of each iterables and pass each of them to one positional parameter of the function.

So you get an iterator from the map() funtion and jsut pass it to the list() builtin function or use list comprehensions.

Embassy

Answer 7

Using list comprehension in python and basic map function utility, one can do this also:

chi = [x for x in map(chr,[66,53,0,94])]

Rhatany

chi list will be containing, the ASIC value of the given elements. – Rhatany

Why the redundant [x for x in map(chr, ...)] when you could just use [chr(x) for x in ...]? Anywhere you use [target for target in iterable], just use list(iterable), there is no point in using a list comprehension then. – Feigned

Answer 8

You can try getting a list from the map object by just iterating each item in the object and store it in a different variable.

a = map(chr, [66, 53, 0, 94])
b = [item for item in a]
print(b)
>>>['B', '5', '\x00', '^']

Lozengy

Answer 9

Another option is to create a shortcut, returning a list:

from functools import reduce
_compose = lambda f, g: lambda *args: f(g(*args))
lmap = reduce(_compose, (list, map))

>>> lmap(chr, [66, 53, 0, 94])
['B', '5', '\x00', '^']

Parr

Answer 10

Best Way to do this in pyton3.X

Simply you can do this in single line

#Devil
input_list = [66, 53, 0, 94]
out = [chr(x) for x in input_list]
print(out)

# you will get the desire output in out list
# ['B', '5', '\x00', '^']

#------------------------------
#To retrieve your list use 'ord'

original_list = [ord(x) for x in out]
print(original_list )
#[66, 53, 0, 94]

Basel

Answer 11

In addition to above answers in Python 3, we may simply create a list of result values from a map as

li = []
for x in map(chr,[66,53,0,94]):
    li.append(x)

print (li)
>>>['B', '5', '\x00', '^']

We may generalize by another example where I was struck, operations on map can also be handled in similar fashion like in regex problem, we can write function to obtain list of items to map and get result set at the same time. Ex.

b = 'Strings: 1,072, Another String: 474 '
li = []
for x in map(int,map(int, re.findall('\d+', b))):
    li.append(x)

print (li)
>>>[1, 72, 474]

Sewing

@miradulo I supposed in Python 2, a list was returned, but in Python 3, only type is returned and I just tried to give in same format. If you think its needless, there maybe people like me who can find it useful and thats why I added. – Sewing

When there is already a list comprehension, a list function, and an unpacking answer, an explicit for loop doesn’t add much IMHO. – Rori

@Rori this answer should be just downvoted. Another comment under a silly answer doesn't add much. – Parr