Transposing (pivoting) a dict of lists in python [duplicate]

Question

I have a dictionary that looks like:

{'x': [1, 2, 3], 'y': [4, 5, 6]}

I want to transform it to the following format:

[{'x': 1, 'y': 4}, {'x': 2, 'y': 5}, {'x': 3, 'y': 6}] 

I can do it by explicit loops but is there a good pythonic way to do it?

Edit: Turns out there is a similar question here and one of the answers is same as the accepted answer here but the author of that answer writes "I do not condone the use of such code in any kind of real system". Can someone explain why such code is bad? It appears very elegant to me.

Linnea

Do you know the keys in the dict? Or do you want a solution that doesn't assume known keys? In this case, I think that a solution with known keys could end up being much easier to read (but obviously less general). – Koal

@mgilson: Yes, I know the keys – Linnea

@mgilson: I should have been more clear. I know the keys as in I have a list of them but they can grow as in there might be 'x', 'y', 'z' as keys. I have a list, say l = [ 'x', 'y', 'z' ] though – Linnea

@Linnea as you acknowledge in your edit, this is a duplicate. If you have a different question, then you should ask that as a separate question, not as a secondary question here. – Pellmell

Answer 1

Use zip() a few times, counting on the fact that both dict.items() and iteration directly over a dict return elements in the same order as long as the dictionary is not mutated in between:

[dict(zip(d, col)) for col in zip(*d.values())]

The zip(*d.values()) call transposes the list values, and the zip(d, col) call pairs up each column with the keys in the dictionary again.

The above is the equivalent of spelling out the keys manually:

[dict(zip(('x', 'y'), col)) for col in zip(d['x'], d['y'])]

without having to spell out the keys manually.

Demo:

>>> d = {'x': [1, 2, 3], 'y': [4, 5, 6]}
>>> [dict(zip(d, col)) for col in zip(*d.values())]
[{'x': 1, 'y': 4}, {'x': 2, 'y': 5}, {'x': 3, 'y': 6}]

Jotham

That's a totally awesome answer! Didn't know about the star operator – Linnea

I'm still trying to decide how I feel about this answer. I think that it would probably take me somewhere between 1 and 10 minutes to figure out what this statement is doing if I didn't know it already ... – Koal

I will say that this is nice and general ... The more that I've thought about it (given that OP doesn't just have 2 keys in the original dict), I'm not sure that there is a much better way to do it... Sometimes you have to put the general, somewhat hard to read thing in the code and just comment what it does so your future self can figure it out... – Koal

Answer 2

I don't think there is any easy to read, pithy 1-liner for this (though it's probably not hard to come up with a difficult to read, pithy 1-liner ... ;) -- at least not in the general case (arbitrary number of dict items where the keys are unknown).

If you know the keys of the dict, I think it's probably easiest to work with that (especially since there are only 2 of them in the example).

Build the new dicts in 2 passes. The first pass fills in x, the second pass fills in y.

new_dicts = [{'x': x} for x in d['x']]
for new_dict, y in zip(new_dicts, d['y']):
    new_dict['y'] = y

If you'd rather do it in 1 pass, I think this isn't too bad either:

new_dicts = [{'x': x, 'y': y} for x, y in zip(d['x'], d['y'])]

---

If you have a list of keys, I might do it slightly differently ...

import operator
value_getter = operator.itemgetter(*list_of_keys)
new_dicts_values = zip(*value_getter(d))
new_dicts = [
    dict(zip(list_of_keys, new_dict_values))
    for new_dict_values in new_dicts_values]

This is pretty much the same strategy taken in Martijn's answer ... However, I think that breaking it out a little and giving things names helps make it a little more clear what is going on. Also, this takes away the mental overhead of having to convince yourself that zipping an unordered dict with an unordered list of column values is OK since they're unordered in the same way...

And, of course, if you don't actually have a list of keys, you can always get one by

list_of_keys = list(d)

Koal

I should have been more clear. I know the keys as in I have a list of them but they can grow as in there might be 'x', 'y', 'z' as keys. I have a list, say l = [ 'x', 'y', 'z' ] though – Linnea

Answer 3

if

d = {'x': [1, 2, 3], 'y': [4, 5, 6]}

One could try:

keys = d.keys()

print map(lambda tupl: map(lambda k,v: {k:v}, keys, tupl), zip(*d.itervalues()))

Looks pythonic but for larger entries the overhead of lambda calls each time map calls lambda function, will increase.

Bleacher

FWIW, it doesn't look pythonic to me ... I think that some people have the idea that "pythonic" means that you managed to fit the entire thing on one line. In reality, "pythonic" means using common python idioms to accomplish a task as simply as possible. – Koal

if you want to use map/lambda, I think map(lambda *vals: dict(zip(d, vals)), *d.values()) is better, but still suffers from the same mental overhead that I find intractable in Martijn's answer (though I don't know that I like mine that much better...) – Koal