Pandas df.to_csv() saves dict values as string. How can I get the dicts back when calling pd.read_csv()?

Question

I load a DataFrame from a database and have a column that is a dict, like so:

id   some_value   ...  coordinates
15         34.7        {'type': 'Point', 'coordinates': [-3.2, 37.0]}

However, when I save my DataFrame to disk using pd.to_csv() and then re-read it, the column containing the coordinates is not a dict, but a string:

id   some_value   ...  coordinates
15         34.7        "{'type': 'Point', 'coordinates': [-3.2, 37.0]}"

How can I tell Pandas to read this column as a dict, or how can I convert this column back into a dict?

Frontispiece

df.coordinates or df['coordinates'] ? – Frontispiece

This worked as expected. – Frontispiece

Answer 1

Use df['coordinates'].map(ast.literal_eval)

In [2333]: import ast

In [2334]: type(df.coordinates[0])
Out[2334]: str

In [2335]: df['coordinates'] = df['coordinates'].map(ast.literal_eval)

In [2336]: type(df.coordinates[0])
Out[2336]: dict

In [2337]: df
Out[2337]:
   id  some_value                                        coordinates
0  15        34.7  {u'type': u'Point', u'coordinates': [-3.2, 37.0]}

Monocotyledon

Answer 2

Use pd_to_pickle if you can. so many change when use to_csv for saving dataframe. to_csv is just in case you want to opening with another tools

Blend