JSON serialize a dictionary with tuples as key

Question

Is there a way in Python to serialize a dictionary that is using a tuple as key?

e.g.

a = {(1, 2): 'a'}

simply using json.dumps(a) raises this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/json/__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.6/json/encoder.py", line 367, in encode
    chunks = list(self.iterencode(o))
  File "/usr/lib/python2.6/json/encoder.py", line 309, in _iterencode
    for chunk in self._iterencode_dict(o, markers):
  File "/usr/lib/python2.6/json/encoder.py", line 268, in _iterencode_dict
    raise TypeError("key {0!r} is not a string".format(key))
TypeError: key (1, 2) is not a string

Windom

possible duplicate of Best way to encode tuples with json – Marco

Does this answer your question? Best way to encode tuples with json – Giguere

Answer 1

You can't serialize that as json, json has a much less flexible idea about what counts as a dict key than python.

You could transform the mapping into a sequence of key, value pairs, something like this:

import json
def remap_keys(mapping):
    return [{'key':k, 'value': v} for k, v in mapping.iteritems()]
... 
json.dumps(remap_keys({(1, 2): 'foo'}))
>>> '[{"value": "foo", "key": [1, 2]}]'

Discreet

Could you add the code to unremap the keys after loading the JSON? That would make for a much more complete answer. – Kreit

Please note that Python 3 renamed dict.iteritems -> dict.items – Kass

@kvothe I have added an answer that also addresses your request – Kass

Answer 2

from json import loads, dumps
from ast import literal_eval

x = {(0, 1): 'la-la la', (0, 2): 'extricate'}

# save: convert each tuple key to a string before saving as json object
s = dumps({str(k): v for k, v in x.items()})

# load in two stages:
# (i) load json object
obj = loads(s)

# (ii) convert loaded keys from string back to tuple
d = {literal_eval(k): v for k, v in obj.items()}

See https://mcmap.net/q/197388/-saving-dictionary-whose-keys-are-tuples-with-json-python.

Alphorn

This also works for dicts as keys! Or anything where literal_eval(str(x)) == x! As a note, literal_eval() is a security risk, it will execute arbitrary code, so only use this if you trust the JSON string you're loading. You could avoid this by replacing str(k) => json.dumps(k) when saving, and literal_eval(k) => tuple(json.loads(k)) when loading. – Quantize

This is insecure, if you cannot trust the JSON input – Robson

Answer 3

You could just use str((1,2)) as key because json only expects the keys as strings but if you use this you'll have to use a[str((1,2))] to get the value.

Duong

I think this is the best option if we want to preserve the way we key the items. – Misreport

Answer 4

JSON only supports strings as keys. You'll need to choose a way to represent those tuples as strings.

Cantonment

This is not strictly correct with respect to the question. Python types that map to JSON keys must be str, int, float, bool or None, so OP only needs to figure out how to map to one of those types – Albatross

Answer 5

This solution:

• Avoids the security risk of eval().
• Is short.
• Is copy-pastable as save and load functions.
• Keeps the structure of tuple as the key, in case you are editing the JSON by hand.
• Adds ugly \" to the tuple representation, which is worse than the other str()/eval() methods here.
• Can only handle tuples as keys at the first level for nested dicts (as of this writing no other solution here can do better)

def json_dumps_tuple_keys(mapping):
    string_keys = {json.dumps(k): v for k, v in mapping.items()}
    return json.dumps(string_keys)

def json_loads_tuple_keys(string):
    mapping = json.loads(string)
    return {tuple(json.loads(k)): v for k, v in mapping.items()}

m = {(0,"a"): "first", (1, "b"): [9, 8, 7]}
print(m)      # {(0, 'a'): 'first', (1, 'b'): [9, 8, 7]}
s = json_dumps_tuple_keys(m)
print(s)      # {"[0, \"a\"]": "first", "[1, \"b\"]": [9, 8, 7]}
m2 = json_loads_tuple_keys(s)
print(m2)     # {(0, 'a'): 'first', (1, 'b'): [9, 8, 7]}
print(m==m2)  # True

Quantize

Answer 6

json can only accept strings as keys for dict, what you can do, is to replace the tuple keys with string like so

with open("file", "w") as f:
    k = dic.keys() 
    v = dic.values() 
    k1 = [str(i) for i in k]
    json.dump(json.dumps(dict(zip(*[k1,v]))),f) 

And than when you want to read it, you can change the keys back to tuples using

with open("file", r) as f:
    data = json.load(f)
    dic = json.loads(data)
    k = dic.keys() 
    v = dic.values() 
    k1 = [eval(i) for i in k] 
    return dict(zip(*[k1,v])) 

Zink

Answer 7

Here is one way to do it. It will require the key to be json decoded after the main dictionary is decoded and the whole dictionary re-sequenced, but it is doable:

    import json

    def jsonEncodeTupleKeyDict(data):
        ndict = dict()
        # creates new dictionary with the original tuple converted to json string
        for key,value in data.iteritems():
            nkey = json.dumps(key)
            ndict[nkey] =  value

        # now encode the new dictionary and return that
        return json.dumps(ndict)

    def main():
        tdict = dict()
        for i in range(10):
            key = (i,"data",5*i)
            tdict[key] = i*i

        try:
            print json.dumps(tdict)
        except TypeError,e:
            print "JSON Encode Failed!",e

        print jsonEncodeTupleKeyDict(tdict)

    if __name__ == '__main__':
        main()

I make no claim to any efficiency of this method. I needed this for saving some joystick mapping data to a file. I wanted to use something that would create a semi-human readable format so it could be edited if needed.

Wilds

Answer 8

Here are two functions you could use to convert a dict_having_tuple_as_key into a json_array_having_key_and_value_as_keys and then de-convert it the way back

import json

def json_dumps_dict_having_tuple_as_key(dict_having_tuple_as_key):
    if not isinstance(dict_having_tuple_as_key, dict):
        raise Exception('Error using json_dumps_dict_having_tuple_as_key: The input variable is not a dictionary.')  
    list_of_dicts_having_key_and_value_as_keys = [{'key': k, 'value': v} for k, v in dict_having_tuple_as_key.items()]
    json_array_having_key_and_value_as_keys = json.dumps(list_of_dicts_having_key_and_value_as_keys)
    return json_array_having_key_and_value_as_keys

def json_loads_dictionary_split_into_key_and_value_as_keys_and_underwent_json_dumps(json_array_having_key_and_value_as_keys):
    list_of_dicts_having_key_and_value_as_keys = json.loads(json_array_having_key_and_value_as_keys)
    if not all(['key' in diz for diz in list_of_dicts_having_key_and_value_as_keys]) and all(['value' in diz for diz in list_of_dicts_having_key_and_value_as_keys]):
        raise Exception('Error using json_loads_dictionary_split_into_key_and_value_as_keys_and_underwent_json_dumps: at least one dictionary in list_of_dicts_having_key_and_value_as_keys ismissing key "key" or key "value".')
    dict_having_tuple_as_key = {}
    for dict_having_key_and_value_as_keys in list_of_dicts_having_key_and_value_as_keys:
        dict_having_tuple_as_key[ tuple(dict_having_key_and_value_as_keys['key']) ] = dict_having_key_and_value_as_keys['value']
    return dict_having_tuple_as_key

usage example:

my_dict = {
    ('1', '1001', '2021-12-21', '1', '484'): {"name": "Carl", "surname": "Black", "score": 0},
    ('1', '1001', '2021-12-22', '1', '485'): {"name": "Joe", "id_number": 134, "percentage": 11}
}

my_json = json_dumps_dict_having_tuple_as_key(my_dict)
print(my_json)

[{'key': ['1', '1001', '2021-12-21', '1', '484'], 'value': {'name': 'Carl', 'surname': 'Black', 'score': 0}}, 
 {'key': ['1', '1001', '2021-12-22', '1', '485'],  'value': {'name': 'Joe', 'id_number': 134, 'percentage': 11}}]

my_dict_reconverted = json_loads_dictionary_split_into_key_and_value_as_keys_and_underwent_json_dumps(my_json)
print(my_dict_reconverted)

{('1', '1001', '2021-12-21', '1', '484'): {'name': 'Carl', 'surname': 'Black', 'score': 0}, 
 ('1', '1001', '2021-12-22', '1', '485'): {'name': 'Joe', 'id_number': 134, 'percentage': 11}}

# proof of working 1

my_dict == my_dict_reconverted

True

# proof of working 2

my_dict == json_loads_dictionary_split_into_key_and_value_as_keys_and_underwent_json_dumps(
json_dumps_dict_having_tuple_as_key(my_dict)
)

True

(Using concepts expressed by @SingleNegationElimination to answer @Kvothe comment)

Kass

A beautiful example of how amazing functionality can become unusable through poor naming ;) the functions are really great.. I'm sure if you simplify the names, e.g. variable "remapped_dict" and the function "load_dict", and remove the assertions that were also not part of the question, it would have 50 updates already – Montparnasse

Answer 9

You can actually not serialize tuples as key to json, but you can convert the tuple to a string and recover it, after you have deserialized the file.

with_tuple = {(0.1, 0.1): 3.14} ## this will work in python but is not serializable in json
{(0.1, 0.1): 3.14}

But you cannot serialize it with json. However, you can use

with_string = {str((0.1, 0.1))[1:-1]: 3.14} ## the expression [1,-1] removes the parenthesis surrounding the tuples in python. 

{'0.1, 0.1': 3.14} # This is serializable

With a bit of cheating, you will recover the original tuple (after having deserialized the whole file) by treating each key (as str) separately

tuple(json.loads("["+'0.1, 0.1'+"]")) ## will recover the tuple from string
(0.1, 0.1)

It is a bit of overload to convert a string to a tuple using json.loads, but it will work. Encapsulate it and you are done.

Peace out and happy coding!

Nicolas

Medical

Answer 10

Here's a complete example to encode/decode nested dictionaries with tuple keys and values into/from json. tuple key will be a string in JSON.

values of types tuple or set will be converted to list

def JSdecoded(item:dict, dict_key=False):
    if isinstance(item, list):
        return [ JSdecoded(e) for e in item ]
    elif isinstance(item, dict):
        return { literal_eval(key) : value for key, value in item.items() }
    return item

def JSencoded(item, dict_key=False):
    if isinstance(item, tuple):
        if dict_key:
            return str(item)
        else:
            return list(item)
    elif isinstance(item, list):
        return [JSencoded(e) for e in item]
    elif isinstance(item, dict):
        return { JSencoded(key, True) : JSencoded(value) for key, value in item.items() }
    elif isinstance(item, set):
        return list(item)
    return item

usage

import json
pydata = [
    { ('Apple','Green') : "Tree",
      ('Orange','Yellow'):"Orchard",
      ('John Doe', 1945) : "New York" }
    ]
jsstr= json.dumps(JSencoded(pydata), indent='\t')
print(jsstr)
#[
#   {
#       "('Apple', 'Green')": "Tree",
#       "('Orange', 'Yellow')": "Orchard",
#       "('John Doe', 1945)": "New York"
#   }
#]
data = json.loads(jsstr) #string keys
newdata = JSdecoded(data) #tuple keys
print(newdata)
#[{('Apple', 'Green'): 'Tree', ('Orange', 'Yellow'): 'Orchard', ('John Doe', 1945): 'New York'}]

Bullpen

This is nice! If you need to practice recursion, try to figure this out! To be clear, this can handle nested dictionaries, but only the first level of keys can be tuples. If keys at lower levels are tuples, they won't get evaled back to tuples in JSdecoded. – Quantize

Answer 11

def stringify_keys(d):
    if isinstance(d, dict):
        return {str(k): stringify_keys(v) for k, v in d.items()}
    if isinstance(d, (list, tuple)):
        return type(d)(stringify_keys(v) for v in d)
    return d

json.dumps(stringify_keys(mydict))

Achilles