Why does JSON serialization of datetime objects in Python not work out of the box for datetime objects?

Question

Why does the JSON serialization not work for datetime objects? As I understand JSON serialization, the basic idea for any object can be called the __str__ built-in function and then URL encode the object that you get as a response. But in case of datetime, I get the following error

TypeError: datetime.datetime(2012, 5, 23, 18, 38, 23, 37566) is not JSON serializable

while there is a __str__, i.e., a way of stringifying the object already available , But it seems like a conscious decision to not do it , why would that be the case?

Antipathy

You're mis-understanding how json works. It has nothing to do with the str method. JSON doesn't have a date-time type, period, so it is impossible to losslessly encode a date-time in JSON, without some sort of special logic on the receiving end. Thus the library (logically) makes you do it yourself by converting to a Unix timestamp or ISO date string or something and making it explicit that conversion is necessary. – Ranson

@TylerEaves This is anything but logical. It is possible to losslessly encode a datetime to a string or int, and many use cases that call for converting from dict, to json, to dict again never leaving the python ecosystem, yet the json module can't handle this case without a custom datetime handler? Really?! Looking at the large number of stackoverflow questions on the topic, I'd say I'm not alone in my incredulity. – Salivation

Answer 1

No, it doesn't work that way in the json module. The module provides you with a default encoder: json.JSONEncoder. You need to extend this to provide your implementation of default method to serialize objects. Something like this:

import json
import datetime
from time import mktime

class MyEncoder(json.JSONEncoder):

    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return int(mktime(obj.timetuple()))

        return json.JSONEncoder.default(self, obj)

print json.dumps(obj, cls=MyEncoder)

As others correctly pointed out, the reason is that the standard for JSON does not specify how date time can be represented.

Gamages

how to decode this datetime at the front end? i am asking about the decoding of this decimal. – Artisan

@Clayton, what you'll get in JSON response is unix timestamp. This question answers how to convert unix timestamp to date. – Gamages

@Gamages Thanks. I followed your code but I slightly modified the isinstance check part to if isinstance (obj, datetime.datetime): return obj.strftime ('%Y/%m/%d/%H/%M/%S') elif isinstance (obj, datetime.date): return obj.strftime ('%Y/%m/%d'). – Shrine

Well using mktime does work I found what I feel is a better way. JSON uses ISO 8601 formatting for its time stamps. To get the same kind of time formatting out of python all you need to do is take your datetime object and add .isoformat() to the end. – Reportage

Answer 2

How would you like them to be serialized?

JSON doesn't specify how to handle dates, so the python json library cannot make the decision on how to then represent these for you. That completely depends on how the other side (browser, script, whatever) handles dates in JSON as well.

Again

With a default serialization one doesn't necessarily care. You just need to be able to get into the json format and back out without too much trouble. Certainly failing on datetime entirely is guaranteed to be the wrong answer. – Salivation

@JoshuaKolden: but in an interchange format you do need to care. How does the JSON library in Java or in a JavaScript app want the date? – Again

Then that is when one would need to customize. Out of the box if I'm just going from python to json to python, not working by default on common data types just adds unnecessary overhead. Plus those other languages have standard conversions to json as well, so one could simply start with that as a good default. – Salivation

No, there is no such default. The standard doesn't specify any. – Again

And that prevents Python from encoding datetimes why? Other languages have no such problem. Json's data abstraction makes it more type flexible, not less so. Most languages choose (for themselves) a default datetime string encoding, and then have at least the ability to read and write datetime values to json consistently within the language. Python forces incompatibility not only with external systems, but even two different Python apps may handle datetime/json differently. – Salivation

But there is no interoperability here. Python's philosophy include the phrases refuse the temptation to guess and explicit is better than implicit. Without a standard there is no sensible default, Python refuses to guess here and forces you to explicitly pick one instead. The majority of usecases for JSON is in communicating with other unknown clients that even if they had a default datetime serialisation all have a different idea about the format. – Again

It is not hard to pick a format, but that is not all that one must do. As an example it is non-trivial to get sqlalchemy to work with dictionaries (containing datetimes at various nesting levels) stuffed into postgresql JSON columns. A trivial task in go, ruby, and javascript. This is just dictionary/hash serialization after all, yet python can't do it. Python does make a choice in terms of what format is acceptable for creating datetime from strings, and there are also general standards, so the 'python philosophy' seems at best circular logic in this case. – Salivation

@JoshuaKolden: if json.dumps would encode datetimes automatically, then json.loads would need to decode them automatically. So JSON strings are always Python strings, unless they happen to be formatted in some way that json.loads decides is a datetime, then they become datetimes. Even if they were just strings in the first place. – Esta

@Esta Precisely so. This is how JSON works. Context defines the meaning of the 6 JSON types. Without context a string is just a string, with context you know what type you're expecting. In JSON having functional symmetry is unimportant, being functional is. – Salivation

Answer 3

A simple way to patch the json module such that serialization would support datetime.

import json
import datetime

json.JSONEncoder.default = lambda self,obj: (obj.isoformat() if isinstance(obj, datetime.datetime) else None)

Then use JSON serialization as you always do, this time with datetime being serialized as isoformat.

json.dumps({'created':datetime.datetime.now()})

Resulting in: '{"created": "2015-08-26T14:21:31.853855"}'

See more details and some words of caution at: Stack Overflow: JSON datetime between Python and JavaScript

Huh

Best solution I've seen. One line patch. – Midgard

Answer 4

If you want to get encoding and decoding of datetimes without having to implement it, you can use json_tricks, which is a wrapper that adds encoding and decoding for various popular types. Just install:

pip install json_tricks

and then import from json_tricks instead of json, e.g.:

from json_tricks import dumps, loads
json = dumps({'name': 'MyName', 'birthday': datetime.datetime(1992, 5, 23, 18, 38, 23, 37566)})
me = loads(json)

Disclaimer: it's made by me. Because I had the same problem.

---

If you want to automatically serialize anything that can be stringified, you can do that with just the standard implementation very easily:

dumps(obj, default=str)

But note that this has disadvantages, e.g., none of it will be deserialized without extra effort, and maybe sometimes you just don't want to serialize something (like a function of a big NumPy array), but get a warning instead, which this method will silence.

Monazite

Nice for mentioning default=str. Easiest fix for most common situations. – Demivolt