JSON datetime between Python and JavaScript

Question

I want to send a datetime.datetime object in serialized form from Python using JSON and de-serialize in JavaScript using JSON. What is the best way to do this?

Actiniform

Do you prefer to use a library or do you want to code this yourself? – Categorize

Answer 1

You can add the 'default' parameter to json.dumps to handle this:

date_handler = lambda obj: (
    obj.isoformat()
    if isinstance(obj, (datetime.datetime, datetime.date))
    else None
)
json.dumps(datetime.datetime.now(), default=date_handler)
'"2010-04-20T20:08:21.634121"'

Which is ISO 8601 format.

A more comprehensive default handler function:

def handler(obj):
    if hasattr(obj, 'isoformat'):
        return obj.isoformat()
    elif isinstance(obj, ...):
        return ...
    else:
        raise TypeError, 'Object of type %s with value of %s is not JSON serializable' % (type(obj), repr(obj))

Update: Added output of type as well as value.
Update: Also handle date

Galactometer

this solution is much simpler than mdorseif's. Working great in my django app. – Woof

The problem is that if you have some other objects in list/dict this code will convert them to None. – Treasatreason

json.dumps won't know how to convert those either, but the exception is being supressed. Sadly a one line lambda fix has it's shortcomings. If you would rather have an exception raised on the unknowns (which is a good idea) use the function I've added above. – Galactometer

the full output format should have timezone on it as well... and isoformat() does not provide this functionality... so you should make sure to append that info on the string before returning – Brawny

This is the best way to go. Why was this not selected as the answer? – Tether

The answer seems to have an error in it - in the last line where the TypeError is raised, the string substitution part (type(Obj), repr(Obj)) refers to something named Obj, whereas it should be obj (note capitalisation). – Kooky

great answer, you can also get timestamp you can use something like that: date2ts = lambda obj: time.mktime(obj.timetuple()) if isinstance(obj, datetime.datetime) else None – Venterea

Better raise TypeError when the type is not datetime - otherwise data may be lost without you noticing! – Cressler

The lambda can be adapted to call the base implementation on non-datetime types, so TypeError can be raised if needed: dthandler = lambda obj: obj.isoformat() if isinstance(obj, datetime) else json.JSONEncoder().default(obj) – Elissaelita

A minor simplification in your lambda one-liner: if isinstance(obj, (datetime.datetime, datetime.date)) will eliminate the need for the or – Doctor

@NickFranceschina I just tested isoformat(), and it's outputting the timezone offset. >>> datetime.datetime(2012, 10, 20, tzinfo=tzoffset(None, 3600*12)).isoformat() '2012-10-20T00:00:00+12:00' – Ventura

The JSON from needs to include a timezone, and preferably UTC with 'Z'. Without that, a time like 2010-04-20T20:08:21.634121 will be assumed to be in the web browser's local time. – Rissa

return _iterencode(o, 0) TypeError: date_handler() takes 1 positional argument but 2 were given I get this error. Any help regarding this? – Univalence

Answer 2

For cross-language projects, I found out that strings containing RFC 3339 dates are the best way to go. An RFC 3339 date looks like this:

  1985-04-12T23:20:50.52Z

I think most of the format is obvious. The only somewhat unusual thing may be the "Z" at the end. It stands for GMT/UTC. You could also add a timezone offset like +02:00 for CEST (Germany in summer). I personally prefer to keep everything in UTC until it is displayed.

For displaying, comparisons and storage you can leave it in string format across all languages. If you need the date for calculations easy to convert it back to a native date object in most language.

So generate the JSON like this:

  json.dump(datetime.now().strftime('%Y-%m-%dT%H:%M:%SZ'))

Unfortunately, JavaScript's Date constructor doesn't accept RFC 3339 strings, but there are many parsers available on the Internet.

huTools.hujson tries to handle the most common encoding issues you might come across in Python code including date/datetime objects while handling time zones correctly.

Gilgamesh

This date formatting mechanism is natively supported, both by datetime: datetime.isoformat() and by simplejson, which will dump datetime objects as isoformat strings by default. No need for manual strftime hacking. – Gaptoothed

@Gaptoothed - I'm not getting automatic conversion from datetime objects to the isoformat string. For me, simplejson.dumps(datetime.now()) yields TypeError: datetime.datetime(...) is not JSON serializable – Graphology

json.dumps(datetime.datetime.now().isoformat()) is where the magic happens. – Illdefined

The beauty of simplejson is that if I have a complex data structure, it will parse it and turn it into JSON. If I have to do json.dumps(datetime.datetime.now().isoformat()) for every datetime object, I lose that. Is there a way to fix this? – Woof

superjoe30: see #456080 on how to do that – Gilgamesh

yes, #456080 is the better solution – Brawny

Most recent Javascript Date constructors accept RfC3339 Strings nowadays – Gilgamesh

That Z for GMT/UTC is a military radio code, Zulu time: en.wikipedia.org/wiki/List_of_military_time_zones – Altazimuth

Answer 3

I've worked it out.

Let's say you have a Python datetime object, d, created with datetime.now(). Its value is:

datetime.datetime(2011, 5, 25, 13, 34, 5, 787000)

You can serialize it to JSON as an ISO 8601 datetime string:

import json
json.dumps(d.isoformat())

The example datetime object would be serialized as:

'"2011-05-25T13:34:05.787000"'

This value, once received in the JavaScript layer, can construct a Date object:

var d = new Date("2011-05-25T13:34:05.787000");

As of JavaScript 1.8.5, Date objects have a toJSON method, which returns a string in a standard format. To serialize the above JavaScript object back to JSON, therefore, the command would be:

d.toJSON()

Which would give you:

'2011-05-25T20:34:05.787Z'

This string, once received in Python, could be deserialized back to a datetime object:

datetime.strptime('2011-05-25T20:34:05.787Z', '%Y-%m-%dT%H:%M:%S.%fZ')

This results in the following datetime object, which is the same one you started with and therefore correct:

datetime.datetime(2011, 5, 25, 20, 34, 5, 787000)

Heptateuch

Timezones will muck this right up. Lets assume your working in UTC in python (only an insane person does otherwise) - JSON output from python has no timezone so JavaScript will interpret it as local timezone. JavaScript d.toJSON will convert to UTC, again. So for your example date (2011-04-25) on a browser in the UK (Summer time so UTC+1) python outputs 13:34 - JS interprets this as local time zone or UTC 12:34 - JS then outputs UTC so 12:34. Python would interpret this as 12:34. You've lost an hour. (or a whole day if you're working just with dates and not times). Except in winter. – Dragone

Answer 4

Using json, you can subclass JSONEncoder and override the default() method to provide your own custom serializers:

import json
import datetime

class DateTimeJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return obj.isoformat()
        else:
            return super(DateTimeJSONEncoder, self).default(obj)

Then, you can call it like this:

>>> DateTimeJSONEncoder().encode([datetime.datetime.now()])
'["2010-06-15T14:42:28"]'

Polyanthus

Minor enhancement - use obj.isoformat(). You can also use the more common dumps() call, which takes other useful args (like indent): simplejson.dumps(myobj, cls=JSONEncoder, ...) – Renaerenaissance

Because that would call JSONEncoder's parent's method, not DateTimeJSONEncoder's parent's method. IE, you'd be going up two levels. – Scamper

Answer 5

Here's a fairly complete solution for recursively encoding and decoding datetime.datetime and datetime.date objects using the standard library json module. This needs Python >= 2.6 since the %f format code in the datetime.datetime.strptime() format string is only supported in since then. For Python 2.5 support, drop the %f and strip the microseconds from the ISO 8601 date string before trying to convert it, but you'll lose microseconds precision, of course. For interoperability with ISO 8601 date strings from other sources, which may include a time zone name or UTC offset, you may also need to strip some parts of the date string before the conversion. For a complete parser for ISO 8601 date strings (and many other date formats) see the third-party dateutil module.

Decoding only works when the ISO 8601 date strings are values in a JavaScript literal object notation or in nested structures within an object. ISO 8601 date strings, which are items of a top-level array will not be decoded.

I.e. this works:

date = datetime.datetime.now()
>>> json = dumps(dict(foo='bar', innerdict=dict(date=date)))
>>> json
'{"innerdict": {"date": "2010-07-15T13:16:38.365579"}, "foo": "bar"}'
>>> loads(json)
{u'innerdict': {u'date': datetime.datetime(2010, 7, 15, 13, 16, 38, 365579)},
u'foo': u'bar'}

And this too:

>>> json = dumps(['foo', 'bar', dict(date=date)])
>>> json
'["foo", "bar", {"date": "2010-07-15T13:16:38.365579"}]'
>>> loads(json)
[u'foo', u'bar', {u'date': datetime.datetime(2010, 7, 15, 13, 16, 38, 365579)}]

But this doesn't work as expected:

>>> json = dumps(['foo', 'bar', date])
>>> json
'["foo", "bar", "2010-07-15T13:16:38.365579"]'
>>> loads(json)
[u'foo', u'bar', u'2010-07-15T13:16:38.365579']

Here's the code:

__all__ = ['dumps', 'loads']

import datetime

try:
    import json
except ImportError:
    import simplejson as json

class JSONDateTimeEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, (datetime.date, datetime.datetime)):
            return obj.isoformat()
        else:
            return json.JSONEncoder.default(self, obj)

def datetime_decoder(d):
    if isinstance(d, list):
        pairs = enumerate(d)
    elif isinstance(d, dict):
        pairs = d.items()
    result = []
    for k,v in pairs:
        if isinstance(v, basestring):
            try:
                # The %f format code is only supported in Python >= 2.6.
                # For Python <= 2.5 strip off microseconds
                # v = datetime.datetime.strptime(v.rsplit('.', 1)[0],
                #     '%Y-%m-%dT%H:%M:%S')
                v = datetime.datetime.strptime(v, '%Y-%m-%dT%H:%M:%S.%f')
            except ValueError:
                try:
                    v = datetime.datetime.strptime(v, '%Y-%m-%d').date()
                except ValueError:
                    pass
        elif isinstance(v, (dict, list)):
            v = datetime_decoder(v)
        result.append((k, v))
    if isinstance(d, list):
        return [x[1] for x in result]
    elif isinstance(d, dict):
        return dict(result)

def dumps(obj):
    return json.dumps(obj, cls=JSONDateTimeEncoder)

def loads(obj):
    return json.loads(obj, object_hook=datetime_decoder)

if __name__ == '__main__':
    mytimestamp = datetime.datetime.utcnow()
    mydate = datetime.date.today()
    data = dict(
        foo = 42,
        bar = [mytimestamp, mydate],
        date = mydate,
        timestamp = mytimestamp,
        struct = dict(
            date2 = mydate,
            timestamp2 = mytimestamp
        )
    )

    print repr(data)
    jsonstring = dumps(data)
    print jsonstring
    print repr(loads(jsonstring))

Riley

If you print the date like datetime.datetime.utcnow().isoformat()[:-3]+"Z" it will be exactly like what JSON.stringify() produces in javascript – Kristopherkristos

Answer 6

If you're certain that only JavaScript will be consuming the JSON, I prefer to pass JavaScript Date objects directly.

The ctime() method on datetime objects will return a string that the JavaScript Date object can understand.

import datetime
date = datetime.datetime.today()
json = '{"mydate":new Date("%s")}' % date.ctime()

JavaScript will happily use that as an object literal, and you've got your Date object built right in.

Empressement

Technically not valid JSON, but it is a valid JavaScript object literal. (For the sake of principle I would set the Content-Type to text/javascript instead of application/json.) If the consumer will always and forever be only a JavaScript implementation, then yeah, this is pretty elegant. I would use it. – Quisling

.ctime() is a VERY bad way to pass time information, .isoformat() is much better. What .ctime() does is throw away timezone and daylight saving like they don't exist. That function should be killed. – Optics

Years later: please don't ever consider doing this. This will only ever work if you eval() your json in Javascript which you really shouldn't... – Doubly

Answer 7

A very simple solution is to patch the json module default.

For example:

import json
import datetime

json.JSONEncoder.default = lambda self,obj: (obj.isoformat() if isinstance(obj, datetime.datetime) else None)

Now, you can use json.dumps() as if it had always supported datetime...

json.dumps({'created':datetime.datetime.now()})

This makes sense if you require this extension to the json module to always kick in and wish to not change the way you or others use JSON serialization (either in existing code or not).

Note that some may consider patching libraries in that way as bad practice.

Special care need to be taken in case you may wish to extend your application in more than one way. In such a case, I suggest to use the solution by ramen or JT and choose the proper JSON extension in each case.

Unconformable

This silently eats non-serializable objects and turns them into None. You may want to throw an exception instead. – Hassler

Answer 8

Not much to add to the community wiki answer, except for timestamp!

JavaScript uses the following format:

new Date().toJSON() // "2016-01-08T19:00:00.123Z"

Python side (for the json.dumps handler, see the other answers):

>>> from datetime import datetime
>>> d = datetime.strptime('2016-01-08T19:00:00.123Z', '%Y-%m-%dT%H:%M:%S.%fZ')
>>> d
datetime.datetime(2016, 1, 8, 19, 0, 0, 123000)
>>> d.isoformat() + 'Z'
'2016-01-08T19:00:00.123000Z'

If you leave that Z out, frontend frameworks, like Angular, can not display the date in browser-local timezone:

> $filter('date')('2016-01-08T19:00:00.123000Z', 'yyyy-MM-dd HH:mm:ss')
"2016-01-08 20:00:00"
> $filter('date')('2016-01-08T19:00:00.123000', 'yyyy-MM-dd HH:mm:ss')
"2016-01-08 19:00:00"

Octagonal

Answer 9

On the Python side:

import time, json
from datetime import datetime as dt
your_date = dt.now()
data = json.dumps(time.mktime(your_date.timetuple())*1000)
return data # Data send to JavaScript

On the JavaScript side:

var your_date = new Date(data)

where data is the result from Python.

Frizzell

Answer 10

My advice is to use a library. There are several available at pypi.org.

I use asjson, and it works well.

Categorize

Answer 11

For the Python to JavaScript date conversion, the date object needs to be in specific ISO format, i.e., ISO format or Unix number. If the ISO format lacks some information, then you can convert to the Unix number with Date.parse() first. Moreover, Date.parse works with React as well while new Date might trigger an exception.

In case you have a DateTime object without milliseconds, the following needs to be considered:

  var unixDate = Date.parse('2016-01-08T19:00:00')
  var desiredDate = new Date(unixDate).toLocaleDateString();

The example date could equally be a variable in the result.data object after an API call.

For options to display the date in the desired format (e.g., to display long weekdays), check out the MDN documentation.

Blayze

is there a way to do that like python for arrays? I have array of datetime objects and want to chop the hour – Hypothecate

What is a "Unix number"? Unix time? – Flyblown

Answer 12

Apparently The “right” JSON (well JavaScript) date format is 2012-04-23T18:25:43.511Z - UTC and "Z". Without this JavaScript will use the web browser's local timezone when creating a Date() object from the string.

For a "naive" time (what Python calls a time with no timezone and this assumes is local) the below will force local timezone so that it can then be correctly converted to UTC:

def default(obj):
    if hasattr(obj, "json") and callable(getattr(obj, "json")):
        return obj.json()
    if hasattr(obj, "isoformat") and callable(getattr(obj, "isoformat")):
        # date/time objects
        if not obj.utcoffset():
            # add local timezone to "naive" local time
            # https://mcmap.net/q/25359/-python-figure-out-local-timezone
            tzinfo = datetime.now(timezone.utc).astimezone().tzinfo
            obj = obj.replace(tzinfo=tzinfo)
        # convert to UTC
        obj = obj.astimezone(timezone.utc)
        # strip the UTC offset
        obj = obj.replace(tzinfo=None)
        return obj.isoformat() + "Z"
    elif hasattr(obj, "__str__") and callable(getattr(obj, "__str__")):
        return str(obj)
    else:
        print("obj:", obj)
        raise TypeError(obj)

def dump(j, io):
    json.dump(j, io, indent=2, default=default)

why is this so hard.

Rissa

Answer 13

Simply do this:

r = json.dumps(your_json_data, default=str)
your_json_data = json.loads(r)

Enunciation

Please review Why not upload images of code/errors when asking a question? (e.g., "Images should only be used to illustrate problems that can't be made clear in any other way, such as to provide screenshots of a user interface.") and do the right thing (it covers answers and text input/output as well). Thanks in advance. – Flyblown