Converting datetime.date to UTC timestamp in Python
Asked Answered
W

11

383

I am dealing with dates in Python and I need to convert them to UTC timestamps to be used inside Javascript. The following code does not work:

>>> d = datetime.date(2011,01,01)
>>> datetime.datetime.utcfromtimestamp(time.mktime(d.timetuple()))
datetime.datetime(2010, 12, 31, 23, 0)

Converting the date object first to datetime also does not help. I tried the example at this link from, but:

from pytz import utc, timezone
from datetime import datetime
from time import mktime
input_date = datetime(year=2011, month=1, day=15)

and now either:

mktime(utc.localize(input_date).utctimetuple())

or

mktime(timezone('US/Eastern').localize(input_date).utctimetuple())

does work.

So general question: how can I get a date converted to seconds since epoch according to UTC?

Woaded answered 8/1, 2012 at 13:3 Comment(5)
possible duplicate of How do I convert local time to UTC in Python?Bathurst
I'm not sure I would agree with marking it as a duplicate. While the solutions are similar the questions are not. One (this one) is attempting to create a timestamp from a datetime.date, the other is attempting to convert a string representation from one timezone to another. As someone looking for a solution to this problem, I may not conclude that the latter will provide the answer I'm looking for.Mazzard
datetime(date.year,date.month,date.day).timestamp()Anneal
Possible duplicate of In Python, how do you convert a `datetime` object to seconds?Constriction
Beware: mktime is up to 10x slower than other approaches. Since it's not unlikely you're doing this in a hot path, it matters.Alfano
B
547

If d = date(2011, 1, 1) is in UTC:

>>> from datetime import datetime, date
>>> import calendar
>>> timestamp1 = calendar.timegm(d.timetuple())
>>> datetime.utcfromtimestamp(timestamp1)
datetime.datetime(2011, 1, 1, 0, 0)

If d is in local timezone:

>>> import time
>>> timestamp2 = time.mktime(d.timetuple()) # DO NOT USE IT WITH UTC DATE
>>> datetime.fromtimestamp(timestamp2)
datetime.datetime(2011, 1, 1, 0, 0)

timestamp1 and timestamp2 may differ if midnight in the local timezone is not the same time instance as midnight in UTC.

mktime() may return a wrong result if d corresponds to an ambiguous local time (e.g., during DST transition) or if d is a past(future) date when the utc offset might have been different and the C mktime() has no access to the tz database on the given platform. You could use pytz module (e.g., via tzlocal.get_localzone()) to get access to the tz database on all platforms. Also, utcfromtimestamp() may fail and mktime() may return non-POSIX timestamp if "right" timezone is used.


To convert datetime.date object that represents date in UTC without calendar.timegm():

DAY = 24*60*60 # POSIX day in seconds (exact value)
timestamp = (utc_date.toordinal() - date(1970, 1, 1).toordinal()) * DAY
timestamp = (utc_date - date(1970, 1, 1)).days * DAY

How can I get a date converted to seconds since epoch according to UTC?

To convert datetime.datetime (not datetime.date) object that already represents time in UTC to the corresponding POSIX timestamp (a float).

Python 3.3+

datetime.timestamp():

from datetime import timezone

timestamp = dt.replace(tzinfo=timezone.utc).timestamp()

Note: It is necessary to supply timezone.utc explicitly otherwise .timestamp() assume that your naive datetime object is in local timezone.

Python 3 (< 3.3)

From the docs for datetime.utcfromtimestamp():

There is no method to obtain the timestamp from a datetime instance, but POSIX timestamp corresponding to a datetime instance dt can be easily calculated as follows. For a naive dt:

timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)

And for an aware dt:

timestamp = (dt - datetime(1970,1,1, tzinfo=timezone.utc)) / timedelta(seconds=1)

Interesting read: Epoch time vs. time of day on the difference between What time is it? and How many seconds have elapsed?

See also: datetime needs an "epoch" method

Python 2

To adapt the above code for Python 2:

timestamp = (dt - datetime(1970, 1, 1)).total_seconds()

where timedelta.total_seconds() is equivalent to (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10**6) / 10**6 computed with true division enabled.

Example

from __future__ import division
from datetime import datetime, timedelta

def totimestamp(dt, epoch=datetime(1970,1,1)):
    td = dt - epoch
    # return td.total_seconds()
    return (td.microseconds + (td.seconds + td.days * 86400) * 10**6) / 10**6 

now = datetime.utcnow()
print now
print totimestamp(now)

Beware of floating-point issues.

Output

2012-01-08 15:34:10.022403
1326036850.02

How to convert an aware datetime object to POSIX timestamp

assert dt.tzinfo is not None and dt.utcoffset() is not None
timestamp = dt.timestamp() # Python 3.3+

On Python 3:

from datetime import datetime, timedelta, timezone

epoch = datetime(1970, 1, 1, tzinfo=timezone.utc)
timestamp = (dt - epoch) / timedelta(seconds=1)
integer_timestamp = (dt - epoch) // timedelta(seconds=1)

On Python 2:

# utc time = local time              - utc offset
utc_naive  = dt.replace(tzinfo=None) - dt.utcoffset()
timestamp = (utc_naive - datetime(1970, 1, 1)).total_seconds()
Bountiful answered 8/1, 2012 at 15:2 Comment(36)
For the Python 2 case, why not: timestamp = (dt - datetime.fromtimestamp(0)).total_seconds() ?Achernar
@m01: datetime(1970, 1, 1) is more explicit. fromtimestamp() is incorrect here (dt is in UTC so utcfromtimestamp() should be used instead).Bountiful
why does it return a floating point number? what if the script only accept an integer?Dimenhydrinate
@ribot: to return the timestamp as a single number with precision that could be better than one second (notice microseconds above). Depending on your application you could return the timestamp measured in microsecond to get an integer or just round the float result.Bountiful
when using Python 2.7.2. it compains TypeError: unsupported operand type(s) for /: 'datetime.timedelta' and 'datetime.timedelta'Blasting
@LymanZerga: the code between Python < 3.3 and Python 2 headers is for Python 3. / is not defined for timedelta in Python 2.Bountiful
@dannysauer wrong. Read the corresponding Python issue that is linked in the answer.Bountiful
Note total_seconds() was introduced in Python 2.7. Is there any approach for <2.7?Mackintosh
@Mackintosh look at totimestamp() function in the answer.Bountiful
as much as I love python, its date-time lib is seriously broken.Benjamin
@Realfun: timezones, DST transitions, the tz database, leap seconds exist independent of Python.Bountiful
Sorry to bother but is there a reason why (dt - datetime(1970, 1, 1, tzinfo=dt.tzinfo)).total_seconds() would not work?Circumference
@Circumference yes. There are multiple reasons why it may fail for various dt.Bountiful
@Scott: timezone.utc exists. Don't make such changes without a comment.Bountiful
On 3.5.2 I get "AttributeError: 'function' object has no attribute 'utc' ", with regard to timezone.utc. I'll take your word that it works on 3.3. Roger regarding change w/o comment. My apologies.Pasia
Now that I look at your link it makes sense. I was using pytz.timezone(). It might help if you specify that you are using datetime.timezone.Pasia
Just use arrow , it is a good tip by Mathieu Longtin (an alternative answer)Mockery
@Mockery I don't see how datetime(2011, 1, 1, tzinfo=timezone.utc).timestamp() is worse than arrow.get(2011, 1, 1).timestamp answer (btw, you might have seen my comment from 2016 there). You don't need 3rd party modules to work with the utc timezone. In general, I prefer pendulum if I need simple consistent datetime API and predictable results. See Why not Arrow?Bountiful
@Bountiful , thanks for pendulum - it is a good shot, I will try to use. About datetime(2011, 1, 1, tzinfo=timezone.utc).timestamp() - python2 can't play with timestamp() - it is not implemented there... docs.python.org/2/library/datetime.html#datetime.datetime But your answer is very good, comprehensive and educational. I like such answers on SOMockery
@maxkoryukov: the answer provides solutions for legacy python versions too.Bountiful
@Bountiful , thanks again, it is the informative article. But I don't want to bother with all these rules, legacy stuff, Python's quirks. I just need a working solution. With arrow I need just install and write one line — and this line will work for all versions of Python (I required for me). So, between [read the article and write your own awesome solution without deps] and [use arrow] I prefer the last one;) Probably, it is very close to NodeJS/NPM approach.Mockery
@maxkoryukov: you don't need to write anything. Working solutions are already in the answer. If you don't want to read the full answer, choose the first solution you see: calendar.timegm(d.timetuple()) (it works for all versions of Python).Bountiful
Let us continue this discussion in chat.Mockery
I don't get why the time libs deal with timezones at all. Best to do everything in UTC and convert to/from a user's timezone only on the frontend!Minstrelsy
This article has a great explanation of why you should make your datetime instances offset aware: howchoo.com/g/ywi5m2vkodk/…Grammar
@Pasia ugh at 3rd party modulesTropical
Potential bug: For me (local timezone CET) the calculation in Python 2.7.15rc1 was off by one hour. Once I replaced datetime(1970, 1, 1) by datetime.fromtimestamp(0) it was working.Disjunct
@Disjunct please, understand that UTC != CET (dt must be in UTC in the formula) and therefore fromtimestamp(0) is wrong. It might work by accident (if you pass local time as dt (again, local time may differ from UTC) and if it happens that the utc offset for your local timezone is the same as it was on Jun 1 1970 (obviously, it may be different)). See earliest comments above.Bountiful
Perhaps I should've been more detailed here. The datetime object I'm comparing with has also been created using datetime.fromtimestamp(), and though this as well as datetime(1970, 1, 1) were reported as naive objects they showed that offset. I've got no idea why it is that way, but the workaround should be transparent regarding the local timezone and thus valid for any TZ setting.Disjunct
@Disjunct do not subtract naive datetime objects that represent time in different time zones. It is like subtracting miles from kilometers — the result is non-sense.Bountiful
Why do you need that play around time zones in Py3? datetime.datetime(2020, 9, 18, 4, 21, 34, 359000, tzinfo=tzutc()) and datetime.datetime(2020, 9, 18, 4, 21, 34, 359000, tzinfo=tzoffset(None, 7200)) correctly works with .timestamp(). Did I miss the point?Flowage
@0andriy: Do you mean dt.replace(tzinfo=timezone.utc).timestamp() code? the input in the answer (dt) is a naive datetime object that represents time in UTC (the following NOTE in the answer should have made it clear)Bountiful
where's dt defined?Languor
@baxx: d is datetime.date instance (the input from the question). dt is the corresponding datetime.datetime instance.Bountiful
@Bountiful Would dt.astimezone() work instead of utc_naive = dt.replace(tzinfo=None) - dt.utcoffset()? I think I am missing something important here. I get that with dt.replace above, the resultant datetime obj is not timezone aware, while it is in case of astimezone, but I don't understand why that is a problem. (This is for python 2.7 specifically, python3 is a lot cleaner now)Berte
@Berte dt.astimezone() is a type error on Python 2.Bountiful
C
115

For unix systems only:

>>> import datetime
>>> d = datetime.date(2011, 1, 1)
>>> d.strftime("%s")  # <-- THIS IS THE CODE YOU WANT
'1293832800'

Note 1: dizzyf observed that this applies localized timezones. Don't use in production.

Note 2: Jakub Narębski noted that this ignores timezone information even for offset-aware datetime (tested for Python 2.7).

Coady answered 30/1, 2016 at 19:27 Comment(9)
A good answer if you know what system your code will run under, but it's important to note that the '%s' format string does not exist on all OSes, so this code is not portable. If you want to check whether it's supported you can check man strftime on Unix-like systems.Jackjackadandy
Thanks for the comment! I was not aware of this. You are welcome to edit the answer accordingly.Coady
It should be mentioned that this drops the fractional seconds part (microseconds or whatever).Rathbone
WARNING: This applies localized timezones! You will get different strftime behavior for identical datetime objects depending on the machine timeVirescence
Moreover this ignores timezone information and assumes that datetime is in local timezone even for offset-aware datetime, with tzinfo set. Well, at least in Python 2.7.Ecclesiastic
javascript timestamp = python timestamp * 1000Goldshlag
Even though everyone else dislikes this answer, this was the only one I could get to properly set the timezone on my linux box. everything else seemed to be offset by some amount.Acinaciform
d = datetime.date(2011,1,1) to avoid "SyntaxError: invalid token"Aldana
@Aldana Nice catch! Must be something new in Python. Answer updated.Coady
M
44
  • Assumption 1: You're attempting to convert a date to a timestamp, however since a date covers a 24 hour period, there isn't a single timestamp that represents that date. I'll assume that you want to represent the timestamp of that date at midnight (00:00:00.000).

  • Assumption 2: The date you present is not associated with a particular time zone, however you want to determine the offset from a particular time zone (UTC). Without knowing the time zone the date is in, it isn't possible to calculate a timestamp for a specific time zone. I'll assume that you want to treat the date as if it is in the local system time zone.

First, you can convert the date instance into a tuple representing the various time components using the timetuple() member:

dtt = d.timetuple() # time.struct_time(tm_year=2011, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=5, tm_yday=1, tm_isdst=-1)

You can then convert that into a timestamp using time.mktime:

ts = time.mktime(dtt) # 1293868800.0

You can verify this method by testing it with the epoch time itself (1970-01-01), in which case the function should return the timezone offset for the local time zone on that date:

d = datetime.date(1970,1,1)
dtt = d.timetuple() # time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=-1)
ts = time.mktime(dtt) # 28800.0

28800.0 is 8 hours, which would be correct for the Pacific time zone (where I'm at).

Mazzard answered 8/1, 2012 at 16:53 Comment(2)
I'm not sure why this has been downvoted. It addresses the OP's issue (the working code the OP labelled as "not working"). It doesn't answer my question (convert UTC datetime.datetime to timestamp), but still… upvoting.Breann
Be careful. The assumption that "you want to treat the date as if it is in the local system timezone is the root of all issues with timezone in python. Unless you're 100% sure of the localization of the machine that will run your script and especially if serving clients that can come from anywhere in the world, ALWAYS assume dates are in UTC, and do the conversion as early as possible. It will keep hairs on your head, trust me.Rooftree
C
10

I defined my own two functions

  • utc_time2datetime(utc_time, tz=None)
  • datetime2utc_time(datetime)

here:

import time
import datetime
from pytz import timezone
import calendar
import pytz


def utc_time2datetime(utc_time, tz=None):
    # convert utc time to utc datetime
    utc_datetime = datetime.datetime.fromtimestamp(utc_time)

    # add time zone to utc datetime
    if tz is None:
        tz_datetime = utc_datetime.astimezone(timezone('utc'))
    else:
        tz_datetime = utc_datetime.astimezone(tz)

    return tz_datetime


def datetime2utc_time(datetime):
    # add utc time zone if no time zone is set
    if datetime.tzinfo is None:
        datetime = datetime.replace(tzinfo=timezone('utc'))

    # convert to utc time zone from whatever time zone the datetime is set to
    utc_datetime = datetime.astimezone(timezone('utc')).replace(tzinfo=None)

    # create a time tuple from datetime
    utc_timetuple = utc_datetime.timetuple()

    # create a time element from the tuple an add microseconds
    utc_time = calendar.timegm(utc_timetuple) + datetime.microsecond / 1E6

    return utc_time
Cumin answered 22/6, 2018 at 11:12 Comment(3)
I browsed straight past your example.. Way to messy and complicated for such a simple thing.. Not pythonic at all.Topographer
@Mayhem: I cleaned it up a little and added comments. If you have a better and more generic solution to convert from time to datetime and back beeing able to set time zone - please let us know. Also let me know what a more pythonic example of my code should look like.Cumin
datetime is a poor choice of name for a function parameter in datetime2utc_time(datetime). It shadows the datetime import.Odiliaodille
W
8

follow the python2.7 document, you have to use calendar.timegm() instead of time.mktime()

>>> d = datetime.date(2011,01,01)
>>> datetime.datetime.utcfromtimestamp(calendar.timegm(d.timetuple()))
datetime.datetime(2011, 1, 1, 0, 0)
Whitesmith answered 30/6, 2014 at 6:51 Comment(1)
how is it different from my answer if d is in UTC?Bountiful
C
6

A complete time-string contains:

  • date
  • time
  • utcoffset [+HHMM or -HHMM]

For example:

1970-01-01 06:00:00 +0500 == 1970-01-01 01:00:00 +0000 == UNIX timestamp:3600

$ python3
>>> from datetime import datetime
>>> from calendar import timegm
>>> tm = '1970-01-01 06:00:00 +0500'
>>> fmt = '%Y-%m-%d %H:%M:%S %z'
>>> timegm(datetime.strptime(tm, fmt).utctimetuple())
3600

Note:

UNIX timestamp is a floating point number expressed in seconds since the epoch, in UTC.


Edit:

$ python3
>>> from datetime import datetime, timezone, timedelta
>>> from calendar import timegm
>>> dt = datetime(1970, 1, 1, 6, 0)
>>> tz = timezone(timedelta(hours=5))
>>> timegm(dt.replace(tzinfo=tz).utctimetuple())
3600
Chrissychrist answered 8/1, 2012 at 14:6 Comment(2)
@user908088 Convert 1970-01-01 06:00:00 +0500 to 3600 as you asked.Chrissychrist
@user908088 There's no timezone in python2, you can do the calculation(06:00:00 - +0500 = 01:00:00) manually.Chrissychrist
K
3

Using the arrow package:

>>> import arrow
>>> arrow.get(2010, 12, 31).timestamp
1293753600
>>> time.gmtime(1293753600)
time.struct_time(tm_year=2010, tm_mon=12, tm_mday=31, 
    tm_hour=0, tm_min=0, tm_sec=0, 
    tm_wday=4, tm_yday=365, tm_isdst=0)
Kerstin answered 9/3, 2016 at 16:17 Comment(6)
note: arrow uses dateutil and dateutil has known for many years bugs related to timezone handling. Don't use it if you need correct results for timezones with a non-fixed UTC offset (it is ok to use it for UTC timezone but the stdlib handles UTC case ok too).Bountiful
@J.F.Sebastian That bug only occurs during a transition, in an ambiguous time.Clubfoot
Looks like the bug was fixed on March 28thKerstin
Looks, like it is the best and cross-python solution. Just use arrow. Thanks;)Mockery
@MathieuLongtin dateutil may have other issuesBountiful
I mean at the end of the day, this is garbage in garbage out. The question leaves a lot of ambiguity. It totally depends on where it was recorded. So this is an answer that is as correct as the question that was served. A better answer might include the caveat that any conversion between timezones must come from a date that is aware of the timezone in which it was recorded. The above does not do that.Sidnee
W
2

the question is a little confused. timestamps are not UTC - they're a Unix thing. the date might be UTC? assuming it is, and if you're using Python 3.2+, simple-date makes this trivial:

>>> SimpleDate(date(2011,1,1), tz='utc').timestamp
1293840000.0

if you actually have the year, month and day you don't need to create the date:

>>> SimpleDate(2011,1,1, tz='utc').timestamp
1293840000.0

and if the date is in some other timezone (this matters because we're assuming midnight without an associated time):

>>> SimpleDate(date(2011,1,1), tz='America/New_York').timestamp
1293858000.0

[the idea behind simple-date is to collect all python's date and time stuff in one consistent class, so you can do any conversion. so, for example, it will also go the other way:

>>> SimpleDate(1293858000, tz='utc').date
datetime.date(2011, 1, 1)

]

Weisler answered 12/7, 2013 at 23:10 Comment(2)
SimpleDate(date(2011,1,1), tz='America/New_York') raises NoTimezone, while pytz.timezone('America/New_York') doesn't raise any exceptions (simple-date==0.4.8, pytz==2014.7).Bountiful
I get an error on pip install simple-date, looks like it is python3/pip3 only.Escapee
B
1

This works for me, pass through a function.

from datetime import timezone, datetime, timedelta 
import datetime


def utc_converter(dt):
    dt = datetime.datetime.now(timezone.utc)
    utc_time = dt.replace(tzinfo=timezone.utc)
    utc_timestamp = utc_time.timestamp()
    return utc_timestamp



# create start and end timestamps
_now = datetime.datetime.now()
str_start = str(utc_converter(_now))
_end = _now + timedelta(seconds=10)
str_end = str(utc_converter(_end))
Bernt answered 7/5, 2021 at 13:31 Comment(0)
M
0

i'm impressed of the deep discussion.

my 2 cents:

from datetime import datetime
import time

the timestamp in utc is:

timestamp = \
    (datetime.utcnow() - datetime(1970,1,1)).total_seconds()

or,

timestamp = time.time()

if now results from datetime.now(), in the same DST

utcoffset = (datetime.now() - datetime.utcnow()).total_seconds()
timestamp = \
    (now - datetime(1970,1,1)).total_seconds() - utcoffset
Mealy answered 6/10, 2019 at 13:23 Comment(0)
J
0

To convert a date into a UTC timestamp, it's best to specify explicitly the timezone before generating the timestamp:

from datetime import date, datetime, time, timezone


def utc_date_to_timestamp(utc_date: date):
    dt = datetime.combine(utc_date, time(tzinfo=timezone.utc))
    return dt.timestamp()

Note that in the above snippet, if we were to omit the timezone specification, we would have a potential problem if the local timezone is different from GMT (which is aligned with UTC). Consider this example:

my_date = date(2021, 1, 1)
wrong_timestamp = datetime.combine(my_date, time()).timestamp()
print(datetime.utcfromtimestamp(wrong_timestamp))  # 2020-12-31 18:00:00

Note that the specific value printed will differ depending on the local machine running the snippet.

Jettiejettison answered 29/5, 2023 at 11:59 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.