Weird timezone issue with pytz

Question

>>> import pytz
>>> pytz.timezone('Asia/Hong_Kong')
<DstTzInfo 'Asia/Hong_Kong' LMT+7:37:00 STD>

A seven hour and 37 minute offset? This is a little strange, does anyone experience the same issue?

In fact I'm getting different behavior between

import pytz
from datetime import datetime
hk = pytz.timezone('Asia/Hong_Kong')

dt1 = datetime(2012,1,1,tzinfo=hk)
dt2 = hk.localize(datetime(2012,1,1))
if dt1 > dt2:
   print "Why?"

Faria

what's the problem with being precise? The Hong Kong Time was first set to Local Mean Time (GMT+7:36:41) on 1 January 1885 at 13:00 by the then Royal Observatory Hong Kong – Westernism

See #11442683 – Lingua

Possible duplicate of Python datetime object show wrong timezone offset – Schizo

Answer 1

Time zones and offsets change over the years. The default zone name and offset delivered when pytz creates a timezone object are the earliest ones available for that zone, and sometimes they can seem kind of strange. When you use localize to attach the zone to a date, the proper zone name and offset are substituted. Simply using the datetime constructor to attach the zone to the date doesn't allow it to adjust properly.

Lingua

Answer 2

Coming here nearly 10 years later, I think it's worth a note that we can now exclusively utilize the Python 3.9+ standard library to handle time zones, without a "localize trap".

Use the zoneinfo module to set and replace the tzinfo however you like, ex:

from datetime import datetime
from zoneinfo import ZoneInfo

hk = ZoneInfo('Asia/Hong_Kong')
print(repr(hk))
# zoneinfo.ZoneInfo(key='Asia/Hong_Kong')

dt1 = datetime(2012,1,1,tzinfo=hk)
print(dt1)
# 2012-01-01 00:00:00+08:00

# set tz to a naive datetime object (pytz localize):
dt2 = datetime(2012,1,1).replace(tzinfo=hk)
print(dt2)
# 2012-01-01 00:00:00+08:00

• there is a deprecation shim for pytz

Alternatives, if you're not able to use zoneinfo:

• for Python < 3.9, there's backports.zoneinfo
• you could also use dateutil, which follows the same semantics as zoneinfo

Note for pandas users:

• pandas (v1.4.1) is still using pytz internally, and seems to have some trouble with ZoneInfo timezone objects

Marimaria

I think the interface between datetime and tzinfo was extended to allow this to work. When pytz was created it was stuck working the way it did. – Lingua

@MarkRansom right, I think I remember having looked into this in the context of another question. So it's not just an additional library and done, it's a bit of an evolution. – Marimaria

Answer 3

While I'm sure historic changes in timezones are a factor, passing pytz timezone object to the DateTime constructor results in odd behavior even for timezones that have experienced no changes since their inception.

import datetime
import pytz 

dt = datetime.datetime(2020, 7, 15, 0, 0, tzinfo= pytz.timezone('US/Eastern'))

produces

2020-07-15 00:00:00-04:56

Creating the datetime object then localizing it produced expected results

import datetime
import pytz 

dt = datetime.datetime(2020, 7, 15, 0, 0)
dt_local = timezone('US/Eastern').localize(dt)

produces

2020-07-15 00:00:00-04:00

Cousin

Prior to November 1883, there weren't any standard time zones in the US; time was local: en.wikipedia.org/wiki/Time_in_the_United_States#History. I expect that the timezone database is returning New York time. – Lingua

If you look at example data from the Olson Database you can see that New York had an offset of 4:56:02 prior to 1883 November 18, 12:03:58. So it's incorrect that the New York time zone never changed. – Lingua

I stand corrected. That would explain this behavior for datetimes in 1883, but not 2020 – Cousin

See my answer. The tzinfo object doesn't know what date you're going to use with it unless you use localize. For some reason pytz assumes 1883 is just as likely as 2020. – Lingua

True but it is still reasonable to expect datetime to use the passed timezone object appropriately with the data parameters passed to its constructor. – Cousin

No, because the tzinfo definition doesn't include a setup function that can be called from the constructor. It is forced to use it in its default state. – Lingua

Take utmost with the DateTime constructor. Check this: with t_now_secs = time.time() doing float(datetime.datetime.fromtimestamp(t_now_secs, pytz.UTC).strftime('%s')) - t_now_secs gives -3600. The local offset is dragged into this. – Legalism

@YtsendeBoer yes the local offset is applied in your example, but it doesn't come from a tzinfo instance. – Lingua

For us mere mortals this pytz behavior is not what one would expect. Wonder how many subtle timezone bugs exists out there due to this. – Yak