Python 3.9: Construct DST valid timestamp using standard library

Question

I would like to contruct DST-valid timestamps only using the standard library in Python 3.9 and was hoping this was possible with this version.

In my timezone "Europe/Berlin", the DST crossings for 2020 are:
2020-03-29 at 02:00 the clock switches to 03:00 (there is no hour 2!)
2020-10-25 at 03:00 the clock switches back to 02:00 (the hour 2 exists two times!)

My script yields the following output:
MARCH
2020-03-29 01:59:00+01:00 CET plus 1 h: 2020-03-29 02:59:00+01:00 CET
(should be 03:59:00 CEST since there is no hour 2!)

OCTOBER
2020-10-25 02:00:00+02:00 CEST plus 1 h: 2020-10-25 03:00:00+01:00 CET
(seems OK- EDIT: should be 02:00 CET!!!)

Example code is provided below. Windows user may need to "pip install tzdata" to make it work.

Any advise would be greatly appreciated!

'''
Should work out of the box with Python 3.9

Got a fallback import statement.

BACKPORT (3.6+)
pip install backports.zoneinfo

WINDOWS (TM) needs:
pip install tzdata
'''

from datetime import datetime, timedelta
from time import tzname
try:
    from zoneinfo import ZoneInfo
except ImportError:
    from backports import zoneinfo
    ZoneInfo = zoneinfo.ZoneInfo


tz = ZoneInfo("Europe/Berlin")
hour = timedelta(hours=1)

print("MARCH")
dt_01 = datetime(2020, 3, 29, 1, 59, tzinfo=tz)

dt_02 = dt_01 + hour
print(f"{dt_01} {dt_01.tzname()} plus 1 h: {dt_02} {dt_02.tzname()}")


print("\nOCTOBER")
dt_01 = datetime(2020, 10, 25, 2, 0, tzinfo=tz)
dt_02 = dt_01 + hour
print(f"{dt_01} {dt_01.tzname()} plus 1 h: {dt_02} {dt_02.tzname()}")

Diameter

Answer 1

Although it is counter-intuitive, this is as expected. See this blog post for more details on how datetime arithmetic works. The reason for this is that adding a timedelta to a datetime should be thought of as "advance the calendar/clock by X amount" rather than "what will the calendar/clock say after this amount of time has elapsed". Note that the first question might result in a time that doesn't even occur in the local time zone!

If you want, "What datetime represents what time it will be after the amount of time represented by this timedelta has elapsed?" (which it seems you do), you should do something equivalent to converting to UTC and back, like so:

from datetime import datetime, timedelta, timezone

def absolute_add(dt: datetime, td: timedelta) -> datetime:
    utc_in = dt.astimezone(timezone.utc)  # Convert input to UTC
    utc_out = utc_in + td  # Do addition in UTC
    civil_out = utc_out.astimezone(dt.tzinfo)  # Back to original tzinfo
    return civil_out

I believe you can create a timedelta subclass that overrides __add__ to do this for you (I'd kind of like to introduce something like this to the standard library if I can).

Note that if dt.tzinfo is None, this will use your system local time zone to determine how to do absolute addition, and it will return an aware time zone. Running this in America/New_York:

>>> absolute_add(datetime(2020, 11, 1, 1), timedelta(hours=1))
datetime.datetime(2020, 11, 1, 1, 0, tzinfo=datetime.timezone(datetime.timedelta(days=-1, seconds=68400), 'EST'))

If you want this to do civil addition for naïve datetimes and absolute addition for aware datetimes, you can check whether or not it's naïve in the function:

def absolute_add_nolocal(dt: datetime, td: timedelta) -> datetime:
    if dt.tzinfo is None:
        return dt + td
    return absolute_add(dt, td)

Also, to be clear, this is not something to do with zoneinfo. This has always been the semantics of datetimes in Python, and it wasn't something we could change in a backwards-compatible way. pytz does work a little differently, because adding pytz-aware datetimes does the wrong thing, and requires a normalize step after the arithmetic has occurred, and the pytz author decided that normalize should use absolute-time semantics.

absolute_add also works with pytz and dateutil, since it uses operations that work well for all time zone libraries.

Dasha

I've encountered this discussion before, thanks for the refresher ^^ - So Python's timedelta tries to represent the overloaded concept of a duration, e.g. a DST transition day with 23/25 hours as well as a duration being an absolute quantity (1 day = 86400 s)? And if that results in datetimes that don't even exist on the "wall clock" (the one that is adjusted to DST), that's just confusing to me, not only counter-intuitive... – Lelialelith

@Paul: Very thorough and helpful! Thanks a lot for your effort! – Diameter

@MrFuppes timedelta always represents an offset on an ideal, linear timeline. The main problem is that it's divorced from the context that generated it, so different-zone subtraction results in "here's how much of an offset to apply in UTC", and same-zone subtraction results in "here's the difference in calendar time in your local time". – Dasha

"divorced from the context" - ok that's another way of putting it but I still fail to see the point of reaching local times that don't exist (or only in case you forgot to change the clock after DST transition...). Btw. I think pandas does the addition/subtraction in UTC by default. So - should timedelta not have married in the first place, I mean stayed in two separate types, "duration" (no overloading), and something like "timezonetimedelta"? – Lelialelith

I mean, the "point" is just that that's how arithmetic is defined? It's a valid definition, even if many people won't find it useful. For example, if you want wall-time addition but you prevent invalid local times from being reached dt + td + td might not be equivalent to dt + 2 * td if dt + td would give an imaginary local time. A lot of low level math to implement time zones and other datetime stuff also relies on the current mathematical properties. – Dasha