datetime timestamp using Python with microsecond level accuracy

Question

I am trying to get timestamps that are accurate down to the microsecond on Windows OS and macOS in Python 3.10+.

On Windows OS, I have noticed Python's built-in time.time() (paired with datetime.fromtimestamp()) and datetime.datetime.now() seem to have a slower clock. They don't have enough resolution to differentiate microsecond-level events. The good news is time functions like time.perf_counter() and time.time_ns() do seem to use a clock that is fast enough to measure microsecond-level events.

Sadly, I can't figure out how to get them into datetime objects. How can I get the output of time.perf_counter() or PEP 564's nanosecond resolution time functions into a datetime object?

Note: I don't need nanosecond-level stuff, so it's okay to throw away out precision below 1-μs).

---

Current Solution

This is my current (hacky) solution, which actually works fine, but I am wondering if there's a cleaner way:

import time
from datetime import datetime, timedelta
from typing import Final

IMPORT_TIMESTAMP: Final[datetime] = datetime.now()
INITIAL_PERF_COUNTER: Final[float] = time.perf_counter()


def get_timestamp() -> datetime:
    """Get a high resolution timestamp with μs-level precision."""
    dt_sec = time.perf_counter() - INITIAL_PERF_COUNTER
    return IMPORT_TIMESTAMP + timedelta(seconds=dt_sec)

Heckelphone

It's impossible to use the output of time.perf_counter to get an absolute time. From the docs: "The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid." – Chiropteran

@Chiropteran please see the updated question. It's possible to adapt to work – Heckelphone

Answer 1

That's almost as good as it gets, since the C module, if available, overrides all classes defined in the pure Python implementation of the datetime module with the fast C implementation, and there are no hooks.
Reference: python/cpython@cf86e36

Note that:

1. There's an intrinsic sub-microsecond error in the accuracy equal to the time it takes between obtaining the system time in datetime.now() and obtaining the performance counter time.
2. There's a sub-microsecond performance cost to add a datetime and a timedelta.

Depending on your specific use case if calling multiple times, that may or may not matter.

A slight improvement would be:

INITIAL_TIMESTAMP: Final[float] = time.time()
INITIAL_TIMESTAMP_PERF_COUNTER: Final[float] = time.perf_counter()

def get_timestamp_float() -> float:
    dt_sec = time.perf_counter() - INITIAL_TIMESTAMP_PERF_COUNTER
    return INITIAL_TIMESTAMP + dt_sec

def get_timestamp_now() -> datetime:
    dt_sec = time.perf_counter() - INITIAL_TIMESTAMP_PERF_COUNTER
    return datetime.fromtimestamp(INITIAL_TIMESTAMP + dt_sec)

---

Anecdotal numbers

Windows:

# Intrinsic error
timeit.timeit('datetime.now()', setup='from datetime import datetime')/1000000  # 0.31 μs
timeit.timeit('time.time()', setup='import time')/1000000                       # 0.07 μs

# Performance cost
setup = 'from datetime import datetime, timedelta; import time'
timeit.timeit('datetime.now() + timedelta(1.000001)', setup=setup)/1000000            # 0.79 μs
timeit.timeit('datetime.fromtimestamp(time.time() + 1.000001)', setup=setup)/1000000  # 0.44 μs

# Resolution
min get_timestamp_float() delta: 239 ns

Windows and macOS:

	Windows	macOS
# Intrinsic error
timeit.timeit('datetime.now()', setup='from datetime import datetime')/1000000	0.31 μs	0.61 μs
timeit.timeit('time.time()', setup='import time')/1000000	0.07 μs	0.08 μs
# Performance cost
setup = 'from datetime import datetime, timedelta; import time'	-	-
timeit.timeit('datetime.now() + timedelta(1.000001)', setup=setup)/1000000	0.79 μs	1.26 μs
timeit.timeit('datetime.fromtimestamp(time.time() + 1.000001)', setup=setup)/1000000	0.44 μs	0.69 μs
# Resolution
min time() delta (benchmark)	x ms	716 ns
min get_timestamp_float() delta	239 ns	239 ns

239 ns is the smallest difference that float allows at the magnitude of Unix time, as noted by Kelly Bundy in the comments.

x = time.time()
print((math.nextafter(x, 2*x) - x) * 1e9)  # 238.4185791015625

Script

Resolution script, based on https://www.python.org/dev/peps/pep-0564/#script:

import math
import time
from typing import Final

LOOPS = 10 ** 6

INITIAL_TIMESTAMP: Final[float] = time.time()
INITIAL_TIMESTAMP_PERF_COUNTER: Final[float] = time.perf_counter()

def get_timestamp_float() -> float:
    dt_sec = time.perf_counter() - INITIAL_TIMESTAMP_PERF_COUNTER
    return INITIAL_TIMESTAMP + dt_sec

min_dt = [abs(time.time() - time.time())
          for _ in range(LOOPS)]
min_dt = min(filter(bool, min_dt))
print("min time() delta: %s ns" % math.ceil(min_dt * 1e9))

min_dt = [abs(get_timestamp_float() - get_timestamp_float())
          for _ in range(LOOPS)]
min_dt = min(filter(bool, min_dt))
print("min get_timestamp_float() delta: %s ns" % math.ceil(min_dt * 1e9))

Obola

Your first sentence in the answer "That's almost as good as it gets" is all I needed. Saved me time. Thanks. – Photoneutron

Answer 2

All credit here to the prior posts! This is a hair splitting optimization. I only bother to post this nitpick regurgitation of the prior solutions since we're all seeking the most precise datetime value, with the least possible impact on the very thing we are trying to measure.

from datetime import datetime
from time import time, perf_counter
from typing import Final

_DT_NOW_ADDEND: Final[float] = time() - perf_counter()
def datetime_now() -> datetime: 
    return datetime.fromtimestamp( _DT_NOW_ADDEND + perf_counter() )

Arnaud