Python logger output dates in IS8601 format

Question

Is there a way to make the Python logger output dates in ISO8601 format?

My logger is set up like this...

logging.basicConfig(
    format="%(message)s - %(asctime)s)

From the Python docs (located here: https://docs.python.org/2/howto/logging.html) you can see the following:

The default format for date/time display (shown above) is ISO8601. If you need more control over the formatting of the date/time, provide a datefmt argument to basicConfig, as in this example:

The only problem is that the date output is not ISO8601 format. The date output by the above formatter is:

2018-06-15 11:07:41,454

This is not ISO8601 format as defined here: https://en.wikipedia.org/wiki/ISO_8601

What is the easiest way to get the date in the correct format? Can this be done out of the box or do I need to import a package to do it?

I've tried adding a date formatter e.g. datefmt="%Y-%m-%dT%H:%M:%S.%f %Z" but some of the formatting characters were not recognised - namely %f and %Z gave a textual description of the timezone and not a numeric offset.

Dick

I think you can build on this answer https://mcmap.net/q/504404/-python-logging-module-emits-wrong-timezone-information – Methodology

"some of the formatting characters were not recognised." - which? – Thundery

Example update. I had %f in there which was not recognised. and %Z gives a textual zone description not the numeric offset. – Dick

Answer 1

This works for me in most situations:

logging.basicConfig(
    format="%(asctime)s %(message)s",
    datefmt="%Y-%m-%dT%H:%M:%S%z"
)

The output looks like:

2019-11-09T01:18:13-0800 Something logged here

Update:

A one-liner I've been using to include miliseconds:

logging.Formatter.formatTime = (lambda self, record, datefmt=None: datetime.datetime.fromtimestamp(record.created, datetime.timezone.utc).astimezone().isoformat(sep="T",timespec="milliseconds"))

The output looks like

2021-08-05T22:43:02.985614+00:00 Something logged here

Elfredaelfrida

how can the milliseconds portion be added to the timestamp, i.e. "%f"? logging doesn't recognize the %f formatter when I try. – Sandrocottus

For the milliseconds, you can fiddle with datefmt and default_msec_format . But imho it is better to subclass loggin.Formater. – Tubman

self should not be in the lambda function. If it is, then when .formatTime is called the value intended for record will instead be in self. – Cordless

Answer 2

A simple solution if you don't want to subclass logging.Formatter is to overwrite/monkeypatch logging.Formatter.formatTime() directly like so:

import logging
from datetime import datetime

def formatTime(self, record, datefmt=None):
    return datetime.fromtimestamp(record.created).astimezone().isoformat(timespec='milliseconds')

logging.Formatter.formatTime = formatTime

logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(levelname)s %(name)s %(message)s')
    
logger = logging.getLogger(__name__) # __name__

myvar = 1
logger.info("test %s", myvar)

which will print the timestamp as a ISO8601 with milliseconds and UTC offset:

2024-01-15T19:31:29.303+01:00 INFO __main__ test 1

---

Or you can subclass logging.Formatter like his

import logging
import logging.config
from datetime import datetime

class MyCustomFormatter(logging.Formatter):
    def formatTime(self, record, datefmt=None):
        return datetime.fromtimestamp(record.created).astimezone().isoformat(timespec='milliseconds')


logging.config.dictConfig({
    'version': 1,
    'formatters': {
        'mycustomformatter': {
            '()': MyCustomFormatter,
            'format': '%(asctime)s %(levelname)s %(name)s %(message)s',
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'mycustomformatter',
        },
    },
    'loggers': {
        'root': {
            'level': 'DEBUG',
            'handlers': ['console'],
        },
    }

    })
    
logger = logging.getLogger(__name__)

myvar = 1
logger.info("test %s", myvar)

which produces the exact same output with ISO8601 timestamps:

2024-01-15T19:31:29.303+01:00 INFO __main__ test 1

---

In both cases the formatTime() code uses:

• record.created is the LogRecord attribute representing the number of seconds (float) since the epoch returned by time.time()
• datetime.fromtimestamp() is a class method that converts the epoch seconds to a datetime (naive)
• astimezone() converts the naive datetime to timezone-aware datetime in the local timezone
• isoformat(): converts the timezone aware datetime to a ISO8601 string.

---

Senskell