Add information to every log message in Python logging

Question

I am using Python with logging module and would like to add the socket.hostname() to every log message, I have to run this query every message and can not use

name = socket.hostname() 

and then logging format with the name

I am looking into this example of using logging filter, But what I need here is not a filter, it is a simple manipulation of every log message.

How can I achieve the wanted outcome?

Messick

How about to decorate logging function? – Jewfish

Answer 1

This builds upon the answer by Philippe while using dictConfig. The contextual filter demonstrated in this answer uses psutil to log the current CPU and memory usage percentage in each log message.

Save this file in say mypackage/util/logging.py:

"""logging utiliies."""
import logging

from psutil import cpu_percent, virtual_memory


class PsutilFilter(logging.Filter):
    """psutil logging filter."""

    def filter(self, record: logging.LogRecord) -> bool:
        """Add contextual information about the currently used CPU and virtual memory percentages into the given log record."""
        record.psutil = f"c{cpu_percent():02.0f}m{virtual_memory().percent:02.0f}"  # type: ignore
        return True

Note that a filter function didn't work for me; only a filter class worked.

Next, update your logging config dict based on this answer as below:

LOGGING_CONFIG = {
    ...,
    "filters": {"psutil": {"()": "mypackage.util.logging.PsutilFilter"}},
    "handlers": {"console": {..., "filters": ["psutil"]}},
    "formatters": {
        "detailed": {
            "format": "%(asctime)s %(levelname)s %(psutil)s %(process)x:%(threadName)s:%(name)s:%(lineno)d:%(funcName)s: %(message)s"
        }
    },
}

Try logging something, and see sample output such as:

2020-05-16 01:06:08,973 INFO c68m51 3c:MainThread:mypackage.mymodule:27:myfunction: This is my log message.

In the above message, c68m51 means 68% CPU and 51% memory usage.

Rubbico

Answer 2

You can use filter to add information to every message :

import logging
import socket

class ContextFilter(logging.Filter):
    def filter(self, record):
        record.hostname = socket.gethostname() 
        return True

if __name__ == '__main__':
    levels = (logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, logging.CRITICAL)
    logging.basicConfig(level=logging.DEBUG,
                        format='%(asctime)-15s hostname: %(hostname)-15s : %(message)s')
    a1 = logging.getLogger('a.b.c')
    f = ContextFilter()
    a1.addFilter(f)
    a1.debug('A debug message')

Transience