Python logging how to track hostname in logs?

Question

I am using Python logging in this manner:

logger = logging.getLogger("my_logger")
logger.info("Some text.")

I have a bunch of IoT devices all running (making logs). They stream their log data to a database. To differentiate the source, I need to include the source IP address.

Is there a way to get hostname using logging? Is IP Address or hostname tracked/trackable in LogRecords?

In general, what is the best way to add in hostname to a LogRecord?

Unionize

They're adding a custom filter. That's all you have to do. – Packsaddle

Answer 1

import logging
from logging import Formatter
from logging.handlers import SysLogHandler
from socket import gethostname

hostname = socket.gethostname()
syslog = SysLogHandler(('my.logging.domain', 601))
syslog.setFormatter(
    Formatter(
        '{asctime} %s {name}: {levelname:8} {message}' % hostname,
        datefmt='%b %d %H:%M:%S',
        style='{'
    )
)

logger = logging.getLogger('myident')
logger.addHandler(syslog)
logger.setLevel(logging.DEBUG)
logger.info('Hello, world!')

Based on 1 and 2 but skipping the filter/dynamic hostname. Tested with Python 3.9.15 and Papertrail.

Therapeutic

Answer 2

You can do that by adding a custom log filter and formatter that puts the host name in the log messages.

import logging, platform

class HostnameFilter(logging.Filter):
    hostname = platform.node()

    def filter(self, record):
        record.hostname = HostnameFilter.hostname
        return True

handler = logging.StreamHandler()
handler.addFilter(HostnameFilter())
handler.setFormatter(logging.Formatter('%(asctime)s %(hostname)s: %(message)s', datefmt='%b %d %H:%M:%S'))

logger = logging.getLogger()
logger.addHandler(handler)
logger.setLevel(logging.INFO)

logger.info('Hello, world!')

Packsaddle

Where do I get the hostname from to put into the formatter? – Unionize

You don't. The logging module will automatically inject it into your logs. Same with time and other fields. – Packsaddle

Check out the docs for the log record object: docs.python.org/3/library/logging.html#logging.LogRecord – Packsaddle

Check out this blog for an example help.papertrailapp.com/kb/configuration/… – Packsaddle

It seems in the blog they use the external module socket.gethostname(), and in the LogRecord docs they manually type out 192.168.0.1. So basically, I have to get hostname from an external source – Unionize

It looks like @gobernador had edited the question months later and updated it to include an actual hostname – Unionize

Answer 3

A simpler solution would be to pre-populate the format line with the hostname. For example:

logging.basicConfig(
   #... your logging configuration,
   format="%(asctime)s {} %(message)s".format(socket.gethostname())
)

would generate a format line like

"%(asctime)s some-hostname %(message)s"

The drawback is that if the host name changes you'd need to restart your service/application. This is usually not a problem for most applications, especially if running on docker/k8s

Muddlehead