Replace default handler of Python logger

Question

I've got the following code running on each request of a wsgi (web2py) application:

import logging, logging.handlers
from logging import StreamHandler, Formatter

def get_configured_logger(name):

    logger = logging.getLogger(name)

    if (len(logger.handlers) == 0):
        # This logger has no handlers, so we can assume it hasn't yet been configured (Django uses similiar trick)

        # === Configure logger ===

        # Create Formatted StreamHandler:
        FORMAT = "%(process)s %(thread)s: %(message)s"
        formatter = logging.Formatter(fmt=FORMAT)
        handler = logging.StreamHandler()
        handler.setFormatter(formatter)
        logger.addHandler(handler)
        logger.setLevel(logging.DEBUG)
        logger.debug('CONFIGURING LOGGER')

    return logger

# Get app specific logger:
logger = get_configured_logger(request.application)
logger.debug("TEST")

It's meant to configure the logger once, with the formatted handler I want. It works, except that I'm getting double entries in my stdout:

81893 4329050112: CONFIGURING LOGGER
DEBUG:dummy:CONFIGURING LOGGER
81893 4329050112: TEST
DEBUG:dummy:TEST

How do I use my new formatted handler and get rid of/hide the default one?

Vixen

I have never used this class before, but don't you think logger.setLevel(logging.DEBUG) and logger.debug is sort of redundant? – Schulein

lkjoel- No, they're two different things. calling .debug(..) creates log entries at the debug level, while setting the logging level tells the handler to handle log entries at that level. – Vixen

Are you calling logging.basicConfig anywhere? If so, comment that out. – Misplace

unutbu- the web2py framework does call basicConfig in it's core, so can't comment it out. Can I overwrite it instead? – Vixen

Answer 1

Perhaps the following example will help. Basically you can either remove the handlers of the logger you'd like to disable, or don't propagate with the logger you are logging from.

$ cat testlog.py
import logging
logging.basicConfig(filename='foo', level=logging.DEBUG)
root_logger = logging.getLogger()
root_logger.debug('bar')

my_logger = logging.getLogger('my_logger')
FORMAT = "%(process)s %(thread)s: %(message)s"
formatter = logging.Formatter(fmt=FORMAT)
handler = logging.StreamHandler()
handler.setFormatter(formatter)

my_logger.addHandler(handler)
my_logger.setLevel(logging.DEBUG)
my_logger.info('baz')

my_logger.propagate = False
my_logger.info('foobar')

my_logger.propagate = True
my_logger.info('foobaz')
root_logger.handlers = []
my_logger.info('barbaz')

$ python testlog.py
5927 140735224465760: baz
5927 140735224465760: foobar
5927 140735224465760: foobaz
5927 140735224465760: barbaz

$ cat foo
DEBUG:root:bar
INFO:my_logger:baz
INFO:my_logger:foobaz

Sapajou

Derek- setting logger.propagate = False was all I needed- thanks! – Vixen

root_logger.handlers = [] is what I need – Laicize

I needed both :) – Deafmute

none is what I need :( I want to override the default handler because one goes to file and one to stdout but the format is different for each – Farceur

For Python 3.8.5, the only thing that worked for me was to set logger.propagate = False. Removing or replacing the handlers list had no effect on the number of emissions per log entry. – Race

Answer 2

You can remove the default handler from the getLogger() using this:

logging.getLogger().removeHandler(logging.getLogger().handlers[0])

Or clear the existing handlers first before adding handlers that you want:

logging.getLogger().handlers.clear()

After doing so, these logs will no longer display except the new handlers you have added:

DEBUG: Do stuff
WARNING: Do stuff

Archiplasm

it doesn't seem to be working... I did removeHandler and handlers=[] but they seem to be ignored somehow – Farceur

Neither of these changes seem to work with Python 3.8+ (and possibly earlier). Multiple log entries are emitted. I think it's because of the "propagation" logic that passes log entries up to the parent logger by default, regardless of what handlers you have on your own logger instance. If you set the property propagate to False, as per the other answer, this no longer seems to happen and you only get the entries emitted from your own logger. – Race