Python logger per function or per module

Question

I am trying to start using logging in python and have read several blogs. One issue that is causing confusion for me is whether to create the logger per function or per module. In this Blog: Good logging practice in Python it is recommended to get a logger per function. For example:

import logging

def foo():
    logger = logging.getLogger(__name__)
    logger.info('Hi, foo') 

class Bar(object):
    def __init__(self, logger=None):
        self.logger = logger or logging.getLogger(__name__)

    def bar(self):
        self.logger.info('Hi, bar')

The reasoning given is that

The logging.fileConfig and logging.dictConfig disables existing loggers by default. So, those setting in file will not be applied to your logger. It’s better to get the logger when you need it. It’s cheap to create or get a logger.

The recommended way I read everywhere else is like as shown below. The blog states that this approach "looks harmless, but actually, there is a pitfall".

import logging
logger = logging.getLogger(__name__)

def foo():
    logger.info('Hi, foo') 

class Bar(object):
    def bar(self):
        logger.info('Hi, bar')

I find the former approach to be tedious as I would have to remember to get the logger in each function. Additionally getting the logger in each function is surely more expensive than once per module. Is the author of the blog advocating a non-issue? Would following logging best practices avoid this issue?

Ries

The logging.fileConfig and logging.dictConfig disables existing loggers by default; does this mean that if you use these functions to initialize loggers, other loggers initializing other loggers that are not configure by the dict/file will not work? – Courtney

Answer 1

I would agree with you; getting logger in each and every function you use creates too much unnecessary cognitive overhead, to say the least.

The author of the blog is right about the fact that you should be careful to properly initialize (configure) your logger(s) before using them.

But the approach he suggests makes sense only in the case you have no control over your application loading and the application entry point (which usually you do).

To avoid premature (implicit) creation of loggers that happens with a first call to any of the message logging functions (like logging.info(), logging.error(), etc.) if a root logger hasn't been configured beforehand, simply make sure you configure your logger before logging.

Initializing the logger from the main thread before starting other threads is also recommended in Python docs.

Python's logging tutorial (basic and advanced) can serve you as a reference, but for a more concise overview, have a look at the logging section of The Hitchhiker's Guide to Python.

A simple blueprint for logging from multiple modules

Have a look at this modified example from Python's logging tutorial:

# myapp.py
import logging
import mylib

# get the fully-qualified logger (here: `root.__main__`)
logger = logging.getLogger(__name__)    

def main():
    logging.basicConfig(format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
                        level=logging.DEBUG)
    # note the `logger` from above is now properly configured
    logger.debug("started")
    mylib.something()

if __name__ == "__main__":
    main()

And

# mylib.py
import logging

# get the fully-qualified logger (here: `root.mylib`)
logger = logging.getLogger(__name__)

def something():
    logger.info("something")

Producing this on stdout (note the correct name):

$ python myapp.py
2017-07-12 21:15:53,334 __main__     DEBUG    started
2017-07-12 21:15:53,334 mylib        INFO     something

Kwh

Thank you for the great explanation. Your link to the basicConfig docs lead me to read The functions debug(), info(), warning(), error() and critical() will call basicConfig() automatically if no handlers are defined for the root logger. and This function does nothing if the root logger already has handlers configured for it. I also appreciate the emphasis on make sure you configure your logger before logging. If logging.info('hello') is added right before the basicConfig function the logging will print nothing. – Ries

This answer has led me to believe getting the logger object within each function demonstrates a misunderstanding of how the logging module works. If you follow the best practice of "make sure you configure your logger before logging" there should be no issue with per module loggers – Ries

You're welcome, I'm glad we got that cleared up. :-) – Kwh

Underappreciated answer. – Groan