Python - Avoid passing logger reference between functions?

Question

I have a simple Python script that uses the in-built logging.

I'm configuring logging inside a function. Basic structure would be something like this:

#!/usr/bin/env python
import logging
import ...

def configure_logging():
    logger = logging.getLogger("my logger")
    logger.setLevel(logging.DEBUG)
    # Format for our loglines
    formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
    # Setup console logging
    ch = logging.StreamHandler()
    ch.setLevel(logging.DEBUG)
    ch.setFormatter(formatter)
    logger.addHandler(ch)
    # Setup file logging as well
    fh = logging.FileHandler(LOG_FILENAME)
    fh.setLevel(logging.DEBUG)
    fh.setFormatter(formatter)
    logger.addHandler(fh)
    return logger

def count_parrots():
    ...
    logger.debug??

if __name__ == '__main__':
    logger = configure_logging()
    logger.debug("I'm a log file")
    parrots = count_parrots()

I can call logger fine from inside __main__. However, how do I call logger from inside the count_parrots() function? What's the most pythonic way of handling configuring a logger like this?

Ajaajaccio

Answer 1

You can either use the root (default) logger, and thus the module level functions logging.debug, ... or get your logger in the function using it. Indeed, the getLogger function is a factory-like function with a registry (singleton like), i.e. it always returns the same instance for the given logger name. You can thus get your logger in count_parrots by simply using

logger = logging.getLogger("my logger") 

at the beginning. However, the convention is to use a dotted hierarchical name for your logger. See http://docs.python.org/library/logging.html#logging.getLogger

EDIT:

You can use a decorator to add the logging behaviour to your individual functions, for example:

def debug(loggername):
    logger = logging.getLogger(loggername) 
    def log_(enter_message, exit_message=None):
        def wrapper(f):
            def wrapped(*args, **kargs):
                logger.debug(enter_message)
                r = f(*args, **kargs)
                if exit_message:
                    logger.debug(exit_message)
                return r
            return wrapped
        return wrapper
    return log_

my_debug = debug('my.logger')

@my_debug('enter foo', 'exit foo')
def foo(a, b):
    return a+b

you can "hardcode" the logger name and remove the top-level closure and my_debug.

Agone

Ok, so I can just call logging.getLogger at the beginning of every function that needs to log. Seems a bit wasteful and repetitive surely? Fair enough. Or would I be better going object-oriented, and trying to shoehorn the whole lot into a class? (It's a very general question I know, I'm just looking for what's the done thing in the Python world). – Ajaajaccio

You can put your function in a class with the logger as instance variable, or (wich I prefer) create a decorator to add the logging functionnality to your individual functions – Agone

This answer shows pretty much all that is wrong with the Python logging module... – Haft

I'm curious what alternative folks think is desirable for this sort of cross cutting module? I'm at a loss to see what's the problem with logging.getLogger from a generic design perspective. Seems pretty boilerplate stuff. – Belton

Answer 2

You can just do :

logger = logging.getLogger("my logger") 

in your count_parrots() method. When you pass the name that was used earlier (i.e. "my logger") the logging module would return the same instance that was created corresponding to that name.

Update: From the logging tutorial (emphais mine)

getLogger() returns a reference to a logger instance with the specified name if it is provided, or root if not. The names are period-separated hierarchical structures. Multiple calls to getLogger() with the same name will return a reference to the same logger object.

Stater

Answer 3

The typical way to handle logging is to have a per-module logger stored in a global variable. Any functions and methods within that module then just reference that same logger instance.

This is discussed briefly in the intro to the advance logging tutorial in the documentation: http://docs.python.org/howto/logging.html#advanced-logging-tutorial

You can pass logger instances around as parameters, but doing so is typically rare.

Isaac

I thought the standard practice was to use logger=logging.getLogger("logger.name") – Smelt

At the module level, sure. Using separate loggers for different functions and methods in the same module is typically overkill though. One exception is that using separate loggers can be a very easy way to record which threads are logging particular events. – Isaac

Ah. I thought you meant actually using the global keyword. – Smelt

@MatthewCornell You mean logger = logging.getLogger(__name__). – Stridor

Answer 4

I got confused by how global variables work in Python. Within a function you only need to declare global logger if you were doing something like logger = logging.getLogger("my logger") and hoping to modify the global logger.

So to modify your example, you can create a global logger object at the start of the file. If your module can be imported by another one, you should add the NullHandler so that if the importer of the library doesn't want logging enabled, they don't have any issues with your lib (ref).

#!/usr/bin/env python
import logging
import ...

logger = logging.getLogger("my logger").addHandler(logging.NullHandler())

def configure_logging():
    logger.setLevel(logging.DEBUG)
    # Format for our loglines
    formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
    # Setup console logging
    ch = logging.StreamHandler()
    ch.setLevel(logging.DEBUG)
    ch.setFormatter(formatter)
    logger.addHandler(ch)
    # Setup file logging as well
    fh = logging.FileHandler(LOG_FILENAME)
    fh.setLevel(logging.DEBUG)
    fh.setFormatter(formatter)
    logger.addHandler(fh)

def count_parrots():
    ...
    logger.debug('counting parrots')
    ...
    return parrots

if __name__ == '__main__':
    configure_logging()
    logger.debug("I'm a log file")
    parrots = count_parrots()

Calabar

Answer 5

If you don't need the log messages on your console, you can use in a minimalist way.

Alternatively you can use tail -f myapp.log to see the messages on the console.

import logging

logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', \
    filename='myapp.log', \
    level=logging.INFO)

def do_something():
    logging.info('Doing something')

def main():
    logging.info('Started')
    do_something()
    logging.info('Finished')

if __name__ == '__main__':
    main()

Socorrosocotra

Answer 6

You can give logger as argument to count_parrots() Or, what I would do, create class parrots and use logger as one of its method.

Uncircumcision