Dynamically changing log level without restarting the application

Question

Is it possible to change the log level using fileConfig without restarting the application? If it cannot be achieved through fileConfig, is there some other way to get the same result?

This is for an application running on a server, I want sys admins to be able to change a config file that is picked during run time by the application and change the log level dynamically.

Ghat

Everyone using threads and timers to reset the logging level. This is what signals were invented for. You can signal a process using "SIGHUP" (which is the standard on *nix) to reload configuration. In python you can install a signal handler that will catch this (event drive) and thus reload or reset a configuration. – Oyez

I agree about using signals. Suggest using SIGUSR1 or SIGUSR2 for this purpose instead of SIGHUP. The latter is sent when a process's controlling terminal disconnects. – Grouse

SIGUSR1, SIGUSR2 are only available for Unix-like system. You could use logging.config.listen() to listen for new configurations (docs.python.org/3/library/…) – Homegrown

Answer 1

fileConfig is a mechanism to configure the log level for you based on a file; you can dynamically change it at any time in your program.

Call .setLevel() on the logging object for which you want to change the log level. Usually you'd do that on the root:

logging.getLogger().setLevel(logging.DEBUG)

Lagomorph

Also see the answer by @sfinkens (and my comment there) in case you only want to change the setLevel for a particular handler. – Tennies

Does this mean, if I change loglevel in the file whilst the program is already running, the change will take effect? Does it use inotify to watch the conf file for changes? – Lowry

@0xc0de: no, the file is not being re-parsed. You can change the log level of the Logger objects that are currently used in a running Python program. – Lagomorph

Answer 2

In addition to the accepted answer: Depending on how you initialized the logger, you might also have to update the logger's handlers:

import logging

level = logging.DEBUG
logger = logging.getLogger()
logger.setLevel(level)
for handler in logger.handlers:
    handler.setLevel(level)

Azobenzene

Useful: If you used a dictConfig to specify your Loggers, Handlers, etc, and you only want to setLevel on one particular Handler, you can use the .get_name() function in such a loop as is shown by @Azobenzene here, and make sure you only change the level of the one you want. – Tennies

Why would handlers need to have a logLevel? Moreover, why can their level be set independently from the logger that owns them? This is terribly confusing, and bad design. – Flam

@CodeKid what if you want to log to file at a DEBUG level, but to the console at a different level? Would you not attach two handlers with different levels? – Rozek

Answer 3

Expanding on sfinken's answer, and Starman's subsequent comment, you can also check the type of the handler to target a specific outputter - for instance:

import logging
logger = logging.getLogger()
for handler in logger.handlers:
    if isinstance(handler, type(logging.StreamHandler())):
        handler.setLevel(logging.DEBUG)
        logger.debug('Debug logging enabled')

Sinatra

just what I needed to automatically enable debug logging if I'm running without the -o flag – Rota

Thanks for this! A small suggestion tho: isinstance(handler, logging.StreamHandler) – Detergent

Answer 4

This might be what you are looking for:

import logging
logging.getLogger().setLevel(logging.INFO)

Note that getLogger() called without any arguments returns the root logger.

Damle

This is the same as Martjin's answer, which doesn't seem to affect existing loggers (at least in Python2.7). sfinken's answer does affect existing loggers. – Hines

Answer 5

It is certainly possible to use fileConfig() to change logging configuration on the fly, though for simple changes a programmatic approach as suggested in Martijn Pieters' answer might be appropriate. Logging even provides a socket server to listen for config changes using the listen() / stopListening() APIs, as documented here. To get logging to listen on a particular port, you use

t = logging.config.listen(PORT_NUMBER)
t.start()

and to stop listening, call

logging.config.stopListening()

To send data to the server, you can use e.g.

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('localhost', PORT_NUMBER))
with open(CONFIG_FILE) as f:
    data_to_send = f.read()
s.send(struct.pack('>L', len(data_to_send)))
s.send(data_to_send)
s.close()

Update: Due to backwards-compatibility constraints, the internal implementation of the fileConfig() call means that you can't specify disable_existing_loggers=False in the call, which makes this feature less useful in certain scenarios. You can use the same API to send a JSON file using the dictConfig schema, which will allow better control over the reconfiguration. This requires Python 2.7/3.2 or above (where dictConfig() was added). Or, you can use the stdlib code to implement your own listener which works in the same way but which is tailored to your specific needs.

Ra

Wow I saw this answer earlier today, but didn't notice the update. I struggled with exactly that issue. There is a bug filed to add the option. bugs.python.org/issue26533 Now I got around the issue in an underhanded way: – Nix

python def my_fileConfig(fname, defaults=None, disable_existing_loggers=False): """ This does _NOTHING_ but call fileConfig with disable_existing_loggers set to False instead of true. It is intended to monkey patcn it out, so that the config listener can be used without disabling all non-root loggers. """ orig_fileConfig(fname, defaults, disable_existing_loggers) orig_fileConfig = logging.config.fileConfig # HACK: Patch out fileConfig for my version. logging.config.fileConfig = my_fileConfig logging.config.fileConfig(config_path) – Nix

Answer 6

I finally settled with using inotify and gevent to check for the file write operation, and once I know the file has been changed then I go and set the level for each logger I have based on the config.

import gevent
import gevent_inotifyx as inotify
from gevent.queue import Queue

class FileChangeEventProducer(gevent.Greenlet):
    def __init__(self, fd, queue):
        gevent.Greenlet.__init__(self)
        self.fd = fd
        self.queue = queue

    def _run(self):
        while True:
            events = inotify.get_events(self.fd)
            for event in events:
                self.queue.put(event)
                gevent.sleep(0)


class FileChangeEventConsumer(gevent.Greenlet):
    def __init__(self, queue, callBack):
        gevent.Greenlet.__init__(self)
        self.queue = queue
        self.callback = callBack

    def _run(self):
        while True:
            _ = self.queue.get()
            self.callback()
            gevent.sleep(0)


class GeventManagedFileChangeNotifier:
    def __init__(self, fileLocation, callBack):
        self.fileLocation = fileLocation
        self.callBack = callBack
        self.queue = Queue()
        self.fd = inotify.init()
        self.wd = inotify.add_watch(self.fd, self.fileLocation, inotify.IN_CLOSE_WRITE)


    def start(self):
        producer = FileChangeEventProducer(self.fd, self.queue)
        producer.start()
        consumer = FileChangeEventConsumer(self.queue, self.callBack)
        consumer.start()
        return (producer, consumer)

The above code gets used like below,

    def _setUpLoggingConfigFileChangeNotifier(self):
        loggingFileNameWithFullPath = self._getFullPathForLoggingConfig()
        self.gFsNotifier = GeventManagedFileChangeNotifier(loggingFileNameWithFullPath, self._onLogConfigChanged)
        self.fsEventProducer, self.fsEventConsumer = self.gFsNotifier.start()


    def _onLogConfigChanged(self):
        self.rootLogger.info('Log file config has changed - examining the changes')
        newLoggingConfig = Config(self.resourcesDirectory, [self.loggingConfigFileName]).config.get('LOG')
        self.logHandler.onLoggingConfigChanged(newLoggingConfig)

Once I have the new log file config I can wire in the right logging level for each logger from config. I just wanted to share the answer and it might help someone if they are trying to use it with gevent.

Ghat

Answer 7

Use logging.basicConfig with force=True.

Example:

logging.basicConfig(level=logging.DEBUG)

logger.info('debug')
logger.info('info')
logger.warning('warning')
print()
logging.basicConfig(level=logging.WARNING, force=True)

logger.info('debug')
logger.info('info')
logger.warning('warning')
exit()

And the output:

INFO:root:debug
INFO:root:info
WARNING:root:warning

WARNING:root:warning

Conjuration

Thank you for an answer for newer Python (>=3.8). For anyone else, the documentation for this is here. – Cartercarteret

Answer 8

Depending on your app, you first need to find a way for reloading that file or resetting the log level based on your own config file during execution.

Easiest way would be to use a timer. Either use threading to do that, or make your async framework to do that (if you use any; they usually implement it).

Using threading.Timer:

import threading
import time


def reset_level():
    # you can reload your own config file or use logging.config.fileConfig here
    print 'Something else'
    pass


t = threading.Timer(10, reset_level)
t.start()

while True:
    # your app code
    print 'Test'
    time.sleep(2)

Output:

Test
Test
Test
Test
Test
Something else
Test
Test

Update: Please check the solution proposed by Martijn Pieters.

Drama

I don't think this fully answers the question. – Viscous

I've used the above solution and the problem i'm facing (on rhel5 with python 2.4.3 and using ConcurrentLogHandler) is that for every reload of the config file, it add's a file handle and after running for a while it throws a "Too many open files" exception. So i agree with Travis Bear that the anwser from Martijn Pieters should be the accepted one. – Gregorygregrory

@Gregorygregrory did you close that file anyway? Use context manager to open a file rather than using the standalone open method. – Bothnia