Changing logging's 'basicConfig' which is already set

Question

I am using the logging module in python as:

import logging, sys
logger= logging.getLogger(__file__)
logging.basicConfig(stream = sys.stderr, level=logging.DEBUG, format='%(filename)s:%(lineno)s %(levelname)s:%(message)s')
logger.debug("Hello World")

Now, after I have set the basic configuration on line 3, I want to have a command line argument that can change the output stream from sys.stderr to a file.

I have read the doc and it says that if both filename and stream are present at the same time, the stream is ignored.

Now, I wanna know how do change the stream to a file after I have already done the basicConfig thing in line 3?

Subfusc

Answer 1

If you look in the Python sources for logging/__init__.py, you'll see that basicConfig() sets the handlers on the root logger object by calling addHandler(). If you want to start from scratch, you could remove all existing handlers and then call basicConfig() again.

# Example to remove all root logger handlers and reconfigure. (UNTESTED)
import logging

# Remove all handlers associated with the root logger object.
for handler in logging.root.handlers[:]:
    logging.root.removeHandler(handler)

# Reconfigure logging again, this time with a file.
logging.basicConfig(filename = 'myfile.log', level=logging.DEBUG, format='%(filename)s:%(lineno)s %(levelname)s:%(message)s')

Bureaucracy

Also, use list comprehension remove all handlers here like the following code would be more simple :) [logging.root.removeHandler(handler) for handler in logging.root.handlers[:]] – Enervated

@KevinGuan List comprehensions are for constructing lists. When you want to run a statement in a loop, use a loop. – Tangential

This was very helpful. I used this solution inside a function that set the logger with basicConfig. Now when I call this inside loops, I can log to different single logfiles in each iteration. – Grice

Since Python 3.8, basicConfig now accept a force argument to remove existing root handlers. – Waterfowl

@Spack, are you saying that we can remove lines 5+6 above and set line 9 to something like logging.basicConfig(filename='myfile.log', ..., force = True) ? – Signor

Answer 2

Attribution to be given in Spack, for his comment. I am just expanding the idea in a proper answer

In 2022 and using Python 3.8, we can use the force input of basicConfig method

As per Python documentation

force

If this keyword argument is specified as true, any existing handlers attached to the root logger are removed and closed, before carrying out the configuration as specified by the other arguments.

Based on OP code sample, can just add the following line

logging.basicConfig(filename = 'my_file.log', level = logging.DEBUG, format = '%(filename)s:%(lineno)s %(levelname)s:%(message)s', force = True)

Fezzan