Is Python's logging module thread safe?

Question

If you call the same logging handler from two different python threads, is there a need for locking?

Indebted

Answer 1

The logging module is thread-safe; it handles the locking for you. See the docs.

Encase

Ohh so as for now still not fixed? I got the logging module go crazy ... behhh. Did not expect that. – Chromato

The bug is talking about "fork", which involves processes rather than threads. So I guess the thread-safe statement still stands? – Ricardo

Yes, the thread-safety stands @Iglym. Without forking the logging process, the bug mentioned by Ibolla will not happen. When thinking about this bug, it should be mentioned, that fixing the bug would not make logging process-safe. For logging from multiple processes take a look at the Python Logging Cookbook. – Interdict

Answer 2

If you call the same handler from different threads, it is thread safe. And I'll just extend a little bit for this question. In fact, it depends on how you use the logging module. There still are some not-thread-safe circumstances when logging in multiple threads.

If multiple threads use different logger instances (e.g., calling the logging.getLogger() with different name in each threads), and those logger instances have their own FileHandler that points to the same file, it will result in a race condition and thus not thread-safe anymore.

Also, if multiple logger instances in different threads hold their own RotatingFileHandler or TimedRotatingFileHandler, while they are pointing to the same file (i.e., giving the same file name when instantiating them), the logics they rotate the file are not thread safe, neither. When they need to rotate the file, something strange could happen. (You can refer to the question Python TimedRotatingFileHandler - logs are missing, which cause by a similar reason)

Under the hood, in the logging module, each handler instance holds an threading.RLock instance, hence different loggers that hold different handlers will hold different RLocks, so those locks cannot avoid the race condition when loggers try to write to the same file in different threads -- loggers are still able to write bytes at the same time to the same file even they have acquired their own RLock.

You can refer to the Logging Cookbook - Opening the same log file multiple times part for more information.

Muzzle