Change level logged to IPython/Jupyter notebook

Question

I have a package that relies on several different modules, each of which sets up its own logger. That allows me to log where each log message originates from, which is useful.

However, when using this code in an IPython/Jupyter notebook, I was having trouble controlling what got printed to the screen. Specifically, I was getting a lot of DEBUG-level messages that I didn't want to see.

How do I change the level of logs that get printed to the notebook?

More info:

I've tried to set up a root logger in the notebook as follows:

# In notebook
import logging
logging.basicConfig()
logger = logging.getLogger()
logger.setLevel(logging.INFO)
# Import the module
import mymodule

And then at the top of my modules, I have

# In mymodule.py
import logging
logger = logging.getLogger('mypackage.' + __name__)
logger.setLevel(logging.DEBUG)
logger.propagate = True
# Log some messages
logger.debug('debug')
logger.info('info')

When the module code is called in the notebook, I would expect the logs to get propagated up, and then for the top logger to only print the info log statement. But both the debug and the info log statement get shown.

Related links:

• From this IPython issue, it seems that there are two different levels of logging that one needs to be aware of. One, which is set in the ipython_notebook_config file, only affects the internal IPython logging level. The other is the IPython logger, accessed with get_ipython().parent.log.
• https://github.com/ipython/ipython/issues/8282
• https://github.com/ipython/ipython/issues/6746

Ctenoid

Answer 1

With current ipython/Jupyter versions (e.g. 6.2.1), the logging.getLogger().handlers list is empty after startup and logging.getLogger().setLevel(logging.DEBUG) has no effect, i.e. no info/debug messages are printed.

Inside ipython, you have to change an ipython configuration setting (and possibly work around ipython bugs), as well. For example, to lower the logging threshold to debug messages:

# workaround via specifying an invalid value first
%config Application.log_level='WORKAROUND'
# => fails, necessary on Fedora 27, ipython3 6.2.1
%config Application.log_level='DEBUG'
import logging
logging.getLogger().setLevel(logging.DEBUG)
log = logging.getLogger()
log.debug('Test debug')

For just getting the debug messages of one module (cf. the __name__ value in that module) you can replace the above setLevel() call with a more specific one:

logging.getLogger('some.module').setLevel(logging.DEBUG)

Directly

Late to the party but this work as of today (oct 2022). Can't believe it! you have to force an error to be able to configure logs! – Sandwich

Answer 2

Adding another solution because the solution was easier for me. On startup of the Ipython kernel:

import logging
logging.basicConfig(level=20)

Then this works:

logging.getLogger().info("hello")
>> INFO:root:hello

logging.info("hello")
>> INFO:root:hello

And if I have similar logging code in a function that I import and run, the message will display as well.

Singleminded

Instead of hardcoding the number 20, you can use level=logging.INFO. – Scrotum

Answer 3

The root cause of this issue (from https://github.com/ipython/ipython/issues/8282) is that the Notebook creates a root logger by default (which is different from IPython default behavior!). The solution is to get at the handler of the notebook logger, and set its level:

# At the beginning of the notebook
import logging
logger = logging.getLogger()
assert len(logger.handlers) == 1
handler = logger.handlers[0]
handler.setLevel(logging.INFO)

With this, I don't need to set logger.propagate = True in the modules and it works.

Ctenoid

@Wesam, with newer ipython versions the referenced issue/the workaround doesn't apply anymore. With newer ipython versions the debug messages aren't printed, by default, for other reasons. See also my answer. – Directly