Rich logging output in Jupyter IPython Notebook

Question

I am running Python notebooks on Jupyter Notebook server. I am using Python logging module for logging, currently wired up to log to stdout like any console application, do display logging messages in Jupyter Notebook output.

But the default stdout based logging output feels limited. There is so much more you can do with HTML output over plain text/ANSI output.

Are there any advanced Jupyter Notebook logging handlers and formatters that would understand that the output is HTML and adjust accordingly? E.g. offer richer formatting options with colors and font sizes and interactively explore logging message context parameters like Sentry allows one to do?

Malloch

Answer 1

Never thought to try this before, but yes you can do this using IPython.display.display and a custom logging.Handler which calls it:

import logging
from IPython.display import display, HTML

class DisplayHandler(logging.Handler):
    def emit(self, record):
        message = self.format(record)
        display(message)

This could be used to display anything the notebook can display, including HTML, Markdown, images???, audio???? (if you want your notebook to read your logs to you).

I combined this with a custom logging.Formatter that outputs an HTML object for passing to display(). It's not pretty but you can take the basic concept and improve it, or combine both classes into a single NotebookHTMLHandler class or something like that:

class HTMLFormatter(logging.Formatter):
    level_colors = {
        logging.DEBUG: 'lightblue',
        logging.INFO: 'dodgerblue',
        logging.WARNING: 'goldenrod',
        logging.ERROR: 'crimson',
        logging.CRITICAL: 'firebrick'
    }
    
    def __init__(self):
        super().__init__(
            '<span style="font-weight: bold; color: green">{asctime}</span> '
            '[<span style="font-weight: bold; color: {levelcolor}">{levelname}</span>] '
            '{message}',
            style='{'
        )
    
    def format(self, record):
        record.levelcolor = self.level_colors.get(record.levelno, 'black')
        return HTML(super().format(record))

Put all together you can use it like this:

log = logging.getLogger()
handler = DisplayHandler()
handler.setFormatter(HTMLFormatter())
log.addHandler(handler)
log.setLevel(logging.DEBUG)

Here's an example of how it looks:

For the HTML version you could tidy this up further, if the HTML grows complicated enough, by combining with an HTML templating engine like Jinja, or add JavaScript/Widgets to make log messages expandable to show more of the log record context, per your idea. A full solution I think would be beyond the scope of this answer.

Enounce

Marvellous. Thank you so much @Iguananaut. I am surprised IPython does not offer this functionality out of the box. Might be a nice feature to add. – Malloch

@MikkoOhtamaa Thanks! Yeah, I really liked your idea of rich logging with interactive widgets. Might be a good idea to expand into a Jupyter extension, though I don't personally have a strong need at the moment to go any further with it... – Enounce

Answer 2

Assuming you know what type of formatted output you wish to print, you can use the IPython.core.display package.

For example, to print HTML-formatted output you could do something like this:

from IPython.core.display import HTML
HTML('<a href="http://example.com">link</a>')

To print Markdown-formatted output you could do:

from IPython.core.display import Markdown
Markdown('# This will be an H1 title')

I'm not sure what exactly you mean by "exploring context parameters", so maybe an example here will clear things.

Surveyor