Jupyter notebook. How to direct the output to a specific cell?

Question

Is there a way to specify the output cell where a function should print its output?

In my specific case, I have some threads running, each with a logger. The logger output is printed on any running cell, interfering with that cell's intended output. Is there a way I can force the logger to print only on cell #1, for example?

Simonasimonds

I'd say it's easier to use an ipywidget and push the logger changes there. See: Output Widget. – Glorianna

redirect the logs to a file or turn off the logger – Zelma

This is a general question; the details are not necessary. As specified, I have a few threads running; I would like to redirect their output to a specific cell if possible. I am not interested in using log files. I know I can dump everything in logfiles, but I want the output printed directly in a specific cell. – Simonasimonds

Answer 1

You could use the following approach:

• Redirect all log messages in the root logger (which you will get by calling getLogger()) to a QueueHandler to accumulate the log messages in a queue.Queue.
• In the intended output cell, start a QueueListener that wraps a StreamHandler. The QueueListener, as its name implies, will listen to new items on the logging queue. It will pass new items to the StreamHandler, which will actually print them.

Assuming we want to print below cell 1, this could look as follows:

# Cell 1
import logging, queue, threading, time
from logging.handlers import QueueHandler, QueueListener

log_queue = queue.Queue(-1)

logging.getLogger().addHandler(QueueHandler(log_queue))

listener = QueueListener(log_queue, logging.StreamHandler())
listener.start()

In cell 2, we will simulate some activity:

# Cell 2
def log_activity_1():
    while True:
        logging.getLogger().warning("Activity 1")
        time.sleep(1)

threading.Thread(target=log_activity_1, daemon=True).start()

And likewise, in cell 3:

# Cell 3
def log_activity_2():
    while True:
        logging.getLogger().warning("Activity 2")
        time.sleep(2)

threading.Thread(target=log_activity_2, daemon=True).start()

The output will happen, basically in real time, under the cell that contains the listener.start() call, thus under cell 1 (and only there) in our case. It will look as expected: For each logged "Activity 2", we will see "Activity 1" logged in alternation and approximately twice as often, as we sleep 2 seconds in the former, and 1 second in the latter:

Once processing has finished, we can stop the QueueListener (either programmatically or manually) via listener.stop() – or rather, we should stop the listener this way, following its documentation: if you don’t call [stop()] before your application exits, there may be some records still left on the queue, which won’t be processed.

Llano

It doesn't work as I expected. In my case, the output is still shown on the active cell, not the listener cell. – Simonasimonds

@Simonasimonds If you say, "it doesn't work", do you mean: (1) with my code, as posted above without changes or (2) with your own code, adapted as proposed above? Also: are you only talking about logging outputs (which the code above is supposed to handle) or also about regular print() outputs (which the code above doesn't handle)? – Llano

With your code. I get the output on the active cell, not on the listener's cell – Simonasimonds

Check this pasteboard.co/UbsIRLY7mklt.png – Simonasimonds

Thanks for sharing, let me try to reproduce this. – Llano

As you can see, as soon I print() in another cell, the logger's output goes to that cell. Instead, It should have gone to the listener's cell. – Simonasimonds

That is weird. Doesn't happen for me. Maybe it is system-dependent (which, I agree, would not be a good solution). I am on Linux here. – Llano

I am on Linux, too. But I am using jupyter notebook, not lab. That could perhaps be the culprit. – Simonasimonds

I just tried in both: lab and notebook. Makes no difference. – Llano

I don't know, then... – Simonasimonds

I don't know either, I am sorry. I also tried two browsers now (Chromium and Firefox) and it works in both for me. In that case, I guess you should keep the bounty :) – Llano

Last try: Maybe it interferes with your Jupyter theme? From your screenshot above, I saw you used a dark mode, but afaik Jupyter by default doesn't have a dark mode. Can you try with theming turned off / default theme? – Llano

Resetting the theme to the default one didn't change anything. I am running Jupyter Notebook 6.4.12. Which version are you on? – Simonasimonds

jupyter notebook 7.2.1 here (on Python 3.11.9) – Llano

I just tried with jupyter notebook 6.4.12 (Python 3.9.19, Ipython 7.33.0) on Firefox. Still works for me. – Llano

It works on a new installation. I have accepted the solution. Thank you! – Simonasimonds

So glad to hear that! Very weird phenomenon though. In any case: I am glad that it worked out in the end! – Llano

Answer 2

You can create a wrapper that will redirect anything printed to another cell.

# Cell 1

import logging
from io import StringIO
from IPython.display import display, HTML
import sys

# Set up a StringIO object to capture output
output_capture = StringIO()
ch = logging.StreamHandler(output_capture)
ch.setLevel(logging.INFO)

# Configure the logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.addHandler(ch)

# Create a display area
display_id = display(HTML(''), display_id=True)

# Function to update the display
def update_display():
    output_contents = output_capture.getvalue()
    display_id.update(HTML(f"<pre>{output_contents}</pre>"))

def capture_output(func):
    def wrapper(*args, **kwargs):
        # Redirect stdout to our StringIO object
        old_stdout = sys.stdout
        sys.stdout = output_capture
        
        try:
            # Call the original function
            result = func(*args, **kwargs)
        finally:
            # Restore stdout
            sys.stdout = old_stdout
            
        # Update the display
        update_display()
        
        return result
    return wrapper

In a subsequent cell:

# Cell 2
@capture_output
def my_function():
    print('Hello world!')

# Now when you call my_function(), anything printed within the function will be displayed in the first cell
my_function()

Orbit

This is a beautiful solution, but it doesn't work when the function runs into threads. That is a requirement. – Simonasimonds