How to dynamically update a plot in a loop in IPython notebook (within one cell)

Question

Environment: Python 2.7, Matplotlib 1.3, IPython notebook 1.1, Linux, and Chrome. The code is in one single input cell, using --pylab=inline.

I want to use IPython notebook and Pandas to consume a stream and dynamically update a plot every five seconds.

When I just use a print statement to print the data in text format, it works perfectly fine: the output cell just keeps printing data and adding new rows. But when I try to plot the data (and then update it in a loop), the plot never shows up in the output cell. But if I remove the loop, and just plot it once, it works fine.

Then I did some simple test:

i = pd.date_range('2013-1-1',periods=100,freq='s')
while True:
    plot(pd.Series(data=np.random.randn(100), index=i))
    #pd.Series(data=np.random.randn(100), index=i).plot() also tried this one
    time.sleep(5)

The output will not show anything until I manually interrupt the process (Ctrl + M + I). And after I interrupt it, the plot shows correctly as multiple overlapped lines. But what I really want is a plot that shows up and gets updated every five seconds (or whenever the plot() function gets called, just like what print statement outputs I mentioned above, which works well). Only showing the final chart after the cell is completely done is not what I want.

I even tried to explicitly add the draw() function after each plot(), etc. None of them works. How can I dynamically update a plot by a for/while loop within one cell in IPython notebook?

Yare

Answer 1

Use the IPython.display module:

%matplotlib inline
import time
import pylab as pl
from IPython import display
for i in range(10):
    pl.plot(pl.randn(100))
    display.clear_output(wait=True)
    display.display(pl.gcf())
    time.sleep(1.0)

Dworman

this is not smooth option, the plot is recreated from scratch with cell going up and down in between – Electrothermal

Adding clear_output(wait=True) solves this problem. See wabu's answer below. – Trauma

You can do better these days with %matplotlib nbagg which gives you a live figure to play with. – Semolina

@tcaswell I've added a new question asking how one uses nbagg to achieve this. (Pinging you in case you're interested in answering it.) #34487142 – Decretive

this works but also destroys anything else in the cell like the printed measures. Is there a way really just updating the plot and keeping everything else in place? – Whipstock

Rather than sleeping for 1 second between graph updates, is there a way to make the figure only update after a button press? – Funny

Why is this not working correctly if I use subplots? – Ovolo

This almost works perfectly, but starts to slow down appreciably about 20 images are displayed. – Emrich

Answer 2

A couple of improvement's on HYRY's answer:

• call display before clear_output so that you end up with one plot, rather than two, when the cell is interrupted.
• catch the KeyboardInterrupt, so that the cell output isn't littered with the traceback.

import matplotlib.pylab as plt
import pandas as pd
import numpy as np
import time
from IPython import display
%matplotlib inline

i = pd.date_range('2013-1-1',periods=100,freq='s')

while True:
    try:
        plt.plot(pd.Series(data=np.random.randn(100), index=i))
        display.display(plt.gcf())
        display.clear_output(wait=True)
        time.sleep(1)
    except KeyboardInterrupt:
        break

Henden

Indeed, display.display(gcf()) should go BEFORE display.clear_output(wait=True) – Lyrate

Thanks, @csta. Added it. – Henden

@Lyrate Why should display be called before clear_output? Shouldn't you first clear the output and then display the new data, instead of doing it the other way around? – Phia

I am still getting a screen flicker with the graph updates, however it's not all the time. Is there a workaround to this? – Necktie

If you are also trying to print text at the beginning of the loop, I find that this causes the graph to disappear, so that it is only visible for a split second. I do not have this problem when the display() call is placed after clear_output(). – Naivete

Answer 3

You can further improve this by adding wait=True to clear_output:

display.clear_output(wait=True)
display.display(pl.gcf())

Elrod

+1. This is very important. I think HYRY's answer should be updated with this info. – Trauma

This is good, but has the annoying side effect of clearing the print output as well. – Chon

Answer 4

I tried many methods, but I found this as the simplest and the easiest way -> to add clear_output(wait=True), for example,

from IPython.display import clear_output

for i in range(n_iterations):
     clear_output(wait=True)
     x = some value
     y = some value
     plt.plot(x, y, '-r')
     plt.show()

This overwrites on the same plot, and gives an illusion of plot animation

Nigh

Answer 5

Adding a label to the other solutions posted here will keep adding new labels in every loop. To deal with that, clear the plot using clf.

For example:

for t in range(100):
   if t % refresh_rate == 0:

     plt.clf()
     plt.plot(history['val_loss'], 'r-', lw=2, label='val')
     plt.plot(history['training_loss'], 'b-', lw=1, label='training')
     plt.legend()
     display.clear_output(wait=True)
     display.display(plt.gcf())

Ursal

Thanks plt.clf() works. However is there anyway to get rid of the flicker from the updates? – Necktie

Answer 6

Try to add show() or gcf().show() after the plot() function. These will force the current figure to update (gcf() returns a reference for the current figure).

Aerodyne

thanks. gcf().show() also works. Need to add the clear_output() suggested by HYRY to show stuff on the same fig – Yare

Is this in addition to "display.display(pl.gcf())"? – Necktie

Answer 7

You can do it like this. It accepts x,y as list and output a scatter plot plus a linear trend on the same plot.

from IPython.display import clear_output
from matplotlib import pyplot as plt
%matplotlib inline
    
def live_plot(x, y, figsize=(7,5), title=''):
    clear_output(wait=True)
    plt.figure(figsize=figsize)
    plt.xlim(0, training_steps)
    plt.ylim(0, 100)
    x = [float(i) for i in x]
    y = [float(i) for i in y]
    
    if len(x) > 1:
        plt.scatter(x,y, label='axis y', color='k') 
        m, b = np.polyfit(x, y, 1)
        plt.plot(x, [x * m for x in x] + b)

    plt.title(title)
    plt.grid(True)
    plt.xlabel('axis x')
    plt.ylabel('axis y')
    plt.show();

You just need to call live_plot(x, y) inside a loop. Here's how it looks:

Leblanc

Answer 8

A nice solution has been proposed by @BlackHC in a related post. It consists in using IPython.display.display with display_id=True to obtain a handle and use the update() method on it.

For instance,

import time

from IPython.display import display
from matplotlib import pyplot as plt

import numpy as np


hdisplay_img = display(display_id=True)
hdisplay_txt = display(display_id=True)

fig = plt.figure()
ax = fig.add_subplot(111)
im = ax.imshow(np.random.random((10,10,3)))
plt.close()

def update(i):
    im.set_data(np.random.random((10,10,3)))
    ax.add_image(im)
    hdisplay_img.update(fig)
    hdisplay_txt.update(f"update {i}")

for f in range(10):
    update(f)
    time.sleep(1)

Landman

Answer 9

Minimal modern solution for notebooks in Jupyter Lab (version 3.6.1):

from matplotlib import pyplot as plt

fig, ax = plt.subplots(1)
ax.set(xlabel=f'Epochs', ylabel='Value',
       title='Dynamics')

for i in range(n_step):
   ...
   ax.plot(...)
   
   if i==0:
      ax.legend(labels, loc='upper right')
   else:
      display(fig, clear=True);

Notice! No need for neither %matplotlib inline nor from IPython.display import clear_output, display because by default a new notebook already has a function with signature:

display(
    *objs,
    include=None,
    exclude=None,
    metadata=None,
    transient=None,
    display_id=None,
    raw=False,
    clear=False,
    **kwargs,
)

Note the keyword argument clear! Setting it to True makes a deal.

Imaginable