What is the currently correct way to dynamically update plots in Jupyter/iPython?

Question

In the answers to how to dynamically update a plot in a loop in ipython notebook (within one cell), an example is given of how to dynamically update a plot inside a Jupyter notebook within a Python loop. However, this works by destroying and re-creating the plot on every iteration, and a comment in one of the threads notes that this situation can be improved by using the new-ish %matplotlib nbagg magic, which provides an interactive figure embedded in the notebook, rather than a static image.

However, this wonderful new nbagg feature seems to be completely undocumented as far as I can tell, and I'm unable to find an example of how to use it to dynamically update a plot. Thus my question is, how does one efficiently update an existing plot in a Jupyter/Python notebook, using the nbagg backend? Since dynamically updating plots in matplotlib is a tricky issue in general, a simple working example would be an enormous help. A pointer to any documentation on the topic would also be extremely helpful.

To be clear what I'm asking for: what I want to do is to run some simulation code for a few iterations, then draw a plot of its current state, then run it for a few more iterations, then update the plot to reflect the current state, and so on. So the idea is to draw a plot and then, without any interaction from the user, update the data in the plot without destroying and re-creating the whole thing.

Here is some slightly modified code from the answer to the linked question above, which achieves this by re-drawing the whole figure every time. I want to achieve the same result, but more efficiently using nbagg.

%matplotlib inline
import time
import pylab as pl
from IPython import display
for i in range(10):
    pl.clf()
    pl.plot(pl.randn(100))
    display.display(pl.gcf())
    display.clear_output(wait=True)
    time.sleep(1.0)

Flower

Answer 1

Here is an example that updates a plot in a loop. It updates the data in the figure and does not redraw the whole figure every time. It does block execution, though if you're interested in running a finite set of simulations and saving the results somewhere, it may not be a problem for you.

The %matplotlib widget magic requires the ipympl Matplotlib Jupyter Extension package. You can install a working environment with pip install jupyter ipympl

%matplotlib widget
import numpy as np
import matplotlib.pyplot as plt
import time

def pltsin(ax, colors=['b']):
    x = np.linspace(0,1,100)
    if ax.lines:
        for line in ax.lines:
            line.set_xdata(x)
            y = np.random.random(size=(100,1))
            line.set_ydata(y)
    else:
        for color in colors:
            y = np.random.random(size=(100,1))
            ax.plot(x, y, color)
    fig.canvas.draw()

fig,ax = plt.subplots(1,1)
ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_xlim(0,1)
ax.set_ylim(0,1)
plt.show()

# run this cell to dynamically update plot
for f in range(5):
    pltsin(ax, ['b', 'r'])
    time.sleep(1)

I put this up on nbviewer here, and here's a direct link to the gist

Disvalue

Great, that seems to work nicely. The lack of interactivity while it's running is not a big problem for me. One slightly odd thing: if I change the while True: to a for loop, when the loop ends I get two static images of the last plot, rather than an interactive nbagg one. Any idea why that is? – Flower

I changed the while to a for loop and tried it on tmpnb.org, and but I'm not seeing a second image or a loss of interactivity. Shot in the dark, but you could try moving the loop around the call to the function, rather than having the loop in the function. for f in range(10): pltsin(ax) time.sleep(1) – Disvalue

How does one plot two lines overlaid together on this plot? – Piles

@Piles you'd need to add as many ax.lines as you want plots, then use line.set_data, and call fig.canvas.draw afterwards to update the figure. I modified the example to show how to do that. – Disvalue

@Disvalue Unfortunately it has some problems with Matplotlib 2.0 on Retina display: in the loop plots are twice smaller that usually. – Dagon

this doesn't do anything on Windows 7. no plots shown at all – Innerdirected

@Aksakal can you plot other things? If I paste the code above at try.jupyter.org, it works okay – Disvalue

It seems the figure is not given the time to resize itself correctly. So I had a much better experience when putting a plt.show() and moving the for-loop to the next cell. – Cutter

The %matplotlib notebook backend is not supported in jupyter lab because they disabled appending raw javascript to the cells. It works fine in regular jupyter notebook, still. – Disvalue

Anyone know what lines is? – Ignitron

@Ignitron it's the list of lines contained by the Axes. lines is a private API, probably should call get_lines instead. – Disvalue

Be sure that you have the %matplotlib notebook in the same jupyter notebook cell as your plot - I spent over 2 hours today troubleshooting this because I had %matplotlib notebook in the first cell with the import statments – Model

Can someone confirm this does no work on Google Colab? – Drooff

@aguazul, I tried to put %matplotlib notebook in the first cell, and it still works on my jupyter notebook. My colleague also tried on Binder with %matplotlib notebook in the first cell, it still works on Binder. May I ask what is the version of your jupyter notebook? Mine is 5.2.2 – Flog

In jupyter lab I had to use %matplotlib widget instead of %matplotlib notebook. Also the plot changes dynamically only if I add plt.show() and move the for-loop to the next cell (under the plot) as @Cutter suggested. – Xanthic

@JulienM on Google Colab, %matplotlib notebook and %matplotlib widget show no output, even when you !pip install ipympl when using widget. I get output with %matplotlib inline, but that backend cannot dynamically update plots – Disvalue

Answer 2

I'm using jupyter-lab and this works for me (adapt it to your case):

from IPython.display import clear_output
from matplotlib import pyplot as plt
import numpy as np
import collections
%matplotlib inline

def live_plot(data_dict, figsize=(7,5), title=''):
    clear_output(wait=True)
    plt.figure(figsize=figsize)
    for label,data in data_dict.items():
        plt.plot(data, label=label)
    plt.title(title)
    plt.grid(True)
    plt.xlabel('epoch')
    plt.legend(loc='center left') # the plot evolves to the right
    plt.show();

Then in a loop you populate a dictionary and you pass it to live_plot():

data = collections.defaultdict(list)
for i in range(100):
    data['foo'].append(np.random.random())
    data['bar'].append(np.random.random())
    data['baz'].append(np.random.random())
    live_plot(data)

make sure you have a few cells below the plot, otherwise the view snaps in place each time the plot is redrawn.

Appendectomy

this creates a new plot each time rather than updating the existing plot – Disvalue

Correct. I haven't found a better way of having a dynamical plot in jupyter-lab. – Appendectomy

Is there a way to set how long it waits between iterations? rather than just having a 'wait = True' – Schellens

Every time the plot is redrawn, the graph flickers. Is there a way to fix this problem? I have a few empty cells under the plot, but that doesn't seem to help. – Ignitron

@Ignitron see "Flickering and jumping output" in buildmedia.readthedocs.org/media/pdf/ipywidgets/latest/… – Clementia

from IPython.display import clear_output is the secret that works well for me, thanks! – Aloud

This takes 0.087s per frame for me (11hz) fwiw – Viccora

Answer 3

If you don't want to clear all outputs, you can use display_id=True to obtain a handle and use .update() on it:

import numpy as np
import matplotlib.pyplot as plt
import time

from IPython import display

def pltsin(ax, *,hdisplay, colors=['b']):
    x = np.linspace(0,1,100)
    if ax.lines:
        for line in ax.lines:
            line.set_xdata(x)
            y = np.random.random(size=(100,1))
            line.set_ydata(y)
    else:
        for color in colors:
            y = np.random.random(size=(100,1))
            ax.plot(x, y, color)
    hdisplay.update(fig)


fig,ax = plt.subplots(1,1)
hdisplay = display.display("", display_id=True)

ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_xlim(0,1)
ax.set_ylim(0,1)
for f in range(5):
    pltsin(ax, colors=['b', 'r'], hdisplay=hdisplay)
    time.sleep(1)
    
plt.close(fig)

(adapted from @pneumatics)

Forworn

This one works for Jupyter notebook, nice and smooth. 🤝 – Rovit

Answer 4

I've adapted @Ziofil answer and modified it to accept x,y as list and output a scatter plot plus a linear trend on the same plot.

from IPython.display import clear_output
from matplotlib import pyplot as plt
%matplotlib inline
    
def live_plot(x, y, figsize=(7,5), title=''):
    clear_output(wait=True)
    plt.figure(figsize=figsize)
    plt.xlim(0, training_steps)
    plt.ylim(0, 100)
    x= [float(i) for i in x]
    y= [float(i) for i in y]
    
    if len(x) > 1:
        plt.scatter(x,y, label='axis y', color='k') 
        m, b = np.polyfit(x, y, 1)
        plt.plot(x, [x * m for x in x] + b)

    plt.title(title)
    plt.grid(True)
    plt.xlabel('axis x')
    plt.ylabel('axis y')
    plt.show();

you just need to call live_plot(x, y) inside a loop. here's how it looks:

Barracuda

This code runs very slowly for me :/ For plotting x=np.linspace(0, 2*3.141592, 100), y=sin(x + phase) for 100 loops without sleeping, it took 8 seconds. Note that this is with the ployfit removed – Viccora

(0.087s per frame or 11hz) – Viccora

Answer 5

The canvas.draw method of the figure dynamically updates its graphs, for the current figure:

from matplotlib import pyplot as plt

plt.gcf().canvas.draw()

Exceptionable