Figure to image as a numpy array

Question

I'm trying to get a numpy array image from a Matplotlib figure and I'm currently doing it by saving to a file, then reading the file back in, but I feel like there has to be a better way. Here's what I'm doing now:

from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from matplotlib.figure import Figure

fig = Figure()
canvas = FigureCanvas(fig)
ax = fig.gca()

ax.text(0.0,0.0,"Test", fontsize=45)
ax.axis('off')

canvas.print_figure("output.png")
image = plt.imread("output.png")

I tried this:

image = np.fromstring( canvas.tostring_rgb(), dtype='uint8' )

from an example I found but it gives me an error saying that 'FigureCanvasAgg' object has no attribute 'renderer'.

Jeramyjerba

Answer 1

In order to get the figure contents as RGB pixel values, the matplotlib.backend_bases.Renderer needs to first draw the contents of the canvas.

You can do this by manually calling canvas.draw():

from matplotlib.figure import Figure

fig = Figure()
canvas = fig.canvas
ax = fig.gca()

ax.text(0.0,0.0,"Test", fontsize=45)
ax.axis('off')

canvas.draw()  # Draw the canvas, cache the renderer

image_flat = np.frombuffer(canvas.tostring_rgb(), dtype='uint8')  # (H * W * 3,)
# NOTE: reversed converts (W, H) from get_width_height to (H, W)
image = image_flat.reshape(*reversed(canvas.get_width_height()), 3)  # (H, W, 3)

See here for more info on the Matplotlib API.

Seedcase

I get the img as 1d array, you can fix that using: width, height = fig.get_size_inches() * fig.get_dpi() and img = np.fromstring(canvas.to_string_rgb(), dtype='uint8').reshape(height, width, 3) – Mcgrody

I sometimes get an error where height and width are floats, converting them to integers is an easy fix though. – Simitar

I edited the answer to include @Mcgrody 's suggestion. – Antalya

Do we really have to call canvas.draw() to make this work? – Irony

@RishabhAgrahari One way or another the contents of the canvas need to have been rendered at least once before you can fetch the pixel values. Rendering can happen as a side-effect of other operations, e.g. if the canvas belongs to a pyplot figure and you call plt.show() to display it then the canvas will be rendered. However in the example above, if you get rid of the call to canvas.draw you'll get AttributeError: 'FigureCanvasAgg' object has no attribute 'renderer' (try it). – Seedcase

NOTE tostring_rgb is deprecated. A much better solution is here, https://mcmap.net/q/322227/-figure-to-image-as-a-numpy-array – Brandwein

Answer 2

For people who are searching an answer for this question, this is the code gathered from previous answers. Keep in mind that the method np.fromstring is deprecated and np.frombuffer is used instead.

#Image from plot
ax.axis('off')
fig.tight_layout(pad=0)

# To remove the huge white borders
ax.margins(0)

fig.canvas.draw()
image_from_plot = np.frombuffer(fig.canvas.tostring_rgb(), dtype=np.uint8)
image_from_plot = image_from_plot.reshape(fig.canvas.get_width_height()[::-1] + (3,))

Overwind

@rayryeng-ReinstateMonica Thank you for the changes that improves the answer significantly – Overwind

@rayreng Is it possible to get a grayscale output? I don't see a method similar to tostring_rgb on the canvas – Buttery

thanks for compiling answers! I should also add that the order of commands (especially fig.canvas.draw()) is very important. My code didn't work out initially due to wrong ordering. – Cart

Answer 3

from the docs:

https://matplotlib.org/gallery/user_interfaces/canvasagg.html#sphx-glr-gallery-user-interfaces-canvasagg-py

fig = Figure(figsize=(5, 4), dpi=100)
# A canvas must be manually attached to the figure (pyplot would automatically
# do it).  This is done by instantiating the canvas with the figure as
# argument.
canvas = FigureCanvasAgg(fig)

# your plotting here

canvas.draw()
s, (width, height) = canvas.print_to_buffer()

# Option 2a: Convert to a NumPy array.
X = np.fromstring(s, np.uint8).reshape((height, width, 4))

Museology

Answer 4

I think there is some update, which is easier.

canvas.draw()
buf = canvas.buffer_rgba()
X = np.asarray(buf)

Updated version from the docs:

from matplotlib.backends.backend_agg import FigureCanvasAgg
from matplotlib.figure import Figure
import numpy as np

# make a Figure and attach it to a canvas.
fig = Figure(figsize=(5, 4), dpi=100)
canvas = FigureCanvasAgg(fig)

# Do some plotting here
ax = fig.add_subplot(111)
ax.plot([1, 2, 3])

# Retrieve a view on the renderer buffer
canvas.draw()
buf = canvas.buffer_rgba()
# convert to a NumPy array
X = np.asarray(buf)

Wendish

This is the version that worked for me. np.fromstring is deprecated, and without specifying FigureCanvasAgg some platforms give an error, on macOS for exampleFigureCanvasMac has no renderer attribute. I find incredible how complicated such an operation is :( – Knotting

To render an image of a specific size (for example, a 1024 by 512 image), do fig = Figure(figsize=(1024, 512), dpi=1) when constructing figure. – Tita

@Tita : no, because you'll get RuntimeError: In set_size: Could not set the fontsize (error code 0x97) because the dpi is set too low to render fonts. prefer a fig = Figure(figsize=(10.24, 5.12), dpi=100.0) instead, it won't change the final picture size, but it pleases matplotlib better this way. – Sparing

Answer 5

To fix the large margins Jorge references, add ax.margins(0). See here for details.

Shortening

also you can use plt.tight_layout(pad=-10) which allows negative numbers for extreme cropping. I have no idea what are the units of pad are, perhaps tenths of an inch? – Brandwein

Answer 6

Another approach is to use a byte stream as follows:

import matplotlib.pyplot as plt
import numpy as np

# Create sample plot
x = np.linspace(-1, 1, 101)
y = x**2 
fig = plt.figure()
plt.plot(x, y)

# Save figure in PNG format to byte stream
from io import BytesIO
b = BytesIO()
fig.savefig(b, format='png')

# Read back from byte stream
b.seek(0)
img = plt.imread(b)

# Dispose of the stream to save memory
b.close()
del b

Northey