Convert a C or numpy array to a Tkinter PhotoImage with a minimum number of copies

Question

I know a recipe for displaying an MxNx3 numpy array as an RGB image via Tkinter, but my recipe makes several copies of the array in the process:

a = np.random.randint(low=255, size=(100, 100, 3), dtype=np.uint8) # Original
ppm_header = b'P6\n%i %i\n255\n'%(a.shape[0], a.shape[1])
a_bytes = a.tobytes() # First copy
ppm_bytes = ppm_header + a_bytes # Second copy https://en.wikipedia.org/wiki/Netpbm_format
root = tk.Tk()
img = tk.PhotoImage(data=ppm_bytes) # Third and fourth copies?
canvas = tk.Canvas(root, width=a.shape[0], height=a.shape[1])
canvas.pack()
canvas.create_image(0, 0, anchor=tk.NW, image=img) # Fifth copy?
root.mainloop()

How can I achieve an equivalent result with a minimum number of copies?

Ideally, I would create a numpy array which was a view of the same bytes that the Tkinter PhotoImage object was using, effectively giving me a PhotoImage with mutable pixel values, and making it cheap and fast to update the Tkinter display. I don't know how to extract this pointer from Tkinter.

Perhaps there's a way via ctypes, as hinted at here?

The PhotoImage.put() method seems very slow, but maybe I'm wrong, and that's a path forward?

I tried making a bytearray() containing the ppm header and the image pixel values, and then using numpy.frombuffer() to view the image pixel values as a numpy array, but I think the PhotoImage constructor wants a bytes() object, not a bytearray() object, and also I think Tkinter copies the bytes of its data input into its internal format (32-bit RGBA?). I guess this saves me one copy compared to the recipe above?

Rozier

FWIW, the underlying data storage of the Tk photo image is RGBA, with 8 bits per pixel per channel. The maximally efficient approach is definitely going to involve starting with that, though I don't know how to actually make that work in Tkinter… – Salamone

@donal-fellows I have some experience using ctypes to talk to C dlls, but I don't know enough about how tkinter is implemented to get started. Is there something like a tkinter shared library with a documented API that Python makes calls to? – Rozier

The underlying library API is here, and the API call you need is Tk_PhotoPutBlock() (here). Assembling the arguments for it… well, that's a bit trickier. The handle can be got from Tk_FindPhoto(), but the interp is buried inside Tkinter (the Python-to-Tcl/Tk wrapping library) and I don't know that well enough to help. Also, you MUST use that code from only one thread or you get major trouble (a common Python/Tkinter issue). – Salamone

Answer 1

I can reduce it to 1 (maybe 2) copies by using PIL and a Label:

import numpy as np
import tkinter as tk
from PIL import Image, ImageTk

a = np.random.randint(low=255, size=(100, 100, 3), dtype=np.uint8) # Original
root = tk.Tk()
img = ImageTk.PhotoImage(Image.fromarray(a)) # First and maybe second copy.
lbl = tk.Label(root, image=img)
lbl.pack()
root.mainloop()

However that's still not mutable. If you want that I think you need to reinvent an image by placing a pixel on the canvas yourself. I did that once with this project and found that the fastest update was a matplotlib animation, which works really well for you since you are already using np arrays.

My code for using a tk.Canvas, a PIL Image(using putpixel()), and matplotlib.

Kamalakamaria

You actually only saved one copy compared to the original code: a_bytes = a.tobytes(); ppm_bytes = ppm_header + a_bytes -> Image.fromarray(a). – Indoxyl

ImageTk.PhotoImage involves two copies: it delegates to tkinter.BitmapImage which involves two copies. – Indoxyl

Answer 2

• you can eliminate the 1st and 2nd copies

You get a numpy.ndarray over arbitrary data with numpy.frombuffer:

shape=(100,100,3)
ppm_header = b'P6\n%i %i\n255\n'%(shape[0], shape[1])
ppm_bytes = ppm_header + b'\0'*(shape[0]*shape[1]*shape[2])
array_image = np.frombuffer(ppm_bytes, dtype=np.uint8, offset=len(ppm_header)).reshape(shape)

• the 3rd and 4th copies are inevitable (see below), but the 3rd one is discarded right after the call

• the 5th copy is not actually made (also see below)

• the drawing stage involves a copy to the screen via the windowing system's drawing API which is also inevitable.

---

Tcl is a safe, garbage-collected language like Python, and Tcl objects don't support either a "buffer protocol", or using memory for their data that they don't own (though objects can be shared.

• img = tk.PhotoImage(data=ppm_bytes) # Third and fourth copies?
  

  When making most Tcl calls, Python variables are first converted into Equivalent Tcl objects (first into C values which doesn't involve copying for a bytestring which are then passed to Tcl constructors which does involve a copy), then these objects are passed to Tcl_EvalObjv.

  On the Tcl side, photo (which PhotoImage() wraps) also parses input data (-data string argument) and as such, also cannot reuse its memory block. Even if it's a raw bitmap 'cuz Tcl strings have no "view" functionality.

  So, for a bytestring, there's one mandatory copy of the bitmap data involved at Pythob-to-Tcl call stage, and one more at image construction stage. (On the bright side, one copy (the argument string for the Tcl call) is discarded after the call.)

• canvas.create_image(0, 0, anchor=tk.NW, image=img) # Fifth copy?
  

  A copy is not involved here, the canvas just saves a reference to the image object in its composition data.

  Now, canvas create image requires a Tcl image identifier as image argument; so there's no way around PhotoImage(), either.

• (drawing stage)

  xlib's commands to draw a bitmap are used, no additional copies are involved.

  The thing to note here is that a canvas doesn't even have direct access to the resulting screen pixels. Instead, xlib uses backend's graphical API for drawing (like WinGDI's GetDC(), BitBlt() etc that you might be familiar with). E.g. in Windows, xlib's XCopyArea uses BitBlt.

Indoxyl

I think what he's looking for is a way to pass his binary blob to the C API call; numpy can certainly generate the right format of data. That'll let the copies that are done at least be a straight memcpy() under the hood. – Salamone

@DonalFellows Tk C API is unavailable from Tkinter. Moreover, photo create is implemented with tkImgPhoto.c:ImgPhotoCreate which isn't even exposed in the API. – Indoxyl

Yes. This is why this is an expert-level need for work. However, if you can get the handle name (a small string) then you should be able to do the rest with a fairly small chunk of C code. It'd probably be a bit of a hack under the covers as it is bridging two thoroughly-different memory management schemes (Python and Tcl), but it could be made to work and would be actually fast. – Salamone

@DonalFellows It won't be just a "hack", it will be tremendously fragile. Since Tcl's objects don't support either a "buffer protocol", or using memory for their data that they don't own. You can get to the data buffer by parsing the memory -- in Tcl/Tk version-, architecture-, compiler- and compilation options-specific way, -- forge that buffer's data -- in likewise way -- and even then that buffer can be moved or discarded at any moment. So as an expert, I cannot recommend this venue. – Indoxyl

Answer 3

Some time ago I implemented a "ripple tank" simulator in numpy+TkInter that I believe to be quite optimized in terms of speed. It is freely available here:

https://gist.github.com/FilipDominec/14761052f42d80d283bd3adcf7eb5347

Ultramicroscope