How to show an image direct from memory on RPI?

Question

I want to create an image in PILLOW and show it on the display on a Raspberry Pi. Until now I have saved the image and have used omxiv to show the image. This is rather time consuming.

from PIL import Image
import os

img = Image.new('RGB', size=(150, 50), color=(0, 0, 255))
save img
....
os.system('omxiv img')

Is there a better and less time consuming way to do this, for example writing direct to the framebuffer?

Drinker

Welcome to StackOverflow! If you use three backticks (the ` character, often to the left of the 1 key) to separate your code, the site will format it nicely so you can benefit from syntax highlighting. – Scarron

Is there some particular reason to use omxiv? Or are other viewers acceptable? Why Python 2? It is End of Life in 2 months and Python 3 has been available for 10 years? – Zanze

The reason why I am using omxiv, is that I am drawing and displaying several images in different windows on my RPI's display, and changing sone if them at different times. This is easily done with Pillow and omxiv. I would be gratefull to know if there is a more effective method to write the images to the framebuffer. I have no X installed. – Drinker

Answer 1

I did some experiments with the framebuffer on my Raspberry Pi 4. Here is what I managed to work out...

You can get the screen resolution using the fbset command like this:

fbset -fb /dev/fb0 

Sample Output

mode "1280x1024"
    geometry 1280 1024 1280 1024 32
    timings 0 0 0 0 0 0 0
    accel true
    rgba 8/16,8/8,8/0,0/0
endmode

That tells me the screen is 1280 px wide and 1024 px high and I need to write 4 bytes per pixel in the order BGRA888.

---

So, I can do a quick test with ImageMagick to see if I can fill the screen, like this:

# Write to screen buffer - BGRA8888, width=1280, height=1024
convert -size 1280x1024 -depth 8 gradient:lime-magenta  bgra:/dev/fb0

and it fills the screen with a lime-magenta gradient. Excellent!

---

So, having acquired that knowledge and a smattering of confidence, let's try Python...

#!/usr/bin/env python3

import numpy as np

# Map the screen as Numpy array
# N.B. Numpy stores in format HEIGHT then WIDTH, not WIDTH then HEIGHT!
# c is the number of channels, 4 because BGRA
h, w, c = 1024, 1280, 4
fb = np.memmap('/dev/fb0', dtype='uint8',mode='w+', shape=(h,w,c)) 

# Fill entire screen with blue - takes 29 ms on Raspi 4
fb[:] = [255,0,0,255]

# Fill top half with red - takes 15 ms on Raspi 4
fb[:h//2] = [0,0,255,255]

# Fill bottom right quarter with green - takes 7 ms on Raspi 4
fb[h//2:, w//2:] = [0,255,0,255] 

I then tried displaying an image - Lena, of course. So, just for brevity and simplicity, I made Lena exactly the right size and added an alpha channel with ImageMagick:

convert lena.png -resize 1280x1024\! -alpha opaque png32:lena1280.png

Then carried on as follows in the Python session I started above:

from PIL import Image

# Load Lena image
im = Image.open('/home/pi/lena1280.png') 

# Convert from PIL Image to Numpy array
n = np.array(im)

# Blit to screen - takes 30ms
fp[:] = n

Note that you would probably do better using OpenCV to load the image, with cv.imread(...,cv.IMREAD_UNCHANGED), because that will deliver you a Numpy array directly without needing conversion and the BGR ordering will already match that of the frame buffer.

---

Other useful commands - for my own reference!

# Retrieve EDID settings from monitor and write into a file called "edid"
tvservice -d edid

# Parse the file we just created to see what the attached monitor is capable of
edidparser edid

Sample Output

Enabling fuzzy format match...
Parsing edid...
HDMI:EDID version 1.3, 0 extensions, screen size 38x30 cm
HDMI:EDID features - videodef 0x80 standby suspend active off; colour encoding:RGB444|YCbCr444|YCbCr422; sRGB is default colourspace; preferred format is native; does not support GTF
HDMI:EDID found monitor S/N descriptor tag 0xff
HDMI:EDID found monitor name descriptor tag 0xfc
HDMI:EDID monitor name is DELL_1907FP
HDMI:EDID found monitor range descriptor tag 0xfd
HDMI:EDID monitor range offsets: V min=0, V max=0, H min=0, H max=0
HDMI:EDID monitor range: vertical is 56-76 Hz, horizontal is 30-81 kHz, max pixel clock is 140 MHz
HDMI:EDID monitor range does not support GTF
HDMI:EDID found preferred DMT detail timing format: 1280x1024p @ 60 Hz (35)
HDMI:EDID established timing I/II bytes are A5 4B 00
HDMI:EDID found DMT format: code 4, 640x480p @ 60 Hz in established timing I/II
HDMI:EDID found DMT format: code 6, 640x480p @ 75 Hz in established timing I/II
HDMI:EDID found DMT format: code 9, 800x600p @ 60 Hz in established timing I/II
HDMI:EDID found DMT format: code 11, 800x600p @ 75 Hz in established timing I/II
HDMI:EDID found DMT format: code 16, 1024x768p @ 60 Hz in established timing I/II
HDMI:EDID found DMT format: code 18, 1024x768p @ 75 Hz in established timing I/II
HDMI:EDID found DMT format: code 36, 1280x1024p @ 75 Hz in established timing I/II
HDMI:EDID standard timings block x 8: 0x714F 8180 0101 0101 0101 0101 0101 0101 
HDMI:EDID found DMT format: code 21, 1152x864p @ 75 Hz (4:3) in standard timing 0
HDMI:EDID found DMT format: code 35, 1280x1024p @ 60 Hz (5:4) in standard timing 1
HDMI:EDID filtering formats with pixel clock unlimited MHz or h. blanking unlimited
HDMI:EDID best score mode initialised to DMT (4) 640x480p @ 60 Hz with pixel clock 25 MHz (score 0)
HDMI:EDID best score mode is now DMT (4) 640x480p @ 60 Hz with pixel clock 25 MHz (score 36864)
HDMI:EDID DMT mode (6) 640x480p @ 75 Hz with pixel clock 31 MHz has a score of 11520
HDMI:EDID best score mode is now DMT (9) 800x600p @ 60 Hz with pixel clock 40 MHz (score 57600)
HDMI:EDID DMT mode (11) 800x600p @ 75 Hz with pixel clock 49 MHz has a score of 18000
HDMI:EDID best score mode is now DMT (16) 1024x768p @ 60 Hz with pixel clock 65 MHz (score 94370)
HDMI:EDID DMT mode (18) 1024x768p @ 75 Hz with pixel clock 78 MHz has a score of 29491
HDMI:EDID DMT mode (21) 1152x864p @ 75 Hz with pixel clock 108 MHz has a score of 62324
HDMI:EDID best score mode is now DMT (35) 1280x1024p @ 60 Hz with pixel clock 108 MHz (score 5260929)
HDMI:EDID DMT mode (36) 1280x1024p @ 75 Hz with pixel clock 135 MHz has a score of 49152
HDMI0:EDID preferred mode remained as DMT (35) 1280x1024p @ 60 Hz with pixel clock 108 MHz
HDMI:EDID has only DVI support and no audio support
edidparser exited with code 0

---

You can turn off/disbale the text cursor in the console like this:

sudo sh -c "TERM=linux setterm -foreground black -clear all >/dev/tty0"

and re-enable it like this:

sudo sh -c "TERM=linux setterm -foreground white -clear all >/dev/tty0"

Keywords: Raspberry Pi, RasPi, framebuffer, fb0, /dev/fb0, Python, Numpy, ImageMagick, direct frame buffer access, edid, HDMI, DVI, monitor capabilities, features, tvservice, edidparser, resolution, bgra8888, blit, bit-blit, cursor

Zanze

Did you try this at all? Any luck? – Zanze

Note that you can use scikit-imageif you want to draw rectangles, lines, polygons, circles, ellipses in a Numpy array before blitting it to the screen as above... scikit-image.org/docs/dev/api/… – Zanze

It was a great improvement in speed. I am very grateful. – Drinker

this is amazing. kudos to you Mark ! been struggling with this for a while! – Howler

Just logged in after a long time to say that this is a brilliant solution! I was fighting with various approaches to write the framebuffer from python without a GUI and this just works (with root permissions). Also works with a 16 bit framebuffer: fb = np.memmap('/dev/fb0', dtype='uint16',mode='w+', shape=(576,720)) – Accommodation

Answer 2

The easiest way to do this is to make sure your /tmp filesystem is mounted on tmpfs which is purely memory-based and therefore is not written to your SD card. Note that means that the contents are lost on each reboot.

So, you need to become root and use your favourite editor to edit /etc/fstab, in my case that would be:

sudo vi /etc/fstab

And then you need to add a line like this:

tmpfs   /tmp    tmpfs   defaults,noatime,nosuid 0   0

Then save the file and reboot your RasPi. If you then run df you will see that /tmp is on tmpfs:

df
tmpfs             966620       0    966620   0% /tmp

So, now to your code. If this is a new project, which I guess it must be if you don't know how to get started, please consider using Python3 which has been out 10 years rather than Python2 which is being discontinued in 2 months.

Now you need to use this code:

#!/usr/bin/env python3

from PIL import Image
import sys, os

# Create a new 640x480 magenta image
img = Image.new('RGB', size=(640, 480), color=(255, 0, 255))

filename = '/tmp/image.jpg'
img.save(filename)
os.system('omxiv ' + filename)

---

Note that you "can" pass an image to omxiv on the command-line:

cat image.jpg | omxiv

or, with a Python program:

WriteImageWithPIL.py | omxiv

But, that produces two problems. Firstly, it no longer reads any keys you type on the keyboard because it is reading from stdin. Secondly, PIL gets upset writing to omxiv like that because it closes the pipe before PIL can flush the data, so you'd probably have to start using stdbuffer in there and that will make big mess.

Zanze

Thanks, but I had hoped that if omxiv could get the file direct from memory (tmpfs?), it would run faster, but unfortuneatly it does not. – Drinker

It depends what you are trying to achieve. Your question implies you are stuck with even getting started on generating an image with Python and passing it to omxiv yet your comments imply you have some sophisticated multi-window app running. Generating a PNG is time-consuming because of the compression. You may be better off writing to the framebuffer directly but it depends if you are generating shapes or displaying photos. It is unclear why you use omxiv. You could modify it to share memory with your program. It's hard to say because I don't really know your aim. – Zanze

I am writing menus with text, lines and logos to the display. Using an IR-remote and lirc, I can switch between the menus, and scroll. I am drawing the menus in Pillow, and after saving them, omxiv puts them on the right place on the display. Since I am new to python, I felt this was a quite simple solution, but I think there must be more faster methods. I have looked for a method to write directly from Pillow to the framebuffer, but I have not found any yet, so I am stuck to omxiv. – Drinker