How to get RGB pixel values from Linux framebuffer?

Question

I want to get RGB values of screen pixels in the most efficient way, using Linux. So I decided to use the framebuffer library in C (fb.h) to access the framebuffer device (/dev/fb0) and read from it directly.

This is the code:

#include <stdint.h>
#include <linux/fb.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/ioctl.h>
#include <sys/mman.h>

int main() {

    int fb_fd;
    struct fb_fix_screeninfo finfo;
    struct fb_var_screeninfo vinfo;
    uint8_t *fb_p;

    /* Open the frame buffer device */
    fb_fd = open("/dev/fb0", O_RDWR);
    if (fb_fd < 0) {
        perror("Can't open /dev/fb0\n");
        exit(EXIT_FAILURE);
    }

    /* Get fixed info */
    if (ioctl(fb_fd, FBIOGET_FSCREENINFO, &finfo) < 0) {
        perror("Can't get fixed info\n");
        exit(EXIT_FAILURE);
    }

    /* Get variable info */
    if (ioctl(fb_fd, FBIOGET_VSCREENINFO, &vinfo) < 0) {
        perror("Can't get variable info\n");
        exit(EXIT_FAILURE);
    }

    /* To access to the memory, it can be mapped*/
    fb_p = (uint8_t *) mmap(0, finfo.smem_len, PROT_READ | PROT_WRITE, MAP_SHARED, fb_fd, 0);
    if (fb_p == MAP_FAILED) {
        perror("Can't map memory\n");
        exit(EXIT_FAILURE);
    }

    /* Print each byte of the frame buffer */
    for (int i = 0; i < finfo.smem_len; i++) {
        printf("%d\n", *(fb_p + i));

        // for (int j = 0; j < 500000000; j++);       /* Delay */
    }

    munmap(fb_p, 0);
        close(fb_fd);

    return 0;
}

But when I print the values, I don't get what I was expecting...

If I pick the RGB value of the pixel (0, 0) with a tool like grabc, I get this:

#85377e
133,55,126

But the first printings with my code are:

126
145
198
...

It looks like I am obtaining well the first value of the first pixel, corresponding to blue, but the rest is wrong.

Crossgrained

I've no experience with Linux framebuffers, but could it be that the pixel buffer you get does not start from the (0,0) you think it's starting from? E.g. you picked the top left value but the buffer starts from bottom right. – Perique

Are you in the X Window System when you are trying to read the framebuffer data? Perhaps it is not possible. – Anniceannie

The key problem is that XWindow may not use framebuffer in kernel at all. As I know, gpu like amd or nvidia do not relay on fb driver in kernel. But when switching to console by ctrl+alt+fn , I'm sure you can get correct rgb value of framebuffer. – Hawsepiece

Does the fbgrab -b 24 test.png give you what you expected? – Anniceannie

Anyway, this should help if you use X Window System. Just open /tmp/fbe_buffer instead of /dev/fb0. – Anniceannie

Answer 1

If you look at the grabc source code (https://www.muquit.com/muquit/software/grabc/grabc.html), you'll see that he's querying X Windows (and NOT the hardware frame buffer per se).

root_window=XRootWindow(display,XDefaultScreen(display));
target_window=selectWindow(display,&x,&y);
...
ximage=XGetImage(display,target_window,x,y,1,1,AllPlanes,ZPixmap);
...
color->pixel=XGetPixel(ximage,0,0);
XDestroyImage(ximage);

For whatever it's worth - and for several different reasons - I'd strongly encourage you to consider doing the same.

You might also be interested in:

• Intro to Low-Level Graphics on Linux

• DirectFB

Burgher

I think the question relates to Linux console; not to X Window System. – Anniceannie

I am not interested in this solution. I want to read it directly from the device... I don't know if it's possible. – Crossgrained

Yes, it's possible. It just presents some "challenges".. The two links I cited both deal with low-level "Frame buffer I/O". The DirectFB library might help simplify your task. I tried the same thing myself a few years ago ... and ultimately wound up using Xlib instead of FB. One more link: ummon.eu/Linux/API/Devices/framebuffer.html – Burgher

@Crossgrained /dev/fb isn't "directly from the device". It's from a driver. A driver that may be clueless about what the device is actually doing - because the X server is talking directly to the device. – Ruelas

@Wumpus Q. Wumbley - Excellent points. Additionally, it's worth emphasizing that /dev/fb is STILL an "abstraction layer" (like XLib), that HIDES (possibly useful) details, and incurs a PERFORMANCE PENALTY. Also note that /dev/fb might not be configured to be involved with any of the Xlib (i.e. visible) graphics anyway! – Burgher

Answer 2

In some rare buggy device using fbdev you may need to align the mmap returned pointer to the next system page in order to get the first pixel correctly, and by doing so you may also need to mmap one more page.

I don't think that's your case, it seems like you're trying to get the xserver desktop pixels by using linux fbdev, unless your xserver is configured to use the fbdev driver (and even so I'm not sure would work) isn't gonna work. If reading the desktop pixels is your target you should look into some screenshot tool.

Beside, fbdev drivers are usually really basic, is a low-level access but there's nothing efficient about mmapping and reading pixels, the Xlib or any higher level graphic system may be aware and support your graphic card accelerations and be far more efficient (perhaps using DMA to read the whole framebuffer to your system memory without loading the cpu).

Corrigendum

Answer 3

Working on iOS there was a private framework iOMobileFrameBuffer and created a lovely recording software for the screen. I've dabbled in this before so I will give you what I know. iOS is obviously different than Linux FrameBuffer (lack of documentation is pretty similar...) but I would personally take a look at this answer. Although the question doesn't fit what you ask, that answer explains the PixelBuffer along with a small example of exactly what you asked. Give it a read and reply with any questions, I will answer to the best of my ability.

Fidelfidela