Correct RGB values for AVFrame

Question

I have to fill the ffmpeg AVFrame->data from a cairo surface pixel data. I have this code:

/* Image info and pixel data */
width  = cairo_image_surface_get_width( surface );
height = cairo_image_surface_get_height( surface );
stride = cairo_image_surface_get_stride( surface );
pix    = cairo_image_surface_get_data( surface );

for( row = 0; row < height; row++ )
{
    data = pix + row * stride;
    for( col = 0; col < width; col++ )
    {
        img->video_frame->data[0][row * img->video_frame->linesize[0] + col] = data[0];
        img->video_frame->data[1][row * img->video_frame->linesize[1] + col] = data[1];
        //img->video_frame->data[2][row * img->video_frame->linesize[2] + col] = data[2];
        data += 4;
    }
    img->video_frame->pts++;
}

But the colors in the exported video are wrong. The original heart is red. Can someone point me in the right direction? The encode.c example is useless sadly and on the Internet there is a lot of confusion about Y, Cr and Cb which I really don't understand. Please feel free to ask for more details. Many thanks.

Neile

Can you please post a complete code sample - something we can compile and execute? You may have to be creative - consider posting a 8x8 input image as a C array (initialized by code). Can you describe the data arrangement of pix (is it row major RGBA with width, height and stride)? In which part YCbCr pixel format is used? – Husain

Thank you so much for replying. I do appreciate. The code is a part of a GTK3 GUI which turns a series of pictures into a movie. It's a project I have been developing since 2009. I can't provide a code snippet sadly but the software is opensource. The cairo surface is RGB24. The AV context is created as AV_PIX_FMT_YUV420P because if I use AV_PIX_FMT_RGB24 I get the error message during the enconding: "Input width is greater than stride". Hope this helps, please feel free to ask for more info. – Neile

My current understanding of YCbCr is that it is made of Luminance/Red-diff/Blue-diff where Luminance is brightness, and Red-diff and Blue-diff are chroma values with information about Red/green and Blue/green respectively. It works by combining a black and white image with the two chroma channels – Proclaim

Answer 1

You need to use libswscale to convert the source image data from RGB24 to YUV420P.

Something like:

int width  = cairo_image_surface_get_width( surface );
int height = cairo_image_surface_get_height( surface );
int stride = cairo_image_surface_get_stride( surface );
uint8_t *pix = cairo_image_surface_get_data( surface );

uint8_t *data[1] = { pix };
int linesize[1]  = { stride };

struct SwsContext *sws_ctx = sws_getContext(width, height, AV_PIX_FMT_RGB24 ,
                                            width, height, AV_PIX_FMT_YUV420P,
                                            SWS_BILINEAR, NULL, NULL, NULL);

sws_scale(sws_ctx, data, linesize, 0, height, 
          img->video_frame->data, img->video_frame->linesize);

sws_freeContext(sws_ctx);

See the example here: scaling_video

Undrape

Thanks for your reply. I also used sws_scale, no errors during the encoding but the video doesn't even play and VLC shows a bunch of these useless messages in the terminal window: [h264 @ 0x7f1508c82700] co located POCs unavailable [h264 @ 0x7f1508c9f400] mmco: unref short failure [h264 @ 0x7f1508cd8f80] co located POCs unavailable The example wasn't helpful as it convert FROM YUV420P but thanks I appreciated it. If you would like to know more I can commit the changes to the SVN server and you can give a look, it's a nice GUI and a nice software, just underrated sadly. – Neile

@Neile Could you dump a sample cairo image to a file somewhere and post the format, width & height? – Undrape

is this code correct for dumping the image: /* Image info and pixel data */ width = cairo_image_surface_get_width( surface ); height = cairo_image_surface_get_height( surface ); stride = cairo_image_surface_get_stride( surface ); pix = cairo_image_surface_get_data( surface ); FILE *fp = fopen("cairo_image.raw","w"); fwrite (pix, 4*width, sizeof(pix), fp); fclose(fp); – Neile