opencv read jpeg image from buffer

Question

I have an unsigned char* buffer containing data of a jpeg image. I would like to display that image using c++ and opencv. If i do:

Mat img(Size(640, 480), CV_8UC3, data);
namedWindow("image", 1);
imShow("image", img);

I get a noisy mess of pixels.

I suppose it's because the data is jpeg (with a header). Because this works:

Mat imgbuf(Size(640, 480), CV_8UC3, data);
Mat img = imdecode(imgbuf, CV_LOAD_IMAGE_COLOR);

BUT I cannot use the imdecode function as it is from highgui.h which is based upon GTK 2, and in my project I use GTK 3.

So, how can I display the buffer data? Is there a way to decode the jpeg image other than imdecode in opencv, if that's the problem. I don't really want to have to rebuild opencv with Qt...

Any other suggestions?

(Using Linux)

Lexy

Can't you use libjpeg for decoding and be done with it ? What is the problem with that ? – Gonzalogoo

@Gonzalogoo I will try libjpeg, just thought I could do the same thing without including another library. – Lexy

opencv won't load jpeg image if you don't have libjpeg, so there is no other library being included. – Gonzalogoo

Wow what you are doing is really not okay. In your Mat imgbuf(...) line you read in data in 640x480x3 fields. Since jpegs are compressed, the jpeg will indeed fit in there with ease. So the rest of the spots are then filled in with random data from memory, that's very, very bad. After that you load in the Mat, that includes the JPEG and a bunch of random memory data, and decode it. Instead, first put your unsigned char buffer in a vector or mat exactly the size of your buffer and THEN feed it in imdecode. – Jokjakarta

I guess things have changed in 2022 because imdecode is part of "imgcodecs.hpp", not highgui, so now this is the best way. I have the data in an std::string (why not), so I'm using this: cv::Mat data(1, (int)str.size(), CV_8UC1, (void*)str.c_str()); followed by cv::imdecode(data, 0); – Medrek

Answer 1

I have decompressed the JPEG image using libjpeg using the standard procedure described in the libjpeg API documentation under 'Decompression details'.

After having decompressed the data you can use it to construct the cv::Mat. Mind you, the decompressed image is in RGB format, whereas openCV uses a BGR format so a cvtColor() operation with format CV_RGB2BGR is needed.

Lexy

Answer 2

I have seen many responses to this question around on the net saying that you should call libjpeg directly and bypass OpenCV's imread() routine.

This is NOT necessary! You can use imdecode() to decode a raw image buffer from memory. The way to do it is NOT intuitive, and isn't documented enough to help people trying to do this for the first time.

If you have a pointer/size for your raw file data (fread() directly from the .jpg, .png, .tif, files, etc...

int    nSize = ...       // Size of buffer
uchar* pcBuffer = ...    // Raw buffer data


// Create a Size(1, nSize) Mat object of 8-bit, single-byte elements
Mat rawData( 1, nSize, CV_8UC1, (void*)pcBuffer );

Mat decodedImage  =  imdecode( rawData /*, flags */ );
if ( decodedImage.data == NULL )   
{
    // Error reading raw image data
}

That's IT!

Hope this helps someone in the future.

Db

It did help someone. Note that you used 'size' instead of 'nsize', and you could have initialized rawData directly from the constructor. – Keener

This did NOT work for me. Reading images from a network stream, saving to file works but calling imdecode gives me a grey window with cv::imshow("Window", decodedImage) – Castanets

@Castanets that's probably because you forgot to put cv2.waitKey(0) after imshow() – Aorist

This was from a really long time ago, but i could display other images (loaded from file) so I dont think the waitKey was the issue. – Castanets

no need for (void*), and also set the required flag (cv::IMREAD_COLOR ) in imdecode should do the trick. – Sibley