How to read raw png from an array in python opencv?

Question

I'm streaming a png image from my iPhone to my MacBook over tcp. The MacBook code is from http://docs.python.org/library/socketserver.html#requesthandler-objects. How can the image be converted for use with OpenCV? A png was selected because they are efficient, but other formats could be used.

I wrote a test program that reads the rawImage from a file, but not sure how to convert it:

# Read rawImage from a file, but in reality will have it from TCPServer
f = open('frame.png', "rb")
rawImage = f.read()
f.close()

# Not sure how to convert rawImage
npImage = np.array(rawImage)
matImage = cv2.imdecode(rawImage, 1)

#show it
cv.NamedWindow('display')
cv.MoveWindow('display', 10, 10)
cv.ShowImage('display', matImage)
cv. WaitKey(0)

Endplay

Is something wrong with the code you posted? Seems everything alright. – Ranna

Answer 1

I figured it out:

# Read rawImage from a file, but in reality will have it from TCPServer
f = open('frame.png', "rb")
rawImage = f.read()
f.close()

# Convert rawImage to Mat
pilImage = Image.open(StringIO(rawImage));
npImage = np.array(pilImage)
matImage = cv.fromarray(npImage)

#show it
cv.NamedWindow('display')
cv.MoveWindow('display', 10, 10)
cv.ShowImage('display', matImage)
cv. WaitKey(0) 

Endplay

Answer 2

@Andy Rosenblum's works, and it might be the best solution if using the outdated cv python API (vs. cv2).

However, because this question is equally interesting for users of the latest versions, I suggest the following solution. The sample code below may be better than the accepted solution because:

1. It is compatible with newer OpenCV python API (cv2 vs. cv). This solution is tested under opencv 3.0 and python 3.0. I believe only trivial modifications would be required for opencv 2.x and/or python 2.7x.
2. Fewer imports. This can all be done with numpy and opencv directly, no need for StringIO and PIL.

Here is how I create an opencv image decoded directly from a file object, or from a byte buffer read from a file object.

import cv2
import numpy as np

#read the data from the file
with open(somefile, 'rb') as infile:
     buf = infile.read()

#use numpy to construct an array from the bytes
x = np.fromstring(buf, dtype='uint8')

#decode the array into an image
img = cv2.imdecode(x, cv2.IMREAD_UNCHANGED)

#show it
cv2.imshow("some window", img)
cv2.waitKey(0)

Note that in opencv 3.0, the naming convention for the various constants/flags changed, so if using opencv 2.x, you will need to change the flag cv2.IMREAD_UNCHANGED. This code sample also assumes you are loading in a standard 8-bit image, but if not, you can play with the dtype='...' flag in np.fromstring.

Tenorrhaphy

This answer should be gold plated. – Ruckus

It's better to use np.frombuffer when working with binary buffers. – Hyetography

actually AFAICT the dtype of the numpy array should always be uint8, as you're just making an bytes object that openCV can work with. Changes in dtype of the actual image are handled by cv2.imdecode. – Swept

Answer 3

another way,

also in the case of a reading an actual file this will work for a unicode path (tested on windows)

with open(image_full_path, 'rb') as img_stream:
    file_bytes = numpy.asarray(bytearray(img_stream.read()), dtype=numpy.uint8)
    img_data_ndarray = cv2.imdecode(file_bytes, cv2.CV_LOAD_IMAGE_UNCHANGED)
    img_data_cvmat = cv.fromarray(img_data_ndarray) #  convert to old cvmat if needed

Thordis

Maybe it's because I am using a newer version of OpenCV than the one used to test the above snippet, but I had to change the cv2.CV_LOAD_IMAGE_UNCHANGED flag to cv2.IMREAD_UNCHANGED. – Centiare

Answer 4

This works for me (these days):

import cv2
import numpy as np

data = open('016e263c726a.raw').read()
x = np.frombuffer(data, dtype='uint8').reshape(2048,2448)
cv2.imshow('x',x); cv2.waitKey(); cv2.destroyAllWindows()

But it reads a RAW image saved without any specific format.

Mannish

This is probably only valid for greyscale images. If the image is a png raw (RGBA), you will need reshape(height, width, 4) Otherwise, great answer – Teazel

Answer 5

(Your question seems to be tagged objective-c but you ask for Python and so is your example, so I'll use that.) My first post on Stack Overflow!

The cv.LoadImageM method seems to be what you are looking for.

http://opencv.willowgarage.com/documentation/python/reading_and_writing_images_and_video.html

Example use: http://opencv.willowgarage.com/wiki/PythonInterface/

LoadImage(filename, iscolor=CV_LOAD_IMAGE_COLOR) → None

Loads an image from a file as an IplImage.
Parameters:   

    filename (str) – Name of file to be loaded.
    iscolor (int) –

    Specific color type of the loaded image:
        CV_LOAD_IMAGE_COLOR the loaded image is forced to be a 3-channel color image
        CV_LOAD_IMAGE_GRAYSCALE the loaded image is forced to be grayscale
        CV_LOAD_IMAGE_UNCHANGED the loaded image will be loaded as is.

The function cvLoadImage loads an image from the specified file and returns the pointer to the loaded image. Currently the following file formats are supported:

Windows bitmaps - BMP, DIB
JPEG files - JPEG, JPG, JPE
Portable Network Graphics - PNG
Portable image format - PBM, PGM, PPM
Sun rasters - SR, RAS
TIFF files - TIFF, TIF

Note that in the current implementation the alpha channel, if any, is stripped from the output image, e.g. 4-channel RGBA image will be loaded as RGB.

Amphetamine

The issue is that the png is already loaded in memory from tcp. LoadImageM requires saving to a file and then loading the file.#3397657 seems to be the right idea. Sorry for the confusion with the tag. – Endplay

All these willowgarage links are broken :( – Ruckus

Answer 6

When you have to load from file, this simple solution does the job (tested with opencv-python-3.2.0.6):

import cv2

img = cv2.imread(somefile)

Breeder