Convert Mat to Array/Vector in OpenCV

Question

I am novice in OpenCV. Recently, I have troubles finding OpenCV functions to convert from Mat to Array. I researched with .ptr and .at methods available in OpenCV APIs, but I could not get proper data. I would like to have direct conversion from Mat to Array(if available, if not to Vector). I need OpenCV functions because the code has to be undergo high level synthesis in Vivado HLS. Please help.

Corinacorine

Answer 1

If the memory of the Mat mat is continuous (all its data is continuous), you can directly get its data to a 1D array:

std::vector<uchar> array(mat.rows*mat.cols*mat.channels());
if (mat.isContinuous())
    array = mat.data;

Otherwise, you have to get its data row by row, e.g. to a 2D array:

uchar **array = new uchar*[mat.rows];
for (int i=0; i<mat.rows; ++i)
    array[i] = new uchar[mat.cols*mat.channels()];

for (int i=0; i<mat.rows; ++i)
    array[i] = mat.ptr<uchar>(i);

---

UPDATE: It will be easier if you're using std::vector, where you can do like this:

std::vector<uchar> array;
if (mat.isContinuous()) {
  // array.assign(mat.datastart, mat.dataend); // <- has problems for sub-matrix like mat = big_mat.row(i)
  array.assign(mat.data, mat.data + mat.total()*mat.channels());
} else {
  for (int i = 0; i < mat.rows; ++i) {
    array.insert(array.end(), mat.ptr<uchar>(i), mat.ptr<uchar>(i)+mat.cols*mat.channels());
  }
}

p.s.: For cv::Mats of other types, like CV_32F, you should do like this:

std::vector<float> array;
if (mat.isContinuous()) {
  // array.assign((float*)mat.datastart, (float*)mat.dataend); // <- has problems for sub-matrix like mat = big_mat.row(i)
  array.assign((float*)mat.data, (float*)mat.data + mat.total()*mat.channels());
} else {
  for (int i = 0; i < mat.rows; ++i) {
    array.insert(array.end(), mat.ptr<float>(i), mat.ptr<float>(i)+mat.cols*mat.channels());
  }
}

---

UPDATE2: For OpenCV Mat data continuity, it can be summarized as follows:

• Matrices created by imread(), clone(), or a constructor will always be continuous.
• The only time a matrix will not be continuous is when it borrows data (except the data borrowed is continuous in the big matrix, e.g. 1. single row; 2. multiple rows with full original width) from an existing matrix (i.e. created out of an ROI of a big mat).

Please check out this code snippet for demonstration.

Complain

better if you use a std::vector. With naked pointers you have also to free memory. – Photostat

@blackibiza Good point. Updated the answer to use std::vector. :-) – Complain

I guess that mat.cols should be multiplied by mat.channels when copying data with array.insert. Also conversion from uchar* to float* can be omitted when using float as template parameter: mat.ptr<float> – Hawkbill

Is there a relevant performance benefit from using assign or could I just use only the else part for all cases to keep the code simple? – Drawknife

@Drawknife There should be a speed benefit as 1 operation vs. N (rows) ones. – Complain

@Complain If the original matrix is 1 dimensional ... aka a single row... is the data guaranteed to be continuous? Thanks for the SO Answer! – Serapis

@WilderField Single row matrix is guaranteed to be continuous, but not single col matrix when its data is borrowed from another big matrix (created out of a ROI of the big matrix). I have created a code snippet for better demonstration. – Complain

This won't work for colored images, at least not float type images with 3 channels (which I'm currently having trouble with) – Lerner

@Lerner Can you give a mimimum working example and show what's the actual case here? – Complain

@Complain I don't know how to easily create C++ drafts online, but just load a colored image, let's say, 100x100 into mat. Then do mat.total() and tell me what number you get. In my case I get 10000, and not 30000, which renders this method wrong, because somehow it's only using 1 channel. I tried exactly this method and always get 10000. – Lerner

@Lerner I know what you mean here. Yes, the original answer is assuming the mat is one-channel. I have refined the answer to work with multiple channels. Please check it out. – Complain

@Complain Ok, so to work for multiple channels you just multiply the mat.total() by the mat.channels(). Got it :) – Lerner

Answer 2

Can be done in two lines :)

Mat to array

uchar * arr = image.isContinuous()? image.data: image.clone().data;
uint length = image.total()*image.channels();

Mat to vector

cv::Mat flat = image.reshape(1, image.total()*image.channels());
std::vector<uchar> vec = image.isContinuous()? flat : flat.clone();

Both work for any general cv::Mat.

Explanation with a working example

    cv::Mat image;
    image = cv::imread(argv[1], cv::IMREAD_UNCHANGED);   // Read the file
    cv::namedWindow("cvmat", cv::WINDOW_AUTOSIZE );// Create a window for display.
    cv::imshow("cvmat", image );                   // Show our image inside it.

    // flatten the mat.
    uint totalElements = image.total()*image.channels(); // Note: image.total() == rows*cols.
    cv::Mat flat = image.reshape(1, totalElements); // 1xN mat of 1 channel, O(1) operation
    if(!image.isContinuous()) {
        flat = flat.clone(); // O(N),
    }
    // flat.data is your array pointer
    auto * ptr = flat.data; // usually, its uchar*
    // You have your array, its length is flat.total() [rows=1, cols=totalElements]
    // Converting to vector
    std::vector<uchar> vec(flat.data, flat.data + flat.total());
    // Testing by reconstruction of cvMat
    cv::Mat restored = cv::Mat(image.rows, image.cols, image.type(), ptr); // OR vec.data() instead of ptr
    cv::namedWindow("reconstructed", cv::WINDOW_AUTOSIZE);
    cv::imshow("reconstructed", restored);

    cv::waitKey(0);     

Extended explanation:

Mat is stored as a contiguous block of memory, if created using one of its constructors or when copied to another Mat using clone() or similar methods. To convert to an array or vector we need the address of its first block and array/vector length.

Pointer to internal memory block

Mat::data is a public uchar pointer to its memory.
But this memory may not be contiguous. As explained in other answers, we can check if mat.data is pointing to contiguous memory or not using mat.isContinous(). Unless you need extreme efficiency, you can obtain a continuous version of the mat using mat.clone() in O(N) time. (N = number of elements from all channels). However, when dealing images read by cv::imread() we will rarely ever encounter a non-continous mat.

Length of array/vector

Q: Should be row*cols*channels right?
A: Not always. It can be rows*cols*x*y*channels.
Q: Should be equal to mat.total()?
A: True for single channel mat. But not for multi-channel mat
Length of the array/vector is slightly tricky because of poor documentation of OpenCV. We have Mat::size public member which stores only the dimensions of single Mat without channels. For RGB image, Mat.size = [rows, cols] and not [rows, cols, channels]. Mat.total() returns total elements in a single channel of the mat which is equal to product of values in mat.size. For RGB image, total() = rows*cols. Thus, for any general Mat, length of continuous memory block would be mat.total()*mat.channels().

Reconstructing Mat from array/vector

Apart from array/vector we also need the original Mat's mat.size [array like] and mat.type() [int]. Then using one of the constructors that take data's pointer, we can obtain original Mat. The optional step argument is not required because our data pointer points to continuous memory. I used this method to pass Mat as Uint8Array between nodejs and C++. This avoided writing C++ bindings for cv::Mat with node-addon-api.

References:

• Create memory continuous Mat
• OpenCV Mat data layout
• Mat from array

Off

the "Mat to vector" suggestion will not work if the matrix is not continuous, because reshape will throw an exception. – Loisloise

Have you tried it and it failed? .clone() ensures you get a continuous flat array which you can reshape later. – Off

Yes, I did. You are right about clone returns a continuous array, but the point is that you've suggested to reshape the mat image BEFORE clone it, so if image is not continuous, it will throw an exception. Anyway, it helped me find the solution to my case. Thank you. – Loisloise

In my case .clone().reshape(...) solves the 'not continuous' exception. – Kenwood

Answer 3

Here is another possible solution assuming matrix have one column( you can reshape original Mat to one column Mat via reshape):

Mat matrix= Mat::zeros(20, 1, CV_32FC1);
vector<float> vec;
matrix.col(0).copyTo(vec);

Outspoken

Why this code cannot run normally? – Pecksniffian

@Pecksniffian img is of type double whereas vec is of type int. Also, vec is not 1D in your case. – Antepast

Answer 4

None of the provided examples here work for the generic case, which are N dimensional matrices. Anything using "rows" assumes theres columns and rows only, a 4 dimensional matrix might have more.

Here is some example code copying a non-continuous N-dimensional matrix into a continuous memory stream - then converts it back into a Cv::Mat

#include <iostream>
#include <cstdint>
#include <cstring>
#include <opencv2/opencv.hpp>

int main(int argc, char**argv)
{
    if ( argc != 2 )
    {
        std::cerr << "Usage: " << argv[0] << " <Image_Path>\n";
        return -1;
    }
    cv::Mat origSource = cv::imread(argv[1],1);

    if (!origSource.data) {
        std::cerr << "Can't read image";
        return -1;
    }

    // this will select a subsection of the original source image - WITHOUT copying the data
    // (the header will point to a region of interest, adjusting data pointers and row step sizes)
    cv::Mat sourceMat = origSource(cv::Range(origSource.size[0]/4,(3*origSource.size[0])/4),cv::Range(origSource.size[1]/4,(3*origSource.size[1])/4));

    // correctly copy the contents of an N dimensional cv::Mat
    // works just as fast as copying a 2D mat, but has much more difficult to read code :)
    // see https://mcmap.net/q/297380/-how-do-i-get-the-size-of-a-multi-dimensional-cv-mat-mat-or-matnd
    // copy this code in your own cvMat_To_Char_Array() function which really OpenCV should provide somehow...
    // keep in mind that even Mat::clone() aligns each row at a 4 byte boundary, so uneven sized images always have stepgaps
    size_t totalsize = sourceMat.step[sourceMat.dims-1];
    const size_t rowsize = sourceMat.step[sourceMat.dims-1] * sourceMat.size[sourceMat.dims-1];
    size_t coordinates[sourceMat.dims-1] = {0};
    std::cout << "Image dimensions: ";
    for (int t=0;t<sourceMat.dims;t++)
    {
        // calculate total size of multi dimensional matrix by multiplying dimensions
        totalsize*=sourceMat.size[t];
        std::cout << (t>0?" X ":"") << sourceMat.size[t];
    }
    // Allocate destination image buffer
    uint8_t * imagebuffer = new uint8_t[totalsize];
    size_t srcptr=0,dptr=0;
    std::cout << std::endl;
    std::cout << "One pixel in image has " << sourceMat.step[sourceMat.dims-1] << " bytes" <<std::endl;
    std::cout << "Copying data in blocks of " << rowsize << " bytes" << std::endl ;
    std::cout << "Total size is " << totalsize << " bytes" << std::endl;
    while (dptr<totalsize) {
        // we copy entire rows at once, so lowest iterator is always [dims-2]
        // this is legal since OpenCV does not use 1 dimensional matrices internally (a 1D matrix is a 2d matrix with only 1 row)
        std::memcpy(&imagebuffer[dptr],&(((uint8_t*)sourceMat.data)[srcptr]),rowsize);
        // destination matrix has no gaps so rows follow each other directly
        dptr += rowsize;
        // src matrix can have gaps so we need to calculate the address of the start of the next row the hard way
        // see *brief* text in opencv2/core/mat.hpp for address calculation
        coordinates[sourceMat.dims-2]++;
        srcptr = 0;
        for (int t=sourceMat.dims-2;t>=0;t--) {
            if (coordinates[t]>=sourceMat.size[t]) {
                if (t==0) break;
                coordinates[t]=0;
                coordinates[t-1]++;
            }
            srcptr += sourceMat.step[t]*coordinates[t];
        }
   }

   // this constructor assumes that imagebuffer is gap-less (if not, a complete array of step sizes must be given, too)
   cv::Mat destination=cv::Mat(sourceMat.dims, sourceMat.size, sourceMat.type(), (void*)imagebuffer);

   // and just to proof that sourceImage points to the same memory as origSource, we strike it through
   cv::line(sourceMat,cv::Point(0,0),cv::Point(sourceMat.size[1],sourceMat.size[0]),CV_RGB(255,0,0),3);

   cv::imshow("original image",origSource);
   cv::imshow("partial image",sourceMat);
   cv::imshow("copied image",destination);
   while (cv::waitKey(60)!='q');
}

Vigil

Answer 5

Instead of getting image row by row, you can put it directly to an array. For CV_8U type image, you can use byte array, for other types check here.

Mat img; // Should be CV_8U for using byte[]
int size = (int)img.total() * img.channels();
byte[] data = new byte[size];
img.get(0, 0, data); // Gets all pixels

Repand

Answer 6

byte * matToBytes(Mat image)
{
   int size = image.total() * image.elemSize();
   byte * bytes = new byte[size];  //delete[] later
   std::memcpy(bytes,image.data,size * sizeof(byte));
}

Stockton

While this code may answer the question, providing additional context regarding why and/or how this code answers the question improves its long-term value. – Maryalice

Answer 7

You can use iterators:

Mat matrix = ...;

std::vector<float> vec(matrix.begin<float>(), matrix.end<float>());

Dryly

Answer 8

cv::Mat m;
m.create(10, 10, CV_32FC3);

float *array = (float *)malloc( 3*sizeof(float)*10*10 );
cv::MatConstIterator_<cv::Vec3f> it = m.begin<cv::Vec3f>();
for (unsigned i = 0; it != m.end<cv::Vec3f>(); it++ ) {
    for ( unsigned j = 0; j < 3; j++ ) {
        *(array + i ) = (*it)[j];
        i++;
    }
}

Now you have a float array. In case of 8 bit, simply change float to uchar, Vec3f to Vec3b and CV_32FC3 to CV_8UC3.

Tundra

In case of this can you recreate the opencv mat with this: A = Mat(10, 10, CV_32FC3, &array); ? – Fibrin

I am not sure, but I think you can do it. I can check it later. So, you want to validate the conversion to the array that is created by cv::Mat m , by again converting them into Mat? – Tundra

Yes, I think I tried (but it was long ago, I am not sure) and it worked! – Fibrin

ok, the question mark in the end of your first comment looked like you were asking a question. – Tundra

Answer 9

If you know that your img is 3 channel, than you can try this code

 Vec3b* dados = new Vec3b[img.rows*img.cols];
    for (int i = 0; i < img.rows; i++)
        for(int j=0;j<img.cols; j++)
            dados[3*i*img.cols+j] =img.at<Vec3b>(i,j);

If you wanna check the (i,j) vec3b you can write

std::cout << (Vec3b)img.at<Vec3b>(i,j) << std::endl;
    std::cout << (Vec3b)dados[3*i*img.cols+j] << std::endl;

Gabrila

Answer 10

Since answer above is not very accurate as mentioned in its comments but its "edit queue is full", I have to add correct one-liners.

Mat(uchar, 1 channel) to vector(uchar):

std::vector<uchar> vec = (image.isContinuous() ? image : image.clone()).reshape(1, 1); // data copy here

vector(any type) to Mat(the same type):

Mat m(vec, false); // false(by default) -- do not copy data 

Strigil