cv::Mat conversion to Eigen-Matrix and back

Question

I have several feature vectors stored in a cv::Mat where, each row is a feature vector (several rows like this one here: [ x1 y1 x2 y2 x3 y3.... ]). I have to apply SVD on each feature vector and for that I use Eigen library. But, before applying SVD the feature matrix has to be converted to Eigen::Matrix form.

Later, I have to convert the SVD result back to cv::Mat.

Could anyone please suggest a nice way to do this? The reason I need it in cv::Mat form is because I have to input it to a Neural Network in OpenCV and only cv::Mat inputs matrices are allowed.

Thanks!!!

Hoag

possible duplicate of OpenCV CV::Mat and Eigen::Matrix. No need to copy the data, use Eigen::Map. Look at the answer here: #14783829 – Insolate

Answer 1

An example from http://forum.kde.org/viewtopic.php?f=74&t=97516:

#include <opencv2/core/eigen.hpp>
cv::Mat_<float> a = Mat_<float>::ones(2,2);
Eigen::Matrix<float,Dynamic,Dynamic> b;
cv2eigen(a,b);

Also, OpenCV CV::Mat and Eigen::Matrix has a solution using an Eigen::Map.

Neona

Answer 2

Try this code for eigen to cv:

template<typename _Tp, int _rows, int _cols, int _options, int _maxRows, int _maxCols>
void eigen2cv(const Eigen::Matrix<_Tp, _rows, _cols, _options, _maxRows, _maxCols>& src, cv::Mat& dst)
{
    if (!(src.Flags & Eigen::RowMajorBit))
    {
        cv::Mat _src(src.cols(), src.rows(), cv::DataType<_Tp>::type,
            (void*)src.data(), src.stride() * sizeof(_Tp));
        cv::transpose(_src, dst);
    }
    else
    {
        cv::Mat _src(src.rows(), src.cols(), cv::DataType<_Tp>::type,
            (void*)src.data(), src.stride() * sizeof(_Tp));
        _src.copyTo(dst);
    }
}

As you can see this performs a copy. With a matrix that small you shouldn't care, but you could change the code. to get the first column, use cv::Mat::column().

Try one of these methods for cv to eigen:

template<typename _Tp, int _rows, int _cols, int _options, int _maxRows, int _maxCols>
void cv2eigen( const Mat& src,
               Eigen::Matrix<_Tp, _rows, _cols, _options, _maxRows, _maxCols>& dst )
{
    CV_DbgAssert(src.rows == _rows && src.cols == _cols);
    if( !(dst.Flags & Eigen::RowMajorBit) )
    {
        Mat _dst(src.cols, src.rows, DataType<_Tp>::type,
                 dst.data(), (size_t)(dst.stride()*sizeof(_Tp)));
        if( src.type() == _dst.type() )
            transpose(src, _dst);
        else if( src.cols == src.rows )
        {
            src.convertTo(_dst, _dst.type());
            transpose(_dst, _dst);
        }
        else
            Mat(src.t()).convertTo(_dst, _dst.type());
        CV_DbgAssert(_dst.data == (uchar*)dst.data());
    }
    else
    {
        Mat _dst(src.rows, src.cols, DataType<_Tp>::type,
                 dst.data(), (size_t)(dst.stride()*sizeof(_Tp)));
        src.convertTo(_dst, _dst.type());
        CV_DbgAssert(_dst.data == (uchar*)dst.data());
    }
}

template<typename _Tp>
void cv2eigen( const Mat& src,
               Eigen::Matrix<_Tp, Eigen::Dynamic, Eigen::Dynamic>& dst )
{
    dst.resize(src.rows, src.cols);
    if( !(dst.Flags & Eigen::RowMajorBit) )
    {
        Mat _dst(src.cols, src.rows, DataType<_Tp>::type,
             dst.data(), (size_t)(dst.stride()*sizeof(_Tp)));
        if( src.type() == _dst.type() )
            transpose(src, _dst);
        else if( src.cols == src.rows )
        {
            src.convertTo(_dst, _dst.type());
            transpose(_dst, _dst);
        }
        else
            Mat(src.t()).convertTo(_dst, _dst.type());
        CV_DbgAssert(_dst.data == (uchar*)dst.data());
    }
    else
    {
        Mat _dst(src.rows, src.cols, DataType<_Tp>::type,
                 dst.data(), (size_t)(dst.stride()*sizeof(_Tp)));
        src.convertTo(_dst, _dst.type());
        CV_DbgAssert(_dst.data == (uchar*)dst.data());
    }
}


template<typename _Tp>
void cv2eigen( const Mat& src,
               Eigen::Matrix<_Tp, Eigen::Dynamic, 1>& dst )
{
    CV_Assert(src.cols == 1);
    dst.resize(src.rows);

    if( !(dst.Flags & Eigen::RowMajorBit) )
    {
        Mat _dst(src.cols, src.rows, DataType<_Tp>::type,
                 dst.data(), (size_t)(dst.stride()*sizeof(_Tp)));
        if( src.type() == _dst.type() )
            transpose(src, _dst);
        else
            Mat(src.t()).convertTo(_dst, _dst.type());
        CV_DbgAssert(_dst.data == (uchar*)dst.data());
    }
    else
    {
        Mat _dst(src.rows, src.cols, DataType<_Tp>::type,
                 dst.data(), (size_t)(dst.stride()*sizeof(_Tp)));
        src.convertTo(_dst, _dst.type());
        CV_DbgAssert(_dst.data == (uchar*)dst.data());
    }
}


template<typename _Tp>
void cv2eigen( const Mat& src,
               Eigen::Matrix<_Tp, 1, Eigen::Dynamic>& dst )
{
    CV_Assert(src.rows == 1);
    dst.resize(src.cols);
    if( !(dst.Flags & Eigen::RowMajorBit) )
    {
        Mat _dst(src.cols, src.rows, DataType<_Tp>::type,
                 dst.data(), (size_t)(dst.stride()*sizeof(_Tp)));
        if( src.type() == _dst.type() )
            transpose(src, _dst);
        else
            Mat(src.t()).convertTo(_dst, _dst.type());
        CV_DbgAssert(_dst.data == (uchar*)dst.data());
    }
    else
    {
        Mat _dst(src.rows, src.cols, DataType<_Tp>::type,
                 dst.data(), (size_t)(dst.stride()*sizeof(_Tp)));
        src.convertTo(_dst, _dst.type());
        CV_DbgAssert(_dst.data == (uchar*)dst.data());
    }
}

Source: This code is taken from OpenCV itself, they use it internally as OpenCV can use libeigen for some tasks internally. I don't understand why format conversions to such libs and Qt are not exposed through the API.

Indole

thanks for the suggestion! could you please tell me what is the "template" thing here? I am not yet at advanced level... and also how do I convert cv::Mat to Eigen::Matrix....thank you for your inputs!!! @Indole – Hoag

and what is this "typename _Tp" and DataType<_Tp> Please explain a little more....its very important... – Hoag

You need to learn templates yourself. But using this code is rather easy and does not involve templates. E.g.: cv::Mat_<float> src; Eigen::MatrixXf dest(src.rows, src.cols); cv2eigen(src, dest); – Indole

I'll try this! Please also tell me a little on what is this "typename _Tp" and DataType<_Tp>....I don't understand... Thank you... @Indole – Hoag

Just read your favorite book about C++ templates. It is very basic C++ concept! About DataType see docs.opencv.org/modules/core/doc/basic_structures.html – Indole

Alright...I will go through some text. And any specific header files to use?? Other than Eigen/Dense ? @Indole – Hoag

If there would be any I would've stated it. Obviously you need using namespace cv; and the OpenCV core include. – Indole

Answer 3

Take a look on Mapping data from Eigen to OpenCV and back article. It describe how to map data with less overhead. In the simplest case there would be no copy at all. It also deal with Eigen expressions as well:

// Unsharp mask
Eigen::ArrayXXd img, blur;    
eigen2cv(img) = cv::imread("lena.jpg");
cv::GaussianBlur(eigen2cv(img), eigen2cv(blur));

cv::imshow("sharpened", eigen2cv(1.5 * img - 0.5 * blur));

Adenovirus

They the new header file you have been talking in your blog does not work with OpenCV 2.4.9. I just included the header Eigen2CV.h and g++ throws a bunch of weird errors. – Goddaughter