How to write/read an Eigen matrix from binary file

Question

To write Eigen::Matrix to file I really like to use the following:

typedef Eigen::Matrix<Scalar, Eigen::Dynamic, Eigen::Dynamic> Matrix_MxN;
Matrix_MxN J = Matrix_MxN::Zeros(10,10);      
std::ofstream("matrix.txt") << J;

But unfortunately, something that can do the opposite is not defined:

std::ifstream("matrix.txt") >> J;

To circumvent this problem, how can you read/write an Eigen::Matrix to binary file instead?

Inspissate

You can also use a serialization library, e.g. Cereal. See: #22884716 – Okwu

Answer 1

You can define these methods:

namespace Eigen{
template<class Matrix>
void write_binary(const char* filename, const Matrix& matrix){
    std::ofstream out(filename, std::ios::out | std::ios::binary | std::ios::trunc);
    typename Matrix::Index rows=matrix.rows(), cols=matrix.cols();
    out.write((char*) (&rows), sizeof(typename Matrix::Index));
    out.write((char*) (&cols), sizeof(typename Matrix::Index));
    out.write((char*) matrix.data(), rows*cols*sizeof(typename Matrix::Scalar) );
    out.close();
}
template<class Matrix>
void read_binary(const char* filename, Matrix& matrix){
    std::ifstream in(filename, std::ios::in | std::ios::binary);
    typename Matrix::Index rows=0, cols=0;
    in.read((char*) (&rows),sizeof(typename Matrix::Index));
    in.read((char*) (&cols),sizeof(typename Matrix::Index));
    matrix.resize(rows, cols);
    in.read( (char *) matrix.data() , rows*cols*sizeof(typename Matrix::Scalar) );
    in.close();
}
} // Eigen::

and you can test their usage with:

typedef Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic, Eigen::ColMajor> Matrix_MxN; 
Matrix_MxN J = Matrix_MxN::Random(10,5);
Eigen::write_binary("matrix.dat",J);
std::cout << "\n original \n" << J << std::endl;
Matrix_MxN J_copy;
Eigen::read_binary("matrix.dat",J_copy);
std::cout << "\n copy \n" << J_copy << std::endl;
cout.flush();'

If you know of a better way, suggestions are welcome!

Inspissate

What if you were reading from an edge file? The edge file contains source and destination id pairs, assuming the file is sorted by source id. Is there an elegant way read a file like this and create a matrix in Eigen? – Reservist

The sub-question is a bit OT. But there is something called "triplets" loading for matrices in Eigen (i index, j index, and value) for sparse matrices. – Inspissate

Notice that if you're using Eigen with another library which uses it (like OpenCV) this is gonna generate a lot of compile errors. In general it's a not a good idea inject new functions in a library namespace. – Earthshaker

This doesnt work if your Matrix has a null terminator equivalent binary – Claimant

Answer 2

modifications to avoid intruding namespace Eigen, using zlib for bigger Matrices.

#include <string>
#include <zlib.h>
#include <Eigen/Core>

namespace myEigen{
template<class Derived>
void write_binary(const std::string &filename,
    const Eigen::PlainObjectBase<Derived> &matrix)
{
    typedef typename Derived::Index Index;
    typedef typename Derived::Scalar Scalar;

    gzFile out = gzopen(filename.c_str(), "wb");
    Index rows=matrix.rows(), cols=matrix.cols();

    gzwrite(out, (char*) (&rows), sizeof(Index));
    gzwrite(out, (char*) (&cols), sizeof(Index));
    gzwrite(out, (char*) matrix.data(), rows*cols*sizeof(Scalar) );
    gzclose(out);
}

template<class Derived>
void read_binary(const std::string &filename,
    Eigen::PlainObjectBase<Derived> &matrix)
{
    typedef typename Derived::Index Index;
    typedef typename Derived::Scalar Scalar;

    gzFile in = gzopen(filename.c_str(), "rb");
    Index rows=0, cols=0;
    gzread(in, (char*) (&rows),sizeof(Index));
    gzread(in, (char*) (&cols),sizeof(Index));
    matrix.resize(rows, cols);
    gzread(in, (char*) matrix.data(), rows*cols*sizeof(Scalar) );
    gzclose(in);
}
} // myEigen::

Faradism

Add some explanation rather than pure code would be perferred – Francoise