Eigen library --> initialize matrix with data from file or existing std::vector<string> content (c++)

Question

My question is how to initialize an eigen Matrix, but NOT this way:

matrix << 1,0,1,0,
          1,0,1,0,
          1,0,1,0,

I have a Matrix that looks like the above one ( commas or no commas doesnt matter) stored in a txt file.

I already wrote a function to read in each line and put it into a vector now I want to create a matrix with this data

But it doesn' work and I cant find any page that explains how to assign data to a matrix without writing just the values.(like the example above)

All I need is the data from my file in an eigen Matrix

What I tried so far: (PS: had the idea with the iterators but i guess it will take too long with really big matrices, I just tried this example with a 1-2 dimensional matrix)

int readFromFile (const char * path, vector <string> & mv)
{
    fstream file;
    string line;
    file.open(path);

    while (getline(file,line))
    {
        mv.push_back(line);
    }
    file.close();
    return 0;
}


typedef Matrix <int, 1, 2> MyMatrix;

int fromVectoEigen (vector<string> & source, MyMatrix & target)
{   //for (int i = source.size(); i<0 ; i--)
    //{

        string valuerow = source.back();
        string::iterator it = valuerow.begin();
        target.row(0)<< *it;
        target.row(0)<<*it+1;
        //source.pop_back();
    //}

    return 0;
}

Unfortunately cant just say Matrix.row(i) = vector.back() that doesnt work.

Underthrust

What do you mean by "it doesn't work"? – Depute

it's not possible it gives the following error: keine bekannte Umwandlung für Argument 1 von »std::string {aka std::basic_string<char>}« nach »const Eigen::BlockImpl<Eigen::Matrix<int, 4, 4>, 1, 4, false, Eigen::Dense>&« – Underthrust

This code is not giving the error. Somewhere you're calling a function where you're passing a string but it should in fact be a BlockImpl object. Can you show the code where the first argument you're passing to a function is a string? – Depute

yes its because i want to read in from file with getline im passing the line string ... – Underthrust

Answer 1

The following code works with files containing matrices of arbitrary size:

#include <iostream>
#include <fstream>
#include <string>
#include <Eigen/Dense>

using namespace std;
using namespace Eigen;

#define MAXBUFSIZE  ((int) 1e6)

MatrixXd readMatrix(const char *filename)
    {
    int cols = 0, rows = 0;
    double buff[MAXBUFSIZE];

    // Read numbers from file into buffer.
    ifstream infile;
    infile.open(filename);
    while (! infile.eof())
        {
        string line;
        getline(infile, line);

        int temp_cols = 0;
        stringstream stream(line);
        while(! stream.eof())
            stream >> buff[cols*rows+temp_cols++];

        if (temp_cols == 0)
            continue;

        if (cols == 0)
            cols = temp_cols;

        rows++;
        }

    infile.close();

    rows--;

    // Populate matrix with numbers.
    MatrixXd result(rows,cols);
    for (int i = 0; i < rows; i++)
        for (int j = 0; j < cols; j++)
            result(i,j) = buff[ cols*i+j ];

    return result;
    };

Regards.

Standfast

I dont need the eigen library anymore, but your answer is what I have been looking for! Thanks – Underthrust

@antonio L. When I call your function with the following statement: MatrixXd abc = readMatrix("C:\\data.txt");, I get an error message that the program stopped working. What could be wrong (I am calling it correctly, right?) – Fabiano

Can you paste the data.txt file? Thanks! – Standfast

@AntonioL. It's quite a large datafile, so I don't think it'd be handy to paste everything in here, but a typical line in it looks like this: -0.016514677 0.500312859 -0.005198081 -0.020745313 6.968308391 -0.001030603 -0.006184097 – Fabiano

The stringstream parsing seems to break when reading the last number -0.006184097. I printed the whole line of numbers as you pasted it, and found unicode characters at that number: -0.016514677 0.500312859 -0.005198081 -0.020745313 6.968308391 -0.001030603 -0.<U+200C><U+200B>006184097. When I delete them, the code seems to work correctly. Could you check that this is the problem? Regards. – Standfast

@AntonioL. Thanks for your reply :). How odd. I think that that is indeed the error then. I don't have time now, but I'll let you know if I can get it to work later. Do you have any idea how it could be that those unicode characters appear in there? I simply created the datafile by saving an excel file as a .txt. – Fabiano

@AntonioL. Sorry to say, but I tried with other data files and it also does not work then. For instance even with a simple data file such as 5 10 20 45 2.4 7 3.4 -1 I instantly get a message that the program has stopped working. – Fabiano

Hello @dreamer: sorry, but I cannot reproduce your error. I compiled the code (linux machine) and tested with your line. It worked ok for me. Maybe there is a problem with Windows. If you are still interested in solving the problem, what is the error you get when you execute the code? Regards. – Standfast

@AntonioL. Hmmh, that's weird. I constantly just get an error message that the program 'has stopped working', so I don't get any message from the compiler. But I wrote my own code to do this task, and that works now :)! But thanks anyway. – Fabiano

I think it is better to check all rows have the same number of columns. if (cols == 0) cols = temp_cols; else assert(cols == tmp_cols); – Coryza

Can't make it to work. I'm getting SO just after calling the method. – Category

Not working for me either! I'm getting SO just after calling the method. – Quesenberry

The reason why a stack overflow happens with this method is because of MAXBUFSIZE macro- the memory the function initially asks for is too large. If you change 1e6 to something lower, the code works just fine. – Newsmagazine

Answer 2

I just released an extension of the Eigen package that does some of what you want. It defines the >> operator, so you can say:

MatrixXd A(5,5); cin >> A;

It also lets you assign a VectorXd to equal a std::vector. The extended version of Eigen can be found here. However, it doesn't (yet) let you copy a std::vector into a MatrixXd object that isn't a vector. The functionality you want is the Map function in Eigen.

Lacagnia

This seems to fail if A is e.g. a MatrixXi – Farmyard

@Farmyard How so? I just re-tested it, and it appears to work fine with MatrixXi. – Lacagnia

It seems the link is broken ... – Blankbook

Answer 3

I think I found a solution! Its not fast or efficient but it works:

#include "topo.h"
#include <iostream>
#include <fstream>
#include <vector>
#include <Eigen/Dense>
#include <Eigen/Sparse>
#include <iterator>
#include <algorithm>


using namespace std;
using namespace Eigen;

/**Read data from File and store it in vector as string**/
int readFromFile (const char * path, vector <string> & mv) // muss vector vorher resized werden? wenn ja lese zeilenanzahl
{
    fstream file;
    string line;
    file.open(path);

    while (getline(file,line)) // lese zeile für zeile
    {
        mv.push_back(line); //fülle vector von hinten last in first
    }
    file.close();
    return 0;
}


typedef Matrix <int, 4, 4> MyMatrix; // Matrix später dynamisch

/**Parsing data to be used as Eigen Matrix**/
int fromVectoEigen (vector<string> & source, MyMatrix & target)
{   /**convert string to int and write it to the two dimensional array **/
    int array [4][4]; // noch resize nach vectorsize -->matrizen sind quadratisch
    int i = source.size();
    for ( i= i-1 ; i >= 0 ; i-- ) // da nur von hintern auf vector zugreifbar auch von hinten bei array anfangen
    {
        string myString = source.back(); // leztzes Element von Vector als String
        stringstream ssin(myString);
        int j = 0;
        while (ssin.good() && j < 4) // auch hier vectorsize später dynamisch
            {
            ssin >> array[j][i]; // fülle spalten in i.ter zeile
            ++j;
            }


        source.pop_back(); //lösche letztes element

    }
//        cout<<array[0][0]<<array[1][0]<<array[2][0]<<array[3][0]<<'\n';
//        cout<<array[0][1]<<array[1][1]<<array[2][1]<<array[3][1]<<'\n';
//        cout<<array[0][2]<<array[1][2]<<array[2][2]<<array[3][2]<<'\n';
//        cout<<array[0][3]<<array[1][3]<<array[2][3]<<array[3][3]<<'\n';
//
    /** from 2 dimensional array to one dimensional array**/
    int newarray [16]; // vectorsize * vectorsize
    int k = 0;
    for ( int i = 0 ; i< 4 ; i++) // vectorsize
    {   for (int j = 0 ; j<4; j++) // vectorsize
            {
                newarray[k]=array[j][i];
                k++;
            }

    }
    /**create Eigen Matrix from Array**/
    target= Map<Matrix4i>(newarray);
    target.transposeInPlace();
    cout<<target<<'\n';



return 0 ;
}

Underthrust

Answer 4

I used element-wise initialization (assuming we know nrows and ncols):

MatrixXf X = MatrixXf::Zero(nrows,ncols);
ifstream fin ("./data.txt");

if (fin.is_open())
{
    for (int row = 0; row < nrows; row++)
        for (int col = 0; col < ncols; col++)
        {
            float item = 0.0;
            fin >> item;
            X(row, col) = item;
        }
    fin.close();
}
cout << "X = " << endl << X << endl;

Relict

Answer 5

A variation using only Eigen::Map for mapping data from std::vector, based on examples from https://eigen.tuxfamily.org/dox/group__TutorialMapClass.html

#include <vector>
#include <Eigen/Dense>

std::vector<double> myStdVector;

// Insert code for filling myStdVector here
// ....

// Detect or set number of rows/columns
size_t numRows = 3;
size_t numCols = 7;

typedef Eigen::Map<Eigen::MatrixXd> Mapper;
Mapper myMatrix(&myStdVector.data()[0], numRows, numCols);

Frydman

Answer 6

Here is my solution:

#include <istream>
#include <string>
#include <sstream>
#include <vector>
#include <fstream>
#include <iostream>
#include <eigen>

using namespace std;
using namespace Eigen;

// load matrix from an ascii text file.
vector<vector<double>> LoadMatrix(istream* filePath, const string &delim = " \t")
{
    string line;
    string strnum;

    auto data = vector<vector<double>>();

    // clear first
    data.clear();

    // parse line by line
    while (getline(*filePath, line))
    {
        data.push_back(vector<double>());

        for (string::const_iterator i = line.begin(); i != line.end(); ++i)
        {
            // If i is not a delim, then append it to strnum
            if (delim.find(*i) == string::npos)
            {
                strnum += *i;
                if (i + 1 != line.end()) // If it's the last char, do not continue
                    continue;
            }

            // if strnum is still empty, it means the previous char is also a
            // delim (several delims appear together). Ignore this char.
            if (strnum.empty())
                continue;

            // If we reach here, we got a number. Convert it to double.
            double       number;

            istringstream(strnum) >> number;
            data.back().push_back(number);

            strnum.clear();
        }
    }
    return data;
}

Eigen::MatrixXd ConvertToEigenMatrix(std::vector<std::vector<double>> data)
{
    Eigen::MatrixXd eMatrix(data.size(), data[0].size());
    for (int i = 0; i < data.size(); ++i)
        eMatrix.row(i) = Eigen::VectorXd::Map(&data[i][0], data[0].size());
    return eMatrix;
}
MatrixXd LoadEigenMatrix(istream* filePath, const string &delim = " \t")
{
    auto data = LoadMatrix(filePath, delim);
    return ConvertToEigenMatrix(data);
}

Category

Answer 7

I used iterators to collect the data in a vector and then initialize the matrix. The conversion to the vector<double> seem to be the time-consuming part of the method, which approximately has the same speed as the solutions above. Ideas on how to improve this would be interesting.

template <class T>
using Tmat = Eigen::Matrix<T,Dynamic,Dynamic>;

Tmat<double> txt_to_mat(std::string path, int rows, int cols)
{
    std::ifstream fstr(path.c_str());
    std::vector<double> data_vec = std::vector<double>{
        std::istream_iterator<double>(fstr),
        std::istream_iterator<double>()
    };

    Tmat<double> mat(rows, cols);
    for(int i=0; i<rows; i++){
        for(int j=0; j<cols; j++){
            mat(i,j) = data_vec[i*cols + j];
        }
    }

    return mat;
}

Chaps

Answer 8

A templated stream operator can be used for reading all Eigen dense matrix types from a stream:

#include <Eigen/Dense>
#include <fstream>

template <typename Derived>
std::ifstream& operator>>(std::ifstream& in, Eigen::MatrixBase<Derived>& b)
{
  for(int row = 0; row < b.rows(); ++row)
    for(int col = 0; col < b.cols(); ++col)
      in >> b(row, col);

  return in;
}

This will instantiate for any of Eigen's dense matrix types. Here's an example reading camera intrinsics and extrinsics from a file for Eigen::Matrix3d and Eigen::Vector3d using the above templated stream operator:

std::ifstream input(krtPath);

Eigen::Matrix3d K, R;
Eigen::Vector3d t;

input >> K >> R >> t;

Eigen::Vector3d C = -R.transpose() * t;

Ecumenicist