Matrix with unknown number of rows and columns Eigen library

Question

I want to read data from a file into a matrix in Eigen. I have coded everything but there is one problem that I encounter. I don't know yet beforehand how many data points are in the file, so I want to be able to initialize a matrix without specifying its size. I know that the following way of intializing a matrix works in Eigen:

MatrixXd A;

But now if I then do for instance

A << 1, 2,
     4, 7;

It doesn't work. I had hoped that it would recognise it as a 2x2 matrix in this example, so that I could then work with it. So basically my question is, how can I add data to A, without having to specify its size?

Tellez

What do you mean it doesn't work? Here you have an identical example in the documentation. – Epigone

@Epigone No, there they specify the size: a 3x3 matrix. Doesn't work means, that it does not accept that as valid syntax; in other words, the program crashes. – Tellez

Have you clicked the here? Matrix3f m; m << 1, 2, 3, 4, 5, 6, 7, 8, 9; std::cout << m; that is the code. Can you tell what do you mean by doesn't work. – Epigone

@Epigone Yes, of course. Then which example do you mean? The example there intializes with Matrix3f, that's not the same, the 3 already specifies the size. – Tellez

@Epigone The difference is that they directly specify that it is a 3x3 matrix through Matrix3f. I don't want to do that. If you look at my syntax, I specified MatrixXd so I have an 'X' where they have a '3' indicating that the size is unknown. – Tellez

OK, my bad. But it is impossible to do what you ask about. – Epigone

@Epigone It is possible, look at the answer below. – Tellez

Answer 1

If what you want is read data from a file which does not specify the matrix size explicitly, then I would recommend to push back the entries in a std::vector and at the end of the parsing copy the data from the std::vector using Map:

MatrixXf A;
std::vector<float> entries;
int rows(0), cols(0);
while(...) { entries.push_back(...); /* update rows/cols*/ }
A = MatrixXf::Map(&entries[0], rows, cols);

This will be much more efficient than calling conservativeResize every times.

Midway

That sounds really interesting! Could you maybe show me some more elaborate code on how to do that (because you have the dots now)? If this works that sounds like a really great way! – Tellez

Just replace the line with the dots with your own code parsing the file. You said that you already implemented it. – Midway

Yes, but I just tried and there is somehow something still not working when I read in a data file. Could I post my code in the OP so that you can look at it? Thanks a lot in advance. – Tellez

I've added my code in the OP. If you could look at it, that'd be great. An example of a line of data that I'd like to read in from a file is -0.016514677 0.500312859 -0.005198081 -0.020745313 6.968308391 -0.001030603 -0.006184097 – Tellez

The problems that I currently have are the following: (1) For values such as the one I posted in the comment above I get all zeros (it works for integer values though). (2) The map class reads in the data in the wrong 'order': I want the data in the eigen matrix to have the exact same shape as that in the data file, but now it reads in the lines vertically (column-major order I think its called). Do you know what I could do to change this? – Tellez

If your data is row-major, then map it as a row-major matrix: Map<Matrix<float,Dynamic,Dynamic,RowMajor> >(&entries[0], rows, cols); – Midway

You declared xas a int. It should be a float. – Midway

First of all, thanks a lot! You're a hero! Everything works fine now! The only problem that I have still is that, even when I use long double everywhere, when I print out the matrix it still shows that it hasn't saved all digits. (The data is the same as that in a few comments above). Do you have any clue how that could be possible? – Tellez

By default, all digits are not printed out. Check the c++ doc: cplusplus.com/reference/ios/ios_base/precision – Midway

Answer 2

From the Eigen Tutioral about Matrix

---

Of course, Eigen is not limited to matrices whose dimensions are known at compile time. The RowsAtCompileTime and ColsAtCompileTime template parameters can take the special value Dynamic which indicates that the size is unknown at compile time, so must be handled as a run-time variable. In Eigen terminology, such a size is referred to as a dynamic size; while a size that is known at compile time is called a fixed size. For example, the convenience typedef MatrixXd, meaning a matrix of doubles with dynamic size, is defined as follows:

typedef Matrix<double, Dynamic, Dynamic> MatrixXd;

And similarly, we define a self-explanatory typedef VectorXi as follows:

typedef Matrix<int, Dynamic, 1> VectorXi;

You can perfectly have e.g. a fixed number of rows with a dynamic number of columns, as in:

Matrix<float, 3, Dynamic>

---

Here is an example I just quickly made:

Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic> myMatrix;
myMatrix.resize(1, 1);
myMatrix(0, 0) = 1.0;
myMatrix.resize(2, 2);
myMatrix(1, 1) = 1.0;
myMatrix.resize(3, 3);
myMatrix(2, 2) = 1.0;
myMatrix.resize(4, 4);
myMatrix(3, 3) = 1.0;

Kun

Thanks. Yes I read that too, but then now how do I fill the matrix that I defined as MatrixXd A; as in my example? – Tellez

@Tellez The tutiroal says to create it like this Matrix<float, 3, Dynamic> – Kun

Could you show me how I could do it for the example that I listed in the OP then for instance? Because I tried that but it still doesn't work for me, so I think I am missing something. – Tellez

Ah, that's nice :), that's what I needed. Thanks a lot :)! – Tellez

Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic> is not needed it is MatixXf. – Epigone

@Epigone True, but that is just a typdef that does the same thing. It is less typing though. – Kun

Answer 3

So I had similar (but not exactly the same problem) and thanks to this plot and answers to another problems from this forum I created that code. I have a file with unknown matrix size as an input, which I need to operate on later.

We first read the first line of the file, get the data, put into vector, read the number of columns. Then we read all the rest data from the file, continue parsing to the vector.

void getTimeSeries(const char* TimeSeriesFileName1)
{

std::ifstream ifile(TimeSeriesFileName1, std::ios::in);
std::vector<double> LineOfTimeSeriesFile;
std::string line;
int rows(0), cols(0);
double number;

//get number of columns you have to read the first line
if (ifile.good())
{
  std::getline(ifile, line);
  std::stringstream ss(line);
  while (ss>>number) 
  {
    ++cols;
    LineOfTimeSeriesFile.push_back(number);
  }
  std::cout << cols << std::endl;
}
// consume and discard token from stream.
if (ifile.fail())
{
  ifile.clear();
  std::string token;
  ifile >> token;
}

//Read all the data into a one big vector
do
{
    // read as many numbers as possible.
    for (number = 0; ifile >> number;) {
        LineOfTimeSeriesFile.push_back(number);
    }
    // consume and discard token from stream.
    if (ifile.fail())
    {
        ifile.clear();
        std::string token;
        ifile >> token;
    }
}
while (!ifile.eof());

//get the number of rows
rows = LineOfTimeSeriesFile.size()/cols;
for (std::size_t i=0; i < LineOfTimeSeriesFile.size(); ++i) 
{
    std::cout << LineOfTimeSeriesFile[i] << std::endl;
    std::cout << "Rows: " << cols << ", Cols: " << rows << ", points: " << LineOfTimeSeriesFile.size() << std::endl;
}    

//Here we map the matrix
timeSeriesMatrix = MatrixXd::Map(&LineOfTimeSeriesFile[0], rows, cols);

I would be happy to see some review of this code in terms of efficiency, but I think it is currently OK. Input files are matrices "tab/space"-separated.

Rattat

Answer 4

It is impossible to do what you ask for. How do you want library to guess the shape of the array? How can it be sure its not 4x1 or 1x4. New lines do not matter for c++ compilers, so your code is the same as m << 1,2,3,4;.

The dynamic arrays are dynamic in the sense their size is not know at a compile time but run-time. However, you still need to set the size/shape of the matrix before putting elements in, because the library needs to know how much space is needs to allocate for the matrix.

By default Matrix m; is of a size 0x0. You need to resize it before putting in elements.

Eg.

#include<Eigen/Core>
#include<iostream>

int main(){
Eigen::MatrixXd m;
m.resize(2,2);
m << 1, 2, 3, 4;
std::cout << m << '\n';
m.resize(1,4);
std::cout << m << '\n';
}

Epigone