Matrix of unknown length in MATLAB?

Question

I'm trying to set up a zero matrix of variable length with two columns into which I can output the results of a while loop (with the intention of using it to store the step data from Euler's method with adjusted time-steps). The length will be determined by the number of iterations of the loop.

I'm wondering if there's a way I can do this while I'm running the loop or whether I need to set it up to begin with, and how to go about doing that.

Stiles

Also, if this is for a class assignment and you need to show the iterations; you can use sprintf within your Euler implementation. – Vancevancleave

Another related question: Appending a vector to an empty MATLAB matrix – Simarouba

Answer 1

if the number of columns is fixed you can always add rows to your matrix (inside the loop)

e.g.

while (....)
   .....
   new_row =[x y] ; % new row with values x & y
   mat = [mat ; new_row]; 

of course if you know the number of iterations before the while loop it's more efficient to pre-allocate the matrix

Pehlevi

Thank you very much! This makes sense to me. You think for a programming unit they'd teach us some of it but instead they throw us to the wolves. Thanks for saving me :) – Stiles

Using alternate syntax for the last line of the code above makes it more explicit that you're extending the matrix: mat(end+1,:) = new_row; – Lorenlorena

See some speed tips on #49256809 – Offload

Answer 2

Another approach that has performance in mind while still trying to be space-efficient, is to preallocate memory in large batches, adding more batches as needed. This is well suited if you have to add a large number of items without knowing how many beforehand.

BLOCK_SIZE = 2000;                          % initial capacity (& increment size)
listSize = BLOCK_SIZE;                      % current list capacity
list = zeros(listSize, 2);                  % actual list
listPtr = 1;                                % pointer to last free position

while rand<1-1e-5                           % (around 1e5 iterations on avrg)
  % push items on list
  list(listPtr,:) = [rand rand];            % store new item
  listPtr = listPtr + 1;                    % increment position pointer

  % add new block of memory if needed
  if( listPtr+(BLOCK_SIZE/10) > listSize )  % less than 10%*BLOCK_SIZE free slots
    listSize = listSize + BLOCK_SIZE;       % add new BLOCK_SIZE slots
    list(listPtr+1:listSize,:) = 0;
  end
end
list(listPtr:end,:) = [];                   % remove unused slots

---

EDIT: As a time comparison, consider the following cases:

1. The same code as above done for 50000 iterations.
2. Preallocating the entire matrix beforehand: list = zeros(50000,2); list(k,:) = [x y];
3. Dynamically adding vectors to matrix: list = []; list(k,:) = [x y];

On my machine, the results were:

1) Elapsed time is 0.080214 seconds.
2) Elapsed time is 0.065513 seconds.
3) Elapsed time is 24.433315 seconds.

---

Update:

Following discussions in the comments, I've rerun some tests using the latest R2014b release. The conclusion is that recent versions of MATLAB has greatly improved the performance of automatic array growth!

However there is a catch; the array must be growing across the last dimension (columns in the case of 2D matrices). That's why appending rows like originally intended is still too slow without preallocation. This is where the above proposed solution can really help (by extending the array in batches).

See here for the full set of tests: https://gist.github.com/amroamroamro/0f104986796f2e0aa618

Simarouba

woohoo! an insightful point + measurements to corroborate it. thanks. – Gorgeous

p.s. most variable-size methods (like string classes) don't use a fixed block size, but rather increase the size by a multiplicative factor K (usually K=2). This bounds the # of allocation steps to O(log N), and if you care about memory efficiency you can always pick K = 1.2 or 1.1 and deal with the math calculation hit to trade off efficiency / # of allocation steps. – Gorgeous

you're probably right.. you could easily modify the code to do such. A number of parameters can also be tuned: when to increase size, by how much, perhaps even a growing factor (start at K=1.1 and increase up to 2) – Simarouba

I have a question regarding this, can this be converted to a class? – Shermanshermie

@masad: of course you can! I've seen a few submissions on the File Exchange that does something similar. Here's one by John D'Errico which he briefly discussed here: growdata/growdata2. Internally it's implemented differently than mine, but you could mimic the interface. – Simarouba

One last thing to mention; MATLAB has greatly improved the performance of automatic array growth in recent versions: blogs.mathworks.com/steve/2011/05/20/…, https://mcmap.net/q/537959/-matlab-appending-to-pre-allocated-matrix. So the naive approach is not as bad as it used to be.. – Simarouba

@amro Many thanks for your reply. I have written a class using matlab handles, however its much slower than your method mentioned above. Apart from defining the class as properties and methods I am directly copying your code. It completes in 4.0 secs as compared to 0.05 second for directly using your code above. Here is the link to my implementation: github.com/masad801/dynamicArray any thoughts on this would really help! – Shermanshermie

@masad: I've done some tests of my own. Here is the full code: gist.github.com/amroamroamro/0f104986796f2e0aa618 (see my remarks in the README file) – Simarouba

Answer 3

MATLAB uses dynamic typing with automatic memory management. This means, you don't need to declare a matrix of a fixed size before using it - you can change it as you go along and MATLAB will dynamically allocate memory for you.

BUT it is way more efficient to allocate memory for the matrix first and then use it. But if your programs needs this kind of flexibility, go for it.

I'm guessing you need to keep appending rows to your matrix. The following code should work.

Matrix = [];

while size(Matrix,1) <= 10
    Matrix = [Matrix;rand(1,2)];
end

disp(Matrix);

Here, we're dynamically reallocating the space required for Matrix every time you add a new row. If you know beforehand, say, an upper bound on the number of rows you're going to have, you can declare Matrix = zeros(20,2) and then insert each row into the matrix incrementally.

% Allocate space using the upper bound of rows (20)
Matrix = zeros(20,2);
k = 1;
for k = 1:10
   Matrix(k,:) = rand(1,2);
end
% Remove the rest of the dummy rows
Matrix(k+1:end,:) = [];

Disseisin

+1 I use this all the time. Note that you can also just use a counter, and Matlab will grow the array. – Vancevancleave

I'm starting to see what you're doing and why it's efficient. Very helpful, thank you. – Stiles

Answer 4

Another flavor of the same thing that Jacob posted.

for counter = 1:10
    Matrix(counter,:) = rand(1,2);
end
disp(Matrix);

One "nice" thing about this is you can guess a minimum size to help the performance along.

This might be of interest as well: http://www.mathworks.com/help/matlab/math/resizing-and-reshaping-matrices.html#f1-88760

Vancevancleave