Python Save a (Sparse) Matrix with a variable inside

Question

I have some matrices of decent size (2000*2000) and I wish to have symbolic expressions in the elements of the matrices - i.e. .9**b + .8**b + .7**b ... is an example of an element. The matrices are quite sparse.

I am creating these matrices by adding up intermediate calculations. I would like to store them to disk to be read in later and evaluated with different values of b.

I have played around with sympy and it does exactly what I need it to do however it is mind-numbingly slow to do simple additions. From what I have read it seems theano or tensorflow might be able to do this with Tensors but I could not figure out how to put a symbol in a Tensor.

Can anyone point me in the right direction as to the best tool to use for this task? I'd prefer it to be in python but if something outside python would do the job that'd be nice too.

Pippas

Are you using SparseMatrix? – Wigfall

Yes. I have tried Matrix and SparseMatrix. Both work but they are unusably slow :(. This is what I am trying to do: A_sy = sy.SparseMatrix(900,900,0) b = sy.abc.b A_sy += arr**b. arr in this case is a 900 by 900 numpy array. I have tried converting arr to a sparse format before doing the addition but it doesn't help. It is suprisingly slow - like a minute or two to do that addition. I would need to do thousands of such additions so it's too slow as is. – Pippas

@Wigfall if you are still thinking about this then let me just say don't worry about it - I was overthinking my problem and found a work around. I appreciate the attention and your work! – Pippas

@Pippas You should include your first comment within the question. (this comment will selfdestruct) – Millan

Answer 1

The problem is likely coming from the fact that you are taking a symbolic power. But, for whatever reason, SymPy tries to find an explicit form for a symbolic power. For example:

In [12]: x = Symbol('x')

In [13]: print(Matrix([[1, 2], [3, 4]])**x)
Matrix([[-2*(5/2 + sqrt(33)/2)**x*(-2/((-3/2 + sqrt(33)/2)*(-1/2 + sqrt(33)/6)**2*(sqrt(33)/4 + 11/4)) + 1/(-1/2 + sqrt(33)/6))/(-sqrt(33)/2 - 3/2) + 2*(-sqrt(33)/2 + 5/2)**x/((-3/2 + sqrt(33)/2)*(-1/2 + sqrt(33)/6)*(sqrt(33)/4 + 11/4)), -4*(5/2 + sqrt(33)/2)**x/((-3/2 + sqrt(33)/2)*(-1/2 + sqrt(33)/6)*(-sqrt(33)/2 - 3/2)*(sqrt(33)/4 + 11/4)) - 2*(-sqrt(33)/2 + 5/2)**x/((-3/2 + sqrt(33)/2)*(sqrt(33)/4 + 11/4))], [(5/2 + sqrt(33)/2)**x*(-2/((-3/2 + sqrt(33)/2)*(-1/2 + sqrt(33)/6)**2*(sqrt(33)/4 + 11/4)) + 1/(-1/2 + sqrt(33)/6)) - (-sqrt(33)/2 + 5/2)**x/((-1/2 + sqrt(33)/6)*(sqrt(33)/4 + 11/4)), 2*(5/2 + sqrt(33)/2)**x/((-3/2 + sqrt(33)/2)*(-1/2 + sqrt(33)/6)*(sqrt(33)/4 + 11/4)) + (-sqrt(33)/2 + 5/2)**x/(sqrt(33)/4 + 11/4)]])

Is this actually what you want to do? Do you know the value of b ahead of time? You can leave the expression unevaluated as a power by using MatPow(arr, b).

Wigfall