Symbolic vs Numeric Math - Performance

Question

Do symbolic math calculations (especially for solving nonlinear polynomial systems) cause huge performance (calculation speed) disadvantage compared to numeric calculations? Are there any benchmark/data about this?

Found a related question: Symbolic computation vs. numerical computation

Another one: Computational Efficiency of Forward Mode Automatic vs Numeric vs Symbolic Differentiation

Plasma

Where do you see areas where there is a significant overlap of both techniques? That is, the same questions with the same expectation to the form and accuracy of the results? – Goggleeyed

@LutzL, Solving nonlinear polynomial systems, for example. – Plasma

What kind of result? Enumerating all solutions in whatever encoding, finding one solution, finding one/all solutions in a specified area? And what numerical methods do you propose that work reliable in any or all of these cases? – Goggleeyed

@LutzL In my case, finding one solution I guess. But I'm looking for a generic answer. If you could explain in which cases symbolic math is disadvantageous and in which cases not, it would be great. For example, the relation of performance to problem size, type etc. – Plasma

The question is too broad. Please narrow down the scope in the question itself, not only in comments. – Marketplace

@PeterG. Well I need a generic answer, but OK, I edited the question, now it's limited to nonlinear polynomial systems. – Plasma

Out of curiosity it seems you are asking an either or question, e.g. either symbolic or numeric, have you considered a hybrid approach? – Timepleaser

@GuyCoder I want to know how expensive is it to use symbolic instead of numeric, if the difference is little I can go ahead with symbolic, if there is a huge difference, then no. And about the hybrid approach, no, I haven't. What do you suggest? – Plasma

In the world of neural networks one has to be able to calculate the derivative, however if a derivate can be simplified before calculating then the cost of calculating goes down. Since simplifying the derivative is a one time action while the cost of calculating occurs thousands to millions of times, the simplification is done symbolically and then the calculation is done numerically. One software package I know that does it this way is Theano. I don't believe that Theano works on nonlinear polynomial systems but there might be something out there. – Timepleaser

@GuyCoder Sounds great, thank you. – Plasma

Of interest: Symbolic-numeric computation If that solves your problem as opposed to answering your question then I will post it as an answer. – Timepleaser

@GuyCoder Yes, it is absolutely helpful. You should post an answer. – Plasma

This question would be much better with any code examples in any language. It is way too broad. – Miquelmiquela

Another example of where a problem is first partially solved without an exact value then solved for exact values is with constraint stratification in logic languages, e.g. Constraint Logic Programming. – Timepleaser

Since this can be considered a subjective question I won't post it as a formal question but if someone knows where differential equations are being simplified symbolically then solved numerically with something like fourth order Runge-Kutta (Wikipedia) (fourth order Runge-Kutta) I would be interested in a ping back with an answer. A use for such would be in simulations. – Timepleaser

Answer 1

I am the individual who answered the Scicomp question you reference in your question. I personally am not aware of any empirical metrics performed to compare run-time performance for symbolic versus numerical solutions to systems of polynomial equations.

However, it should be fairly intuitive that symbolic solutions will have a bit more overhead for most aspects of solving the problem due to things such as manipulation of terms in the equation symbolically, searching how to simplify/rearrange equations to make them easier to solve, searching through known closed form solutions, etc. One major issue with symbolic solvers is that you may not have a closed form solution you can find and use, so solving it numerically would have to happen either way.

The only way I can see symbolic solvers outperforming numerical solutions in terms of run-time is if the symbolic solver can quickly enough recognize your problem as one with a known analytical solution or if it arrives at the solution eventually while the numerical solver never does (aka it diverges).

Given you can find a numerical solver that converges, I think the numerical case will generally be much more efficient since there's just much less overhead to make progress in refining your solution. Since you mention solving systems of polynomial equations, I suspect there are also some tailored algorithms for your type of problem that may be superior to typical nonlinear equation solving schemes.

Mundt

Both answers are equally helpful, so I want to split the bounty between two answers. But it seems no way to do this. :/ – Plasma

I decided to award the bounty to you as you have less reputation than @Guy Coder. And I am not accepting any answer to encourage future answers. Thanks again. – Plasma

Answer 2

This is not a direct answer to the question but a suggested course correction.

While it is possible to evaluate math expressions in a purely numeric means or in a purely symbolic means, it is also possible to use a hybrid approach.

This is know as Symbolic-numeric computation

Maple is one software package that has this ability.
Note: I have never used Maple so I can't add more.

Searching for packages

I find I get better results when searching for math packages that use symbolic-numeric computation by searching for the name of the package combined with Symbolic-numeric computation, e.g.

wolfram symbolic-numeric computation

A specific example related to neural networks

In the world of neural networks one has to be able to calculate the derivative, however if a derivate can be simplified before calculating then the cost of calculating goes down. Since simplifying the derivative is a one time action while the cost of calculating occurs thousands to millions of times, the simplification is done symbolically and then the calculation is done numerically. Theano is a software package that does this specifically for use with neural networks.

Timepleaser

Both answers are equally helpful, so I want to split the bounty between two answers. But it seems no way to do this. :/ – Plasma