Ordering operation to maximize double precision

Question

I'm working on some tool that gets to compute numbers that can get close to 1e-25 in the worst cases, and compare them together, in Java. I'm obviously using double precision.

I have read in another answer that I shouldn't expect more than 1e-15 to 1e-17 precision, and this other question deals with getting better precision when ordering operations in a "better" order.

Which double precision operations are more keen to loose precision along the way? Should I try to work with number as big as possible or as small as possible? Do divisions first before multiplications?

I'd rather not use the BigDecimal classes or equivalent, as the code is already slow enough ;) (unless they don't impact speed too much, of course).

Any information will be greatly appreciated!

EDIT: The fact that numbers are "small" in absolute value (1e-25) does not matter, as double can go down to 1e-324. But what matters is that, when they are very similar (both in 1e-25), I have to compare, let's say 4.64563824048517606458e-21 to 4.64563824048517606472e-21 (difference is the 19th and 20th digits). When computing these numbers, the difference is so small that I might hit the "rounding error", where remainder is filled with random numbers.

The question is: "how to order computation so that this loss of precision is minimized?". It might be doing divisions before multiplications, or additions first.

Explanatory

Have you read this? docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html – Impedimenta

Thanks, I'll give it a look and get back to you, but it sounds promising. – Explanatory

The part about cancellation is what I was looking for! Can you post it as an answer so I can accept it? – Explanatory

Glad it helped. You're welcome to answer your own question and accept your own answer. If later users find it helpful, they can upvote both your question and your answer. – Impedimenta

Answer 1

Thanks to @John for pointing out a very complete article about floating point arithmetics.

It turns out that, when precision is needed, operations should be re-ordered, and formulas adapted to avoid loss of precision, as explained in the Cancellation chapter: when comparing numbers that are very close to each other (which is my case), "catastrophic cancellation" may occur, inducing a huge loss of precision. Often, re-writing the formula, or re-ordering operations according to your à-priori knowledge of the operands values can lead to achieving greater accuracy in calculus.

What I'll remember from this article is:

• be careful when substracting two nearly-identical quantities
• try to re-arrange operations to avoid catastrophic cancellation

For the latter case, remember that computing (x - y) * (x + y) gives more accurate results than x * x - y * y.

Explanatory

Answer 2

If it is important to get the correct answer, you should use BigDecimal. It is slower than double, but for most cases it is fast enough. I can't think of a lot of cases where you do a lot of calculations with such small numbers where it does not matter if the answer is correct - at least with Java.

If this is a super performance sensitive application, I would consider using a different language.

Blubbery

Indeed. I'm just comparing my results with another application and they match until a point where I seem to hit the precision of double. If that really is the case, I feel good enough stopping there, but if there are way to re-order computation so that I don't loose (too much) precision (e.g. by doing divisions first?), I'd like to do that. – Explanatory