Understanding floating point precision

Question

Is it the case that:

1. Representable floating point values are densest in the real number line near zero?
2. Representable floating point values grow sparser (exponentially?) as the number line moves away from zero?
3. If the above two are true, does that mean there is less precision farther from zero?

Overall question: Does precision in some way refer to or depend on the density of numbers you can represent (accurately)?

Chaste

Yes, your understanding is correct. – Damnation

The way I understand it is that the precision is roughly the same for all representable numbers (as in, the number of bits that specify the value). The accuracy, however, decreases when moving away from the zero. – Redfield

Answer 1

The term precision usually refers to the number of significant digits (bits) in the represented value. So precision varies with the number of bits (or digits) in the mantissa of representation. Distance from the origin has no role.

What you say is true about the density of floats on the real line. But in this case the right term is accuracy, not precision. FP numbers of small magnitude are far more accurate that larger ones. This contrasts with integers, which have uniform accuracy over their ranges.

I highly recommend the paper What Every Computer Scientist Should Know About Floating Point Arithmetic, which covers this and much more.

Ibson

“accuracy” can refer to absolute accuracy, which, for floats, has the opportunity to be better near zero, or to relative accuracy, which, for floats, has the opportunity to be uniform. The difference between “precision” and “accuracy” has more to do with “the number of digits you have” vs “what (a certain computation) does with them”, I think. The result of a computation can be represented with 24 binary digits of precision but only have a (relative) accuracy of 2^-20 because of compounded errors. – Crusted

The article from your link makes my head explode. – Swiftlet

Answer 2

Floating point numbers are basically stored in scientific notation. As long as they are normalized, they consistently have the same number of significant figures, no matter where you are on the number line.

If you consider density linearly, then the floating point numbers get exponentially more dense as you get closer to 0.

As you get extremely closed to 0, and the exponent reaches its lowest point, the floating point numbers become denormalized. At this point, they have 1 extra significant figure and are thus more precise.

Suave

Answer 3

Answers:

1. Representable floating point values are densest in the real number line near zero? Yes
2. Representable floating point values grow sparser (exponentially? No - It decreases hyperbolically) as the number line moves away from zero? Yes
3. If the above two are true, does that mean there is less precision farther from zero? Yes

Overall question: Does precision in some way refer to or depend on the density of numbers you can represent (accurately)?

See https://stackoverflow.com/a/24179424

I also recommend What Every Computer Scientist Should Know About Floating Point Arithmetic

Gaudreau

Answer 4

Representable floating point values are densest in the real number line near zero?

In a full implementation of IEEE 754 floating point yes.

However in systems that do not support subnormals, there is a gap around zero which is substantially larger than the difference between the smallest nonzero value and the second smallest nonzero value.

Representable floating point values grow sparser (exponentially?) as the number line moves away from zero?

Yes, each time the value passes a power of 2, the gap between adjacent values doubles.

If the above two are true, does that mean there is less precision farther from zero?

That depends on how exactly you define "precision", one can talk about precision in either a relative sense ("significant figures") or an absolute sense ("decimal places").

Which is more appropriate depends on what exactly the numbers are used for. Loss of precision when moving away from zero tends to become a real concern if floating point numbers are used for things like coordinates or timestamps.

Orpiment