Asked 25/2, 2009 at 21:39 Answered 17/11, 2023 at 3:1

Solved math floating-point language-agnostic precision floating-accuracy

3945

Consider the following code:

0.1 + 0.2 == 0.3  ->  false

0.1 + 0.2         ->  0.30000000000000004

Why do these inaccuracies happen?

Submediant answered 25/2, 2009 at 21:39 Comment(13)

Floating point variables typically have this behaviour. It's caused by how they are stored in hardware. For more info check out the Wikipedia article on floating point numbers. – Opt 25/2, 2009 at 21:41

JavaScript treats decimals as floating point numbers, which means operations like addition might be subject to rounding error. You might want to take a look at this article: What Every Computer Scientist Should Know About Floating-Point Arithmetic – Gazetteer 25/2, 2009 at 21:42

Just for information, ALL numeric types in javascript are IEEE-754 Doubles. – Burger 11/4, 2010 at 13:1

@Gary True, although you are guaranteed to have perfect integer precision for integers up to 15 digits, see hunlock.com/blogs/The_Complete_Javascript_Number_Reference – Robynroc 1/8, 2010 at 23:26

Because JavaScript uses the IEEE 754 standard for Math, it makes use of 64-bit floating numbers. This causes precision errors when doing floating point (decimal) calculations, in short, due to computers working in Base 2 while decimal is Base 10. – Conventionalism 7/5, 2018 at 4:57

This isn't an answer but I don't see any other answers that say this. x86 uses extended precision for floating point and then round it to 64 bits for a double. That means the answer should be more precise because of less intermediate rounding en.wikipedia.org/wiki/… – Indomitability 19/6, 2018 at 22:7

@JerryJeremiah most computer languages define the semantics of their typing model very precisely and try to limit any room for interpretation. Let's say JavaScript for instance. In JavaScript, the number type is represented as 64-bit IEEE 754 standard. 64 bits. Not 80 bits. It doesn't matter to anyone who's implementing JavaScript, whether the microprocessor is capable to handle wider floating-point types. What matters is the wanted type, which is 64 bits. The same reasoning can usually be followed for all types of all technologies you're using. If you want 80-bit FP, you need a native ext. – Bilinear 20/6, 2018 at 15:0

@Robynroc This is not a contradiction, IEEE 754 doubles have a 52-bit mantissa, which means that every integer with absolute value of at most 2^53 can be represented exactly, and 10^15 is less than 2^53. – Forester 21/2, 2019 at 14:53

A recent (September 2019) article about Numbers limit how accurately digital computers model chaos. – Matelot 27/9, 2019 at 18:30

What Every Computer Scientist Should Know About Floating-Point Arithmetic – Bay 25/12, 2019 at 19:42

Wrote a blog post that explores this: Floating Point Basics. Shorter than the otherwise great "What every computer scientist should know about floating-point arithmetic" already mentioned. – Triley 23/1, 2021 at 7:17

This question is being discussed on Meta. – Deadpan 2/3, 2023 at 19:40

I have read all the answers and for me its a fault in the software, PhP does this for 7.156 - 5.556 + 0.1 + 1.4 = 8.8817841970013E-16; while the answer should be 0. Meaning that the most simple calculations need work arounds with round(x,3). – Rilda 28/6, 2023 at 6:45

2962

Binary floating point math is like this. In most programming languages, it is based on the IEEE 754 standard. The crux of the problem is that numbers are represented in this format as a whole number times a power of two; rational numbers (such as 0.1, which is 1/10) whose denominator is not a power of two cannot be exactly represented.

For 0.1 in the standard binary64 format, the representation can be written exactly as

0.1000000000000000055511151231257827021181583404541015625 in decimal, or
0x1.999999999999ap-4 in C99 hexfloat notation.

In contrast, the rational number 0.1, which is 1/10, can be written exactly as

0.1 in decimal, or
0x1.99999999999999...p-4 in an analogue of C99 hexfloat notation, where the ... represents an unending sequence of 9's.

The constants 0.2 and 0.3 in your program will also be approximations to their true values. It happens that the closest double to 0.2 is larger than the rational number 0.2 but that the closest double to 0.3 is smaller than the rational number 0.3. The sum of 0.1 and 0.2 winds up being larger than the rational number 0.3 and hence disagreeing with the constant in your code.

A fairly comprehensive treatment of floating-point arithmetic issues is What Every Computer Scientist Should Know About Floating-Point Arithmetic. For an easier-to-digest explanation, see floating-point-gui.de.

Side Note: All positional (base-N) number systems share this problem with precision

Plain old decimal (base 10) numbers have the same issues, which is why numbers like 1/3 end up as 0.333333333...

You've just stumbled on a number (3/10) that happens to be easy to represent with the decimal system, but doesn't fit the binary system. It goes both ways (to some small degree) as well: 1/16 is an ugly number in decimal (0.0625), but in binary it looks as neat as a 10,000th does in decimal (0.0001)** - if we were in the habit of using a base-2 number system in our daily lives, you'd even look at that number and instinctively understand you could arrive there by halving something, halving it again, and again and again.

Of course, that's not exactly how floating-point numbers are stored in memory (they use a form of scientific notation). However, it does illustrate the point that binary floating-point precision errors tend to crop up because the "real world" numbers we are usually interested in working with are so often powers of ten - but only because we use a decimal number system day-to-day. This is also why we'll say things like 71% instead of "5 out of every 7" (71% is an approximation, since 5/7 can't be represented exactly with any decimal number).

So no: binary floating point numbers are not broken, they just happen to be as imperfect as every other base-N number system :)

Side Side Note: Working with Floats in Programming

In practice, this problem of precision means you need to use rounding functions to round your floating point numbers off to however many decimal places you're interested in before you display them.

You also need to replace equality tests with comparisons that allow some amount of tolerance, which means:

Do not do if (x == y) { ... }

Instead do if (abs(x - y) < myToleranceValue) { ... }.

where abs is the absolute value. myToleranceValue needs to be chosen for your particular application - and it will have a lot to do with how much "wiggle room" you are prepared to allow, and what the largest number you are going to be comparing may be (due to loss of precision issues). Beware of "epsilon" style constants in your language of choice. These can be used as tolerance values but their effectiveness depends on the magnitude (size) of the numbers you're working with, since calculations with large numbers may exceed the epsilon threshold.

Prynne answered 25/2, 2009 at 21:39 Comment(19)

I think "some error constant" is more correct than "The Epsilon" because there is no "The Epsilon" which could be used in all cases. Different epsilons need to be used in different situations. And the machine epsilon is almost never a good constant to use. – Mazzard 4/9, 2010 at 23:33

@Mazzard couldn't agree more. I think linking to machine epsilon adds confusion - it's really something else. – Polyvinyl 5/9, 2010 at 2:10

@Rostor @Polyvinyl After noticing your comments six months later, I've removed the link to "Machine epsilon." – Bernettabernette 1/3, 2011 at 13:42

It's not quite true that all floating-point math is based on the IEEE [754] standard. There are still some systems in use that have the old IBM hexadecimal FP, for example, and there are still graphics cards that don't support IEEE-754 arithmetic. It's true to a reasonably approximation, however. – Plasmodium 3/1, 2013 at 23:36

Cray ditched IEEE-754 compliance for speed. Java loosened its adherence as an optimization as well. – Malayalam 12/2, 2013 at 3:12

Here is a VERY well-done and deep (bit-level) but comprehensible 30min video from JSconf EU 2013 about numbers in Javascript. JS number implementation is according to IEEE 754 standard, so not JS specific. Video link: youtube.com/… – Raindrop 27/11, 2013 at 18:40

I think you should add something to this answer about how computations on money should always, always be done with fixed-point arithmetic on integers, because money is quantized. (It may make sense to do internal accounting computations in tiny fractions of a cent, or whatever your smallest currency unit is - this often helps with e.g. reducing round-off error when converting "$29.99 a month" to a daily rate - but it should still be fixed-point arithmetic.) – Rabid 12/5, 2014 at 22:23

@Zack: Actually, the key to doing such calculations accurately isn't using fixed-point arithmetic, but rather arranging calculations to avoid having to round values which are going to be scaled up. For example, to charge someone for 17 days of a 30 day month, multiply the monthly amount by 17 and then divide by 30, rather than dividing by 30 and multiplying by 17. – Tuscany 1/10, 2014 at 15:13

@Tuscany You need to do that also, but it doesn't insulate you from the fact that $0.03 cannot be represented exactly as a binary floating point number. – Rabid 3/10, 2014 at 19:46

@Zack: No, but 3.0 pennies can. Fixed-point code which divides $29.99/month by 30 and then multiplies by 17 will end up with $16.994333333333333333333333334 and sends a bill which is rounded to the penny but doesn't round the result before adding it to the invoice total will be less accurate than floating-point code which multiplies 2999 by 17, divides by 30, and rounds to the nearest unit. – Tuscany 3/10, 2014 at 20:37

But why, e.g. in Javascript, 0.1 shows up as 0.1. (instead of 0.1000000000000000055... ) while 0.1+0.2 shows up as 0.30000000000000004 – Aulea 20/9, 2015 at 17:36

Interesting fact: this very 0.1 not being exactly represented in binary floating-point caused an infamous Patriot missile software bug which resulted in 28 people killed during the first Iraq war. – Contusion 24/9, 2015 at 12:57

Not mentioned is that many languages (SQL, dotNet, etc.) have a decimal data type where all decimal fractions can be represented exactly and this problem does not exist. Of course then binary fractions would be represented as approximations. – Chatty 7/6, 2016 at 19:22

As far as I know gpus may use alternative representations, as well as some spare reimplementations – Timbal 26/11, 2016 at 7:52

" In Javascript, the value Number.EPSILON is provided for you to use as a tolerance." -- do not use Number.EPSILON. It is usable only for numbers close to 1. See the following example: let f1 = 1000000.1 + 0.2; let f2 = 1000000.3; console.log(Math.abs(f1 - f2) < Number.EPSILON); // you get false here – Medardas 11/6, 2019 at 13:11

Note: even though a language agnostic Q/A, In C if (abs(x - y) < myToleranceValue) ... should be if (fabs(x - y) < myToleranceValue) ... to avoid the common mistake of using int abs(int). – Bewley 14/9, 2021 at 14:42

@ArtTaylor Regarding your mention of Java loosening its adherence to the IEEE standard for floating-point, Java 17 seems to have restored strict adherence to the standard. – Fiedler 4/11, 2021 at 4:40

Actually having an == operator for floating point is the thing that is broken. – Roesch 19/12, 2021 at 10:8

@ArtTaylor: Update, 8 years after your comment: In version 17 Java has changed its specification to always follow IEEE 754: openjdk.org/jeps/306 – Olethea 18/12, 2022 at 11:16

727

A Hardware Designer's Perspective

I believe I should add a hardware designer’s perspective to this since I design and build floating point hardware. Knowing the origin of the error may help in understanding what is happening in the software, and ultimately, I hope this helps explain the reasons for why floating point errors happen and seem to accumulate over time.

1. Overview

From an engineering perspective, most floating point operations will have some element of error since the hardware that does the floating point computations is only required to have an error of less than one half of one unit in the last place. Therefore, much hardware will stop at a precision that's only necessary to yield an error of less than one half of one unit in the last place for a single operation which is especially problematic in floating point division. What constitutes a single operation depends upon how many operands the unit takes. For most, it is two, but some units take 3 or more operands. Because of this, there is no guarantee that repeated operations will result in a desirable error since the errors add up over time.

2. Standards

Most processors follow the IEEE-754 standard but some use denormalized, or different standards . For example, there is a denormalized mode in IEEE-754 which allows representation of very small floating point numbers at the expense of precision. The following, however, will cover the normalized mode of IEEE-754 which is the typical mode of operation.

In the IEEE-754 standard, hardware designers are allowed any value of error/epsilon as long as it's less than one half of one unit in the last place, and the result only has to be less than one half of one unit in the last place for one operation. This explains why when there are repeated operations, the errors add up. For IEEE-754 double precision, this is the 54th bit, since 53 bits are used to represent the numeric part (normalized), also called the mantissa, of the floating point number (e.g. the 5.3 in 5.3e5). The next sections go into more detail on the causes of hardware error on various floating point operations.

3. Cause of Rounding Error in Division

The main cause of the error in floating point division is the division algorithms used to calculate the quotient. Most computer systems calculate division using multiplication by an inverse, mainly in Z=X/Y, Z = X * (1/Y). A division is computed iteratively i.e. each cycle computes some bits of the quotient until the desired precision is reached, which for IEEE-754 is anything with an error of less than one unit in the last place. The table of reciprocals of Y (1/Y) is known as the quotient selection table (QST) in the slow division, and the size in bits of the quotient selection table is usually the width of the radix, or a number of bits of the quotient computed in each iteration, plus a few guard bits. For the IEEE-754 standard, double precision (64-bit), it would be the size of the radix of the divider, plus a few guard bits k, where k>=2. So for example, a typical Quotient Selection Table for a divider that computes 2 bits of the quotient at a time (radix 4) would be 2+2= 4 bits (plus a few optional bits).

3.1 Division Rounding Error: Approximation of Reciprocal

What reciprocals are in the quotient selection table depend on the division method: slow division such as SRT division, or fast division such as Goldschmidt division; each entry is modified according to the division algorithm in an attempt to yield the lowest possible error. In any case, though, all reciprocals are approximations of the actual reciprocal and introduce some element of error. Both slow division and fast division methods calculate the quotient iteratively, i.e. some number of bits of the quotient are calculated each step, then the result is subtracted from the dividend, and the divider repeats the steps until the error is less than one half of one unit in the last place. Slow division methods calculate a fixed number of digits of the quotient in each step and are usually less expensive to build, and fast division methods calculate a variable number of digits per step and are usually more expensive to build. The most important part of the division methods is that most of them rely upon repeated multiplication by an approximation of a reciprocal, so they are prone to error.

4. Rounding Errors in Other Operations: Truncation

Another cause of the rounding errors in all operations are the different modes of truncation of the final answer that IEEE-754 allows. There's truncate, round-towards-zero, round-to-nearest (default), round-down, and round-up. All methods introduce an element of error of less than one unit in the last place for a single operation. Over time and repeated operations, truncation also adds cumulatively to the resultant error. This truncation error is especially problematic in exponentiation, which involves some form of repeated multiplication.

5. Repeated Operations

Since the hardware that does the floating point calculations only needs to yield a result with an error of less than one half of one unit in the last place for a single operation, the error will grow over repeated operations if not watched. This is the reason that in computations that require a bounded error, mathematicians use methods such as using the round-to-nearest even digit in the last place of IEEE-754, because, over time, the errors are more likely to cancel each other out, and Interval Arithmetic combined with variations of the IEEE 754 rounding modes to predict rounding errors, and correct them. Because of its low relative error compared to other rounding modes, round to nearest even digit (in the last place), is the default rounding mode of IEEE-754.

Note that the default rounding mode, round-to-nearest even digit in the last place, guarantees an error of less than one half of one unit in the last place for one operation. Using the truncation, round-up, and round down alone may result in an error that is greater than one half of one unit in the last place, but less than one unit in the last place, so these modes are not recommended unless they are used in Interval Arithmetic.

6. Summary

In short, the fundamental reason for the errors in floating point operations is a combination of the truncation in hardware, and the truncation of a reciprocal in the case of division. Since the IEEE-754 standard only requires an error of less than one half of one unit in the last place for a single operation, the floating point errors over repeated operations will add up unless corrected.

Jiggerypokery answered 18/4, 2013 at 11:52 Comment(16)

(1) Floating point numbers do not have error. Every floating point value is exactly what it is. Most (but not all) floating point operations give inexact results. For example, there is no binary floating point value that is exactly equal to 1.0/10.0. Some operations (e.g., 1.0 + 1.0) do give exact results on the other hand. – Ruthenic 10/6, 2014 at 16:31

@james large Thanks for catching that. I edited the reply to clarify that most floating point operations have an error less than 1/2 of one ulp. There are some special cases where the result can be exact (like adding zero). – Jiggerypokery 11/6, 2014 at 10:58

"The main cause of the error in floating point division, are the division algorithms used to calculate the quotient" is a very misleading thing to say. For an IEEE-754 conforming division, the only cause of error in floating-point division is the inability of the result to be exactly represented in the result format; the same result is computed regardless of the algorithm that is used. – Plasmodium 23/2, 2015 at 20:23

@Matt Sorry for the late response. It's basically due to resource/time issues and tradeoffs. There is a way to do long division/more 'normal' division, it's called SRT Division with radix two. However, this repeatedly shifts and subtracts the divisor from the dividend and takes many clock cycles since it only computes one bit of the quotient per clock cycle. We use tables of reciprocals so that we can compute more bits of the quotient per cycle and make effective performance/speed tradeoffs. – Jiggerypokery 1/2, 2016 at 15:33

@james -- I disagree. It is the numbers, not the operations. The ops are exact. But very very few of the decimal fractions we can write have an exact equivalent in the binary radix of floating point fractions. See my (really late, you'll need to scroll way down) answer for a complete explanation. – Amusement 9/6, 2016 at 1:20

@DigitalRoss, we seem to be using different lexicons. What do you mean when you say that a number is "inexact?" I know how an answer can be inexact, but if my program has a floating point number stored in some variable, x, how can that number be anything other than exactly what it is? Your answer speaks of "giving it input that is slightly off from what we wrote." But, if "giving it input" means converting a string of decimal digits into a floating point value in memory, then that's actually a lengthy sequence of floating point operations (at least two per digit.) – Ruthenic 9/6, 2016 at 15:21

@james large, yes, the numbers are what they are, but they aren't what you entered. Just like you cannot actually ever have 1/3 as a decimal fraction, you cannot ever by any mechanism have 0.01 as a binary fraction. You can't have 0.02 either. Never. When you get to 0.25, that's 0.01(2) and now you can have an exact fraction. The very nature of the base 10 radix prevents most decimal fractions from being representable in the IEEE format. I referenced my explanation. Did you read it? – Amusement 9/6, 2016 at 16:58

@DigitalRoss, I read your answer. It explains why there is no binary floating point (BFP) number that represents the real number, 0.01. I don't think we disagree about the reality, only on how to describe it. You say the BFP representation of 0.01 is "inexact." I say it does not exist. I say that when you type the string "0.01" into your computer, the conversion function gives you an inexact result. My way of thinking probably is colored by work I've done in the past on low-level math libraries for machines that did not have floating point hardware. – Ruthenic 9/6, 2016 at 17:32

A max error of 1/2 ulp for the "basic" operations (add, sub, mul, div, and sqrt) means the result is correctly rounded out to the last bit of the mantissa. This answer makes it sound like more accuracy would be possible if speed / power / die-area weren't a concern, but that's not the case. For any given inputs to an operation like ADD, there is exactly one result allowed by the IEEE standard. Hardware has to compute enough extra bits of the exact result to figure out what the correctly rounded result is, but has no choice in "how much error" to leave in the result of ADDSS for example. – Jesselyn 9/12, 2016 at 4:21

This makes bit-exact deterministic computation possible across different hardware. But not in C; even without any -ffast-math options, the C language allows the compiler too much freedom. However, in asm you can run the same code on different implementations of x86 and get bit-exact results. See this answer about FP determinism in C vs. x86 asm. – Jesselyn 9/12, 2016 at 4:24

Anyway, this answer is correct that error-accumulation from repeated operations does happen. Also a good point about "what is an operation"; FMA changes things. It also has some interesting details about how FP division is implemented in hardware, even though as Stephen Canon pointed out those are irrelevant for accuracy. I think I'm going to have to downvote for confusingly implying that better than 0.5 ulp is possible. :/ – Jesselyn 9/12, 2016 at 4:27

@KernelPanik, Interval-arithmetic though, is surprisingly common. Eg putting items within a box, or drawing pixels within a rectangle [screen]. Also, why do you say that normalized mode is "typical"? And do you mean Javascript's mandatory denormalized mode support is atypical? – Amyotonia 17/4, 2017 at 19:13

@Pacerier, Normalized mode allows for accuracy down to machine epsilon for the mantissa or numeric part, which is 2^-53 for double precision. Since many applications depend upon accuracy until machine epsilon, normalized mode is typical (root finding comes to mind). However, it is very common to have support for denormalized mode today, like the JavaScript example on modern computers at the expense of decimal place accuracy. It's less common to support denormalized mode on many embedded systems. – Jiggerypokery 18/4, 2017 at 16:14

The exponent range is separate from the mantissa width. 1 part in 2^53 is the relative precision of IEEE double, and thus epsilon (1 ulp of (double)1.0), but normalized floats have full precision for numbers down to 2^(-1022) ~= 2.2e-308, the smallest normalized float. en.wikipedia.org/wiki/…. – Jesselyn 29/12, 2021 at 20:56

Allowing gradual underflow (non-zero numbers smaller than that, with less precision) does not hurt the precision of normal numbers. Maybe you meant to say that applications only depend on precision for normalized numbers, so hard underflow to +-0.0 instead of gradual underflow to subnormals doesn't hurt most programs? (i.e. the way gcc -ffast-math works on x86, setting setting flush-to-zero and denormals-are-zero. Or like 32-bit ARM NEON SIMD which doesn't support subnormals, @Pacerier). That doesn't make anything worse for non-tiny numbers; they still produce identical results. – Jesselyn 29/12, 2021 at 21:1

Anyway, the default floating-point environment on standard IEEE-conforming systems supports subnormal numbers. Some, such as x86, support flushing denormals to exactly 0.0, because some CPUs handle gradual underflow by taking a microcode exception which is very slow, @Pacerier. Disabling that lets the normal fast-path always happen. – Jesselyn 29/12, 2021 at 21:5

646

Floating point notation is broken in the exact same way the decimal (base-10) notation you learned in grade school and use every day is broken, just for base-2.

To understand, think about representing 2/3 as a decimal value. It's impossible to do exactly! The world will end before you finish writing the 6's after the decimal point, and so instead we write to some number of places, round to a final 7, and consider it sufficiently accurate.

In the same way, 1/10 (decimal 0.1) cannot be represented exactly in base 2 (binary) as a "decimal" value; a repeating pattern after the decimal point goes on forever. The value is not exact, and therefore you can't do exact math with it using normal floating point methods. Just like with base 10, there are other values that exhibit this problem as well.

Hasidism answered 25/2, 2009 at 21:43 Comment(10)

Great and short answer. Repeating pattern looks like 0.00011001100110011001100110011001100110011001100110011... – Quadrireme 16/6, 2012 at 14:22

There ARE methods that yield exact decimal values. BCD (Binary coded decimal) or various other forms of decimal number. However, these are both slower (a LOT slower) and take more storage than using binary floating point. (as an example, packed BCD stores 2 decimal digits in a byte. That's 100 possible values in a byte that can actually store 256 possible values, or 100/256, which wastes about 60% of the possible values of a byte.) – Interlunar 21/6, 2016 at 16:43

@DuncanC: Given the fact, that all x86 compatible CPUs still have dedicated BCD opcodes, they really may not be that slow (for basic arithmetic operations). And of course, unpacked BCDs take a whopping 8 bits to store a single decimal digit. – Rn 7/9, 2016 at 15:34

@IInspectable, for floating point operations, BCD based math is hundreds of times slower than native binary floating point. – Interlunar 8/9, 2016 at 0:20

Wouldn't it make sense for the system to consider 0.1 as 1e-1? Maybe that causes more problems down the line.. – Asphaltite 14/11, 2016 at 15:34

@DuncanC Well, there are methods that yield exact decimal values -- for addition and subtraction. For division, multiplication, etc. they have the same issues as binary methods. That's why BCD is used in accounting since that deals mostly with plus and minus and you can't account for anything smaller than a penny. However something simple like 1/3*3 == 1 fails (evaluates to false) in BCD math, just like it would fail if you used decimal division on paper. – Drag 22/6, 2018 at 21:19

@Joooeey, true. Note that I said "exact decimal values", not exact values. Decimal has numbers it can't express exactly as well. (any fraction were the denominator has a factor of a prime number other than 2 or 5 in it, the square root of 2, pi, and quite a few others.) – Interlunar 22/6, 2018 at 23:3

@Rn BCD is a lot slower than binary floating point, period. BCD machine instructions are a lot slower than binary floating point instructions. Look it up. – Interlunar 22/6, 2018 at 23:4

@DuncanC: "BCD is a lot slower than binary floating point, period." - Uhm, yeah. Unless it isn't. Pretty sure there are architectures, where BCD math is at least as fast (or faster) than IEEE-754 floating point math. But that's besides the point: If you need decimal accuracy, you cannot use IEEE-754 floating point representation. Doing so will achieve one thing only: Calculating the wrong results faster. – Rn 23/6, 2018 at 10:30

If you implement your BCD in FPGA, then those who thought you were SOL will be SOL themselves, because it will not be "slower". ;-) – Hypostyle 5/7, 2022 at 6:45

392

Most answers here address this question in very dry, technical terms. I'd like to address this in terms that normal human beings can understand.

Imagine that you are trying to slice up pizzas. You have a robotic pizza cutter that can cut pizza slices exactly in half. It can halve a whole pizza, or it can halve an existing slice, but in any case, the halving is always exact.

That pizza cutter has very fine movements, and if you start with a whole pizza, then halve that, and continue halving the smallest slice each time, you can do the halving 53 times before the slice is too small for even its high-precision abilities. At that point, you can no longer halve that very thin slice, but must either include or exclude it as is.

Now, how would you piece all the slices in such a way that would add up to one-tenth (0.1) or one-fifth (0.2) of a pizza? Really think about it, and try working it out. You can even try to use a real pizza, if you have a mythical precision pizza cutter at hand. :-)

Most experienced programmers, of course, know the real answer, which is that there is no way to piece together an exact tenth or fifth of the pizza using those slices, no matter how finely you slice them. You can do a pretty good approximation, and if you add up the approximation of 0.1 with the approximation of 0.2, you get a pretty good approximation of 0.3, but it's still just that, an approximation.

For double-precision numbers (which is the precision that allows you to halve your pizza 53 times), the numbers immediately less and greater than 0.1 are 0.09999999999999999167332731531132594682276248931884765625 and 0.1000000000000000055511151231257827021181583404541015625. The latter is quite a bit closer to 0.1 than the former, so a numeric parser will, given an input of 0.1, favour the latter.

(The difference between those two numbers is the "smallest slice" that we must decide to either include, which introduces an upward bias, or exclude, which introduces a downward bias. The technical term for that smallest slice is an ulp.)

In the case of 0.2, the numbers are all the same, just scaled up by a factor of 2. Again, we favour the value that's slightly higher than 0.2.

Notice that in both cases, the approximations for 0.1 and 0.2 have a slight upward bias. If we add enough of these biases in, they will push the number further and further away from what we want, and in fact, in the case of 0.1 + 0.2, the bias is high enough that the resulting number is no longer the closest number to 0.3.

In particular, 0.1 + 0.2 is really 0.1000000000000000055511151231257827021181583404541015625 + 0.200000000000000011102230246251565404236316680908203125 = 0.3000000000000000444089209850062616169452667236328125, whereas the number closest to 0.3 is actually 0.299999999999999988897769753748434595763683319091796875.

P.S. Some programming languages also provide pizza cutters that can split slices into exact tenths. Although such pizza cutters are uncommon, if you do have access to one, you should use it when it's important to be able to get exactly one-tenth or one-fifth of a slice.

(Originally posted on Quora.)

Cyano answered 20/11, 2014 at 2:39 Comment(17)

Note that there are some languages which include exact math. One example is Scheme, for example via GNU Guile. See draketo.de/english/exact-math-to-the-rescue — these keep the math as fractions and only slice up in the end. – Wapiti 20/11, 2014 at 6:40

@FloatingRock Actually, very few mainstream programming languages have rational numbers built-in. Arne is a Schemer, as I am, so these are things we get spoilt on. – Cyano 25/11, 2014 at 16:56

@ArneBabenhauserheide I think it's worth adding that this will only work with rational numbers. So if you're doing some math with irrational numbers like pi, you'd have to store it as a multiple of pi. Of course, any calculating involving pi cannot be represented as an exact decimal number. – Diadiabase 11/3, 2015 at 13:6

@Diadiabase by harnessing something like scmutils, you could use prior simplification, so if you wrote something as (* 5 pi) and used that in an equation, the muiltiplication with pi would be done last, minimizing the loss of precision. See cs.rochester.edu/~gildea/guile-scmutils — this is symbolic math, then: Simplifying the equation before calculating anything. – Wapiti 18/3, 2015 at 20:25

The precision pizza cutter would have to be complemented by a precision pizza rotator. To get exactly 1/10 of a pizza, proceed as follows: Take the full pizza and cut it in half. Keep the pizza cutter in place and rotate the pizza exactly 36 degrees, and let the pizza cutter do the exactly same cut as in step 1. Easy peasy! – Heckle 13/8, 2015 at 14:2

If you feel that this doesn't work because processors and programming languages cannot do to 1 what every idiot can do to a pizza (that is, rotate it), maybe you get an idea why explaining complex mathematical problems with every day examples usually doesn't work :) – Heckle 13/8, 2015 at 14:9

@Heckle Okay. How would you program your pizza rotator to get 36 degrees? What is 36 degrees? (Hint: if you are able to define this in an exact fashion, you also have a slices-an-exact-tenth pizza cutter.) In other words, you can't actually have 1/360 (a degree) or 1/10 (36 degrees) with only binary floating point. – Cyano 13/8, 2015 at 14:50

@Heckle Also, "every idiot" can't rotate a pizza exactly 36 degrees. Humans are too error-prone to do anything quite so precise. – Cyano 13/8, 2015 at 14:51

I just tried and 0.1000000000000000055511151231257827021181583404541015625 + 0.200000000000000011102230246251565404236316680908203125 is not equal to 0.3000000000000000444089209850062616169452667236328125 (ttmath.org/online_calculator) – Ankara 10/12, 2015 at 5:11

@Ankara It is if you round the result to double-precision (53-bit mantissa) afterwards. If you use an arbitrary-precision calculator, like the one you linked to, of course you'll get different results. – Cyano 10/12, 2015 at 5:13

I don't quite understand. I thought 0.1000000000000000055511151231257827021181583404541015625 and 0.200000000000000011102230246251565404236316680908203125 is already rounded to 53-bit mantissa. I'm not sure why we round to 53-bit mantissa again. Could you point me to how it works? Thanks! – Ankara 10/12, 2015 at 5:34

@ChrisJester-Young: Saying "very few mainstream programming languages have rational numbers built-in" as a counter to "Except PHP. It doesn't know any better." is a little misleading. While few languages outside a few functional languages with limited industry use (e.g. Lisp, Haskell) offer built-in, library-free rational numbers with complete language integration equivalent to that of integers, many languages, particularly PHP competitors like Perl, Python and Ruby ship with a rational number library that can be used like a built-in type after import... (1/2) – Nydianye 3/2, 2016 at 18:54

(2/2)...and others, like C/C++ have well known, commonly used third party libraries like GMP/MPFR that add the same features to the language (in the case of C++ and GMP/MPFR, with modern C++ standards that add constexpr, r-value references, and auto for type deduction, the integration is so tight that you can use mpfr_class as a drop in replacement for a primitive type like double. PHP is unusual in that it provides no rational support, I can't even find third party libs for it. And since it lacks operator overloading, add-on types wouldn't be drop in replacements in any event. – Nydianye 3/2, 2016 at 18:57

I think should ommit 0.3000000000000000444089209850062616169452667236328125. Just say 0.1000000000000000055511151231257827021181583404541015625 + 0.200000000000000011102230246251565404236316680908203125 = 0.3000000000000000166533453693773481063544750213623046875, which is 0.0100110011001100110011001100110011001100110011001100111, and the last bit should be ommit, so it's the same with 0.010011001100110011001100110011001100110011001100110100, which is 0.3000000000000000444089209850062616169452667236328125. – Titanite 20/5, 2016 at 7:58

@Nydianye Haskell does include rational numbers as library, it is not built into the base language. (See Data.Ratio for the implementation) – Brodsky 11/12, 2016 at 18:2

A one-foot-diameter ie one-metre-circumference pizza halved 53 times gives a slice roughly the width of an electron. (Now I feel like I know how big that that last bit is.) – Euchromatin 25/10, 2017 at 8:54

Yea ... but the kids will still argue about who got the bigger slice :-) – Parttime 31/12, 2021 at 1:45

233

Floating point rounding errors. 0.1 cannot be represented as accurately in base-2 as in base-10 due to the missing prime factor of 5. Just as 1/3 takes an infinite number of digits to represent in decimal, but is "0.1" in base-3, 0.1 takes an infinite number of digits in base-2 where it does not in base-10. And computers don't have an infinite amount of memory.

Pioneer answered 25/2, 2009 at 21:41 Comment(5)

@Amyotonia Sure, they could use two unbounded-precision integers to represent a fraction, or they could use quote notation. It's the specific notion of "binary" or "decimal" that makes this impossible -- the idea that you have a sequence of binary/decimal digits and, somewhere in there, a radix point. To get precise rational results we'd need a better format. – Pioneer 15/10, 2011 at 19:45

@Pacerier: Neither binary nor decimal floating-point can precisely store 1/3 or 1/13. Decimal floating-point types can precisely represent values of the form M/10^E, but are less precise than similarly-sized binary floating-point numbers when it comes to representing most other fractions. In many applications, it's more useful to have higher precision with arbitrary fractions than to have perfect precision with a few "special" ones. – Tuscany 24/4, 2014 at 16:43

@Tuscany In comparing precision of binary64 and decimal64: the precision are fairly comparable - certainly within a factor of 10 to each other. Granted decimal64 wobbles more than binary64. – Bewley 26/8, 2015 at 19:26

@chux: The difference in precision between binary and decimal types isn't huge, but the 10:1 difference in best-case vs. worst-case precision for decimal types is far greater than the 2:1 difference with binary types. I'm curious whether anyone has built hardware or written software to operate efficiently on either of the decimal types, since neither would seem amenable to efficient implementation in hardware nor software. – Tuscany 26/8, 2015 at 19:47

@DevinJeanpierre I think the point is that "computers" don't have a "specific notion of 'binary' or 'decimal'". Pacerier's point seems to be that it is language designers who have decided to make the jump to "floating point" too early, when storing such numbers as "0.1", "0.2", and "0.3" which can not only be more accurately but also more space-efficiently stored as text (BCD). – Evidence 1/2, 2016 at 14:58

155

My answer is quite long, so I've split it into three sections. Since the question is about floating point mathematics, I've put the emphasis on what the machine actually does. I've also made it specific to double (64 bit) precision, but the argument applies equally to any floating point arithmetic.

Preamble

An IEEE 754 double-precision binary floating-point format (binary64) number represents a number of the form

value = (-1)^s * (1.m₅₁m₅₀...m₂m₁m₀)₂ * 2^e-1023

in 64 bits:

The first bit is the sign bit: 1 if the number is negative, 0 otherwise¹.
The next 11 bits are the exponent, which is offset by 1023. In other words, after reading the exponent bits from a double-precision number, 1023 must be subtracted to obtain the power of two.
The remaining 52 bits are the significand (or mantissa). In the mantissa, an 'implied' 1. is always² omitted since the most significant bit of any binary value is 1.

¹ - IEEE 754 allows for the concept of a signed zero - +0 and -0 are treated differently: 1 / (+0) is positive infinity; 1 / (-0) is negative infinity. For zero values, the mantissa and exponent bits are all zero. Note: zero values (+0 and -0) are explicitly not classed as denormal².

² - This is not the case for denormal numbers, which have an offset exponent of zero (and an implied 0.). The range of denormal double precision numbers is d_min ≤ |x| ≤ d_max, where d_min (the smallest representable nonzero number) is 2^{-1023 - 51} (≈ 4.94 * 10^-324) and d_max (the largest denormal number, for which the mantissa consists entirely of 1s) is 2^{-1023 + 1} - 2^{-1023 - 51} (≈ 2.225 * 10^-308).

Turning a double precision number to binary

Many online converters exist to convert a double precision floating point number to binary (e.g. at binaryconvert.com), but here is some sample C# code to obtain the IEEE 754 representation for a double precision number (I separate the three parts with colons (:):

public static string BinaryRepresentation(double value)
{
    long valueInLongType = BitConverter.DoubleToInt64Bits(value);
    string bits = Convert.ToString(valueInLongType, 2);
    string leadingZeros = new string('0', 64 - bits.Length);
    string binaryRepresentation = leadingZeros + bits;

    string sign = binaryRepresentation[0].ToString();
    string exponent = binaryRepresentation.Substring(1, 11);
    string mantissa = binaryRepresentation.Substring(12);

    return string.Format("{0}:{1}:{2}", sign, exponent, mantissa);
}

Getting to the point: the original question

(Skip to the bottom for the TL;DR version)

Cato Johnston (the question asker) asked why 0.1 + 0.2 != 0.3.

Written in binary (with colons separating the three parts), the IEEE 754 representations of the values are:

0.1 => 0:01111111011:1001100110011001100110011001100110011001100110011010
0.2 => 0:01111111100:1001100110011001100110011001100110011001100110011010

Note that the mantissa is composed of recurring digits of 0011. This is key to why there is any error to the calculations - 0.1, 0.2 and 0.3 cannot be represented in binary precisely in a finite number of binary bits any more than 1/9, 1/3 or 1/7 can be represented precisely in decimal digits.

Also note that we can decrease the power in the exponent by 52 and shift the point in the binary representation to the right by 52 places (much like 10^-3 * 1.23 == 10^-5 * 123). This then enables us to represent the binary representation as the exact value that it represents in the form a * 2^p. where 'a' is an integer.

Converting the exponents to decimal, removing the offset, and re-adding the implied 1 (in square brackets), 0.1 and 0.2 are:

0.1 => 2^-4 * [1].1001100110011001100110011001100110011001100110011010
0.2 => 2^-3 * [1].1001100110011001100110011001100110011001100110011010
or
0.1 => 2^-56 * 7205759403792794 = 0.1000000000000000055511151231257827021181583404541015625
0.2 => 2^-55 * 7205759403792794 = 0.200000000000000011102230246251565404236316680908203125

To add two numbers, the exponent needs to be the same, i.e.:

0.1 => 2^-3 *  0.1100110011001100110011001100110011001100110011001101(0)
0.2 => 2^-3 *  1.1001100110011001100110011001100110011001100110011010
sum =  2^-3 * 10.0110011001100110011001100110011001100110011001100111
or
0.1 => 2^-55 * 3602879701896397  = 0.1000000000000000055511151231257827021181583404541015625
0.2 => 2^-55 * 7205759403792794  = 0.200000000000000011102230246251565404236316680908203125
sum =  2^-55 * 10808639105689191 = 0.3000000000000000166533453693773481063544750213623046875

Since the sum is not of the form 2ⁿ * 1.{bbb} we increase the exponent by one and shift the decimal (binary) point to get:

sum = 2^-2  * 1.0011001100110011001100110011001100110011001100110011(1)
    = 2^-54 * 5404319552844595.5 = 0.3000000000000000166533453693773481063544750213623046875

There are now 53 bits in the mantissa (the 53rd is in square brackets in the line above). The default rounding mode for IEEE 754 is 'Round to Nearest' - i.e. if a number x falls between two values a and b, the value where the least significant bit is zero is chosen.

a = 2^-54 * 5404319552844595 = 0.299999999999999988897769753748434595763683319091796875
  = 2^-2  * 1.0011001100110011001100110011001100110011001100110011

x = 2^-2  * 1.0011001100110011001100110011001100110011001100110011(1)

b = 2^-2  * 1.0011001100110011001100110011001100110011001100110100
  = 2^-54 * 5404319552844596 = 0.3000000000000000444089209850062616169452667236328125

Note that a and b differ only in the last bit; ...0011 + 1 = ...0100. In this case, the value with the least significant bit of zero is b, so the sum is:

sum = 2^-2  * 1.0011001100110011001100110011001100110011001100110100
    = 2^-54 * 5404319552844596 = 0.3000000000000000444089209850062616169452667236328125

whereas the binary representation of 0.3 is:

0.3 => 2^-2  * 1.0011001100110011001100110011001100110011001100110011
    =  2^-54 * 5404319552844595 = 0.299999999999999988897769753748434595763683319091796875

which only differs from the binary representation of the sum of 0.1 and 0.2 by 2^-54.

The binary representation of 0.1 and 0.2 are the most accurate representations of the numbers allowable by IEEE 754. The addition of these representation, due to the default rounding mode, results in a value which differs only in the least-significant-bit.

TL;DR

Writing 0.1 + 0.2 in a IEEE 754 binary representation (with colons separating the three parts) and comparing it to 0.3, this is (I've put the distinct bits in square brackets):

0.1 + 0.2 => 0:01111111101:0011001100110011001100110011001100110011001100110[100]
0.3       => 0:01111111101:0011001100110011001100110011001100110011001100110[011]

Converted back to decimal, these values are:

0.1 + 0.2 => 0.300000000000000044408920985006...
0.3       => 0.299999999999999988897769753748...

The difference is exactly 2^-54, which is ~5.5511151231258 × 10^-17 - insignificant (for many applications) when compared to the original values.

Comparing the last few bits of a floating point number is inherently dangerous, as anyone who reads the famous "What Every Computer Scientist Should Know About Floating-Point Arithmetic" (which covers all the major parts of this answer) will know.

Most calculators use additional guard digits to get around this problem, which is how 0.1 + 0.2 would give 0.3: the final few bits are rounded.

Hydrozoan answered 23/2, 2015 at 17:15 Comment(0)

142

In addition to the other correct answers, you may want to consider scaling your values to avoid problems with floating-point arithmetic.

For example:

var result = 1.0 + 2.0;     // result === 3.0 returns true

... instead of:

var result = 0.1 + 0.2;     // result === 0.3 returns false

The expression 0.1 + 0.2 === 0.3 returns false in JavaScript, but fortunately integer arithmetic in floating-point is exact, so decimal representation errors can be avoided by scaling.

As a practical example, to avoid floating-point problems where accuracy is paramount, it is recommended¹ to handle money as an integer representing the number of cents: 2550 cents instead of 25.50 dollars.

¹ Douglas Crockford: JavaScript: The Good Parts: Appendix A - Awful Parts (page 105).

Patmore answered 9/4, 2010 at 12:25 Comment(3)

The problem is that the conversion itself is inaccurate. 16.08 * 100 = 1607.9999999999998. Do we have to resort to splitting the number and converting separately (as in 16 * 100 + 08 = 1608)? – Oliguria 7/10, 2011 at 19:13

The solution here is to do all your calculations in integer then divide by your proportion (100 in this case) and round only when presenting the data. That will ensure that your calculations will always be precise. – Rainey 8/12, 2011 at 21:38

Just to nitpick a little: integer arithmetic is only exact in floating-point up to a point (pun intended). If the number is larger than 0x1p53 (to use Java 7's hexadecimal floating point notation, = 9007199254740992), then the ulp is 2 at that point and so 0x1p53 + 1 is rounded down to 0x1p53 (and 0x1p53 + 3 is rounded up to 0x1p53 + 4, because of round-to-even). :-D But certainly, if your number is smaller than 9 quadrillion, you should be fine. :-P – Cyano 3/12, 2014 at 13:28

Floating point numbers stored in the computer consist of two parts, an integer and an exponent that the base is taken to and multiplied by the integer part.

If the computer were working in base 10, 0.1 would be 1 x 10⁻¹, 0.2 would be 2 x 10⁻¹, and 0.3 would be 3 x 10⁻¹. Integer math is easy and exact, so adding 0.1 + 0.2 will obviously result in 0.3.

Computers don't usually work in base 10, they work in base 2. You can still get exact results for some values, for example 0.5 is 1 x 2⁻¹ and 0.25 is 1 x 2⁻², and adding them results in 3 x 2⁻², or 0.75. Exactly.

The problem comes with numbers that can be represented exactly in base 10, but not in base 2. Those numbers need to be rounded to their closest equivalent. Assuming the very common IEEE 64-bit floating point format, the closest number to 0.1 is 3602879701896397 x 2⁻⁵⁵, and the closest number to 0.2 is 7205759403792794 x 2⁻⁵⁵; adding them together results in 10808639105689191 x 2⁻⁵⁵, or an exact decimal value of 0.3000000000000000444089209850062616169452667236328125. Floating point numbers are generally rounded for display.

Carangid answered 16/3, 2016 at 5:27 Comment(3)

@Mark Thank you for this Clear explanation but then the question arises why 0.1+0.4 exactly adds up to 0.5 (atleast in Python 3) . Also what is the best way to check equality when using floats in Python 3? – Wakayama 20/1, 2018 at 3:15

@user2417881 IEEE floating point operations have rounding rules for every operation, and sometimes the rounding can produce an exact answer even when the two numbers are off by a little. The details are too long for a comment and I'm not an expert in them anyway. As you see in this answer 0.5 is one of the few decimals that can be represented in binary, but that's just a coincidence. For equality testing see stackoverflow.com/questions/5595425/…. – Carangid 20/1, 2018 at 4:35

@user2417881 your question intrigued me so I turned it into a full question and answer: https://mcmap.net/q/23351/-why-does-0-1-0-4-0-5/5987 – Carangid 22/1, 2018 at 4:27

In short it's because:

Floating point numbers cannot represent all decimals precisely in binary

So just like 10/3 which does not exist in base 10 precisely (it will be 3.33... recurring), in the same way 1/10 doesn't exist in binary.

So what? How to deal with it? Is there any workaround?

In order to offer The best solution I can say I discovered following method:

parseFloat((0.1 + 0.2).toFixed(10)) => Will return 0.3

Let me explain why it's the best solution. As others mentioned in above answers it's a good idea to use ready to use Javascript toFixed() function to solve the problem. But most likely you'll encounter with some problems.

Imagine you are going to add up two float numbers like 0.2 and 0.7 here it is: 0.2 + 0.7 = 0.8999999999999999.

Your expected result was 0.9 it means you need a result with 1 digit precision in this case. So you should have used (0.2 + 0.7).tofixed(1) but you can't just give a certain parameter to toFixed() since it depends on the given number, for instance

0.22 + 0.7 = 0.9199999999999999

In this example you need 2 digits precision so it should be toFixed(2), so what should be the paramter to fit every given float number?

You might say let it be 10 in every situation then:

(0.2 + 0.7).toFixed(10) => Result will be "0.9000000000"

Damn! What are you going to do with those unwanted zeros after 9? It's the time to convert it to float to make it as you desire:

parseFloat((0.2 + 0.7).toFixed(10)) => Result will be 0.9

Now that you found the solution, it's better to offer it as a function like this:

function floatify(number){
           return parseFloat((number).toFixed(10));
        }

Let's try it yourself:

function floatify(number){
       return parseFloat((number).toFixed(10));
    }
 
function addUp(){
  var number1 = +$("#number1").val();
  var number2 = +$("#number2").val();
  var unexpectedResult = number1 + number2;
  var expectedResult = floatify(number1 + number2);
  $("#unexpectedResult").text(unexpectedResult);
  $("#expectedResult").text(expectedResult);
}
addUp();

input{
  width: 50px;
}
#expectedResult{
color: green;
}
#unexpectedResult{
color: red;
}

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<input id="number1" value="0.2" onclick="addUp()" onkeyup="addUp()"/> +
<input id="number2" value="0.7" onclick="addUp()" onkeyup="addUp()"/> =
<p>Expected Result: <span id="expectedResult"></span></p>
<p>Unexpected Result: <span id="unexpectedResult"></span></p>

You can use it this way:

var x = 0.2 + 0.7;
floatify(x);  => Result: 0.9

As W3SCHOOLS suggests there is another solution too, you can multiply and divide to solve the problem above:

var x = (0.2 * 10 + 0.1 * 10) / 10;       // x will be 0.3

Keep in mind that (0.2 + 0.1) * 10 / 10 won't work at all although it seems the same! I prefer the first solution since I can apply it as a function which converts the input float to accurate output float.

FYI, the same problem exists for multiplication, for instance 0.09 * 10 returns 0.8999999999999999. Apply the floatify function as a workaround: floatify(0.09 * 10) returns 0.9

Englebert answered 7/8, 2018 at 9:34 Comment(5)

this made me a real headache. I sum 12 float numbers, then show sum and the average if those numbers. using toFixed() might fix the summing of 2 numbers, but when sum several numbers the leap is significant. – Emilioemily 17/3, 2020 at 12:27

@Nuryagdy Mustapayev I didn't get your intention, as I tested before you can sum 12 float numbers, then use the floatify() function on the result, then do whatever you want on it, I observed no issue using it. – Englebert 18/3, 2020 at 19:21

I am just saying in my situation where I have around 20 parameters and 20 formulas where the result of each formula depends on others this solution did not help. – Emilioemily 20/3, 2020 at 9:15

Some pedantry: binary floating-point can't represent exact decimals. Systems using decimal floating-point have no problem here (but have other compromises, notably that precision and range are smaller than for binary). Systems with native decimal fp include IBM z9 and POWER6 processors. – Icaria 17/1, 2023 at 8:53

But the asker has already gone wrong at 0.1. – Euchromatin 11/9, 2023 at 23:23

Floating point rounding error. From What Every Computer Scientist Should Know About Floating-Point Arithmetic:

Squeezing infinitely many real numbers into a finite number of bits requires an approximate representation. Although there are infinitely many integers, in most programs the result of integer computations can be stored in 32 bits. In contrast, given any fixed number of bits, most calculations with real numbers will produce quantities that cannot be exactly represented using that many bits. Therefore the result of a floating-point calculation must often be rounded in order to fit back into its finite representation. This rounding error is the characteristic feature of floating-point computation.

Seidule answered 25/2, 2009 at 21:42 Comment(0)

My workaround:

function add(a, b, precision) {
    var x = Math.pow(10, precision || 2);
    return (Math.round(a * x) + Math.round(b * x)) / x;
}

precision refers to the number of digits you want to preserve after the decimal point during addition.

Fluctuation answered 26/12, 2011 at 6:51 Comment(0)

No, not broken, but most decimal fractions must be approximated

Summary

Floating point arithmetic is exact, unfortunately, it doesn't match up well with our usual base-10 number representation, so it turns out we are often giving it input that is slightly off from what we wrote.

Even simple numbers like 0.01, 0.02, 0.03, 0.04 ... 0.24 are not representable exactly as binary fractions. If you count up 0.01, .02, .03 ..., not until you get to 0.25 will you get the first fraction representable in base₂. If you tried that using FP, your 0.01 would have been slightly off, so the only way to add 25 of them up to a nice exact 0.25 would have required a long chain of causality involving guard bits and rounding. It's hard to predict so we throw up our hands and say "FP is inexact", but that's not really true.

We constantly give the FP hardware something that seems simple in base 10 but is a repeating fraction in base 2.

How did this happen?

When we write in decimal, every fraction (specifically, every terminating decimal) is a rational number of the form

a / (2ⁿ x 5^m)

In binary, we only get the 2ⁿ term, that is:

a / 2ⁿ

So in decimal, we can't represent ¹/₃. Because base 10 includes 2 as a prime factor, every number we can write as a binary fraction also can be written as a base 10 fraction. However, hardly anything we write as a base₁₀ fraction is representable in binary. In the range from 0.01, 0.02, 0.03 ... 0.99, only three numbers can be represented in our FP format: 0.25, 0.50, and 0.75, because they are 1/4, 1/2, and 3/4, all numbers with a prime factor using only the 2ⁿ term.

In base₁₀ we can't represent ¹/₃. But in binary, we can't do ¹/₁₀ or ¹/₃.

So while every binary fraction can be written in decimal, the reverse is not true. And in fact most decimal fractions repeat in binary.

Dealing with it

Developers are usually instructed to do < epsilon comparisons, better advice might be to round to integral values (in the C library: round() and roundf(), i.e., stay in the FP format) and then compare. Rounding to a specific decimal fraction length solves most problems with output.

Also, on real number-crunching problems (the problems that FP was invented for on early, frightfully expensive computers) the physical constants of the universe and all other measurements are only known to a relatively small number of significant figures, so the entire problem space was "inexact" anyway. FP "accuracy" isn't a problem in this kind of application.

The whole issue really arises when people try to use FP for bean counting. It does work for that, but only if you stick to integral values, which kind of defeats the point of using it. This is why we have all those decimal fraction software libraries.

I love the Pizza answer by Chris, because it describes the actual problem, not just the usual handwaving about "inaccuracy". If FP were simply "inaccurate", we could fix that and would have done it decades ago. The reason we haven't is because the FP format is compact and fast and it's the best way to crunch a lot of numbers. Also, it's a legacy from the space age and arms race and early attempts to solve big problems with very slow computers using small memory systems. (Sometimes, individual magnetic cores for 1-bit storage, but that's another story.)

Conclusion

If you are just counting beans at a bank, software solutions that use decimal string representations in the first place work perfectly well. But you can't do quantum chromodynamics or aerodynamics that way.

Amusement answered 2/2, 2016 at 23:49 Comment(8)

Rounding to the nearest integer isn't a safe way to solve the comparison problem in all cases. 0.4999998 and 0.500001 round to different integers, so there's a "danger zone" around every rounding cut-point. (I know those decimal strings probably aren't exactly representable as IEEE binary floats.) – Jesselyn 9/12, 2016 at 3:31

Also, even though floating point is a "legacy" format, it's very well designed. I don't know of anything that anyone would change if re-designing it now. The more I learn about it, the more I think it's really well designed. e.g. the biased exponent means consecutive binary floats have consecutive integer representations, so you can implement nextafter() with an integer increment or decrement on the binary representation of an IEEE float. Also, you can compare floats as integers and get the right answer except when they're both negative (because of sign-magnitude vs. 2's complement). – Jesselyn 9/12, 2016 at 3:35

I disagree, the floats should be stored as decimals and not binary and all problems are solved. – Bangka 19/2, 2017 at 19:32

Shouldn't "x / (2^n + 5^n)" be "x / (2^n * 5^n)"? – Hydrozoan 5/2, 2018 at 7:34

@stephen c you will be able to define the precision you want at the compiler settings. But it will just round the result, like in a calculator. – Bangka 16/8, 2018 at 8:45

@RonenFestinger: All problems? No, the fundamental problem remains even when storing as decimal floating point, e.g. (1/3) * 3 != 1 in such a format. – Wardwarde 6/1, 2022 at 13:44

It's better than 0.1 + 0.2 != 0.3 – Bangka 7/1, 2022 at 14:14

Re "Floating point arithmetic is exact": What about division? For example, 22.0 / 7.0. – Acetylate 3/12, 2023 at 18:11

Not all numbers can be represented via floats/doubles. For example, the number "0.2" will be represented as "0.200000003" in single precision in the IEEE 754 float point standard.

The model for storing real numbers under the hood represents float numbers as

Even though you can type 0.2 easily, FLT_RADIX and DBL_RADIX is 2; not 10 for a computer with an FPU which uses "IEEE Standard for Binary Floating-Point Arithmetic (ISO/IEEE Std 754-1985)".

So it is a bit hard to represent such numbers exactly. Even if you specify this variable explicitly without any intermediate calculation.

Cincture answered 5/10, 2014 at 18:39 Comment(0)

Some statistics related to this famous double precision question.

When adding all values (a + b) using a step of 0.1 (from 0.1 to 100) we have ~15% chance of precision error. Note that the error could result in slightly bigger or smaller values. Here are some examples:

0.1 + 0.2 = 0.30000000000000004 (BIGGER)
0.1 + 0.7 = 0.7999999999999999 (SMALLER)
...
1.7 + 1.9 = 3.5999999999999996 (SMALLER)
1.7 + 2.2 = 3.9000000000000004 (BIGGER)
...
3.2 + 3.6 = 6.800000000000001 (BIGGER)
3.2 + 4.4 = 7.6000000000000005 (BIGGER)

When subtracting all values (a - b where a > b) using a step of 0.1 (from 100 to 0.1) we have ~34% chance of precision error. Here are some examples:

0.6 - 0.2 = 0.39999999999999997 (SMALLER)
0.5 - 0.4 = 0.09999999999999998 (SMALLER)
...
2.1 - 0.2 = 1.9000000000000001 (BIGGER)
2.0 - 1.9 = 0.10000000000000009 (BIGGER)
...
100 - 99.9 = 0.09999999999999432 (SMALLER)
100 - 99.8 = 0.20000000000000284 (BIGGER)

*15% and 34% are indeed huge, so always use BigDecimal when precision is of big importance. With 2 decimal digits (step 0.01) the situation worsens a bit more (18% and 36%).

Belding answered 3/1, 2015 at 12:12 Comment(0)

Some high level languages such as Python and Java come with tools to overcome binary floating point limitations. For example:

Python's decimal module and Java's BigDecimal class, that represent numbers internally with decimal notation (as opposed to binary notation). Both have limited precision, so they are still error prone, however they solve most common problems with binary floating point arithmetic.

Decimals are very nice when dealing with money: ten cents plus twenty cents are always exactly thirty cents:
```
  >>> 0.1 + 0.2 == 0.3
  False
  >>> Decimal('0.1') + Decimal('0.2') == Decimal('0.3')
  True
```
Python's decimal module is based on IEEE standard 854-1987.
Python's fractions module and Apache Common's BigFraction class. Both represent rational numbers as (numerator, denominator) pairs and they may give more accurate results than decimal floating point arithmetic.

Neither of these solutions is perfect (especially if we look at performances, or if we require a very high precision), but still they solve a great number of problems with binary floating point arithmetic.

Schweiker answered 21/8, 2015 at 14:53 Comment(1)

We may also use fixed point. For example if cents is your finest granularity, then calculations can be done with integers on number of cents instead of dollars. – Stepdaughter 18/7, 2020 at 21:58

People always assume this to be a computer problem, but if you count with your hands (base 10), you can't get (1/3+1/3=2/3)=true unless you have infinity to add 0.333... to 0.333... so just as with the (1/10+2/10)!==3/10 problem in base 2, you truncate it to 0.333 + 0.333 = 0.666 and probably round it to 0.667 which would be also be technically inaccurate.

Count in ternary, and thirds are not a problem though - maybe some race with 15 fingers on each hand would ask why your decimal math was broken...

Confluent answered 18/3, 2016 at 0:38 Comment(4)

Since humans use decimal numbers, I see no good reason why the floats are not represented as a decimal by default so we have accurate results. – Bangka 19/2, 2017 at 19:27

Humans use many bases other than base 10 (decimals), binary being the one we use most for computing.. the 'good reason' is that you simply cant represent every fraction in every base.. – Confluent 20/2, 2017 at 8:59

@RonenFestinger binary arithmetic is easy to implement on computers because it requires only eight basic operations with digits: say $a$, $b$ in $0,1$ all you need to know is $\operatorname{xor}(a,b)$ and $\operatorname{cb}(a,b)$, where xor is exclusive or and cb is the "carry bit" which is $0$ in all cases except when $a=1=b$, in which case we have one (in fact commutativity of all operations saves you $2$ cases and all you need is $6$ rules). Decimal expansion needs $10\times 11$ (in decimal notation) cases to be stored and $10$ different states for each bit and wastes storage on the carry. – Adopt 25/3, 2018 at 6:36

@RonenFestinger - Decimal is NOT more accurate. That is what this answer is saying. For any base you chose, there will be rational numbers (fractions) that give an infinitely repeating digit sequences. For the record, some of first computers did use base 10 representations for numbers, but the pioneering computer hardware designers soon concluded that base 2 was much easier and more efficient to implement. – Parttime 7/7, 2019 at 5:26

Did you try the duct tape solution?

Try to determine when errors occur and fix them with short if statements. It's not pretty, but for some problems it is the only solution and this is one of them.

 if( (n * 0.1) < 100.0 ) { return n * 0.1 - 0.000000000000001 ;}
                    else { return n * 0.1 + 0.000000000000001 ;}

I had the same problem in a scientific simulation project in C#, and I can tell you that if you ignore the butterfly effect, it's going to turn to a big fat dragon and bite you in the a**.

Weddle answered 1/8, 2012 at 7:2 Comment(0)

Those weird numbers appear because computers use the binary (base 2) number system for calculation purposes, while we use decimal (base 10).

There are a majority of fractional numbers that cannot be represented precisely either in binary or in decimal or both. Result - A rounded up (but precise) number results.

Sprag answered 14/10, 2013 at 16:45 Comment(1)

@Nae I would translate the second paragraph as "The majority of fractions cannot be represented exactly in either decimal or binary. So most results will be rounded off -- although they will still be precise to the number of bits/digits inherent in the representation being used." – Filch 9/3, 2018 at 14:19

Many of this question's numerous duplicates ask about the effects of floating point rounding on specific numbers. In practice, it is easier to get a feeling for how it works by looking at exact results of calculations of interest rather than by just reading about it. Some languages provide ways of doing that - such as converting a float or double to BigDecimal in Java.

Since this is a language-agnostic question, it needs language-agnostic tools, such as a Decimal to Floating-Point Converter.

Applying it to the numbers in the question, treated as doubles:

0.1 converts to 0.1000000000000000055511151231257827021181583404541015625,

0.2 converts to 0.200000000000000011102230246251565404236316680908203125,

0.3 converts to 0.299999999999999988897769753748434595763683319091796875, and

0.30000000000000004 converts to 0.3000000000000000444089209850062616169452667236328125.

Adding the first two numbers manually or in a decimal calculator such as Full Precision Calculator, shows the exact sum of the actual inputs is 0.3000000000000000166533453693773481063544750213623046875.

If it were rounded down to the equivalent of 0.3 the rounding error would be 0.0000000000000000277555756156289135105907917022705078125. Rounding up to the equivalent of 0.30000000000000004 also gives rounding error 0.0000000000000000277555756156289135105907917022705078125. The round-to-even tie breaker applies.

Returning to the floating point converter, the raw hexadecimal for 0.30000000000000004 is 3fd3333333333334, which ends in an even digit and therefore is the correct result.

Kaffiyeh answered 21/12, 2015 at 11:15 Comment(2)

... also, the round to even refers to the binary representation, not the decimal representation. See this or, for instance, this. – Hydrozoan 12/1, 2018 at 19:33

@WaiHaLee I did not apply the odd/even test to any decimal numbers, only hexadecimal. A hexadecimal digit is even if, and only if, the least significant bit of its binary expansion is zero. – Kaffiyeh 25/9, 2019 at 16:39

The kind of floating-point math that can be implemented in a digital computer necessarily uses an approximation of the real numbers and operations on them. (The standard version runs to over fifty pages of documentation and has a committee to deal with its errata and further refinement.)

This approximation is a mixture of approximations of different kinds, each of which can either be ignored or carefully accounted for due to its specific manner of deviation from exactitude. It also involves a number of explicit exceptional cases at both the hardware and software levels that most people walk right past while pretending not to notice.

If you need infinite precision (using the number π, for example, instead of one of its many shorter stand-ins), you should write or use a symbolic math program instead.

But if you're okay with the idea that sometimes floating-point math is fuzzy in value and logic and errors can accumulate quickly, and you can write your requirements and tests to allow for that, then your code can frequently get by with what's in your FPU.

Ation answered 5/10, 2015 at 15:55 Comment(0)

Just for fun, I played with the representation of floats, following the definitions from the Standard C99 and I wrote the code below.

The code prints the binary representation of floats in 3 separated groups

SIGN EXPONENT FRACTION

and after that it prints a sum, that, when summed with enough precision, it will show the value that really exists in hardware.

So when you write float x = 999..., the compiler will transform that number in a bit representation printed by the function xx such that the sum printed by the function yy be equal to the given number.

In reality, this sum is only an approximation. For the number 999,999,999, the compiler will insert in bit representation of the float the number 1,000,000,000.

After the code I attach a console session, in which I compute the sum of terms for both constants (minus PI and 999999999) that really exists in hardware, inserted there by the compiler.

#include <stdio.h>
#include <limits.h>

void
xx(float *x)
{
    unsigned char i = sizeof(*x)*CHAR_BIT-1;
    do {
        switch (i) {
        case 31:
             printf("sign: ");
             break;
        case 30:
             printf("exponent: ");
             break;
        case 23:
             printf("fraction: ");
             break;

        }
        char b = (*(unsigned long long*)x&((unsigned long long)1<<i)) != 0;
        printf("%d ", b);
    } while (i--);
    printf("\n");
}

void
yy(float a)
{
    int sign = !(*(unsigned long long*)&a&((unsigned long long)1<<31));
    int fraction = ((1<<23)-1)&(*(int*)&a);
    int exponent = (255&((*(int*)&a)>>23))-127;

    printf(sign ? "positive" " ( 1+" : "negative" " ( 1+");
    unsigned int i = 1 << 22;
    unsigned int j = 1;
    do {
        char b = (fraction&i) != 0;
        b && (printf("1/(%d) %c", 1 << j, (fraction&(i-1)) ? '+' : ')' ), 0);
    } while (j++, i >>= 1);

    printf("*2^%d", exponent);
    printf("\n");
}

void
main()
{
    float x = -3.14;
    float y = 999999999;
    printf("%lu\n", sizeof(x));
    xx(&x);
    xx(&y);
    yy(x);
    yy(y);
}

Here is a console session in which I compute the real value of the float that exists in hardware. I used bc to print the sum of terms outputted by the main program. One can insert that sum in python repl or something similar also.

-- .../terra1/stub
@ qemacs f.c
-- .../terra1/stub
@ gcc f.c
-- .../terra1/stub
@ ./a.out
sign: 1 exponent: 1 0 0 0 0 0 0 fraction: 0 1 0 0 1 0 0 0 1 1 1 1 0 1 0 1 1 1 0 0 0 0 1 1
sign: 0 exponent: 1 0 0 1 1 1 0 fraction: 0 1 1 0 1 1 1 0 0 1 1 0 1 0 1 1 0 0 1 0 1 0 0 0
negative ( 1+1/(2) +1/(16) +1/(256) +1/(512) +1/(1024) +1/(2048) +1/(8192) +1/(32768) +1/(65536) +1/(131072) +1/(4194304) +1/(8388608) )*2^1
positive ( 1+1/(2) +1/(4) +1/(16) +1/(32) +1/(64) +1/(512) +1/(1024) +1/(4096) +1/(16384) +1/(32768) +1/(262144) +1/(1048576) )*2^29
-- .../terra1/stub
@ bc
scale=15
( 1+1/(2) +1/(4) +1/(16) +1/(32) +1/(64) +1/(512) +1/(1024) +1/(4096) +1/(16384) +1/(32768) +1/(262144) +1/(1048576) )*2^29
999999999.999999446351872

That's it. The value of 999999999 is in fact

999999999.999999446351872

You can also check with bc that -3.14 is also perturbed. Do not forget to set a scale factor in bc.

The displayed sum is what inside the hardware. The value you obtain by computing it depends on the scale you set. I did set the scale factor to 15. Mathematically, with infinite precision, it seems it is 1,000,000,000.

Innervate answered 29/12, 2016 at 10:29 Comment(0)

Since Python 3.5, you have been able to use the math.isclose() function for testing approximate equality:

>>> import math
>>> math.isclose(0.1 + 0.2, 0.3)
True
>>> 0.1 + 0.2 == 0.3
False

Corwin answered 8/8, 2018 at 8:47 Comment(0)

Floating point numbers are represented, at the hardware level, as fractions of binary numbers (base 2). For example, the decimal fraction:

0.125

has the value 1/10 + 2/100 + 5/1000 and, in the same way, the binary fraction:

0.001

has the value 0/2 + 0/4 + 1/8. These two fractions have the same value, the only difference is that the first is a decimal fraction, the second is a binary fraction.

Unfortunately, most decimal fractions cannot have exact representation in binary fractions. Therefore, in general, the floating point numbers you give are only approximated to binary fractions to be stored in the machine.

The problem is easier to approach in base 10. Take for example, the fraction 1/3. You can approximate it to a decimal fraction:

0.3

or better,

0.33

or better,

0.333

etc. No matter how many decimal places you write, the result is never exactly 1/3, but it is an estimate that always comes closer.

Likewise, no matter how many base 2 decimal places you use, the decimal value 0.1 cannot be represented exactly as a binary fraction. In base 2, 1/10 is the following periodic number:

0.0001100110011001100110011001100110011001100110011 ...

Stop at any finite amount of bits, and you'll get an approximation.

For Python, on a typical machine, 53 bits are used for the precision of a float, so the value stored when you enter the decimal 0.1 is the binary fraction.

0.00011001100110011001100110011001100110011001100110011010

which is close, but not exactly equal, to 1/10.

It's easy to forget that the stored value is an approximation of the original decimal fraction, due to the way floats are displayed in the interpreter. Python only displays a decimal approximation of the value stored in binary. If Python were to output the true decimal value of the binary approximation stored for 0.1, it would output:

>>> 0.1
0.1000000000000000055511151231257827021181583404541015625

This is a lot more decimal places than most people would expect, so Python displays a rounded value to improve readability:

>>> 0.1
0.1

It is important to understand that in reality this is an illusion: the stored value is not exactly 1/10, it is simply on the display that the stored value is rounded. This becomes evident as soon as you perform arithmetic operations with these values:

>>> 0.1 + 0.2
0.30000000000000004

This behavior is inherent to the very nature of the machine's floating-point representation: it is not a bug in Python, nor is it a bug in your code. You can observe the same type of behavior in all other languages that use hardware support for calculating floating point numbers (although some languages do not make the difference visible by default, or not in all display modes).

Another surprise is inherent in this one. For example, if you try to round the value 2.675 to two decimal places, you will get

>>> round (2.675, 2)
2.67

The documentation for the round() primitive indicates that it rounds to the nearest value away from zero. Since the decimal fraction is exactly halfway between 2.67 and 2.68, you should expect to get (a binary approximation of) 2.68. This is not the case, however, because when the decimal fraction 2.675 is converted to a float, it is stored by an approximation whose exact value is :

2.67499999999999982236431605997495353221893310546875

Since the approximation is slightly closer to 2.67 than 2.68, the rounding is down.

If you are in a situation where rounding decimal numbers halfway down matters, you should use the decimal module. By the way, the decimal module also provides a convenient way to "see" the exact value stored for any float.

>>> from decimal import Decimal
>>> Decimal (2.675)
>>> Decimal ('2.67499999999999982236431605997495353221893310546875')

Another consequence of the fact that 0.1 is not exactly stored in 1/10 is that the sum of ten values of 0.1 does not give 1.0 either:

>>> sum = 0.0
>>> for i in range (10):
... sum + = 0.1
...>>> sum
0.9999999999999999

The arithmetic of binary floating point numbers holds many such surprises. The problem with "0.1" is explained in detail below, in the section "Representation errors". See The Perils of Floating Point for a more complete list of such surprises.

It is true that there is no simple answer, however do not be overly suspicious of floating virtual numbers! Errors, in Python, in floating-point number operations are due to the underlying hardware, and on most machines are no more than 1 in 2 ** 53 per operation. This is more than necessary for most tasks, but you should keep in mind that these are not decimal operations, and every operation on floating point numbers may suffer from a new error.

Although pathological cases exist, for most common use cases you will get the expected result at the end by simply rounding up to the number of decimal places you want on the display. For fine control over how floats are displayed, see String Formatting Syntax for the formatting specifications of the str.format () method.

This part of the answer explains in detail the example of "0.1" and shows how you can perform an exact analysis of this type of case on your own. We assume that you are familiar with the binary representation of floating point numbers.The term Representation error means that most decimal fractions cannot be represented exactly in binary. This is the main reason why Python (or Perl, C, C ++, Java, Fortran, and many others) usually doesn't display the exact result in decimal:

>>> 0.1 + 0.2
0.30000000000000004

Why? 1/10 and 2/10 are not representable exactly in binary fractions. However, all machines today (July 2010) follow the IEEE-754 standard for the arithmetic of floating point numbers. and most platforms use an "IEEE-754 double precision" to represent Python floats. Double precision IEEE-754 uses 53 bits of precision, so on reading the computer tries to convert 0.1 to the nearest fraction of the form J / 2 ** N with J an integer of exactly 53 bits. Rewrite:

1/10 ~ = J / (2 ** N)

in:

J ~ = 2 ** N / 10

remembering that J is exactly 53 bits (so> = 2 ** 52 but <2 ** 53), the best possible value for N is 56:

>>> 2 ** 52
4503599627370496
>>> 2 ** 53
9007199254740992
>>> 2 ** 56/10
7205759403792793

So 56 is the only possible value for N which leaves exactly 53 bits for J. The best possible value for J is therefore this quotient, rounded:

>>> q, r = divmod (2 ** 56, 10)
>>> r
6

Since the carry is greater than half of 10, the best approximation is obtained by rounding up:

>>> q + 1
7205759403792794

Therefore the best possible approximation for 1/10 in "IEEE-754 double precision" is this above 2 ** 56, that is:

7205759403792794/72057594037927936

Note that since the rounding was done upward, the result is actually slightly greater than 1/10; if we hadn't rounded up, the quotient would have been slightly less than 1/10. But in no case is it exactly 1/10!

So the computer never "sees" 1/10: what it sees is the exact fraction given above, the best approximation using the double precision floating point numbers from the "" IEEE-754 ":

>>>. 1 * 2 ** 56
7205759403792794.0

If we multiply this fraction by 10 ** 30, we can observe the values of its 30 decimal places of strong weight.

>>> 7205759403792794 * 10 ** 30 // 2 ** 56
100000000000000005551115123125L

meaning that the exact value stored in the computer is approximately equal to the decimal value 0.100000000000000005551115123125. In versions prior to Python 2.7 and Python 3.1, Python rounded these values to 17 significant decimal places, displaying “0.10000000000000001”. In current versions of Python, the displayed value is the value whose fraction is as short as possible while giving exactly the same representation when converted back to binary, simply displaying “0.1”.

Perlis answered 3/8, 2020 at 15:3 Comment(0)

The trap with floating point numbers is that they look like decimal but they work in binary.

The only prime factor of 2 is 2, while 10 has prime factors of 2 and 5. The result of this is that every number that can be written exactly as a binary fraction can also be written exactly as a decimal fraction but only a subset of numbers that can be written as decimal fractions can be written as binary fractions.

A floating point number is essentially a binary fraction with a limited number of significant digits. If you go past those significant digits then the results will be rounded.

When you type a literal in your code or call the function to parse a floating point number to a string, it expects a decimal number and it stores a binary approximation of that decimal number in the variable.

When you print a floating point number or call the function to convert one to a string it prints a decimal approximation of the floating point number. It is possible to convert a binary number to decimal exactly, but no language I'm aware of does that by default when converting to a string*. Some languages use a fixed number of significant digits, others use the shortest string that will "round trip" back to the same floating point value.

* Python does convert exactly when converting a floating point number to a "decimal.Decimal". This is the easiest way I know of to obtain the exact decimal equivalent of a floating point number.

Volcano answered 16/9, 2021 at 0:28 Comment(0)

Another way to look at this: Used are 64 bits to represent numbers. As consequence there is no way more than 2**64 = 18,446,744,073,709,551,616 different numbers can be precisely represented.

However, Math says there are already infinitely many decimals between 0 and 1. IEE 754 defines an encoding to use these 64 bits efficiently for a much larger number space plus NaN and +/- Infinity, so there are gaps between accurately represented numbers filled with numbers only approximated.

Unfortunately 0.3 sits in a gap.

Collegian answered 19/12, 2017 at 22:37 Comment(0)

Imagine working in base ten with, say, 8 digits of accuracy. You check whether

1/3 + 2 / 3 == 1

and learn that this returns false. Why? Well, as real numbers we have

1/3 = 0.333.... and 2/3 = 0.666....

Truncating at eight decimal places, we get

0.33333333 + 0.66666666 = 0.99999999

which is, of course, different from 1.00000000 by exactly 0.00000001.

The situation for binary numbers with a fixed number of bits is exactly analogous. As real numbers, we have

1/10 = 0.0001100110011001100... (base 2)

and

1/5 = 0.0011001100110011001... (base 2)

If we truncated these to, say, seven bits, then we'd get

0.0001100 + 0.0011001 = 0.0100101

while on the other hand,

3/10 = 0.01001100110011... (base 2)

which, truncated to seven bits, is 0.0100110, and these differ by exactly 0.0000001.

The exact situation is slightly more subtle because these numbers are typically stored in scientific notation. So, for instance, instead of storing 1/10 as 0.0001100 we may store it as something like 1.10011 * 2^-4, depending on how many bits we've allocated for the exponent and the mantissa. This affects how many digits of precision you get for your calculations.

The upshot is that because of these rounding errors you essentially never want to use == on floating-point numbers. Instead, you can check if the absolute value of their difference is smaller than some fixed small number.

Beaudette answered 20/12, 2018 at 18:27 Comment(0)

Decimal numbers such as 0.1, 0.2, and 0.3 are not represented exactly in binary encoded floating point types. The sum of the approximations for 0.1 and 0.2 differs from the approximation used for 0.3, hence the falsehood of 0.1 + 0.2 == 0.3 as can be seen more clearly here:

#include <stdio.h>

int main() {
    printf("0.1 + 0.2 == 0.3 is %s\n", 0.1 + 0.2 == 0.3 ? "true" : "false");
    printf("0.1 is %.23f\n", 0.1);
    printf("0.2 is %.23f\n", 0.2);
    printf("0.1 + 0.2 is %.23f\n", 0.1 + 0.2);
    printf("0.3 is %.23f\n", 0.3);
    printf("0.3 - (0.1 + 0.2) is %g\n", 0.3 - (0.1 + 0.2));
    return 0;
}

Output:

0.1 + 0.2 == 0.3 is false
0.1 is 0.10000000000000000555112
0.2 is 0.20000000000000001110223
0.1 + 0.2 is 0.30000000000000004440892
0.3 is 0.29999999999999998889777
0.3 - (0.1 + 0.2) is -5.55112e-17

For these computations to be evaluated more reliably, you would need to use a decimal-based representation for floating point values. The C Standard does not specify such types by default but as an extension described in a technical Report.

The _Decimal32, _Decimal64 and _Decimal128 types might be available on your system (for example, GCC supports them on selected targets, but Clang does not support them on OS X).

Lashondra answered 22/4, 2019 at 1:2 Comment(0)

It's actually pretty simple. When you have a base 10 system (like ours), it can only express fractions that use a prime factor of the base. The prime factors of 10 are 2 and 5. So 1/2, 1/4, 1/5, 1/8, and 1/10 can all be expressed cleanly because the denominators all use prime factors of 10. In contrast, 1/3, 1/6, and 1/7 are all repeating decimals because their denominators use a prime factor of 3 or 7. In binary (or base 2), the only prime factor is 2. So you can only express fractions cleanly which only contain 2 as a prime factor. In binary, 1/2, 1/4, 1/8 would all be expressed cleanly as decimals. While, 1/5 or 1/10 would be repeating decimals. So 0.1 and 0.2 (1/10 and 1/5) while clean decimals in a base 10 system, are repeating decimals in the base 2 system the computer is operating in. When you do math on these repeating decimals, you end up with leftovers which carry over when you convert the computer's base 2 (binary) number into a more human readable base 10 number.

From https://0.30000000000000004.com/

Rosalba answered 7/5, 2019 at 19:45 Comment(0)

There are projects on fixing floating point implementation issues.

Take a look at Unum & Posit for example, which showcases a number type called posit (and its predecessor unum) that promises to offer better accuracy with fewer bits. If my understanding is correct, it also fixes the kind of problems in the question. It is a quite interesting project, and the person behind is a mathematician, Dr. John Gustafson.

The whole thing is open source, with many actual implementations in C/C++, Python, Julia and C# (https://hastlayer.com/arithmetics).

Saponaceous answered 22/12, 2017 at 16:39 Comment(1)

That improves the rounding-error vs. size in bytes tradeoff, but Posit and similar ideas are still based on a linear mantissa (aka significand) times a power of 2. They don't fix the fact that 0.2 and 0.3 aren't exactly representable. For that you need decimal floating point or a different representation entirely. (e.g. rational numbers as fractions with bigint numerator / denominator, like in some computer-algebra systems.) – Jesselyn 16/12, 2023 at 0:6

Normal arithmetic is base-10, so decimals represent tenths, hundredths, etc. When you try to represent a floating-point number in binary base-2 arithmetic, you are dealing with halves, fourths, eighths, etc.

In the hardware, floating points are stored as integer mantissas and exponents. Mantissa represents the significant digits. Exponent is like scientific notation but it uses a base of 2 instead of 10. For example 64.0 would be represented with a mantissa of 1 and exponent of 6. 0.125 would be represented with a mantissa of 1 and an exponent of -3.

Floating point decimals have to add up negative powers of 2

0.1b = 0.5d
0.01b = 0.25d
0.001b = 0.125d
0.0001b = 0.0625d
0.00001b = 0.03125d

and so on.

It is common to use a error delta instead of using equality operators when dealing with floating point arithmetic. Instead of

if(a==b) ...

you would use

delta = 0.0001; // or some arbitrarily small amount
if(a - b > -delta && a - b < delta) ...

Leptospirosis answered 20/8, 2020 at 15:38 Comment(0)

I just saw this interesting issue around floating points:

Consider the following results:

error = (2**53+1) - int(float(2**53+1))

>>> (2**53+1) - int(float(2**53+1))
1

We can clearly see a breakpoint when 2**53+1 - all works fine until 2**53.

>>> (2**53) - int(float(2**53))
0

This happens because of the double-precision binary: IEEE 754 double-precision binary floating-point format: binary64

From the Wikipedia page for Double-precision floating-point format:

Double-precision binary floating-point is a commonly used format on PCs, due to its wider range over single-precision floating point, in spite of its performance and bandwidth cost. As with single-precision floating-point format, it lacks precision on integer numbers when compared with an integer format of the same size. It is commonly known simply as double. The IEEE 754 standard specifies a binary64 as having:

Sign bit: 1 bit

Exponent: 11 bits

Significant precision: 53 bits (52 explicitly stored)

The real value assumed by a given 64-bit double-precision datum with a given biased exponent and a 52-bit fraction is

or

Thanks to @a_guest for pointing that out to me.

Ingot answered 5/10, 2019 at 21:46 Comment(0)

Simply ignore the all the nitty-gritty and just think of numeric bases like different geometric shapes—circles, squares, triangles, hexagons, trapezoids (multi-dimensional shapes too), etc.

Say one kind of rectangle has a 3:1 aspect ratio vs. a square — you can fit three squares exactly in that rectangle. The equivalent in the analogy would be larger shape (base) being an integer power of the smaller base,

like 2 vs. 8 = 2 x 2 x 2

And similarly, three of those rectangles stacked on top of each other makes a perfect square that's nine times the area of the original square, which you can also confirm pseudo-mathematically:

2 ^ 9 = 512 = 8 ^ 3

--- (again, this is a different type of non-euclidean analogy, not a 18-dimensional object of a square^9)

Two rectangles, one horizontal 19:13 ratio the other vertical with a 7:13 ratio. Neither maps perfectly into the other with no gap left, but eventually they'll meet since they share the same 13 on one side.

The analogy equivalent would be like numeric bases

216 = 6^3 vs. 7,776 = 6^5

-- meeting at (6 ^ 3) ^ 5 = 470,184,984,576 = (6 ^ 5) ^ 3 - the LCM point of their powers

A circle vs. a hexagon would be equivalent to bases, say, 61 vs. 181. They might get close, but since they simply don't share any prime bases at all, then no matter how many hexagons you have, you simply can’t fit every possible circle within them, and ditto for the reverse.

Then your question is equivalent to asking:

Why doesn't this circle completely fill a finite number of these trapezoids?

Methuselah answered 30/5, 2023 at 3:1 Comment(0)

Unfortunately, there is no elegant way to perform arithmetic operations on float numbers without encountering such errors. This is because processors don't operate with decimal numbers; they use binary ones. Different processors may even work differently, leading to varied errors in calculations. To see what the number 0.2 actually looks like, you can run this code:

(0.2).toPrecision(50)
// 0.20000000000000001110223024625156540423631668090820

It's physically impossible for a computer's memory to store the exact number 0.2. JavaScript's job is to find a binary number that closely approximates 0.2 with a certain number of zeroes following it.

Similarly, there's no elegant way to round these numbers! Even standard JavaScript functions (like toFixed, toPrecision) can be inaccurate. This can vary depending on the computer, as different processors may yield different results.

(8.005).toPrecision(3) // 8.01
(1.005).toPrecision(3) // 1.00
(8.005).toFixed(2) // 8.01
(1.005).toFixed(2) // 1.00
Math.round((8.005) * 100) / 100 // 8.01
Math.round((1.005) * 100) / 100 // 1.00, surprise!

// This is the reason:
(8.005) * 100 = 800.5000000000001
(1.005) * 100 = 100.49999999999999

In my opinion, the simplest way to handle float numbers is to round them frequently, trimming off the accumulating errors from arithmetic operations. This allows us to continue using the standard Number type without complicating our program.

I believe the most reliable rounding method involves converting the number to a string and rounding manually by characters. A number derived from a string will convert back to the same string form, without changes. parseFloat('0.2').toString() === '0.2' This is crucial for eliminating errors, especially considering that all numeric constants in JavaScript code are text, as the entire code is text. JavaScript's compiler converts all numeric constants from the code into numbers just like parseFloat.

Based on these considerations, I wrote functions for correct rounding, based on conversion to and from a string, and optimized performance as much as possible. My solution has been thoroughly tested with different numbers (~130,000 test cases).

You can verify this in your browser console. If you have more elegant or efficient solutions, test them with my test. I would be happy to see such solutions.

My solution

const MAX_FLOAT_PRECISION = 15
const ROUND = 0
const FLOOR = 1
const CEIL = 2
const PRECISION = 0
const FRACTION = 1

function _round(value, digits, digitsType, roundType) {
  if (digitsType === PRECISION && digits <= 0) {
    throw new Error(`Precision digits (${digits}) must be > 0`)
  }
  if (digitsType === FRACTION && digits < 0) {
    throw new Error(`Fraction digits (${digits}) must be >= 0`)
  }
  if (!value) {
    return value
  }
  if ((digitsType === PRECISION && digits < MAX_FLOAT_PRECISION) ||
    digitsType === FRACTION) {
    value = fixFloat(value)
  }
  const negative = value < 0
  const valueAbs = negative ? -value : value
  const str = valueAbs.toExponential()
  const len = str.indexOf('e')
  let precisionDigits = digits
  let exponent
  if (digitsType === FRACTION) {
    exponent = parseInt(str.slice(len + 1))
    precisionDigits += exponent + 1
  }
  let index = precisionDigits > 0 ? precisionDigits + 1 : precisionDigits
  if (index >= len) {
    return value
  }
  let ch = index < 0 ? 0 : str.charCodeAt(index) - 48
  let increment
  switch (roundType) {
    case ROUND:
      increment =
        ch > 5 ||
          (!negative && ch === 5) ||
          (negative && ch === 5 && index < len - 1)
      break
    case FLOOR:
      increment = negative
      break
    case CEIL:
      increment = !negative
      break
  }
  if (precisionDigits <= 0) {
    if (increment) {
      if (typeof exponent === 'undefined') {
        exponent = parseInt(str.slice(len + 1))
      }
      return parseFloat(
        `${negative ? '-' : ''}1e${exponent - precisionDigits + 1}`,
      )
    }
    return negative ? -0 : 0
  }
  if (!increment) {
    return parseFloat(
      (negative ? '-' : '') +
        str.slice(0, index === 2 ? 1 : index) +
        str.slice(len),
    )
  }
  for (let nDigit = precisionDigits - 1; nDigit >= 0; nDigit--) {
    index = nDigit > 0 ? nDigit + 1 : nDigit
    ch = str.charCodeAt(index) - 48
    if (ch < 9) {
      return parseFloat(
        (negative ? '-' : '') + str.slice(0, index) + (ch + 1) + str.slice(len),
      )
    }
  }
  return parseFloat((negative ? '-' : '') + '10' + str.slice(len))
}

function roundPrecision(value, digits) {
  return _round(value, digits, PRECISION, ROUND)
}
function floorPrecision(value, digits) {
  return _round(value, digits, PRECISION, FLOOR)
}
function ceilPrecision(value, digits) {
  return _round(value, digits, PRECISION, CEIL)
}
function roundFraction(value, fractionDigits) {
  return _round(value, fractionDigits || 0, FRACTION, ROUND)
}
function floorFraction(value, fractionDigits) {
  return _round(value, fractionDigits || 0, FRACTION, FLOOR)
}
function ceilFraction(value, fractionDigits) {
  return _round(value, fractionDigits || 0, FRACTION, CEIL)
}
/**
 * @example 0.0000000000001 => 0
 * @example 0.9999999999999 => 1
 */
function fixFloat(value) {
  return roundPrecision(value, MAX_FLOAT_PRECISION)
}

Example of usage

console.log(roundPrecision(100500, 3)); // 101000
console.log(roundPrecision(100499.9, 3)); // 100000
console.log(floorPrecision(100999.9, 3)); // 100000
console.log(ceilPrecision(100000.1, 3)); // 101000
console.log(roundFraction(100.005, 2)); // 100.01
console.log(roundFraction(100.00499, 2)); // 100
console.log(floorFraction(100.00999, 2)); // 100
console.log(ceilFraction(100.00001, 2)); // 100.001

console.log(fixFloat(0.1 + 0.2) === 0.3); // true

Tests

function assertEqual(actual, expected) {
  if (actual !== expected) {
    throw new Error(`Assertion failed: expected ${expected}, got ${actual}`)
  }
}

function assertNotEqual(actual, expected) {
  if (actual === expected) {
    throw new Error(`Assertion failed: expected ${expected}, got ${actual}`)
  }
}

function testRound(func, input, expected) {
  const MIN_EPSILON = 1.1103e-16
  const MAX_EPSILON = 2.8486e-16
  if (input !== 0) {
    assertNotEqual(input - input * MIN_EPSILON, input)
  }
  assertEqual(fixFloat(input + input * MAX_EPSILON), input) // (-0) + (-0) = 0
  assertEqual(
    fixFloat(input - input * MAX_EPSILON),
    Object.is(input, -0) ? 0 : input,
  ) // (-0) - (-0) = 0
  assertEqual(func(input), expected)
  assertEqual(func(input + input * MAX_EPSILON), expected)
  assertEqual(
    func(input - input * MAX_EPSILON),
    Object.is(input, -0) ? 0 : expected,
  )
}

const test_roundPrecision = (input, precision, expected) => {
  testRound(o => roundPrecision(o, precision), input, expected)
}
const test_floorPrecision = (input, precision, expected) => {
  testRound(o => floorPrecision(o, precision), input, expected)
}
const test_ceilPrecision = (input, precision, expected) => {
  testRound(o => ceilPrecision(o, precision), input, expected)
}
const test_roundFraction = (input, fraction, expected) => {
  testRound(o => roundFraction(o, fraction), input, expected)
}
const test_floorFraction = (input, fraction, expected) => {
  testRound(o => floorFraction(o, fraction), input, expected)
}
const test_ceilFraction = (input, fraction, expected) => {
  testRound(o => ceilFraction(o, fraction), input, expected)
}

function test(name, func) {
  func()
  console.log(`✅ ${name}`)
}

test('roundPrecision', () => {
  assertEqual(Math.round(9.5), 10)
  assertEqual(Math.round(-9.5), -9)
  assertEqual(Math.round(9.499999999), 9)
  assertEqual(Math.round(-9.499999999), -9)
  assertEqual(Math.round(9.500000001), 10)
  assertEqual(Math.round(-9.500000001), -10)
  
  test_roundPrecision(0, 1, 0)
  test_roundPrecision(-0, 1, -0)
  test_roundPrecision(9, 1, 9)
  test_roundPrecision(-9, 1, -9)
  test_roundPrecision(9.5, 1, 10)
  test_roundPrecision(-9.5, 1, -9)
  test_roundPrecision(9.499999999, 1, 9)
  test_roundPrecision(-9.499999999, 1, -9)
  test_roundPrecision(9.999999999, 1, 10)
  test_roundPrecision(-9.999999999, 1, -10)
  test_roundPrecision(9.500000001, 1, 10)
  test_roundPrecision(-9.500000001, 1, -10)
  test_roundPrecision(9.0e-50, 1, 9.0e-50)
  test_roundPrecision(-9.0e-50, 1, -9.0e-50)
  test_roundPrecision(9.5e-50, 1, 1.0e-49)
  test_roundPrecision(-9.5e-50, 1, -9.0e-50)
  test_roundPrecision(9.499999999e-50, 1, 9.0e-50)
  test_roundPrecision(-9.499999999e-50, 1, -9.0e-50)
  test_roundPrecision(9.999999999e-50, 1, 1.0e-49)
  test_roundPrecision(-9.999999999e-50, 1, -1.0e-49)
  test_roundPrecision(9.500000001e-50, 1, 1.0e-49)
  test_roundPrecision(-9.500000001e-50, 1, -1.0e-49)
})

test('floorPrecision', () => {
  assertEqual(Math.floor(9.0), 9)
  assertEqual(Math.floor(-9.0), -9)
  assertEqual(Math.floor(9.0000001), 9)
  assertEqual(Math.floor(-9.0000001), -10)
  assertEqual(Math.floor(9.9999999), 9)
  assertEqual(Math.floor(-9.9999999), -10)
  
  test_floorPrecision(0, 1, 0)
  test_floorPrecision(-0, 1, -0)
  test_floorPrecision(9, 1, 9)
  test_floorPrecision(-9, 1, -9)
  test_floorPrecision(9.0, 1, 9)
  test_floorPrecision(-9.0, 1, -9)
  test_floorPrecision(9.0000001, 1, 9)
  test_floorPrecision(-9.0000001, 1, -10)
  test_floorPrecision(9.9999999, 1, 9)
  test_floorPrecision(-9.9999999, 1, -10)
  test_floorPrecision(9.0e-50, 1, 9.0e-50)
  test_floorPrecision(-9.0e-50, 1, -9.0e-50)
  test_floorPrecision(9.0e-50, 1, 9.0e-50)
  test_floorPrecision(-9.0e-50, 1, -9.0e-50)
  test_floorPrecision(9.0000001e-50, 1, 9.0e-50)
  test_floorPrecision(-9.0000001e-50, 1, -1.0e-49)
  test_floorPrecision(9.9999999e-50, 1, 9.0e-50)
  test_floorPrecision(-9.9999999e-50, 1, -1.0e-49)
})

test('ceilPrecision', () => {
  assertEqual(Math.ceil(9.0), 9)
  assertEqual(Math.ceil(-9.0), -9)
  assertEqual(Math.ceil(9.0000001), 10)
  assertEqual(Math.ceil(-9.0000001), -9)
  assertEqual(Math.ceil(9.9999999), 10)
  assertEqual(Math.ceil(-9.9999999), -9)
  
  test_ceilPrecision(0, 1, 0)
  test_ceilPrecision(-0, 1, -0)
  test_ceilPrecision(9, 1, 9)
  test_ceilPrecision(-9, 1, -9)
  test_ceilPrecision(9.0, 1, 9)
  test_ceilPrecision(-9.0, 1, -9)
  test_ceilPrecision(9.0000001, 1, 10)
  test_ceilPrecision(-9.0000001, 1, -9)
  test_ceilPrecision(9.9999999, 1, 10)
  test_ceilPrecision(-9.9999999, 1, -9)
  test_ceilPrecision(9.0e-50, 1, 9.0e-50)
  test_ceilPrecision(-9.0e-50, 1, -9.0e-50)
  test_ceilPrecision(9.0e-50, 1, 9.0e-50)
  test_ceilPrecision(-9.0e-50, 1, -9.0e-50)
  test_ceilPrecision(9.0000001e-50, 1, 1.0e-49)
  test_ceilPrecision(-9.0000001e-50, 1, -9.0e-50)
  test_ceilPrecision(9.9999999e-50, 1, 1.0e-49)
  test_ceilPrecision(-9.9999999e-50, 1, -9.0e-50)
})

test('roundFraction', () => {
  assertEqual(Math.round(-0), -0)
  assertEqual(Math.round(-0.1), -0)
  assertEqual(Math.round(9.5), 10)
  assertEqual(Math.round(-9.5), -9)
  assertEqual(Math.round(9.499999999), 9)
  assertEqual(Math.round(-9.499999999), -9)
  assertEqual(Math.round(9.500000001), 10)
  assertEqual(Math.round(-9.500000001), -10)
  
  test_roundFraction(0, 0, 0)
  test_roundFraction(-0, 0, -0)
  test_roundFraction(0.1, 0, 0)
  test_roundFraction(-0.1, 0, -0)
  test_roundFraction(9, 0, 9)
  test_roundFraction(-9, 0, -9)
  test_roundFraction(9.5, 0, 10)
  test_roundFraction(-9.5, 0, -9)
  test_roundFraction(9.499999999, 0, 9)
  test_roundFraction(-9.499999999, 0, -9)
  test_roundFraction(9.500000001, 0, 10)
  test_roundFraction(-9.500000001, 0, -10)
  test_roundFraction(0, 0, 0)
  test_roundFraction(-0, 0, -0)
  test_roundFraction(0.5, 0, 1)
  test_roundFraction(-0.5, 0, -0)
  test_roundFraction(0.499999999, 0, 0)
  test_roundFraction(-0.499999999, 0, -0)
  test_roundFraction(0.500000001, 0, 1)
  test_roundFraction(-0.500000001, 0, -1)
  test_roundFraction(0.09, 0, 0)
  test_roundFraction(-0.09, 0, -0)
  test_roundFraction(9.0e-10, 0, 0)
  test_roundFraction(-9.0e-10, 0, -0)
})

test('floorFraction', () => {
  assertEqual(Math.floor(-0), -0)
  assertEqual(Math.floor(-0.1), -1)
  assertEqual(Math.floor(9.5), 9)
  assertEqual(Math.floor(-9.5), -10)
  assertEqual(Math.floor(9.499999999), 9)
  assertEqual(Math.floor(-9.499999999), -10)
  assertEqual(Math.floor(9.500000001), 9)
  assertEqual(Math.floor(-9.500000001), -10)
  
  test_floorFraction(0, 0, 0)
  test_floorFraction(-0, 0, -0)
  test_floorFraction(0.1, 0, 0)
  test_floorFraction(-0.1, 0, -1)
  test_floorFraction(9, 0, 9)
  test_floorFraction(-9, 0, -9)
  test_floorFraction(9.5, 0, 9)
  test_floorFraction(-9.5, 0, -10)
  test_floorFraction(9.499999999, 0, 9)
  test_floorFraction(-9.499999999, 0, -10)
  test_floorFraction(9.500000001, 0, 9)
  test_floorFraction(-9.500000001, 0, -10)
  test_floorFraction(0.9, 0, 0)
  test_floorFraction(-0.9, 0, -1)
  test_floorFraction(9.0e-10, 0, 0)
  test_floorFraction(-9.0e-10, 0, -1)
})

test('ceilFraction', () => {
  assertEqual(Math.ceil(-0), -0)
  assertEqual(Math.ceil(-0.1), -0)
  assertEqual(Math.ceil(9.5), 10)
  assertEqual(Math.ceil(-9.5), -9)
  assertEqual(Math.ceil(9.499999999), 10)
  assertEqual(Math.ceil(-9.499999999), -9)
  assertEqual(Math.ceil(9.500000001), 10)
  assertEqual(Math.ceil(-9.500000001), -9)
  
  test_ceilFraction(0, 0, 0)
  test_ceilFraction(-0, 0, -0)
  test_ceilFraction(0.1, 0, 1)
  test_ceilFraction(-0.1, 0, -0)
  test_ceilFraction(9, 0, 9)
  test_ceilFraction(-9, 0, -9)
  test_ceilFraction(9.5, 0, 10)
  test_ceilFraction(-9.5, 0, -9)
  test_ceilFraction(9.499999999, 0, 10)
  test_ceilFraction(-9.499999999, 0, -9)
  test_ceilFraction(9.500000001, 0, 10)
  test_ceilFraction(-9.500000001, 0, -9)
  test_ceilFraction(0.09, 0, 1)
  test_ceilFraction(-0.09, 0, -0)
  test_ceilFraction(9.0e-10, 0, 1)
  test_ceilFraction(-9.0e-10, 0, -0)
})

test('extra', () => {
  assertEqual(fixFloat(49.34000000000001), 49.34)
  
  test_floorFraction(0.00000011111, 2, 0)
  test_floorFraction(0.011111, 2, 0.01)
  test_ceilFraction(0.0000001, 2, 0.01)
  test_ceilFraction(0.0100001, 2, 0.02)
  test_roundFraction(0.0000001, 2, 0)
  test_roundFraction(0.0049999, 2, 0)
  test_roundFraction(0.005, 2, 0.01)
  test_roundFraction(0.0149999, 2, 0.01)
  test_roundFraction(0.015, 2, 0.02)
  test_floorFraction(111 * 1.18, 2, 130.98)
  test_roundPrecision(111, 1, 100)
  test_roundPrecision(111, 2, 110)
  test_roundPrecision(0.1, 1, 0.1)
  test_roundPrecision(0.01, 1, 0.01)
  test_roundPrecision(0.999, 2, 1)
  test_roundPrecision(9.05e-97, 2, 9.1e-97)
  test_roundPrecision(0.01 * 1.01 - 0.01, 13, 0.0001)
  test_roundPrecision(1.05e-200, 5, 1.05e-200)
  test_roundPrecision(1.05e-50, 5, 1.05e-50)
})

_{This answer was intended for a JavaScript question, but it has been closed, marked as a duplicate, and redirected here. But most likely my answer will be suitable for many other languages, since the logic of working with float is approximately the same everywhere.}

Inion answered 17/11, 2023 at 3:1 Comment(0)

-1

In decimal, 1/3 cannot be represented exactly with a finite number of digits.

E.g. 1/3 ~ 0.33333333 and 3 x (1/3) ~ 0.99999999 != 1.

Likewise, in binary 1/5 cannot be represented exactly with a finite number of bits.

E.g. 1/5 ~ 0.00110011b and 101b x 0.00110011b = 0.11111111 != 1.

Bobbiebobbin answered 9/5, 2023 at 6:51 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++