Casting a result to float in method returning float changes result

Question

Why does this code print False in .NET 4? It seems some unexpected behavior is being caused by the explicit cast.

I'd like an answer beyond "floating point is inaccurate" or "don't do that".

float a(float x, float y)
{
  return ( x * y );
}

float b(float x, float y)
{
  return (float)( x * y );
}

void Main()
{
  Console.WriteLine( a( 10f, 1f/10f ) == b( 10f, 1f/10f ) );
}

PS: This code came from a unit test, not release code. The code was written this way deliberately. I suspected it would fail eventually but I wanted to know exactly when and exactly why. The answer proves the validity of this technique because it provides an understanding that goes beyond the usual understanding of floating point determinism. And that was the point of writing this code this way; deliberate exploration.

PPS: The unit test was passing in .NET 3.5, but now fails after the upgrade to .NET 4.

Jannelle

floating point variables are, by definition, not 100% accurate. They're approximate numbers. msdn.microsoft.com/en-us/library/b1e65aza.aspx See this post as well: stackoverflow.com/questions/618535/… There is no guarantee that they'll be the same in any flavor of the runtime. – Unessential

One more useful link: msdn.microsoft.com/en-us/library/ms187912.aspx – Unessential

Nonetheless, it is an interesting question. Functionally one would expect these methods A and B to perform the same operation, even with the floating-point approximation. I'm interested in hearing the explanation. – Dhyana

x86 jitter in the Release build without debugger required. The jitter optimizer completely eliminates the b() method and replaces it with 1.0f. But not a(), it calculates 1.0000000149011611. The lesson to learn here is never compare floating point values for equality. – Punctuate

Instead of comparing a == b, you should do something like Math.Abs(a - b) <= 1e-6 (change the 6 according to your needs, but float has only 7 digits precision, which means any value higher than that will randomly produce wrong results - if you need higher precision, you'll have to use double or decimal). This'll make sure the difference is small enough to be negligible. – Howell

@HansPassant +1, wow! do we have any idea as to why it eliminates one but not the other? – Formica

@MikeNakis: I can tell you exactly why; because when you explicitly cast something of type float to float (or something of type double to double) the C# compiler emits instructions that hint to the runtime that say "take this thing out of extra-high precision mode if you happen to be using that mode". – Upstretched

@Unessential According to the CLI spec, floating-point numbers follow the IEC 60559:1989 standard, aka IEEE-754. Higher precision is allowed, yes, but it's not like there aren't any guarantees. – Caroylncarp

Highly related: Are floating-point numbers consistent in C#? Can they be? – Miracidium

Funny point - remove the (float). – Bandicoot

More information on the tangent people are anxious to discuss here: Floating Point Determinism - gafferongames.com/networking-for-game-programmers/… – Jannelle

Answer 1

David's comment is correct but insufficiently strong. There is no guarantee that doing that calculation twice in the same program will produce the same results.

The C# specification is extremely clear on this point:

---

Floating-point operations may be performed with higher precision than the result type of the operation. For example, some hardware architectures support an “extended” or “long double” floating-point type with greater range and precision than the double type, and implicitly perform all floating-point operations using this higher precision type. Only at excessive cost in performance can such hardware architectures be made to perform floating-point operations with less precision, and rather than require an implementation to forfeit both performance and precision, C# allows a higher precision type to be used for all floating-point operations. Other than delivering more precise results, this rarely has any measurable effects. However, in expressions of the form x * y / z, where the multiplication produces a result that is outside the double range, but the subsequent division brings the temporary result back into the double range, the fact that the expression is evaluated in a higher range format may cause a finite result to be produced instead of an infinity.

---

The C# compiler, the jitter and the runtime all have broad lattitude to give you more accurate results than are required by the specification, at any time, at a whim -- they are not required to choose to do so consistently and in fact they do not.

If you don't like that then do not use binary floating point numbers; either use decimals or arbitrary precision rationals.

I don't understand why casting to float in a method that returns float makes the difference it does

Excellent point.

Your sample program demonstrates how small changes can cause large effects. You note that in some version of the runtime, casting to float explicitly gives a different result than not doing so. When you explicitly cast to float, the C# compiler gives a hint to the runtime to say "take this thing out of extra high precision mode if you happen to be using this optimization". As the specification notes, this has a potential performance cost.

That doing so happens to round to the "right answer" is merely a happy accident; the right answer is obtained because in this case losing precision happened to lose it in the correct direction.

How is .net 4 different?

You ask what the difference is between 3.5 and 4.0 runtimes; the difference is clearly that in 4.0, the jitter chooses to go to higher precision in your particular case, and the 3.5 jitter chooses not to. That does not mean that this situation was impossible in 3.5; it has been possible in every version of the runtime and every version of the C# compiler. You've just happened to run across a case where, on your machine, they differ in their details. But the jitter has always been allowed to make this optimization, and always has done so at its whim.

The C# compiler is also completely within its rights to choose to make similar optimizations when computing constant floats at compile time. Two seemingly-identical calculations in constants may have different results depending upon details of the compiler's runtime state.

More generally, your expectation that floating point numbers should have the algebraic properties of real numbers is completely out of line with reality; they do not have those algebraic properties. Floating point operations are not even associative; they certainly do not obey the laws of multiplicative inverses as you seem to expect them to. Floating point numbers are only an approximation of real arithmetic; an approximation that is close enough for, say, simulating a physical system, or computing summary statistics, or some such thing.

Upstretched

Unfortunately I already knew floating point is not accurate. What I didn't know was everything else about the casting and optimization. It'd be easier to ask the right question if I knew the answer. – Jannelle

I don't like the term 'inaccurate' being applied to floating-point numbers. (float)1 is 100% accurate. Adding small integers and multiplication by a power of two with a normal result is also 100% accurate. The differences in this example are the result not of 'inaccurate' calculations, but (as Eric explained at length) of the freedom of the language and runtime to choose certain implementation details. – Caroylncarp

@JeffreySax: Whatever you want to call it, it's a thing. And it's that thing many people mistakenly or incompletely attributed this issue to. The point of the question was to get to the "other stuff". It took edits to both the question and answer for that to happen. I wish nobody mentioned the basics of floating point 'accuracy', because this issue (as Eric explained at length) is more complicated than what is commonly known of floating point 'accuracy'. That's why comments solely about floating point 'accuracy' are still being upvoted. – Jannelle

@kk For some more background and examples, see also this blog post from (at the time) a member of the JIT team: blogs.msdn.com/b/davidnotario/archive/2005/08/08/449092.aspx – Caroylncarp

I would have never expected that casting float to itself had any effect. Is there a reason why you went with such a surprising feature instead of using a built in method such as Math.ForceToSingle? – Laroy

@CodeInChaos: This feature greatly predates my time here. I found it surprising too. The use of a cast is consistent with the way the feature is implemented in the CLR, which is to emit a conv.r4 or conv.r8 instruction. – Upstretched

Answer 2

I have no Microsoft compiler right now and Mono have no such effect. As far as I know GCC 4.3+ uses gmp and mpfr to calculate some stuff in compile time. C# compiler may do the same for non-virtual, static or private methods in same assembly. Explicit cast may interfere with such optimization (but I see no reason why it can't have same behavior). I.e. it may inline with calculating constant expression to some level (for b() it may be for example up to the cast).

GCC as well have the optimization that promotes operation to more highest precision if that makes sense.

So I'd consider both optimization as potential reason. But for both of them I see no reason why doing explicit casting of result may have some additional meaning like "be closer to standard".

Luganda