Are floating point operations in Delphi deterministic?

Question

Are floating point operations in Delphi deterministic?

I.E. will I get the same result from an identical floating point mathematical operation on the same executable compiled with Delphi Win32 compiler as I would with the Win64 compiler, or the OS X compiler, or the iOS compiler, or the Android compiler?

This is a crucial question as I'm implementing multiplayer support in my game engine, and I'm concerned that the predictive results on the client side could very often differ from the definitive (and authoritative) determination of the server.

The consequence of this would be the appearance of "lag" or "jerkiness" on the client side when the authoritative game state data overrules the predictive state on the client side.

Since I don't realistically have the ability to test dozens of different devices on different platforms compiled with the different compilers in anything resembling a "controlled condition", I figure it's best to put this question to the Delphi developers out there to see if anyone has an internal understanding of floating point determinism at a low-level on the compilers.

Dike

What do you mean by the 'floating point operations'? If you include transcendental functions like sin or cos I don't think you can expect same results. If you limit the question to 4 arithmetic operations on Double or Single type maybe you can achieve the same results but I don't like the idea of relying on it because change of floating operation mode can change result of a floating operation even on a single platform. – Ubald

See The Extended Data Type Is 2 Bytes Smaller on 64-bit Windows Systems. – Aubreyaubrie

I don't see why this is a problem to you. Floating point data types and operations are always approximations. – Despondency

Suggested reading, Floating-Point Rounding Issues and Floating point precision control (Delphi for x64). There are so many ways that can go wrong, relying on the result must be equal on all platforms. – Aubreyaubrie

@DavidHeffernan: False. – Mudra

The concern is that the client-side prediction could be so far out of alignment with the server-side calculation, that the player keeps "snapping" into position when the client-side data is updated with the definitive (authoritative) server-side data. – Dike

@Mudra No. The true result of a calculation may be a number that cannot be represented exactly. – Despondency

@DavidHeffernan: It also may not be. Lots of carefully-written floating-point code relies on floating-point arithmetic working. – Mudra

@Mudra Floating point arithmetic works as defined. But not all numbers can be represented. No amount of careful coding can change that fact. I'd love to see how you evaluate 1.0/10.0 without approximation. – Despondency

@DavidHeffernan: I can evaluate 1.0 / 1.0 just fine, which serves as an excellent counterexample to your broad, sweeping statement about floating-point math. There is no need to consider 1.0 / 10.0 here. – Mudra

@Mudra OK. I see what you mean. What I meant, and completely failed to express, is that given that floating point arithmetic is inexact, does it matter that different hardware behave differently. But I expressed that appallingly badly. – Despondency

@DavidHeffernan: It does for the poster's application; he's wondering whether he can trust client-side floating-point math to reproduce a list of calculations faithfully. Whether two state machines, identical at the Delphi level, could diverge as a result of different platforms handling floating-point differently. – Mudra

@Mudra We all agree that the results will differ. I wonder whether or not it matters. It might not. – Despondency

@DavidHeffernan: Within this context, it clearly does. – Mudra

@tmtklebu I expect that the amount of drift will be negligible. It's certainly plausible that would be the case. Screen resolution is way coarser than double precision, even single precision arithmetic. If don't think you or I know anything much about the actual specifics of this problem. I'm speculating that it is possible that some drift won't matter. I don't think you can be sure that it will matter. – Despondency

FWIW, take a look here: two equal float values (they are the result of exactly the same function with exactly the same parameters) compare as unequal when a piece of code, which only DISPLAYS these values is inserted: #24321765 – Pilgarlic

I'll be testing just how significant the "drift" is shortly (I figure we can argue academics until the cows come home, but really we'll only know by actually testing it) – Dike

Just a suggestion, why don't you let the client be authoritative on the state and then have a margin for error? As long as the client doesn't stray too far off course then the server will correct itself. Players may "jump" in the eyes of other players but this is a lot less jerky than the player themselves jumping. – Anthozoan

The problem with allowing the client to be in any way "authoritative" is that it would enable players to very easily cheat, simply by poking the memory related to their position, health, velocity, and inventory counts. This is why the server must be authoritative, so that any attempt to cheat by players would be fruitless (the server's numbers are fact) – Dike

Answer 1

I think there is no simple answer. Similar task was discussed here. In general, there are two standards for presentation of floating point numbers: IEEE 754-1985 and EEE 754-2008. All modern (and quite old actually) CPUs follow the standards and it guarantees some things:

• Binary presentation of same standard floating type will be equal
• Result of some operations (not all, only basic operations!) is guaranteed to be equal, but only if compiler will use same type of the command, i am not sure it is true.

But if you use some extended operations, such as square root, result may vary even for different models of desktop CPUs. You can read this article for some details: http://randomascii.wordpress.com/2013/07/16/floating-point-determinism/

P.S. As tmyklebu mentioned, square root is also defined by IEEE 754, so it is possible to guarantee same result for same input for Add, Subtract, Multiply, Divide and Square root. Few other operations are also defined by IEEE, but for all details it is better to read IEEE.

Hostess

This is what I was presuming (and wanted to be sure). Annoyingly, the physics engine is using virtually all complex floating point operations (including Sine, Cosine, I'm using Tangent to determine which 3D layer to render the sprites onto, and square root). I may need to rely on "smoothing" based on a back-buffer in order to make the gameplay appear smooth on the client-side, while in reality constantly being "adjusted" (even if by only thousandths of a unit) by the authoritative server calculation. Inelegant, but looking to be essential now. – Dike

sqrt is fine. You're playing with fire with trig and exponential functions. The really scary thing is that, on x86-32, it's typical to use the FPU instead of SSE instructions. In order to get any sort of performance out of that, compilers need to generate code that rounds in a different way from what the program specifies. – Mudra

Are you sure IEEE754 specifies sqrt? – Despondency

@DavidHeffernan Yes. The 6 basic operations +, -, *, /, sqrt, fma must be correctly rounded. – Gaffrigged

To reiterate, sqrt is completely defined by the IEEE 754 standard and does not “vary for different models of desktop CPUs”. – Gaffrigged

@DavidHeffernan: Yes; look at the opening paragraph of section 5. – Mudra

Answer 2

Putting aside the standards for floating point calculations for a moment, consider that the 32 and 64 bit compilers compile to use the old FPU vs the newer SSE instructions. I would find it difficult to trust that every calculation would always come out exactly the same on different hardware implementations. Better go the safe route and assume that if its within a small delta pf difference, you evaluate as equal.

Print

That's the "best case scenario" I'm hoping for. In an ideal world the values would be precise, but since we know that the world is less than ideal, I'm just hoping that the results of the multiplayer test I'm preparing now will show that the differences between values are so miniscule that the rendered result still comes out the same. – Dike

The standard exists to ensure that results are consistent – Despondency

Answer 3

From experience I can tell that the results are different: 32-bit works with Extended precision by default, while 64-bit works with double precision by default.

Consider the statement

x,y,z: double; x := y * z;

in Win32 this will execute as "x := double(Extended(y)*Extended(z)); in Win64 this will execute as "x := double(double(y)*double(z));

you put a lot of effort into ensuring that you use the same precision and mode. However whenever you call 3rd party libraries, you need to consider that they may internally change these flags.

Grayce

I'm quite fortunate in that this game engine uses entirely Double values internally, and Single is used by FMX for rendering the sprites only (no complex math performed on Single values, for that matter) Thanks for the info :) – Dike

@La The answer is talking about intermediate values on x87 that you have no control over – Despondency