Is float slower than double? Does 64 bit program run faster than 32 bit program?

Question

Is using float type slower than using double type?

I heard that modern Intel and AMD CPUs can do calculations with doubles faster than with floats.

What about standard math functions (sqrt, pow, log, sin, cos, etc.)? Computing them in single-precision should be considerably faster because it should require less floating-point operations. For example, single precision sqrt can use simpler math formula than double precision sqrt. Also, I heard that standard math functions are faster in 64 bit mode (when compiled and run on 64 bit OS). What is the definitive answer on this?

Olivares

Which is faster, my Ferrari or your dump truck? It depends - if you're trying to run the quarter-mile, probably the Ferrari. If you're trying to move 5 tons of gravel, probably the dump truck. It depends on what you're doing. This isn't an answerable question. – Agateware

@Ken White: It depends on which one is towing the other, of course! – Hussar

The definitive answer is that there is no definitive answer to such general questions. – Overarch

@Tim Sylvester. Yes, looks like it isn't as simple as I though. I would have to experiment with my code to figure out how to make it faster. – Olivares

@Ken White. It is a math number crunching project. Takes very long time to finish. I am trying to make it run faster. – Olivares

See my answer below for information about timings I made of operations in Java in case it's of use. – Esbensen

@BobaFet: Always start at the algorithm. It is suboptimal. – Korella

Answer 1

The classic x86 architecture uses floating-point unit (FPU) to perform floating-point calculations. The FPU performs all calculations in its internal registers, which have 80-bit precision each. Every time you attempt to work with float or double, the variable is first loaded from memory into the internal register of the FPU. This means that there is absolutely no difference in the speed of the actual calculations, since in any case the calculations are carried out with full 80-bit precision. The only thing that might be different is the speed of loading the value from memory and storing the result back to memory. Naturally, on a 32-bit platform it might take longer to load/store a double as compared to float. On a 64-bit platform there shouldn't be any difference.

Modern x86 architectures support extended instruction sets (SSE/SSE2) with new instructions that can perform the very same floating-point calculations without involving the "old" FPU instructions. However, again, I wouldn't expect to see any difference in calculation speed for float and double. And since these modern platforms are 64-bit ones, the load/store speed is supposed to be the same as well.

On a different hardware platform the situation could be different. But normally a smaller floating-point type should not provide any performance benefits. The main purpose of smaller floating-point types is to save memory, not to improve performance.

Edit: (To address @MSalters comment) What I said above applies to fundamental arithmetical operations. When it comes to library functions, the answer will depend on several implementation details. If the platform's floating-point instruction set contains an instruction that implements the functionality of the given library function, then what I said above will normally apply to that function as well (that would normally include functions like sin, cos, sqrt). For other functions, whose functionality is not immediately supported in the FP instruction set, the situation might prove to be significantly different. It is quite possible that float versions of such functions can be implemented more efficiently than their double versions.

Randyranee

Why floats are not faster on SSE/SSE2? I read that SSE can do 4x32 bit floats and only 2x64 bit doubles at once. I don't use SSE directly, but I think my compiler can vectorize some of the simple loops to use SSE. I am using Intel's compiler, but didn't read the manual thoroughly yet. I think C# can't vectorize any loops. – Olivares

@Boba Fet: I was considering non-vectorized computations only. For vectorized computations things might turn out differently for the reasons you just mentioned. – Randyranee

The memory bus is 64 bit since the Pentium times. The loading of 1 float or 1 double is the same. The difference comes if you load more than 1 value then in float 2 values can be loaded in each transaction. – Zouave

-1. The statement "the calculations are carried out with full 80-bit precision" is misleadingly wrong for the question: "standard math functions (sqrt, pow, log, sin, cos, etc.". Yes, the native x87 operations are done at full precision. But pow isn't a native x87 operation, it's a non-trivial function. A 32 bits implementation of this function might be faster because it uses less 80 bits operations. (note: The problem is far worse for C99's new math functions, source: Mr. Plauger) – Graphology

Answer 2

Your first question has already been answer here on SO.

Your second question is entirely dependent on the "size" of the data you are working with. It all boils down to the low level architecture of the system and how it handles large values. 64-bits of data in a 32 bit system would require 2 cycles to access 2 registers. The same data on a 64 bit system should only take 1 cycle to access 1 register.

Everything always depends on what you're doing. I find there are no fast and hard rules so you need to analyze the current task and choose what works best for your needs for that specific task.

Animatism

Thanks for the link. It surprising that using float can make things slower. Looks like it is more complicated than I thought. – Olivares

Yes, there is a lot of stuff that goes on that we take for granted. It wasn't until I took a microprocessors course that I understood all the work required by the CPU in order to do simple stuff like represent negative numbers, decimals, etc. The larger the data (more precision, bigger numbers) you are working with the more work the CPU has to do. – Animatism

No, since the Pentium all the data busses are 64bit wide. Loading a double (if it is alligned) takes only 1 bus cycle. – Zouave

@PatrickSchlüter Wouldn't a 32-bit compiler be limited to loading a double in 2 separate 32-bit transfers, 1 bus cycle each? In other words, I don't think you can always assume that a 64-bit HW architecture will give you higher performance. I believe it depends on your compiler too. – Fixate

@Michael Ansolis the compiler will emit a FLD instruction which can access memory in 64 bits chunk. There's no reason to emit 32 bits operations. – Zouave

Answer 3

While on most systems double will be the same speed as float for individual values, you're right that computing functions like sqrt, sin, etc. in single-precision should be a lot faster than computing them to double-precision. In C99, you can use the sqrtf, sinf, etc. functions even if your variables are double, and get the benefit.

Another issue I've seen mentioned is memory (and likewise storage device) bandwidth. If you have millions or billions of values to deal with, float will almost certainly be twice as fast as double since everything will be memory-bound or io-bound. This is a good reason to use float as the type in an array or on-disk storage in some cases, but I would not consider it a good reason to use float for the variables you do your computations with.

Provinciality

Answer 4

From some research and empirical measurements I have made in Java:

• basic arithmetic operations on doubles and floats essentially perform identically on Intel hardware, with the exception of division;
• on the other hand, on the Cortex-A8 as used in the iPhone 4 and iPad, even "basic" arithmetic on doubles takes around twice as long as on floats (a register FP addition on a float taking around 4ns vs a register FP on a double taking around 9ns);
• I've made some timings of methods on java.util.Math (trigonometrical functions etc) which may be of interest -- in principle, some of these may well be faster on floats as fewer terms would be required to calculate to the precision of a float; on the other hand, many of these end up being "not as bad as you'd think";

It is also true that there may be special circumstances in which e.g. memory bandwidth issues outweigh "raw" calculation times.

Esbensen

Answer 5

The "native" internal floating point representation in the x86 FPU is 80 bits wide. This is different from both float (32 bits) and double (64 bits). Every time a value moves in or out of the FPU, a conversion is performed. There is only one FPU instruction that performs a sin operation, and it works on the internal 80 bit representation.

Whether this conversion is faster for float or for double depends on many factors, and must be measured for a given application.

Hussar

Answer 6

It depends on the processor. If the processor has native double-precision instructions, it'll usually be faster to just do double-precision arithmetic than to be given a float, convert it to a double, do the double-precision arithmetic, then convert it back to a float.

Enki

Hi. We use Intel Core 2 and newer and AMD Opteron. I observed that switching to float is somewhat slower. – Olivares