Simple math operations faster on double than on float datatype? [duplicate]

Question

Possible Duplicate:
Are doubles faster than floats in c#?

I wrote simple benchmark to check how much performance i can get changing double datatype to float in my application. Here is my code:

    // my form: 
    // one textbox: textbox1 (MultiLine property set to true)
    // one button: button1 with event button1_Click

    private void button1_Click(object sender, EventArgs e)
    {

        int num = 10000000;

        float[] floats1 = new float[num];
        float[] floats2 = new float[num];
        float[] floatsr = new float[num];  // array for results
        double[] doubles1 = new double[num];
        double[] doubles2 = new double[num];
        double[] doublesr = new double[num]; // array for results

        Stopwatch stw = new Stopwatch();

        log("Preparing data");

        Random rnd = new Random();

        stw.Start();

        for (int i = 0; i < num; i++)
        {
            floats1[i] = NextFloat(rnd);
            floats2[i] = NextFloat(rnd);
            doubles1[i] = rnd.NextDouble();
            doubles2[i] = rnd.NextDouble();
        }
        stw.Stop();
        log(stw.Elapsed.TotalMilliseconds.ToString()+"ms");
        stw.Reset();




        log("");


        stw.Start();
        for (int i = 0; i <# i++)
        {
            floatsr[i] = floats1[i] * floats2[i];
        }
        stw.Stop();
        log("Multiplying floats: " + stw.Elapsed.TotalMilliseconds.ToString() + "ms");
        stw.Reset();



        stw.Start();
        for (int i = 0; i < num; i++)
        {
            doublesr[i] = doubles1[i] * doubles2[i];
        }
        stw.Stop();
        log("Multiplying doubles: " + stw.Elapsed.TotalMilliseconds.ToString() + "ms");
        stw.Reset();


        stw.Start();
        for (int i = 0; i < num; i++)
        {
            floatsr[i] = floats1[i] / floats2[i];
        }
        stw.Stop();
        log("Dividing floats: " + stw.Elapsed.TotalMilliseconds.ToString() + "ms");
        stw.Reset();


        stw.Start();
        for (int i = 0; i < num; i++)
        {
            doublesr[i] = doubles1[i] / doubles2[i];
        }
        stw.Stop();
        log("Dividing doubles: " + stw.Elapsed.TotalMilliseconds.ToString() + "ms");
        stw.Reset();

    }

    private void log(string text)
    {
        textBox1.Text = textBox1.Text + text + Environment.NewLine;
    }

    // I found that function somewhere on stackoverflow
    static float NextFloat(Random random)
    {
        double mantissa = (random.NextDouble() * 2.0) - 1.0;
        double exponent = Math.Pow(2.0, random.Next(-126, 128));
        return (float)(mantissa * exponent);
    }

I got results like this (release, no debug, Intel Mobile Core Duo T2500 2.0GHz 2MB CPU):

Preparing data 5275,6862ms

Multiplying floats: 442,7865ms 
Multiplying doubles: 169,4028ms
Dividing floats: 550,7052ms 
Dividing doubles: 164,1607ms

I was suprised, that operations on double are almost 3 times faster than operations on float. I searched for "double float" here, and i found this:

Is using double faster than float?

Best answer is focused on CPU architecture, but I cant agree with that.

I suspect that something else is causing low performance on floats, because my CPU with Intel SSE should be able to multiply or divide 4 floats at once (packed floating point instructions), or 2 doubles at once. So floats should be faster.

Maybe compiler (or clr in .net) is optimizing memory usage somehow?

Is there any way to optimize it and make float faster?

Please don't report duplicate, i saw other questions and they not satisfying me.

---

My results after changing method for generating floats now look fine (suggested by Servy):

Preparing data 1367,0678ms

Multiplying floats: 109,8742ms 
Multiplying doubles: 149,9555ms
Dividing floats: 167,0079ms 
Dividing doubles: 168,6821ms

Shewchuk

Please bear in mind that using floats for any protracted numerical computations runs the risk of accumulating roundoff errors very soon. – Originative

You're NextFloat makes a very different distribution of numbers compared to wheat NextDouble does. I don't know if that has any relevance. But note that NextDouble makes a number between 0.0 and 1.0 that is an integral multiple of 1.0 / 2147483647.0. So the "ending" of the number is not really random, which might be important when rounding the product or quotient calculated. – Intervenient

@JeppeStigNielsen It is significant; see my answer. Changing it such that they both generate numbers between 0 and 1 radically changes the runtimes when I ran the OP's code before and after the modification. – Blim

Also, suppose your computer can multiply 64-bit floating point numbers by itself. The to deal with the 32-bit (float or System.Single case), it might have to first "widen" the numbers read from your array, then multiply, and finally round and "narrow" the number to fit into the 32-bit "slot" of your result array. Whereas the 64-bit numbers will need no such conversions. – Intervenient

Answer 1

It has to do with how you generated the random numbers. Multiplying and dividing floating point numbers are not all the same; the actual values of those numbers matter. In the case of floats, you're populating a value over a fairly large range. If you create your floats such that they are between 0 and 1, just like the doubles, then it comes out more like you'd expect. Just change NextFloat to be this:

static float NextFloat(Random random)
{
    return (float) random.NextDouble();
}

I just ran a few tests, and with that change the floats were 33% faster at multiplication.

Of course, this is just the easiest way to make the comparison "fair". To get a better understanding of how floats really compare to doubles you'd want to generate random floats and doubles each between the full range of the respective types, or better yet, both holding values representing the type of data your program will be using.

Blim

This is correct. If you make this change, the float perforamnce will be modestly better than double performance. – Downcomer

Nice catch! Times after fix: Multiplying floats: 107,5963ms Multiplying doubles: 132,2323ms Dividing floats: 160,1531ms Dividing doubles: 170,8343ms. Now it makes sense for me. – Shewchuk

Maybe it would be even more fair to say: static float NextFloat(Random random) { return random.Next(32767) / 32767f; } The reason is that NextDouble does not use the full precision of a double. It only produces numbers that are a multiple of 1.0 / 2147483647.0, that is a precision of circa ten decimal figures, whereas the precision of a double is circa sixteen decimal figures. – Intervenient

Answer 2

GPU operations on floats are still faster, in some cases by far, because they have 32-bit float hardware.

Your x86 (or x86_64) architecture CPU doesn't have 32-bit support in the math coprocessor. Or even 64-bit support. The x87 floating-point unit uses 80-bit arithmetic.

Now, modern x86 CPUs do have SIMD instructions (MMX, SSE, AVX) with hardware support for 32-bit and 64-bit floating-point operations, with much higher performance -- if you can do everything in the SIMD unit. Moving data between SIMD and the FPU will kill performance.

And .NET doesn't use MMX or SSE or AVX, as of the current version. You might try Mono, which provides intrinsic methods that JIT-compile to SIMD instructions. Or you might use native code for the most performance-sensitive parts, since modern C++ compilers not only allow SIMD usage, but can auto-vectorize ordinary-looking code into SIMD instructions.

Reiko