views:

129

answers:

3

I have noticed an interesting behavior with float rounding / truncation by the C# compiler. Namely, when a float literal is beyond the guaranteed representable range (7 decimal digits), then a) explicitly casting a float result to float (a semantically unnecessary operation) and b) storing intermediate calculation results in a local variable both change the output. An example:

using System;

class Program
{
    static void Main()
    {
        float f = 2.0499999f;
        var a = f * 100f;
        var b = (int) (f * 100f);
        var c = (int) (float) (f * 100f);
        var d = (int) a;
        var e = (int) (float) a;
        Console.WriteLine(a);
        Console.WriteLine(b);
        Console.WriteLine(c);
        Console.WriteLine(d);
        Console.WriteLine(e);
    }
}

The output is:

205
204
205
205
205

In the JITted debug build on my computer, b is calculated as follows:

          var b = (int) (f * 100f);
0000005a  fld         dword ptr [ebp-3Ch] 
0000005d  fmul        dword ptr ds:[035E1648h] 
00000063  fstp        qword ptr [ebp-5Ch] 
00000066  movsd       xmm0,mmword ptr [ebp-5Ch] 
0000006b  cvttsd2si   eax,xmm0 
0000006f  mov         dword ptr [ebp-44h],eax 

whereas d is calculated as

          var d = (int) a;
00000096  fld         dword ptr [ebp-40h] 
00000099  fstp        qword ptr [ebp-5Ch] 
0000009c  movsd       xmm0,mmword ptr [ebp-5Ch] 
000000a1  cvttsd2si   eax,xmm0 
000000a5  mov         dword ptr [ebp-4Ch],eax 

Finally, my question: why is the second line of the output different from the fourth? Does that extra fmul make such a difference? Also note that if the last (already unrepresentable) digit from the float f is removed or even reduced, everything "falls in place".

+4  A: 

Your question can be simplified to asking why these two results are different:

float f = 2.0499999f;
var a = f * 100f;
var b = (int)(f * 100f);
var d = (int)a;
Console.WriteLine(b);
Console.WriteLine(d);

If you look at the code in .NET Reflector you can see that the above code is actually compiled as if it were the following code:

float f = 2.05f;
float a = f * 100f;
int b = (int) (f * 100f);
int d = (int) a;
Console.WriteLine(b);
Console.WriteLine(d);

Floating point calculations cannot always be made exactly. The result of 2.05 * 100f is not exactly equal to 205, but just a little less due to rounding errors. When this intermediate result is converted to an integer is truncated. When stored as a float it is rounded to the nearest representable form. These two methods of rounding give different results.


Regarding your comment to my answer when you write this:

Console.WriteLine((int) (2.0499999f * 100f));
Console.WriteLine((int)(float)(2.0499999f * 100f));

The calculations are done entirely in the compiler. The above code is equivalent to this:

Console.WriteLine(204);
Console.WriteLine(205);
Mark Byers
So you say the reason is that (int) is done by truncation and (float) means rounding. If that's the case, then why is the output different forConsole.WriteLine((int) (2.0499999f * 100f)) andConsole.WriteLine((int) (float) (2.0499999f * 100f)) ?
Alan
@Alan, check my answer. reason is that float can hold only 7 digits. log (2^23) = 6.9
Andrey
@Alan: When you use hard-coded constants the calculations are done entirely in the compiler and using the compiler's rules, not in the .NET runtime.
Mark Byers
@Andrey, thanks. I know that the float exceeds the representable range (see question, I did not edit that part), but then, with your confirmation, this looks a little scary to me - casting a float to float should not make a difference, but in this case, it does.
Alan
@Mark: Are these rules different? And if yes, am I supposed to know this, either from the C# language reference doc or MSDN, or is this just an occasional discrepancy between the compiler and the runtime?
Alan
@Alan: Apparently they are different as your example shows. I think in general though relying on the least significant digit of a floating point calculation is an incredibly bad idea.
Mark Byers
@Mark: Sure it is. Thanks for your reply.
Alan
+2  A: 

Mark is right about compiler. Now let's fool the compiler:

    float f = (Math.Sin(0.5) < 5) ? 2.0499999f : -1;
    var a = f * 100f;
    var b = (int) (f * 100f);
    var c = (int) (float) (f * 100f);
    var d = (int) a;
    var e = (int) (float) a;
    Console.WriteLine(a);
    Console.WriteLine(b);
    Console.WriteLine(c);
    Console.WriteLine(d);
    Console.WriteLine(e);

first expression is meaningless but prevents compiler from optimization. Result is:

205
204
205
204
205

ok, i found the explanation.

2.0499999f can't be stored as float, because it can hold only 7 10-based digits. and this literal is 8 digits, so compiler rounded it because could not store. (should give a warning IMO)

if you change to 2.049999f result will be expected.

Andrey
Thanks Andrey, I chose Mark's reply on the basis of the compiler vs runtime info, but yours is also relevant.
Alan
+2  A: 

In a comment you asked

Are these rules different?

Yes. Or, rather, the rules allow for different behaviour.

And if yes, am I supposed to know this, either from the C# language reference doc or MSDN, or is this just an occasional discrepancy between the compiler and the runtime

It's implied by the specification. Floating point operations have a certain minimum level of precision that must be met, but the compiler or runtime is permitted to use more precision if it sees fit. That can cause large, observable changes when you do operations that magnify small changes. Rounding, for example, can turn an extremely small change into an extremely large one.

This fact leads to fairly frequently asked questions here. For some background on this situation and other situations that can produce similar discrepancies, see the following:

http://stackoverflow.com/questions/2342396

http://stackoverflow.com/questions/2345534

http://stackoverflow.com/questions/2225503

http://stackoverflow.com/questions/2494724

Eric Lippert
Eric, thank you very much. Your last link was especially enlightening. I actually searched for similar scenarios before I posted the question, but apparently my scope was too narrow.
Alan
@Alan: You're welcome!
Eric Lippert