ansaurus

Question

Answer 1

+9 A:

The problem is that when you're taking the value of "number + 0.995", that's not going to be exactly number + 0.995 for the normal reasons of binary floating point not representing decimal values precisely.

Sometims it will be a little over, and sometimes it'll be a little under. When it's under, and you round that result to two decimal places, you end up with the "0.99" bit. It's possible that Math.Round tries to take this into account to some extent, but I'm not sure of the exact details.

The solution - where possible - is to use decimal instead of double for things where the decimal digits matter.

See my articles on binary floating point and decimal floating point in .NET for more information.

Here's a program using my DoubleConverter code which shows the inaccuracy after adding 0.995:

using System;

class Program
{
    static void Main(string[] args)
    {
        for (int i = 0; i < 100; i++)
        {
            double d = i + 0.995;
            Console.WriteLine(DoubleConverter.ToExactString(d));
        }
    }
}

Results:

0.99499999999999999555910790149937383830547332763671875 1.99500000000000010658141036401502788066864013671875 2.99500000000000010658141036401502788066864013671875 3.99500000000000010658141036401502788066864013671875 4.99500000000000010658141036401502788066864013671875 5.99500000000000010658141036401502788066864013671875 6.99500000000000010658141036401502788066864013671875 7.99500000000000010658141036401502788066864013671875 8.9949999999999992184029906638897955417633056640625 9.9949999999999992184029906638897955417633056640625 10.9949999999999992184029906638897955417633056640625 11.9949999999999992184029906638897955417633056640625 12.9949999999999992184029906638897955417633056640625 13.9949999999999992184029906638897955417633056640625 14.9949999999999992184029906638897955417633056640625 15.9949999999999992184029906638897955417633056640625 16.995000000000000994759830064140260219573974609375 17.995000000000000994759830064140260219573974609375 18.995000000000000994759830064140260219573974609375 19.995000000000000994759830064140260219573974609375 20.995000000000000994759830064140260219573974609375 ...

(Note how although it starts going 0.994 at 8, it continues as far as 15 - whereas Math.Round "corrects" itself at 10.)

Jon Skeet 2010-06-18 06:10:02

Thanks Jon. It helped a lot! :)

Jeevan 2010-06-18 06:48:48

ansaurus

tags:

views:

answers:

System.Math.Round bug?

related questions