views:

345

answers:

5

I'm using the following piece of code and under some mysterious circumstances the result of the addition is not as it's supposed to be:

double _west = 9.482935905456543;
double _off = 0.00000093248155508263153;
double _lon = _west + _off;

// check for the expected result
Debug.Assert(_lon == 9.4829368379380981);
// sometimes i get 9.48293685913086 for _lon (which is wrong)

I'm using some native DLLs within my application and i suspect that some DLL is responsible for this 'miscalculation', but i need to figure out which one. Can anyone give me a hint how to figure out the root of my problem?

+3  A: 

double is not completely accurate, try using decimal instead

The advanteage of using double and float over decimal is performance

TT
-1 How can a `Decimal` offer better performance over a `Double`? Your CPU knows how to handle `Singles` and `Doubles` but all operations on a `Decimal` must happen in memory which is slower.
Andrew Hare
Maybe my answer was a bit unclear but that is what I meant
TT
@TT - I don't know how I did it but I completely misread your answer - I apologize! (-1) removed and +1 for pointing out performance concerns!
Andrew Hare
+2  A: 

At first I thought this was a rounding error but actually it is your assertion that is wrong. Try adding the entire result of your calculation without any arbitrary rounding on your part.

Try this:

using System;

class Program
{
    static void Main()
    {
     double _west = 9.482935905456543;
     double _off = 0.00000093248155508263153;
     double _lon = _west + _off;

     // check for the expected result
     Console.WriteLine(_lon == 9.48293683793809808263153);  
    }
}

In the future though it is best to use System.Decimal in cases where you need to avoid rounding errors that are usually associated with the System.Single and System.Double types.

That being said, however, this is not the case here. By arbitrarily rounding the number at a given point you are assuming that the type will also round at that same point which is not how it works. Floating point numbers are stored to their maximum representational capacity and only once that threshold has been reached does rounding take place.

Andrew Hare
I tried using decimals, but converting a double into a decimal failed, because decimal automatically rounded the given value. How could this be avoided?
Andreas Roth
+1 I totally missed the wrong assertion!
n8wrl
A: 

You are being bitten by rounding and precision problems. See this. Decimal might help. Go here for details on conversions and rounding.

From MSDN:

When you convert float or double to decimal, the source value is converted to decimal representation and rounded to the nearest number after the 28th decimal place if required. Depending on the value of the source value, one of the following results may occur:

If the source value is too small to be represented as a decimal, the result becomes zero.

If the source value is NaN (not a number), infinity, or too large to be represented as a decimal, an OverflowException is thrown.

n8wrl
A: 

The problem is that Double only has precision of 15 - 16 digits (and you seem to need more precision in your example) whereas Decimal has precision to 28 - 29. How are you converting between Double and Decimal?

Paul Lydon
A: 

You cannot represent every floating point number in the decimal system with a floating point in a binary system accurately, this isn't even directly related to how "small" the decimal number is, some numbers just don't "fit" in base-2 nicely.

Using a longer bit width can help in most cases, but not always.

To specify your constants in Decimal (128-bit floating point ) precision, use this declaration:

decimal _west = 9.482935905456543m;
decimal _off = 0.00000093248155508263153m;
decimal _lon = _west + _off;
kek444