cocoa - I've discovered what I think is a bug with NSDecimalNumber ...

Question

Here is a simple code that shows what I think is a bug when dealing with double numbers...

double wtf = 36.76662445068359375000;
id xxx = [NSDecimalNumber numberWithDouble: wtf];
NSString *myBug = [xxx stringValue];
NSLog(@"%.20f", wtf);
NSLog(@"%@", myBug);
NSLog(@"-------\n");

the terminal will show two different numbers

36.76662445068359375000 and

36.76662445068359168

Is this a bug or am I missing something?

if the second number is being rounded, it is a very strange rounding btw...

= = = = = = = = = =

I am editing the original question to include one more WTF bug...

try this:

modify the original number and truncate it on 10 decimal digits... so...

double wtf = 36.76662445068359375000;
NSNumberFormatter *formatter = [[NSNumberFormatter alloc] init];
[formatter setMaximumFractionDigits:10];

NSString *valueX = [formatter stringFromNumber:[NSDecimalNumber numberWithDouble:wtf]];
[formatter release];
NSLog(@"%@", valueX);

the answer now is 36.7666244507

now the value is a string with 10 decimal digits... now lets convert it back to double

double myDoubleAgain = [valueX doubleValue];
NSLog(@"%.20f", myDoubleAgain);

the answer is 36.76662445070000018177 ??????

myDoubleAgain has now more digits!!!!

Answer 1

Trying to go beyond roughly 16 digits of decimal precision with a double is generally not recommended. Frankly, I'm surprised that you are able to represent this double (with 19 significant digits) in a way that maintains that precision when logging it out. You may even get different behavior on the iPhone itself, which maps a long double type to just plain double (your Mac may be handling this as a long double behind the scenes).

The rounding you are seeing may be happening at the binary level (see here for more on this), so you won't be seeing the decimal rounding that you are expecting.

It is for these reasons that you will want to work entirely with NSDecimalNumbers or NSDecimals from start to finish if you need this kind of high-precision math. To do this, do not convert to and from floating point types, but instead use NSStrings directly to populate and export the numbers (or store them as NSDecimalNumbers in Core Data).

For example, you could work around the above problems with the following code:

id xxx = [NSDecimalNumber decimalNumberWithString:@"36.76662445068359375000"];

NSDecimalNumbers (and their C struct equivalent NSDecimal) can handle up to 38 significant digits.

Answer 2

Floating point conversions necessarily involve (tiny) rounding errors: What Every Computer Scientist Should Know About Floating-Point Arithmetic

Answer 3

See these conversions http://www.wolframalpha.com/input/?i=convert+36.76662445068359375000+to+binary to understand the rounding http://www.wolframalpha.com/input/?i=convert+36.76662445068359168+to+binary

Answer 4

Normally, I'm the one who comes in and explains to people that the number they entered is not representable as a floating-point number, and where the rounding errors are, blah blah blah.

This question is much more fun than what we usually see, and it illustrates exactly what's wrong with the crowd wisdom of "floating point is inexact, read 'what every computer scientist should know...' lolz".

36.76662445068359375 is not just any 19-digit decimal number. It happens to be a 19 digit decimal number that is also exactly representable in double precision binary floating point. Thus, the initial conversion implicit in:

double wtf = 36.76662445068359375000;

is exact. wtf contains exactly b100100.11000100010000011, and no rounding has occurred.

The spec for NSDecimalNumber says that it represents numbers as a 38 digit decimal mantissa and a decimal exponent in the range [-127,128], so the value in wtf is also exactly representable as an NSDecimalNumber. Thus, we may conclude that numberWithDouble is not delivering a correct conversion. Although I cannot find documentation that claims that this conversion routine is correctly rounded, there is no good reason for it not to be. This is a real bug, please report it.

I note that the string formatters on iPhoneOS seem to deliver correctly rounded results, so you can probably work around this by first formatting the double as a string with 38 digit precision and then using decimalNumberWithString. Not ideal, but it may work for you.