views:

811

answers:

1

I am calculating g with e and s, which are all doubles. After that I want to cut off all digits after the second and save the result in x, for example:

g = 2.123 => x = 2.12

g = 5.34995 => x = 5.34

and so on. I Use...

g = 0.5*e + 0.5*s;
x = floor(g*100)/100;

...and it works fine most of the time. But sometimes I get strange results. For example:

e = 3.0 s = 1.6 g = 2.30 but x = 2.29!!!

So I tried to track down the error:

g = 0.5*e + 0.5*s;
NSLog(@"%f",g);

gives me g = 2.30

g = g * 100;
NSLog(@"%f",g);

gives me g = 230.0

x = floor(g);
NSLog(@"%f",x);

results in x = 229.0 !!!

I don't get it! Help please! :-)

+1  A: 

This will be due to floating point calculations.

Your calculation

g * 100

already brings back

229.99999999999997

From where your issue stems.

Have a look at INFO: Precision and Accuracy in Floating-Point Calculations

Also have a look at Floating point

Accuracy problems

The fact that floating-point numbers cannot precisely represent all real numbers, and that floating-point operations cannot precisely represent true arithmetic operations, leads to many surprising situations. This is related to the finite precision with which computers generally represent numbers.

astander
Thanks a lot. Do you have a quick solution to avoid this problem?
iHilke
Have a look at http://social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/921a8ffc-9829-4145-bdc9-a96c1ec174a5/. try decimal vs double for precision if you can. And maybe http://stackoverflow.com/questions/618535/what-is-the-difference-between-decimal-float-and-double-in-c
astander
outis
Sorry, when i originally posted the answer it also stated c#, please see the revisions to the question.
astander