tags:

views:

100

answers:

3

I'm doing what seems like simple math in an android app, however I keep getting massive rounding errors. For example.

Float.toString(0.4*0.4*12.5);

I would expect to get 2 as the result of the above equation. However I get 2.0000002

+5  A: 

0.4 can't be represented exactly in a 32-bit IEEE-754 float. See:

If you absolutely need arbitrary precision use BigDecimal. Arbitrary precision math is slower, though. Or use 64-bit math (doubles) to improve the precision.

Mauricio Scheffer
Doubles doesn't make much difference in this case. While I cant find a variable precision rounding function in the Math lib for Android I did cobble together. I just dont like having to force results into a specific level of precision. Whats more the math routines I've doing sometimes result in a Double and sometimes a Float. Not found a clean way to force all results into one format or another. And I'm not familiar with BigDecimal. Having a hard time making sense of its usage.
Kujako
@Kujako: I'm not sure I understand the problem. IEEE-754 is a standard implemented by all CPUs I know, you'll have to deal with it.
Mauricio Scheffer
Yea, its just been a while since I've worked with Java. Forgotten some of these annoyances. Still, the functions I detailed in my response/answer do address the problem.
Kujako
@Kujako: this is not about Java, Android or any other platform. This is just how floating-point math is implemented on CPUs.
Mauricio Scheffer
A: 

Floating point is an approximate number system, in your case 0.4 cannot be represented exactly using a floating point number, so it is represented as close as possible, so you don't get the expected result.

Is there a Round function for Android that you can use?

Chris Diver
A: 

So I ended up making two functions, one to round Doubles and one for Floats. Seems like a PITA, but other then being limited to a preset level of precision (which I'll look at making a app setting) it does work.

public static float fRound(float Rval, int Rpl) {
    float p = (float)Math.pow(10,Rpl);
    Rval = Rval * p;
    float tmp = Math.round(Rval);
    return (float)tmp/p;
}
public static Double dRound(double d, int decimalPlace){
    BigDecimal bd = new BigDecimal(Double.toString(d));
    bd = bd.setScale(decimalPlace,BigDecimal.ROUND_HALF_UP);
    return bd.doubleValue();
}
Kujako