How do you handle floating point rounding problems?

Question

Seeing this post about floating point errors on slashdot, I got curious what other kind of solutions exist to deal with such kind of floating point rounding errors. So what was the floating point bug you learned most from, and what did you learn from it?

EDIT: I am working within a project where we have to deal a lot with floating point calculations, so I hope to get some answers that might help me to avoid some things before they are becoming problems. I will accept the answer that gives me the most new insights beyond "compare everything with an epsilon".

Answer 1

I don't think you really have to worry about it unless you're dealing with very small numbers and/or very large numbers... if it's a problem, either use some sort of 'decimal' class, or choose a more stable algorithm.

I don't think this has ever actually bitten me in the past, but I was quite excited when it happened to one of my peers and I knew exactly what was wrong :D

Answer 2

I learned to never ever compare two floats for equality. Instead I always try to phrase my logic so that the comparisons are always a less-than or greater-than condition. Comparing a float with zero is particularly nasty.

Answer 3

Try to avoid adding numbers of dissimilar magnitudes. For example, 10^8+1==10^8 in single-precision arithmetic. You can fix this by moving to double precision, but then 10^8+1.00000001==10^8+1… the fraction gets lost.

If all the numbers going into some linear algebra are biased, remove the bias first. So, given 1000001, 1000003.9, 1000002.5, …, subtract a million before doing anything, and add it back at the end.

To sum a very large sequence of small numbers, sum smaller subsequences first so the numbers at the end don't get unduly rounded.

To multiply a very large sequence of numbers, add their logarithms to avoid overflow or underflow.