Issues with checking the equality of two doubles in .NET -- what's wrong with this method?

Question

Hi,

So I'm just going to dive into this issue... I've got a heavily used web application that, for the first time in 2 years, failed doing an equality check on two doubles using the equality function a colleague said he'd also been using for years.

The goal of the function I'm about to paste in here is to compare two double values to 4 digits of precision and return the comparison results. For the sake of illustration, my values are:

Dim double1 As Double = 0.14625000000000002 ' The result of a calculation
Dim double2 As Double = 0.14625 ' A value that was looked up in a DB

If I pass them into this function:

Public Shared Function AreEqual(ByVal double1 As Double, ByVal double2 As Double) As Boolean

    Return (CType(double1 * 10000, Long) = CType(double2 * 10000, Long))

End Function

the comparison fails. After the multiplication and cast to Long, the comparison ends up being:

Return 1463 = 1462

I'm kind of answering my own question here, but I can see that double1 is within the precision of a double (17 digits) and the cast is working correctly.

My first real question is: If I change the line above to the following, why does it work correctly (returns True)?

Return (CType(CType(double1, Decimal) * 10000, Long) = _
    CType(CType(double2, Decimal) * 10000, Long))

Doesn't Decimal have even more precision, thus the cast to Long should still be 1463, and the comparison return False? I think I'm having a brain fart on this stuff...

Secondly, if one were to change this function to make the comparison I'm looking for more accurate or less error prone, would you recommend changing it to something much simpler? For example:

Return (Math.Abs(double1 - double2) < 0.0001)

Would I be crazy to try something like:

Return (double1.ToString("N5").Equals(double2.ToString("N5")))

(I would never do the above, I'm just curious about your reactions. It would be horribly inefficient in my application.)

Anyway, if someone could shed some light on the difference I'm seeing between casting Doubles and Decimals to Long, that would be great.

Thanks!

Answer 1

What Every Computer Scientist Should Know About Floating-Point Arithmetic

Answer 2

When you use CType, you're telling your program "I don't care how you round the numbers; just make sure the result is this other type". That's not exactly what you want to say to your program when comparing numbers.

Comparing floating-point numbers is a pain and I wouldn't ever trust a Round function in any language unless you know exactly how it behaves (e.g. sometimes it rounds .5 up and sometimes down, depending on the previous number...it's a mess).

In .NET, I might actually use Math.Truncate() after multiplying out my double value. So, Math.Truncate(.14625 * 10000) (which is Math.Truncate(1462.5)) is going to equal 1462 because it gets rid of all decimal values. Using Truncate() with the data from your example, both values would end up being equal because 1) they remain doubles and 2) you made sure the decimal was removed from each.

I actually don't think String comparison is very bad in this situation since floating point comparison is pretty nasty in itself. Granted, if you're comparing numbers, it's probably better to stick with numeric types, but using string comparison is another option.

Answer 3

Relying on a cast in this situation is error prone, as you have discovered - depending upon the rules used when casting, you may not get the number you expect.

I would strongly advise you to write the comparison code without a cast. Your Math.Abs line is perfectly fine.

Regarding your first question:

My first real question is: If I change the line above to the following, why does it work correctly (returns True)?

The reason is that the cast from Double to Decimal is losing precision, resulting in a comparison of 0.1425 to 0.1425.