views:

2723

answers:

4

I have trouble comparing 2 double in Excel VBA

suppose that I have the following code

Dim a as double
Dim b as double
a = 0.15
b = 0.01

After a few manipulations on b, b is now equal to 0.6

however the imprecision related to the double data type gives me headache because

if a = b then
 //this will never trigger
end if

Do you know how I can remove the trailing imprecision on the double type?

A: 

It is never wise to compare doubles on equality.

Some decimal values map to several floating point representations. So one 0.6 is not always equal to the other 0.6.

If we subtract one from the other, we probably get something like 0.00000000051.

We can now define equality as having a difference smaller that a certain error margin.

Gamecat
+4  A: 

You can't compare floating point values for equality. See this article on "Comparing floating point numbers" for a discussion of how to handle the intrinsic error.

It isn't as simple as comparing to a constant error margin unless you know for sure what the absolute range of the floats is beforehand.

Rob Walker
For another reference you can check out http://msdn.microsoft.com/en-us/library/ae382yt8(VS.80).aspx
ZCHudson
A: 

As has been pointed out, many decimal numbers cannot be represented precisely as traditional floating-point types. Depending on the nature of your problem space, you may be better off using the Decimal VBA type which can represent decimal numbers (base 10) with perfect precision up to a certain decimal point. This is often done for representing money for example where 2-digit decimal precision is often desired.

Dim a as Decimal
Dim b as Decimal
a = 0.15
b = 0.01
C. Dragon 76
Is decimal available in vba? I think you need to declare as variant then cast to decimal via Cdec(myVar).
Mark Nold
A: 

The Currency data type may be a good alternative. It handles relatively large numbers with fixed four digit precision.

DJ