How do I trim the zero value after decimal

Question

hi guys this is ridiculous..

as i tried to debug i found that : just as I type in

Dim value As Double value = 0.90000 and then hit enter

it automatically converts 0.9

shouldnt it keep the precision in double in visual basic?

for my calculation, i absolutely need to show the precision :(

thanks!

Answer 1

No, it shouldn't keep the precision. Binary floating point values don't retain this information... and it would be somewhat odd to do so, given that you're expressing the value in one base even though it's being represented in another.

I don't know whether VB6 has a decimal floating point type, but that's probably what you want - or a fixed point decimal type, perhaps. Certainly in .NET, System.Decimal has retained extra 0s from .NET 1.1 onwards. If this doesn't help you, you could think about remembering two integers - e.g. "90000" and "100000" in this case, so that the value you're representing is one integer divided by another, with the associated level of precision.

EDIT: I thought that Currency may be what you want, but according to this article, that's fixed at 4 decimal places, and you're trying to retain 5. You could potentially just multiply by 10, if you always want 5 decimal places - but it's an awkward thing to remember to do everywhere... and you'd have to work out how to format it appropriately. It would also always be 4 decimal places, I suspect, even if you'd specified fewer - so if you want "0.300" to be different to "0.3000" then Currency may not be appropriate. I'm entirely basing this on articles online though...

Answer 2

If precision is required then the Currency data type is what you want to use.

Answer 3

There are at least two representations of your value in play. One is the value you see on the screen -- a string -- and one is the internal representation -- a binary value. In dealing with fractional values, the two are often not equivalent and where they aren't, it's because they can't be.

If you stick with doubles, VB will maintain 53 bits of mantissa throughout your calculations, no matter how they might appear when printed. If you transition through the string domain, say by saving to a file or DB and later retrieving, it often has to leave some of that precision behind. It's inevitable, because the interface between the two domains is not perfect. Some values that can be exactly represented as strings (or Decimals, that is, powers of ten) can't be exactly represented as fractional powers of 2.

This has nothing to do with VB, it's the nature of floating point. The best you can do is control where the rounding occurs. For this purpose your friend is the Format function, which controls how a value appears in string form.

? Format$(0.9, "0.00000") will show you an example.

Answer 4

You are getting what you see on the screen confused with what bits are being set in the Double to make that number.

VB is simply being "helpful", and simply knocking off excess zeros. But for all intents and purposes,

0.9

is identical to

0.90000

If you don't believe me, try doing this comparison:

Debug.Print CDbl("0.9") = CDbl("0.90000")

As has already been said, displayed precision can be shown using the Format$() function, e.g.

Debug.Print Format$(0.9, "0.00000")

Answer 5

You can also enter the value as 0.9# instead. This helps avoid implicit coercion within an expression that may truncate the precision you expect. In most cases the compiler won't require this hint though because floating point literals default to Double (indeed, the IDE typically deletes the # symbol unless the value was an integer, e.g. 9#).

Contrast the results of these:

MsgBox TypeName(0.9)
MsgBox TypeName(0.9!)
MsgBox TypeName(0.9@)