How to fix precision of variable

Question

In c#

double tmp = 3.0 * 0.05;

tmp = 0.15000000000000002

This has to do with money. The value is really $0.15, but the system wants to round it up to $0.16. 0.151 should probably be rounded up to 0.16, but not 0.15000000000000002

What are some ways I can get the correct numbers (ie 0.15, or 0.16 if the decimal is high enough).

Answer 1

Use a fixed-point variable type, or a base ten floating point type like Decimal. Floating point numbers are always somewhat inaccurate, and binary floating point representations add another layer of inaccuracy when they convert to/from base two.

Answer 2

'decimal' type was designed especially for this

Answer 3

In Patterns of Enterprise Application Architecture, Martin Fowler recommends using a Money abstraction

http://martinfowler.com/eaaCatalog/money.html

Mostly he does it for dealing with Currency, but also precision.

You can see a little of it in this Google Book search result:

http://books.google.com/books?id=FyWZt5DdvFkC&pg=PT520&lpg=PT520&dq=money+martin+fowler&source=web&ots=eEys-C_vdA&sig=jckdxgMLSRJtGDYZtcbYST1ak8M&hl=en&sa=X&oi=book_result&resnum=6&ct=result

Answer 4

Money should be stored as decimal, which is a floating decimal point type. The same goes for other data which really is discrete rather than continuous, and which is logically decimal in nature.

Humans have a bias to decimal for obvious reasons, so "artificial" quantities such as money tend to be more appropriate in decimal form. "Natural" quantities (mass, height) are on a more continuous scale, which means that float/double (which are floating binary point types) are often (but not always) more appropriate.

Answer 5

A decimal data type would work well and is probably your choice.

However, in the past I've been able to do this in an optimized way using fixed point integers. It's ideal for high performance computations where decimal bogs down and you can't have the small precision errors of float.

Start with, say an Int32, and split in half. First half is whole number portion, second half is fractional portion. You get 16-bits of signed integer plus 16 bits of fractional precision. e.g. 1.5 as an 16:16 fixed point would be represented as 0x00018000. Or, alter the distribution of bits to suit your needs.

Fixed point numbers can generally be added/sub/mul/div like any other integer, with a little bit of work around mul/div to avoid overflows.

Answer 6

What you faced is a rounding problem, which I had mentioned earlier in another post

Can I use “System.Currency” in .NET?

And refer to this as well Rounding