ansaurus

Question

[Haskell] Odd values when enumerating a list

Answer 1

+8 A:

Because IEEE floating point arithmetic can't express decimal numbers precisely, in general. There is always a rounding error in the binary representation, which sometimes seeps to the surface when displaying the number.

Depending on how GHC converts floating-point numbers to a decimal representation, you might find that on Windows it displays the result as the expected 0.3. This is because Microsoft's runtime libraries are smarter than Linux and Mac about how they present floats.

EDIT: This may not be the case. The number 0.3 will encode as the integer 3fd3333333333333 when using IEEE floats, whereas 0.1 + 0.1 + 0.1 will produce a number that encodes as 3fd3333333333334, and I don't know whether Microsoft's runtime libraries are tolerant enough to round back to 0.3 when displaying this.

In any event, a good example of the different handling is to type 0.3 into a Python interactive shell. If it's Python 2.6, you'll get back 0.29999999999999999, and if it's 2.7, it will display 0.3.

Marcelo Cantos 2010-07-11 09:19:52

Interesting. Apparently I have a lot of wikipedia to read! Thanks.

danportin 2010-07-11 10:27:51

@danportin See also "What Every Computer Scientist Should Know About Floating-Point Arithmetic" at http://docs.sun.com/source/806-3568/ncg_goldberg.html

Greg Bacon 2010-07-11 11:29:16

Answer 2

+1 A:

A better way of doing this is more along the lines of

map (/10) [0 .. 10]

This takes whole numbers, thereby avoiding the float problem, and divides each one by 10.

Paul Johnson 2010-07-11 14:48:15

+1 (I wish it could be more) for a very wise observation.

Marcelo Cantos 2010-07-12 02:15:31

Actually, it doesn't. Just because you aren't putting a decimal in the number doesn't mean it isn't one. So while the numbers may be "whole", you aren't avoiding the float problem. In fact, because of the way you've written it, you risk making the problem worse (because you're trying to convince yourself that the numbers aren't using floating-point).A better way of doing this would be to use `Rational`s (or some sort of `Ratio`-using type.

BMeph 2010-07-12 05:48:57

The basic problem stems from the floating binary representation. 0.1 decimal is a recurring binary fraction, so you get rounding errors. However all integers up some very large value have precise representations as floats, so you are guaranteed not to get rounding errors as long as you stick to them. You can always specify IEE-488 arithmetic in the documentation if it makes you feel more comfortable.

Paul Johnson 2010-07-12 16:38:48

Answer 3

+4 A:

Greg Bacon 2010-07-12 02:41:46

This answer is excellent. The above solution works better than expected (it solved some false Bool values from being reported in a few 'brute force this calc. problem' functions. The definition of 'printApprox' is over my head, but I don't mind reading Rationals (and, of course, the goal is that it /won't/ be over my head in a while).

danportin 2010-07-12 06:57:41

@danportin Thanks. See updated answer.

Greg Bacon 2010-07-12 15:55:42

ansaurus

tags:

views:

answers:

[Haskell] Odd values when enumerating a list

related questions