views:

273

answers:

4

Hey all, let's jump straight to a code sample to show how ECMAScript/JavaScript/AS3 can't do simple math right (AS3 uses a 'IEEE-754 double-precision floating-point number' for the Number class which is supposedly identical to that used in JavaScript)...

trace(1.1);         //'1.1': Ok, fine, looks good.
trace(1.1*100);     //'110.00000000000001': What!?
trace((1.1*100)/100);  //'1.1': Brings it back to 1.1 (since we're apparently multiplying by *approximately* 100 and then dividing by the same *approximate* '100' amount)
trace(1.1*100-110); //'1.4210854715202004e-14': Proof that according to AS3, 1.1*100!=110 (i.e. this isn't just a bug in Number.toString())
trace(1.1*100==110);   //'false': Even further proof that according to AS3, 1.1*100!=110

What gives?

+2  A: 

This is just a side effect of using floating point numbers - these are binary representations of decimal numbers, there will always be some approximations.

Long explanation

Paddy
+7  A: 

Welcome to the wonderful world of floating point calculation accuracy. In general, floating point calculations will give you results that are very very nearly correct, but comparing outputs for absolute equality is unlikely to give you results you expect without the use of rounding functions.

moonshadow
+2  A: 

Floating point inconsistencies are a known problem in many languages. This is because computers aren't designed to handle floating point numbers.

Have fun

Matt Dawdy
+2  A: 

As moonshadow states, you're running into issues with floating point precision. Floating point numbers aren't suited to the task of representing and performing arithmetic upon decimal values in the manner that you would expect. These kinds of problems are seen most often when people try to using floating point numbers for financial calculations. The wikipedia entry is good, but you might get more out of this page, which steps through an error-prone financial calculation: http://c2.com/cgi/wiki?FloatingPointCurrency

To accurately deal with decimal numbers you need a decimal library. I've outlined two BigDecimal-style libraries written in javascript that may suit your needs in another SO post, hopefully you'll find them useful:

http://stackoverflow.com/questions/744099/javascript-bigdecimal-library/1575569

Matt Baker