ansaurus

Question

SQL Server makes up extra precision for floats???

Answer 1

+3 A:

According to Books-On-Line:

Float: Approximate-number data types for use with floating point numeric data. Floating point data is approximate; therefore, not all values in the data type range can be represented exactly.

Emphasis mine.

G Mastros 2009-07-29 21:23:48

you're missing the point of the issue..it's ADDING precision, not SUBTRACTING it. Where could it possibly be getting the extra decimal places from? It's just pulling them out of the sky.

Scott 2009-07-29 21:34:58

@Scott A float can not hold EXACTLY 41.88694340165634, so the float becomes something close such as 41.886943401656339.

Shannon Severance 2009-07-29 22:18:22

Answer 2

A:

I haven't personally seen this, but it might just be that the SQL server management studio shows the "representation" of the float i.e. how the value would be stored in the db. Remember that floats are all approximate anyway.

nagul 2009-07-29 21:25:53

but that would explain the phenomenon of precision LOSS, not precision ADDITION.

Scott 2009-07-29 21:35:46

Not really. As @hova points out, it's not precision per se. It's to do with how floats are stored. Since 41.88694340165634 cannot be stored in the bits available for a float, it stores the nearest avabilable representation. Have a look here to understand how floats are internally represented: http://steve.hollasch.net/cgindex/coding/ieeefloat.html

nagul 2009-07-29 22:16:30

Consider a hypothetical situation:Let's assume that 1.0 cannot be stored in the x bits a hypothetical flaot has. What can we do then ? We can store the closest number to 1 that can be put in a flaot, which would be 0.99999 or 0.999 or 0.9 or 1.000001 or some other such approximation. The important thing here is that we're not increasing precision, just storing the nearest value to 1.0 we can. Hence, flaot is an "approximate" datatype, same as float.

nagul 2009-07-29 22:23:01

Answer 3

+5 A:

It's not adding precision, it's just rounding it to the nearest IEEE floating point representation. When you convert that back to decimal, it only LOOKS like it gained precision.

hova 2009-07-29 21:39:53

thanks for the insights, makes sense now.

Scott 2009-07-30 00:40:35

ansaurus

tags:

views:

answers:

SQL Server makes up extra precision for floats???

related questions