views:

35

answers:

3

According to the entry for decimal and numeric data types in SQL Server 2008 Books Online, precision is:

p (precision) The maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point. The precision must be a value from 1 through the maximum precision of 38. The default precision is 18.

However, the second select below fails with "Arithmetic overflow error converting int to data type numeric."

SELECT CAST(123456789 as decimal(9,0))
SELECT CAST(123456789 as decimal(9,1))
+4  A: 

Correct. Since you're doing decimal(9,1) that means you have 9 total digits, but the ,1 is reserving one of them for the right of the decimal place, so you can do at most 8 to the left and 1 to the right.

Donnie
A: 

try
SELECT CAST(123456789 as decimal(10,1))

VoodooChild
+6  A: 

see here: http://msdn.microsoft.com/en-us/library/aa258832(SQL.80).aspx

decimal[(p[, s])]

p (precision) Specifies the maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point. The precision must be a value from 1 through the maximum precision. The maximum precision is 38. The default precision is 18.

s (scale) Specifies the maximum number of decimal digits that can be stored to the right of the decimal point. Scale must be a value from 0 through p. Scale can be specified only if precision is specified. The default scale is 0; therefore, 0 <= s <= p. Maximum storage sizes vary, based on the precision.

when using: decimal(p,s), think of p as how many total digits (regardless of left or right of the decimal point) you want to store, and s as how many of those p digits should be to the right of the decimal point.

DECIMAL(10,5)=     12345.12345
DECIMAL(10,2)=  12345678.12
DECIMAL(10,10)=         .1234567891
DECIMAL(11,10)=        1.1234567891

your sample code fails:

SELECT CAST(123456789 as decimal(9,1))

because:

9=precision (total number of digits to left and right of decimal)
1=scale (total number of digits to the right of the decimal)
(9-1)=8 (total digits to the left of the decimal)

and your value 123456789 requires 9 digits to the left of the decimal. you will need decimal(10,1) or just decimal(9,0)

KM
Scale is the maximum number to the right -- we had an insert problem that led us to do some tests and conclude that if the entire scale wasn't needed then SQL Server would allow additional digits up to p to the left of the decimal point. Is there a server setting that could change this behavior?
Jamie Ide
@Jamie Ide said `led us to do some tests and conclude that if the entire scale wasn't needed then SQL Server would allow additional digits up to p to the left of the decimal point`. I really find that hard to believe. The Decimal data type's Precision and Scale are well documented and no exceptions are included or mentioned. I think it is more likely you had a problem with your test. Post a script (with sample data) that shows this behavior.
KM
I think I've sorted out my confusion on this. I had a developer who thought decimal(p,s) meant p places to the left, s to the right. I created a test table to demo it and in the course of testing we both discovered that SQL Server rounds to the scale. Somehow we conflated that into the conclusion in my first comment.
Jamie Ide