views:

277

answers:

2

I was wondering if someone could help me, as I have gone blind to what I believe is a simple cause to a simple error.

I have this code:

 doRound1(x1)
  denom1 = 5
  y1 = denom1 - x1 mod denom1
  if y1 <> denom1 then
    x1= x1+y1
  end if

  doRound1=x1
 End function

 'theCalc = 20488888888.684
  theCalc = cDbl(11111111111) * 1.844
  doRound1(theCalc)

I get this error

Microsoft VBScript runtime  error '800a0006'
Overflow: 'x1'

Caused by this line in the above code:

 y1 = denom1 - x1 mod denom1

Any ideas? As I say, I have gone blind this p.m.

+1  A: 

I believe that when you don't specifically define the values, ASP Classic assumes a numeric value to be an int (which is only 32767). Try forcing your values to be longs

 Function doRound1(x1)
  x1 = CDbl(x1)
  denom1 = CDbl(5)
  y1 = denom1 - x1 mod denom1
  if y1 <> denom1 then
    x1= x1+y1
  end if

  doRound1=x1
 End function

Note: I haven't actually tested this.

C. Ross
The answer on this one (http://stackoverflow.com/questions/316312/misunderstanding-of-long-data-type-in-vba ) goes into a bit more detail about this.
Chris Lively
@C. Ross: x1 = CLng(x1) causes `Overflow: 'cLng'`
Grant Wagner
Actually, those should probably be Doubles. However, I suspect that you have the right answer otherwise. I recommend doubles because a Long in VBScript still caps at 2B; also OP's input appears to be a double based on the comments in the post.
John Rudy
@John: Still doesn't help the OP. `theCalc = 20488888888.684` then `cDbl(theCalc) mod cDbl(5)` still causes `Overflow: 'cDbl(...)'`
Grant Wagner
VBScript has its limitations ... but yes, CDbl makes more sense.
C. Ross
@C. Ross: -1. This untested sample doesn't work. The correct answer is that `mod` causes floating point numbers used in the expression to be converted to Longs first. If the floating point number is too big to fit in a Long, an overflow will occur.
Grant Wagner
+3  A: 

The answer appears to be at PRB: "Overflow" with Integer Division and MOD Operator:

The Visual Basic Help topic for the Mod operator and the integer division operator () explains that if floating point numbers are used in the expression, they are converted to Longs first. Thus, if the floating point number is greater than the maximum value of a Long (2,147,483,647), or less than the minimum value for a long (-2,147,483,648), an overflow error will occur.

The answer is available there as well:

The following code demonstrates how to perform integer division and modulo arithmetic when the size of an operand is sufficiently large to cause overflow:

Dim dblX as Double
Dim dblY as Double
dblX = 2147483648                ' numerator
dblY = 123                       ' denominator

' round off the numerator and denominator (ensure number is .0)
dblX = INT(dblX + .5)         
dblY = INT(dblY + .5)      

' Emulate integer division
MsgBox FIX(dblX / dblY)             
' Emulate modulo arithmetic
MsgBox dblX - ( dblY * FIX(dblX / dblY) )
Grant Wagner
Any reason Int(x + .5) is used instead of CInt(x) ?
AnthonyWJones
Also it would be better to edit the code to VBScript since this is a classic ASP question not a VB6 question. +1 though
AnthonyWJones