power and modulo on the fly for big numbers

Question

I raise some basis b to the power p and take the modulo m of that.

Let's assume b=55170 or 55172 and m=3043839241 (which happens to be the square of 55171). The linux-calculator bc gives the results (we need this for control):

echo "p=5606;b=55171;m=b*b;((b-1)^p)%m;((b+1)^p)%m" | bc
2734550616
309288627

Now calculating 55170^5606 gives a somewhat large number, but since I have to do a modulooperation, I can circumvent the usage of BigInt, I thought, because of:

(a*b) % c == ((a%c) * (b%c))%c i.e.
(9*7) % 5 == ((9%5) * (7%5))%5 =>
63 % 5    == (4     *    2) %5 =>
3         == 8 % 5

... and a^d = a^(b+c) = a^b * a^c, therefore I can divide b+c by 2, which gives, for even or odd ds d/2 and d-(d/2), so for 8^5 I can calculate 8^2 * 8^3.

So my (defective) method, which always cut's off the divisor on the fly looks like that:

def powMod (b: Long, pot: Int, mod: Long) : Long = { 
      if (pot == 1) b % mod else {
          val pot2 = pot/2
          val pm1 = powMod (b, pot2, mod)             
          val pm2 = powMod (b, pot-pot2, mod)           
          (pm1 * pm2) % mod 
      } 
}

and feeded with some values,

powMod (55170, 5606, 3043839241L) 
res2: Long = 1885539617
powMod (55172, 5606, 3043839241L) 
res4: Long = 309288627

As we can see, the second result is exactly the same as the one above, but the first one looks quiet different. I'm doing a lot of such calculations, and they seem to be accurate as long as they stay in the range of Int, but I can't see any error. Using a BigInt works as well, but is way too slow:

def calc2 (n: Int, pri: Long) = {
    val p: BigInt = pri
    val p3 = p * p
    val p1 = (p-1).pow (n) % (p3)
    val p2 = (p+1).pow (n) % (p3)
    print ("p1: " + p1 + " p2: " + p2)
}

calc2 (5606, 55171) 
p1: 2734550616 p2: 309288627

(same result as with bc) Can somebody see the error in powMod?

Answer 1

Not familiar with Scala, but...

def powMod (b: Long, pot: Int, mod: Long) : Long = {  
      if (pot == 1) b % mod else { 
          val pot2 = pot/2 
          val pm1 = powMod (b, pot, mod)              
          val pm2 = powMod (b, pot-pot2, mod)            
          (pm1 * pm2) % mod  
      }  
} 

Did you mean

          val pm1 = powMod (b, pot2, mod) 

Notice the pot2 instead of pot.

Strangely, it seems that this should loop forever/overflow the stack, but who knows what Scala is doing.

Answer 2

I think the answer is here:

scala> math.sqrt(Long.MaxValue).toLong < 3043839241L
res9: Boolean = true

That means you can have a long overflow even for numbers which are less than that particular module value. Let's try to catch it:

scala> def powMod (b: Long, pot: Int, mod: Long) : Long = {
     |       if (pot == 1) b % mod else {
     |           val pot2 = pot/2
     |           val pm1 = powMod (b, pot2, mod)
     |           val pm2 = powMod (b, pot-pot2, mod)
     |           val partial = ((pm1 % mod) * (pm2 % mod)).ensuring(res =>
     |             res > pm1 % mod && res > pm2 % mod, "Long overflow multiplying "+pm1+" by "+pm2)
     |           partial % mod
     |       }
     | }
powMod: (b: Long,pot: Int,mod: Long)Long

scala> powMod (55170, 5606, 3043839241L)
java.lang.AssertionError: assertion failed: Long overflow multiplying 3042625480 by 3042625480

There you have it.

Answer 3

Ok fellows, it took me some time, and finally destroyed a long but never proven assumption, which was, that if you multiply two 64-bit-positive integral values (aka: Longs, and practically only 63-bit, after all), you could overrun, and get negative values, but not get an overrun to reach positive (but wrong) values again.

So I had tried to put a guard into my code, to calculate my value with BigInt, it too big, but the guard was insufficient, because I tested for res < 0. res < pm1 && res < pm2 isn't sufficient too.

To increase the speed I used a mutable.HashMap, and now the code looks like this:

val MVL : Long = Integer.MAX_VALUE
var modPow = new scala.collection.mutable.HashMap [(Long, Int, Long), Long ] () 

def powMod (b: Long, pot: Int, mod: Long) : Long = { 
      if (pot == 1) b % mod else modPow.getOrElseUpdate ((b, pot, mod), {
    val pot2= pot/2
    val pm1 = powMod (b, pot2, mod)             
    val pm2 = powMod (b, pot-pot2, mod)
    val res = (pm1 * pm2) 
    // avoid Long-overrun
    if (pm1 < MVL && pm2 < MVL)
        res % mod else {
            val f1: BigInt = pm1
            val f2: BigInt = pm2
            val erg = (f1 * f2) % mod
            erg.longValue 
        }
      })
}

You might ask yourself, whether the Long-declared MVL is really needed, or whether a

if (pm1 < Integer.MAX_VALUE && ...

would have worked too. No. It wouldn't. :) Another trap to avoid. :)

Finally it is pretty fast and correct and I learned two lessons about overruns and MAX_VALUE - comparision.