You can't have an error bound in general. Either your algorithm works or it doesn't. The reason for this is that a reasonable error bound is obviously much smaller that RAND_MAX. That in turn means that the the low bits are not as random as the higher bits. But a good PRNG makes certain that all bits are equally random.
Consider this slow but mathematically sound example of an RNG algorithm:
int rand() {
state = AES_encrypt(state);
return state % RAND_MAX;
}
void srand(int seed) {
state = AES_encrypt(seed);
}
If you can find any significant correlation between the output sequence and the previous state
, the AES algorithm should be considered broken.