Generating "too perfect" random numbers

Question

A good RNG ought to pass several statistical tests of randomness. For example, uniform real values in the range 0 to 1 can be binned into a histogram with roughly equal counts in each bin, give or take some due to statistical fluctuations. These counts obey some distribution, I don't recall offhand if it's Poisson or binomial or what, but in any case these distributions have tails. Same idea applies to tests for correlations, subtle periodicities etc.

A high quality RNG will occasionally fail a statistical test. It is good advice to be suspicious of RNGs that look to perfect.

Well, I'm crazy and would like to generate (reproducibly) "too perfect" random numbers, ones suspiciously lacking in those random fluctuations in statistical measures. Histograms come out too flat, variances of moving-box averages come out too small, correlations suspiciously close to zero, etc. Looking for RNGs that pass all statistical tests too cleanly. What known RNGs are like this? Is there published research on this idea?

One unacceptable answer: some of the poorer linear congruential counter generators have too flat a distribution, but totally flunk most tests of randomness.

Related to this is the generation of random number streams with a known calibrated amount of imperfection. A lump in the distribution is easy - just generate a nonuniform distribution approximating the idea (e.g see http://stackoverflow.com/questions/977354/generating-non-uniform-random-numbers) but what about introducing calibrated amounts of higher order correlations while maintaining a correct, or too perfect, distribution?

Answer 1

You could try something like:

numbers = [1, 2, 3, 4, 5, 6] * 100
random.shuffle(numbers)

to get a random sequence with a perfect uniform distribution.

Answer 2

By definition, a PRNG (pseudorandom number generator) cannot generate truly random numbers. No matter what trick you use to generate your pseudorandom sequence, there exists a test that will expose the trick, by showing the actual nonrandomness.

Answer 3

You can't. If it is flat in one test this will mean failure in another one, since the flatness shows it is not random.

Answer 4

Apparently the Mersenne Twister, a commonly used random number generator, fails the DieHarder tests by being "too random". In other words, certain tests consistently come too close to their expected value under true randomness.

Answer 5

I think what you're looking for may be a quasi-random sequence. A quasi-random sequence jitters around but in a self-avoiding way, not clumping as much as a random sequence. When you look at how many points fall in different bins, the distribution will work out "too well" compared to a random sequence.

Also, this article may be relevant: When people ask for a random sequence, they’re often disappointed with what they get.

Answer 6

The folks at the National Institutes of Standards and Technology Computer Security Division have an abiding interest in RNGs and being able to measure the degree of randomness. I found this while looking for the old DIEHARD suite of PRNG tests.

The folks at the National Security Agency have an abiding interest in RNGs also, but they aren't going to tell you much.

Answer 7

If you wish to generate a set of random numbers while tied to a set a correlation, you may want to investigate the Cholesky decomposition. I suspect from there you would just need a simple transformation to generate your "too perfect" random numbers.