pr.probability - Is ERNIE output skewed by statistical tests?

Yes. In the same way that flipping a fair coin (with equal probabilities of getting heads of tails) eight times in a row is likely to come up all heads 1/256 times, or all tails 1/256 times. The psychological perception of a sequence with a run of 8 heads or 8 tails is that it is so unlikely as to never occur at all; whereas we mathematicians see the likelihood of a run of 8 in 8 flips as occuring with 2/256 or just under 1% of the time.

The opposite error is true, and also occurs with some frequency in biomedical experiments and medical experiments. The standard for accepting a result in a clinical or medical trial is for $p<0.05$: that there is less than a 5% probability that the results occured by chance. Thus, one in twenty times, it is possible that a random occurence or set of occurences will be perceived or accepted as being statistically valid when it is not.

But it also depends on how much data (how many draws) are in the sample being gauged for randomness. The smaller the sample size, the more likely you are to discard a valid but unreasonable appearing "true" random sequence. So my answer is really a qualified "maybe".

Shouldn't the validity or "true randomness" of the method be the gauge, along with a check to see that the algorithm is properly implemented? The problem, of course, with software is that bugs can creep into the implementation at any point:

• the compiler could be messed up, generating incorrect code,




• the ALU (the arithmetic logic unit) in the central processing unit could be incorrectly implemented, e.g. the Pentium chip had the floating-point co-processor which incorrectly calculated some particular multiply operations depending upon the operands,




• the algorithm specified in the program requirements may be correct, but may be implemented incorrectly,




• the algorithm may be correct for 32-bit integer math calculations, but incorrect if the system uses 64-bit arithmetic, or vice versa,

I wonder if the software code and hardware is vetted as well there as it is in Las Vegas by the Nevada Gaming Commision which oversees gambling and the electronic machinery for slots and electronic poker, etc.