error rate eps of each wrong candidate on the input distribution (100,000 inputs):
  upper-median    0.3462   tests needed to catch with probability 0.95: 8
  set-based       0.2441   tests needed to catch with probability 0.95: 11
  late-bug        0.1104   tests needed to catch with probability 0.95: 26
  mean            0.5653   tests needed to catch with probability 0.95: 4

probability that a wrong candidate passes n i.i.d. tests: simulated (20,000 test sets) vs (1-eps)^n
candidate                  n=1             n=3            n=10            n=30
upper-median     0.6505/0.6538   0.2771/0.2795   0.0152/0.0143   0.0000/0.0000
set-based        0.7531/0.7559   0.4283/0.4320   0.0611/0.0609   0.0003/0.0002
late-bug         0.8857/0.8896   0.7097/0.7039   0.3127/0.3103   0.0289/0.0299
mean             0.4284/0.4347   0.0830/0.0821   0.0003/0.0002   0.0000/0.0000

selecting among the correct program and the four wrong ones: the verifier accepts any candidate that passes all n tests and one of the accepted is returned uniformly
 n tests    P(returned program is wrong)
       1                          0.7125
       2                          0.6242
       3                          0.5272
       5                          0.3910
      10                          0.1740
      20                          0.0462
      50                          0.0010

stratified tests: one test per length 1..9, with a distinct-valued list of that length
  tests needed: 9;  wrong candidates that pass all nine: ['set-based']

memorizer: visible pass rate 1.00, hidden pass rate 0.010
