P(success) = 0.01563 (= 1/64);  |g| = 0.02210

estimator      tokens/sample  rel. bias   variance  variance x tokens
outcome                    6   2.52e-15     0.0308             0.1846
reward-to-go               6   2.39e-15     0.0308             0.1846
advantage                  6   1.88e-15     0.0098             0.0586
MC(N=1)                   27   2.02e-02     0.0839             2.2654
MC(N=2)                   48   1.20e-02     0.0461             2.2130
MC(N=4)                   90   9.38e-03     0.0280             2.5222
MC(N=16)                 342   6.82e-03     0.0143             4.8946

exact per-sample variance of the three estimators as the policy improves (pi_k(c_k) = p at every step)
     p  P(success)    outcome  reward-to-go  advantage      |g|
   0.3     0.00073    0.00285       0.00268    0.00050  0.00144
   0.5     0.01563    0.03076       0.03076    0.00977  0.02210
   0.7     0.11765    0.09467       0.13287    0.08969  0.09983
   0.9     0.53144    0.29105       0.37850    0.36918  0.15031
  0.97     0.83297    0.23580       0.25472    0.25429  0.07068
