true per-turn gradient c p (1-p) = 0.2500
   H |                outcome |           reward-to-go |              immediate | sum over turns of the variance (exact)
     |   sim / exact (turn 1) |   sim / exact (turn 1) |   sim / exact (turn 1) |   outcome     to-go immediate
   1 |       0.251 /    0.250 |       0.251 /    0.250 |       0.251 /    0.250 |      0.25      0.25      0.25
   2 |       0.562 /    0.562 |       0.562 /    0.562 |       0.249 /    0.250 |      1.12      0.81      0.50
   4 |       1.189 /    1.188 |       1.189 /    1.188 |       0.250 /    0.250 |      4.75      2.88      1.00
   8 |       2.433 /    2.438 |       2.433 /    2.438 |       0.250 /    0.250 |     19.50     10.75      2.00
  16 |       4.943 /    4.938 |       4.943 /    4.938 |       0.250 /    0.250 |     79.00     41.50      4.00
  32 |       9.937 /    9.938 |       9.937 /    9.938 |       0.251 /    0.250 |    318.00    163.00      8.00

means of the three estimators for turn 1 at H = 8 (should all equal c p (1-p) = 0.25):
   outcome: 0.2491   reward-to-go: 0.2491   immediate: 0.2506
