Ex 9.2: variance of the leave-one-out advantage for i.i.d. rewards with variance s^2 is s^2 (1 + 1/(G-1))
  G =  2: simulated 2.0059   formula 2.0000
  G =  4: simulated 1.3301   formula 1.3333
  G =  8: simulated 1.1399   formula 1.1429
  G = 32: simulated 1.0302   formula 1.0323

Ex 9.3: Bernoulli(p) reward, logit parameter; exact expectation of the group-normalized estimate
      p      G  E[std-normalized]  sqrt(p(1-p))  E[mean-centered]   p(1-p)
    0.5      8            0.46391       0.50000           0.21875  0.25000
    0.5     64            0.49605       0.50000           0.24609  0.25000
    0.5   1024            0.49976       0.50000           0.24976  0.25000
    0.1      8            0.20949       0.30000           0.07875  0.09000
    0.1     64            0.29293       0.30000           0.08859  0.09000
    0.1   1024            0.29959       0.30000           0.08991  0.09000
   0.02      8            0.05043       0.14000           0.01715  0.01960
   0.02     64            0.11493       0.14000           0.01929  0.01960
   0.02   1024            0.13911       0.14000           0.01958  0.01960

Ex 9.4: with length normalization the baseline no longer drops out: E[psi / len] != 0
  E[psi] = [ 0. -0. -0. -0.];  E[psi / len] = [ 0.0225 -0.06    0.0675 -0.03  ]

Ex 9.6: weight 1/sqrt(p(1-p)) of a prompt under std-normalization, and the chance that a group of G is constant
       p   weight  P(const), G=4  P(const), G=8 P(const), G=16
     0.5     2.00          0.125          0.008          0.000
     0.2     2.50          0.411          0.168          0.028
    0.05     4.59          0.815          0.663          0.440
    0.01    10.05          0.961          0.923          0.851
