  lr lag k                 naive                    is             tis (c=2)              selfnorm
          final reward / unstablefinal reward / unstablefinal reward / unstablefinal reward / unstable
 4.0     0       2.187 /  0.00         2.187 /  0.00         2.187 /  0.00         2.187 /  0.00  
 4.0     4       2.203 /  0.00         2.173 /  0.00         2.194 /  0.00         2.173 /  0.00  
 4.0    16       2.208 /  0.00         2.137 /  0.00         2.205 /  0.00         2.137 /  0.00  
 4.0    64       2.205 /  0.00         2.137 /  0.00         2.110 /  0.00         2.156 /  0.00  
16.0     0       2.152 /  0.00         2.152 /  0.00         2.152 /  0.00         2.152 /  0.00  
16.0     4       2.201 /  0.00         2.067 /  0.00         2.200 /  0.00         2.048 /  0.00  
16.0    16       2.209 /  0.00         2.044 /  0.00         2.204 /  0.00         2.090 /  0.00  
16.0    64       2.209 /  0.00         2.038 /  0.00         2.192 /  0.00         2.056 /  0.00  
64.0     0       2.076 /  0.00         2.076 /  0.00         2.076 /  0.00         2.076 /  0.00  
64.0     4       2.179 /  0.00         1.978 /  0.00         1.958 /  0.00         1.978 /  0.00  
64.0    16       2.201 /  0.00         1.966 /  0.00         1.835 /  0.00         1.970 /  0.00  
64.0    64       2.208 /  0.00         1.958 /  0.00         1.850 /  0.00         1.926 /  0.00  

mean effective sample size fraction of the importance weights (lr = 64.0)
  lag 4: is 0.947, tis 0.897
  lag 16: is 0.820, tis 0.735
  lag 64: is 0.348, tis 0.182

best possible reward (mean over seeds of max_a m_a): 2.209

sequence-level importance weight over T tokens when each token ratio is lognormal with E[rho] = 1 and log-sd 0.1
     T  ESS fraction (sim)   exp(-T sd^2) (theory)   max weight / mean
     1              0.9901                  0.9900                 1.5
    10              0.9043                  0.9048                 3.2
   100              0.3680                  0.3679                33.5
   400              0.0315                  0.0183               282.1
