20.1 list-scheduling bound: sum(L)/S <= continuous steps <= sum(L)/S + (1 - 1/S) max(L); and static utilization via E[max]
  S=8: largest (steps - sum/S)/max(L) over 200 trials = 0.738 (bound 0.875)
  S=64: largest (steps - sum/S)/max(L) over 200 trials = 0.868 (bound 0.984)
  S=16: E[max of S lengths] = 882, E[L] = 275, static utilization E[L]/E[max] = 0.312
  S=64: E[max of S lengths] = 1335, E[L] = 275, static utilization E[L]/E[max] = 0.206

20.2 truncated importance sampling: bias <= max|f| E_mu[(rho - c)^+], second moment <= c max|f|^2
  trial 0: E_mu[rho^2] = 13.46; at c=2: |bias| 0.1509 <= 0.3863
  trial 1: E_mu[rho^2] = 30.90; at c=2: |bias| 0.1885 <= 0.5089
  trial 2: E_mu[rho^2] = 12.94; at c=2: |bias| 0.1317 <= 0.3943

20.3 self-normalized importance sampling has bias of order 1/n
  n=  10: bias -0.20445, n * bias -2.045
  n=  40: bias -0.11335, n * bias -4.534
  n= 160: bias -0.03915, n * bias -6.263
  n= 640: bias -0.01362, n * bias -8.717

20.4 effective sample size of sequence-level weights for per-token log-sd 0.1: exp(-T sd^2) and its failure at large T
  T=   10: simulated 0.9039   exp(-T sd^2) = 0.9048
  T=  100: simulated 0.3594   exp(-T sd^2) = 0.3679
  T=  400: simulated 0.0107   exp(-T sd^2) = 0.0183
  T= 1000: simulated 0.0027   exp(-T sd^2) = 0.0000

20.5 checkpointing memory ratio L / ((L + 1) / a')  for a = 16.1 block coefficient: full = a L, checkpointed = L + a (one block recomputed) in units of B T d elements
  L=  4: full    64.5, checkpointed   20.1, ratio   3.2
  L= 12: full   193.6, checkpointed   28.1, ratio   6.9
  L= 32: full   516.2, checkpointed   48.1, ratio  10.7
  L= 80: full  1290.4, checkpointed   96.1, ratio  13.4

20.6 KV cache arithmetic: tokens per GiB for grouped-query attention with g key/value heads
  kv heads  1:     16 KiB per token,    65536 tokens per GiB
  kv heads  8:    128 KiB per token,     8192 tokens per GiB
  kv heads 32:    512 KiB per token,     2048 tokens per GiB

20.7 balanced split: n_g / n_t = t_gen / t_train, where both are single-device times; effect of the response length
  response  200 tokens: single-device generation    64.4 s, training  1147.1 s, ratio n_g:n_t = 0.06
  response  400 tokens: single-device generation   128.9 s, training  1469.3 s, ratio n_g:n_t = 0.09
  response  800 tokens: single-device generation   257.8 s, training  2113.8 s, ratio n_g:n_t = 0.12
  response 1600 tokens: single-device generation   515.5 s, training  3402.6 s, ratio n_g:n_t = 0.15

20.8 numerical mismatch between two implementations of the same policy: logits rounded to bfloat16 versus float32
  per-token log-ratio: mean -0.0000, sd 0.0147; implied sequence-level ESS fraction exp(-T sd^2): T=100 0.979, T=1000 0.805
