If a model has seen test questions during training, its score reflects memory instead of ability. Contamination is one of the hardest problems in LLM evaluation.
How it happens
Training corpora scraped from the web can include benchmark questions, solutions, and discussions of them. Even paraphrased or translated copies can help the model. Contamination can also occur through fine-tuning data or through repeated tuning against a public test set.
Why it is hard to detect
Training data is often undisclosed, and exact string matching misses rewritten versions. A model can perform well on a leaked item without any obvious sign.
Detection approaches
- N-gram overlap between test items and training data, when the data is available.
- Perplexity comparison: unusually low perplexity on test items compared with fresh similar items can suggest memorization.
- Canary strings and held-back items that should never appear in training.
- Re-written variants: a large drop in score on paraphrased or newly created items is a warning sign.
Prevention
- Keep a private test set for final evaluation.
- Refresh benchmarks over time with new items.
- Limit how often you evaluate against the same test set.
- Document data sources and deduplicate training data against evaluation sets.
Reporting honestly
State what contamination checks you ran and what you could not check. A clean result from a weak check is less reassuring than it seems.
Continue with the book
Explore the full treatment, worked examples, and exercises in Statistical Foundations for LLM Development and Evaluation.
Adapted from Statistical Foundations for LLM Development and Evaluation, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.