A model that says it is 90% confident should be right about 90% of the time. Calibration measures how well stated confidence matches reality.
What calibration means
Group predictions by confidence and compare average confidence with actual accuracy in each group. A perfectly calibrated model lies on the diagonal. Overconfident models fall below it.
Measuring it
A reliability diagram plots accuracy against confidence by bin. Expected calibration error summarizes the gap, though it depends on the binning choice and can be noisy with small samples.
Proper scoring rules
A scoring rule is proper if the best expected score comes from reporting your true beliefs. The Brier score, which is the mean squared error of predicted probabilities, and log loss are proper. Accuracy alone is not, because it ignores confidence. Proper scores reward both calibration and sharpness.
Why LLMs need this
Language models can be confidently wrong. Token probabilities, verbal confidence statements, and judge scores may each be miscalibrated, particularly after instruction tuning.
Improving calibration
- Temperature scaling adjusts logits using a single parameter fitted on held-out data.
- Isotonic regression and other recalibration maps work with more data.
- Evaluate calibration on the distribution you will deploy on, since it can shift.
Takeaway
Report a proper score alongside accuracy whenever predictions carry confidence, and verify calibration on fresh data before trusting probabilities in decisions.
Continue with the book
Explore the full treatment, worked examples, and exercises in Statistical Foundations for LLM Development and Evaluation.
Adapted from Statistical Foundations for LLM Development and Evaluation, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.