Bias and variance describe two different ways a learned model can fail. They are most useful as diagnostic concepts rather than slogans about model complexity.
High bias
A high-bias model cannot capture important structure. Training and validation errors are both poor and often close together. Useful responses include richer features, a more expressive model, or less restrictive regularization.
High variance
A high-variance model fits the training set well but performs much worse on unseen data. More representative data, stronger regularization, augmentation, ensembling, or a simpler hypothesis class may improve generalization.
Use learning curves
Compare training and validation performance as data volume grows. Curves reveal whether collecting more data is likely to help and whether the evaluation split resembles deployment. Always examine subgroup performance and distribution shift; one average error can hide important failures.
Continue with the book
Explore the full treatment, worked examples, and exercises in The AI Algorithms Compendium.
Adapted from The AI Algorithms Compendium, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.