Two problems with the same mathematical answer can behave very differently on a computer. The condition number measures how sensitive a problem is to small changes in its input.
The definition for matrices
For an invertible matrix, the condition number in the 2-norm is the ratio of the largest singular value to the smallest. A value near 1 means the matrix is well conditioned. A very large value means it is nearly singular.
What it predicts
When solving Ax = b, a relative error in b or A can be amplified by about the condition number in the worst case. If the condition number is 10⁸ and the data has about 16 digits of precision in double arithmetic, you may retain only around 8 trustworthy digits.
Symptoms in machine learning
- Linear regression with highly correlated features produces unstable coefficients.
- Gradient descent crawls along narrow valleys when the Hessian is ill conditioned.
- Normal equations square the condition number, which is why solving least squares through the QR decomposition or the SVD is preferred.
Practical remedies
- Standardize features so they have similar scales.
- Add regularization, such as a ridge penalty, which improves conditioning.
- Prefer numerically stable factorizations over explicit matrix inverses.
- Use preconditioning for iterative solvers.
The takeaway
An algorithm can be correct on paper and still fail numerically. Checking the condition number is a quick way to see whether trouble is coming.
Continue with the book
Explore the full treatment, worked examples, and exercises in Mathematics for AI and Machine Learning.
Adapted from Mathematics for AI and Machine Learning, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.