Symmetric matrices appear constantly in machine learning, in covariance matrices, Hessians, and kernel matrices. The spectral theorem explains why they are so well behaved.
The statement
A real symmetric matrix has real eigenvalues, and its eigenvectors can be chosen to be orthonormal. That means A = QΛQᵀ, where Q has orthonormal columns and Λ is diagonal.
Why it matters
A symmetric matrix acts like independent stretches along perpendicular directions. In the eigenvector basis the action is simply multiplication by each eigenvalue. That makes it easy to reason about powers, inverses, and quadratic forms.
Reading the signs
- All eigenvalues positive: the matrix is positive definite, and the quadratic form xᵀAx is bowl-shaped.
- All eigenvalues non-negative: positive semidefinite. Covariance matrices always fall in this class.
- Mixed signs: the surface curves up in some directions and down in others, as at a saddle point.
Connections to practice
The eigenvectors of a covariance matrix are the principal component directions, and the eigenvalues are the variances along them. The eigenvalues of the Hessian describe curvature of a loss surface, which affects how gradient descent behaves.
Quick check
If you need to know whether a symmetric matrix defines a convex quadratic, inspect its smallest eigenvalue. If it is non-negative, the quadratic is convex.
Continue with the book
Explore the full treatment, worked examples, and exercises in Mathematics for AI and Machine Learning.
Adapted from Mathematics for AI and Machine Learning, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.