Convex problems are special because every local minimum is a global minimum. Duality adds a second view that often makes hard problems easier to solve and understand.
What convex means
A function is convex if the line segment between any two points on its graph lies on or above the graph. A convex optimization problem minimizes a convex function over a convex set. Gradient-based methods can find a global solution reliably.
The Lagrangian
To handle constraints, form the Lagrangian by adding each constraint multiplied by a non-negative multiplier. The dual problem maximizes the resulting lower bound over the multipliers. Weak duality says the dual value never exceeds the primal value. Under mild conditions such as Slater's condition, strong duality holds and the two values match.
Worked idea: the hard-margin SVM
A hard-margin support vector machine finds the separating hyperplane with the largest margin. This is a convex quadratic program: minimize ½‖w‖² subject to every training point being classified correctly with margin at least 1. The dual form depends on the data only through inner products, which is what makes kernels possible.
What duality gives you
- A certificate of optimality through the duality gap.
- Insight into which constraints matter. Points with non-zero multipliers are the support vectors.
- Algorithms that work in the dual when it is simpler.
Why it matters beyond SVMs
Regularized regression, many resource-allocation problems, and parts of reinforcement learning use the same convex tools.
Continue with the book
Explore the full treatment, worked examples, and exercises in Mathematics for AI and Machine Learning.
Adapted from Mathematics for AI and Machine Learning, published by Choral Software. Article text and original examples © 2026 Choral Software. All rights reserved.