Maximum Likelihood vs. MAP: Two Ways to Estimate, One Hidden Prior

← Back to Blog & Articles

Both methods pick parameter values that explain data well. The difference is whether they also use prior beliefs about the parameters.

Maximum likelihood estimation

MLE chooses the parameters that make the observed data most probable. For independent data, that means maximizing the product of probabilities, or equivalently the sum of log-probabilities. Many familiar loss functions are negative log-likelihoods: squared error corresponds to Gaussian noise, and cross-entropy corresponds to categorical outcomes.

Maximum a posteriori estimation

MAP adds a prior distribution over parameters and maximizes the posterior, which is proportional to likelihood times prior. In log form, you maximize the log-likelihood plus the log-prior.

The regularization connection

A Gaussian prior on weights adds an L2 penalty, which is ridge regularization.
A Laplace prior adds an L1 penalty, which encourages sparsity.

So when you regularize, you are implicitly stating a prior, whether or not you call it that.

When they differ

With lots of data, the likelihood dominates and the two estimates become close. With little data, the prior can pull the estimate toward sensible values and reduce overfitting.

A limitation

MAP still returns a single point. A fully Bayesian approach keeps the whole posterior and can express uncertainty about parameters.

Continue with the book

Explore the full treatment, worked examples, and exercises in Mathematics for AI and Machine Learning.