Original companion articles exploring the mathematics and engineering of large language models.
← Back to Blog & Articles Training classifiers and language models usually means minimizing cross-entropy. The reason becomes clear once you connect it to […]
← Back to Blog & Articles Why should a model that fits training data also work on new data? Statistical learning theory gives conditions […]
← Back to Blog & Articles Modern training relies on a small family of optimizers. Each one fixes a specific weakness of the one […]
← Back to Blog & Articles Convex problems are special because every local minimum is a global minimum. Duality adds a second view that […]
← Back to Blog & Articles Both methods pick parameter values that explain data well. The difference is whether they also use prior beliefs […]
← Back to Blog & Articles The central limit theorem explains why averages of many independent measurements look approximately bell-shaped, even when the individual […]
← Back to Blog & Articles People often picture training a neural network as rolling into a valley and getting stuck in a bad […]
← Back to Blog & Articles Two problems with the same mathematical answer can behave very differently on a computer. The condition number measures […]
← Back to Blog & Articles Symmetric matrices appear constantly in machine learning, in covariance matrices, Hessians, and kernel matrices. The spectral theorem explains […]
← Back to Blog & Articles The singular value decomposition, or SVD, takes any matrix and expresses it as a rotation, a stretch, and […]
