For anyone serious about the mathematical foundations of machine learning, Christopher Bishop’s Pattern Recognition and Machine Learning (PRML) is nothing short of a rite of passage. Originally published in 2006, it remains a timeless masterpiece. Even in an era dominated by massive large language models and empirical deep learning engineering, the foundational principles of statistical modeling, Bayesian inference, and continuous optimization laid out in PRML are more relevant than ever.

Having spent considerable time teaching this material, I’ve found that while PRML is incredibly elegant, it is also notoriously dense. Bishop has a brilliant way of stating profound truths, but he frequently leaves the algebraic heavy lifting “as an exercise for the reader.”

For a student trying to build intuition, jumping from equation (X) to equation (Y) can sometimes feel like drawing the rest of the owl.

To bridge this gap for my students—and to satisfy my own obsession with systematic modeling and clarity—I have spent the last few terms building a comprehensive, heavily annotated version of the textbook. Today, I’m excited to share these resources publicly on my teaching page.

Why We Still Need to Teach PRML

In modern AI education, there is a strong temptation to focus entirely on implementation—importing PyTorch, calling APIs, and tweaking hyperparameters. But black-box engineering only takes you so far. When a model fails, or when you need to design a novel architecture for complex, high-dimensional scientific data, you have to understand the underlying mechanics.

Teaching PRML allows us to dive deep into:

• The Bayesian Perspective: Shifting away from point estimates to entire probability distributions over parameters, which is vital for understanding uncertainty.
• The Geometry of Spaces: Developing a physical intuition for high-dimensional Gaussians, latent spaces, and kernel functions.
• Elegant Approximations: Realizing that when exact integration is impossible, techniques like Variational Inference and Expectation-Maximization (EM) give us the tools to make intractable problems tractable.

What’s Inside the Annotated PDF?

My goal with these annotations was simple: leave no algebraic leap unexplained. If an equation changes form, I want the reader to see exactly which identity, substitution, or matrix property made it happen.

Some of the key areas where I’ve fleshed out the step-by-step derivations include:

• Chapter 2 (Probability Distributions): Complete expansion of the conditional and marginal distributions for fully joint Gaussian variables. (If you’ve ever gotten lost in the matrix inversions of the linear Gaussian model, these notes are for you!)

• Chapter 3 & 4 (Linear Models): The explicit path from maximum likelihood to regularized least squares, and the exact geometry behind the evidence framework (empirical Bayes).

• Chapter 9 (Mixture Models and EM): A rigorous, step-by-step breakdown of the decomposition of the log marginal likelihood, proving exactly why the E-step and M-step are guaranteed to maximize the lower bound.

• Chapter 10 (Approximate Inference): Fleshing out the coordinate ascent equations for Variational Inference, transforming dense notation into a clear algorithmic flow.

Access the Resource

Learning machine learning shouldn’t feel like deciphering hieroglyphics. I hope these annotations serve as a helpful companion piece, whether you are a student encountering these concepts for the first time, a researcher looking to brush up on the fundamentals, or a fellow educator looking for teaching materials.

You can download the annotated chapters directly below:

• Download the Annotated PRML PDF

I plan to continually update and expand these notes. If you find a typo, an error in a derivation, or a section that could use further clarification, please feel free to reach out. Happy modeling!