All lessons
Machine learning cheat sheet
The key formulas of the course on one page, each with what it means in one line and the lesson that explains it. Print it or keep it open while you work.
Foundations
Straight line Lesson 1.3
w is the slope (change in y per unit of x), b the value at x = 0.
Dot product Lesson 1.4
A weighted sum. Large when the vectors point the same way.
Cosine similarity Lesson 1.4
1 same direction, 0 unrelated, −1 opposite.
Gradient descent step Lesson 3.3
Step every weight a little downhill; η is the learning rate.
Chain rule Lesson 1.5
The slope through a chain of steps is the product of the steps' slopes.
Mean and variance Lesson 1.6
The centre, and the average squared distance from it.
z-score Lesson 2.4
How many standard deviations from the mean. Also how columns are scaled.
Correlation Lesson 2.3
From −1 to 1: how closely two columns follow a straight line.
Regression
Linear model Lesson 3.1
A weighted sum of the inputs plus a bias.
Mean squared error Lesson 3.2
The loss least squares makes as small as possible.
Least-squares line Lesson 3.2
The best straight line, in one step. It passes through the point of means.
Normal equation Lesson 3.4
Least squares with many inputs at once.
MAE and RMSE Lesson 3.5
Typical miss, in the answer's units. RMSE punishes big misses more.
R² Lesson 3.5
Share of the variance explained: 1 perfect, 0 no better than the mean.
Ridge and lasso Lesson 3.8
A penalty on large weights; lasso can set some to exactly 0.
Classification
Sigmoid Lesson 4.2
Squashes any number into a probability between 0 and 1.
Log-odds Lesson 4.2
Logistic regression is a linear model of the log-odds.
Log-loss Lesson 4.3
Punishes confident wrong answers hardest.
Precision and recall Lesson 4.5
Of the flagged, how many were right; of the real cases, how many were caught.
F1 Lesson 4.5
High only when both are high.
Bayes' rule Lesson 4.8
Turns the chance of the evidence into the chance of the cause.
Euclidean distance Lesson 4.7
Straight-line distance, used by k-NN and k-means. Scale the columns first.
Trees, forests and judging models
Gini impurity Lesson 5.2
0 for a pure group; 0.5 for a 50/50 split of two classes.
Entropy Lesson 5.2
Mixture in bits: 0 pure, 1 for a 50/50 split.
Information gain Lesson 5.3
How much a split cuts impurity, weighted by size.
Bootstrap coverage Lesson 6.2
Share of rows that appear in a bootstrap sample.
Gradient boosting Lesson 6.6
Each new tree h fits the residuals of the model so far.
Bias-variance Lesson 7.1
Too simple, too jumpy, and the noise no model can remove.
Standard error Lesson 1.6
How much a mean (such as a cross-validated score) would vary.
Patterns without labels
k-means objective Lesson 8.1
Total squared distance to each point's own centre.
Silhouette Lesson 8.2
a: mean distance within the cluster; b: to the nearest other cluster.
PCA Lesson 8.4
Components are eigenvectors of the covariance matrix; eigenvalues give the variance kept.
Neural networks
A layer Lesson 9.3
A linear map, then a bend. Stack them to make a network.
ReLU Lesson 9.2
The default activation: slope 1 for positive inputs, so gradients do not fade.
Softmax Lesson 9.3
Scores to probabilities that add up to 1.
Backpropagation Lesson 9.4
Each layer's error from the next layer's; a weight's gradient is error × input.
Convolution output size Lesson 9.6
An n × n image with a k × k filter, no padding, stride 1.
Recurrent cell Lesson 9.7
A running memory updated with each new input.
From words to an LLM
TF-IDF Lesson 10.1
Frequent here, rare elsewhere: the words that matter.
Next-word probability Lesson 10.3
A sentence's probability is the product of each word's.
Temperature Lesson 10.3
Below 1 sharpens the odds; above 1 flattens them.
Cross-entropy and perplexity Lesson 10.3
The loss LLMs are trained on, and how many words it is choosing between.
Attention Lesson 10.4
Each token mixes in the values of the tokens whose keys match its query.
Transformer block Lesson 10.5
Attention and a feed-forward network, each with a shortcut and normalisation.
Block size Lesson 10.5
With d numbers per token: 4d² in attention, 8d² in the feed-forward part.
Training compute Lesson 10.6
Arithmetic for N parameters trained on D tokens.