Module 1: Data, models and the maths you need · Lesson 1.6 · about 25 min
Probability, mean, variance and the normal curve
The few ideas from statistics that machine learning uses all the time: probability, the mean, variance and standard deviation, and the bell-shaped normal curve. Draw samples and watch the averages settle.
Module 1 · about 25 minutes · school maths. You’ll be able to compute a mean, variance and standard deviation, read a normal curve, and say why bigger samples give steadier answers.
1. Why this matters
Data is noisy. The same student doesn’t get the same mark twice, and two flats of the same size sell for different prices. Models cope with this wobble by working with probabilities (how likely) and averages (what is typical). Logistic regression outputs a probability (Module 4), naive Bayes multiplies probabilities (Lesson 4.8), regression errors are measured with squared gaps like variance (Lesson 3.2), and a language model’s output is a probability for every possible next word (Lesson 10.3).
2. Probability
A probability is a number from 0 (impossible) to 1 (certain) that says how likely something is. A fair coin lands heads with probability 0.5: in the long run, about half of all tosses. A weather app’s “30 % chance of rain” is 0.3. Some rules:
- The probabilities of all possible outcomes add up to 1.
- .
- For two things that don’t affect each other (independent), . Two heads in a row: 0.5 × 0.5 = 0.25.
- Conditional probability is the chance of A once you know B happened: . “The chance an email is spam, given it contains the word prize” is a conditional probability. Lesson 4.8 builds a spam filter from these.
3. The centre: mean and median
The mean (average) of values is their sum divided by :
The median is the middle value once they are sorted. Lesson 2.2 shows when each one is the better summary.
4. The spread: variance and standard deviation
Think of two bus routes that both take 30 minutes on average. On one, the bus is always within a minute or two. On the other, it can take 15 minutes or 50. The average is the same; the spread is very different, and it is the spread that makes you late.
Two classes can both average 60 % while one has every mark between 55 and 65 and the other runs from 20 to 100. The variance measures spread as the average squared distance from the mean:
Squaring makes every gap positive and counts big gaps much more than small ones. Because squaring also squares the units (marks²), we take the square root to get the standard deviation , back in the original units: “a typical distance from the mean”.
Here it is step by step, with five delivery times:
When the values are a sample used to estimate a bigger population, divide by instead of (this is written ). It corrects a small bias: a sample’s own mean sits closer to its values than the true mean does. For large the difference is tiny.
5. The normal curve
Many measurements pile up in a symmetric bell shape: heights, measurement errors, marks in a large exam. This shape is the normal distribution, set by its mean (the centre) and standard deviation (the width):
The curve is a density: the area under it between two values is the probability of landing between them, and the total area is 1. The useful rule of thumb is 68-95-99.7: about 68 % of values lie within 1σ of the mean, 95 % within 2σ and 99.7 % within 3σ.
Draw 1 value at a time, then 1,000. Small samples look nothing like the curve, and their mean can be well off. Large samples match it closely. This is the law of large numbers, and it is why models need plenty of data.
“After five heads in a row, tails is due.” No. A fair coin has no memory: the next toss is still 50-50. The law of large numbers does not “even things out” by pushing back against a streak. It works because a streak of five becomes a tiny part of the total once you have thousands of tosses. This mistake has a name, the gambler’s fallacy.
Maths and principle
Principle: the mean measures the centre, the variance measures the spread, and the average of many random draws settles near the true mean.
Equations: ; ; the sample mean’s own typical error (its standard error) is
Analogy: asking one person the time gives an answer a few minutes off. Ask 100 people and average: the individual errors mostly cancel.
Try it: with , what is the standard error of the mean for 25 values? For 2,500? (; . A hundred times the data, a tenth of the wobble.)
- x̄ (mean)
- The balance point of the values: add them up and divide by how many there are.
- σ² (variance)
- The average squared distance from the mean. Big gaps count much more than small ones.
- σ (standard deviation)
- The square root of the variance, back in the original units: a typical distance from the mean.
- σ / √n (standard error)
- How much an average of n values wobbles from sample to sample.
- Physical meaning
- The mean tells you where things usually land; the standard deviation tells you how far off they can land. A delivery firm promising "26 minutes" lives or dies by the 5-minute standard deviation, and a pollster asking 1,000 people instead of 10 gets an answer ten times steadier.
6. Worked example
Made-up numbers. Five deliveries took 20, 24, 25, 27 and 34 minutes.
- Mean: (20 + 24 + 25 + 27 + 34) ÷ 5 = 130 ÷ 5 = 26 minutes.
- Gaps from the mean: −6, −2, −1, 1, 8. Squared: 36, 4, 1, 1, 64. Sum: 106.
- Population variance: 106 ÷ 5 = 21.2; standard deviation √21.2 ≈ 4.6 minutes.
- Sample standard deviation (dividing by 4): √(106 ÷ 4) = √26.5 ≈ 5.1 minutes.
If delivery times are roughly normal with mean 26 and σ ≈ 5, about 95 % of deliveries take between 16 and 36 minutes (26 ± 2 × 5). A firm promising “within 35 minutes” would miss a few percent of the time.
7. Practice
- Find the mean and population standard deviation of 3, 7, 7, 11.
- Test marks are normal with mean 65 and σ = 8. About what share of students score above 81?
- A spam filter says . What is ?
Answers
- Mean 7. Squared gaps 16, 0, 0, 16: variance 32 ÷ 4 = 8, σ = √8 ≈ 2.83.
- 81 is 2σ above the mean. 95 % lie within ±2σ, so 5 % lie outside, half of them above: about 2.5 %.
- 0.7.
Try it on your own data: the probability distribution explorer draws normal and other curves and shades any probability you ask for.
New words: probability, independent, conditional probability, mean, median, variance, standard deviation, normal distribution, density, standard error, law of large numbers.
- A probability runs from 0 (impossible) to 1 (certain); all outcomes add to 1.
- The mean is the centre; the variance is the average squared gap from it; the standard deviation is its square root, in the original units.
- For a normal curve, about 68 %, 95 % and 99.7 % of values lie within 1, 2 and 3 standard deviations of the mean.
- An average of n values wobbles like σ ÷ √n, so more data gives steadier answers.
Check yourself
Sources
- James, G., Witten, D., Hastie, T., Tibshirani, R. and Taylor, J. (2023), An Introduction to Statistical Learning with Applications in Python, Springer, Chapters 2 and 3 https://www.statlearning.com/
- Goodfellow, I., Bengio, Y. and Courville, A. (2016), Deep Learning, MIT Press, Chapter 3 (probability and information theory) https://www.deeplearningbook.org/
- Freedman, D., Pisani, R. and Purves, R. (2007), Statistics, 4th ed., W. W. Norton, Chapters 4, 5 and 16 to 17 (average, standard deviation, normal curve, law of averages)
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.