Module 4: Samples · Lesson 4.4 · about 20 min
Standard error: how much a sample result wobbles
The standard error measures how far a statistic typically lands from the truth: σ/√n for a mean, √(p(1−p)/n) for a proportion, and the formula for a difference between two groups. SD versus SE, and the square-root law.
Module 4 · about 20 minutes · school maths. You’ll be able to compute standard errors for a mean, a proportion and a difference, tell SE from SD, and plan with the square-root law.
In everyday terms: measure your own height ten times with a tape and you get slightly different numbers. The SD is how much single measurements scatter; the SE is how much the average of your ten measurements would scatter if you did the whole thing again.
1. The size of the wobble
The standard error (SE) is the standard deviation of a statistic’s sampling distribution: how far, typically, the estimate lands from the truth. It is the number behind every margin of error, confidence interval and test in the rest of the course.
2. SD versus SE
| Standard deviation | Standard error | |
|---|---|---|
| Describes | Individual values | An estimate (a mean, a rate) |
| Question | How different are customers from each other? | How sure are we about the average? |
| With more data | Settles near σ | Shrinks towards 0 |
A common reporting error is to put ”± SE” on a chart where readers expect ”± SD”, or the other way round. Say which one you’re showing.
3. The formulas
Mean:
using the sample SD in place of the unknown .
Proportion:
Difference between two independent groups (A/B tests!): variances add, so
A difference is noisier than either group alone, about √2 times as noisy for equal groups. This is why A/B tests need more traffic than people expect.
4. The square-root law
Because of the , precision gets expensive: 4× the data halves the SE; 100× the data cuts it to a tenth. When planning, ask what precision you need before asking for more data. Going from a ±2-point to a ±1-point margin costs four times the sample.
- SD
- Spread of individual values (customers, orders).
- SE of a mean = s/√n
- Spread of the sample mean across repeated samples.
- SE of a share = √(p(1 − p)/n)
- Spread of a percentage such as a conversion rate.
- SE of a difference = √(SE₁² + SE₂²)
- Wobbles add in squares when you compare two groups.
- Physical meaning
- The SE is the "give or take" on every result you report. To halve it you need four times as much data, which is why precision gets expensive.
“The SE of a difference is SE₁ + SE₂.” Two independent wobbles partly cancel, so they add in squares: √(0.28² + 0.30²) ≈ 0.41 points, not 0.58.
Watch it happen, one step at a time:
Maths and principle
Principle: the SE of an average equals the spread of one value divided by the square root of how many values were averaged.
Equations: ; ; for independent groups .
In code: Excel =STDEV.S(range)/SQRT(COUNT(range)); pandas df["x"].sem(); SQL STDDEV_SAMP(x) / SQRT(COUNT(x)).
5. Worked example
Made-up numbers. A test of a new product page:
| Visitors | Buyers | Rate | |
|---|---|---|---|
| A (old) | 5,000 | 200 | 4.0 % |
| B (new) | 5,000 | 240 | 4.8 % |
- SE(A) = √(0.04 × 0.96 ÷ 5,000) ≈ 0.00277 (0.28 points).
- SE(B) = √(0.048 × 0.952 ÷ 5,000) ≈ 0.00302 (0.30 points).
- SE(difference) = √(0.00277² + 0.00302²) ≈ 0.0041 (0.41 points).
- Observed difference 0.8 points is about 0.8 ÷ 0.41 ≈ 1.95 SEs. Borderline: right at the usual line for “clearly more than noise”. Module 7 takes this further.
6. Practice
- 25 values with s = 10. What is the SE of the mean?
- A rate of 20 % from 1,600 people. What is the SE?
- Group A: mean 50, s = 12, n = 36. Group B: mean 55, s = 16, n = 64. What is the SE of the difference?
Answers
- 10 ÷ 5 = 2.
- √(0.2 × 0.8 ÷ 1,600) = √0.0001 = 0.01, one percentage point.
- √(144/36 + 256/64) = √(4 + 4) = √8 ≈ 2.83. The difference of 5 is about 1.8 SEs.
Try it on your own data: the hypothesis tests tool shows the standard error behind each test it runs; the sample size calculator runs the square-root law backwards.
New words: standard error, SE of a proportion, SE of a difference, square-root law.
- SD describes individuals; SE describes a statistic.
- SE of a mean = s/√n; of a share = √(p(1 − p)/n).
- For a difference of independent groups, add SEs in squares.
- Square-root law: four times the data halves the SE.
Check yourself
Sources
- OpenIntro Statistics, 4th edition (Diez, Cetinkaya-Rundel, Barr), Sections 5.1 and 6.2 (difference of two proportions) https://www.openintro.org/book/os/
- NIST/SEMATECH e-Handbook, 7.2 Comparisons based on data from one process https://www.itl.nist.gov/div898/handbook/prc/section2/prc2.htm
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.