Module 2: Looking at data before modelling · Lesson 2.2 · about 20 min
Summaries and charts that tell the truth
Mean or median, histogram or box plot, and how a cut-off axis can turn a 4 % gap into a landslide. Drag an outlier and move an axis to see how summaries and charts mislead, and how to make honest ones.
Module 2 · about 20 minutes · read first: 1.6. You’ll be able to choose between the mean and median, read a histogram and a box plot, and spot four common ways charts mislead.
1. Why this matters
Before any model, you look. A mechanic listens to the engine before taking it apart; an analyst draws the data before modelling it. Looking catches broken data, shows which features might matter, and hints at which model shapes make sense. But summaries and charts can also hide things or exaggerate them, sometimes by accident. A good analyst knows the tricks so as not to use them, and to catch them in other people’s reports.
2. Mean or median?
- Use the median for skewed data with a long tail: pay, house prices, time on a website, hospital stays.
- Use the mean when values are roughly symmetric, or when totals matter (total revenue = mean × number of sales).
- Report both when they differ a lot. The gap itself tells you the data is skewed.
The same goes for spread: the standard deviation is pulled by outliers too. The interquartile range (IQR), the distance from the first quartile (25 % of values below) to the third (75 % below), is the robust partner of the median.
Here is how a box plot is built from eight waiting times, in minutes:
3. Four charts worth knowing
| Chart | Shows | Use it for |
|---|---|---|
| Histogram | How one number column is spread: bars count values in equal bins | Shape: symmetric, skewed, two peaks, odd spikes |
| Box plot | Median, quartiles, whiskers and outliers | Comparing a number across groups |
| Scatter plot | Two number columns, one dot per row | Relationships, clusters, outliers (Lesson 2.3) |
| Bar chart | A number per category | Counts or totals by group |
Histograms deserve a warning: the bin width changes the picture. Too wide and two peaks merge into one; too narrow and noise looks like structure. Try two or three widths.
4. How charts mislead
- Cut-off bar axis. As above. Bars must start at zero.
- Stretched or squashed scale. The same line rises steeply in a tall narrow chart and gently in a wide one. Compare charts only on the same scale.
- Cherry-picked range. Showing sales from the dip in March to the peak in July makes “growth” out of a normal season. Show whole years.
- Area and 3-D tricks. A picture of a coin twice as tall is four times the area and looks four times bigger. 3-D pie charts make the front slice look larger. Use flat bars.
“A chart can’t lie: it just shows the numbers.” The numbers can be right while the picture is wrong. Bars of 50 and 55 on an axis starting at 45 are drawn 5 and 10 units long, so a 10 % gap looks like 100 %. Edward Tufte called the ratio between the size of an effect in the picture and in the data the “lie factor”; an honest chart keeps it close to 1.
Maths and principle
Principle: robust summaries (median, IQR) barely move when a few values go wild; the mean and standard deviation can move without limit.
Equations: the quartiles split the sorted data into four equal parts, and
and a value is an outlier if
This is Tukey’s rule, used to draw the dots beyond a box plot’s whiskers.
Breakdown point: the share of values you can make arbitrarily wrong before a summary breaks. For the mean it is 0 (one value can drag it anywhere); for the median it is 50 %.
Analogy: the median is the middle runner in a race. It doesn’t matter how far ahead the winner finishes.
Try it: the values 2, 3, 3, 4, 5, 6, 30 have , . Is 30 an outlier? (IQR = 2.5, upper fence 5.5 + 3.75 = 9.25, so yes.)
- Median
- The middle value once sorted: half the values are below, half above.
- Q1, Q3
- The values with a quarter and three quarters of the data below them.
- IQR = Q3 − Q1
- The width of the middle half of the data.
- Q3 + 1.5 IQR
- Tukey's upper fence: values beyond it are flagged for a closer look.
- Physical meaning
- Robust summaries describe the typical case, not the loudest one. When one billionaire walks into a café, the mean wealth of the customers jumps to millions, but the median customer is exactly as rich as before. The median and IQR tell you about the people actually in the room.
5. Worked example
Made-up numbers. A city report says “average commute: 52 minutes”. The underlying 1,000 survey answers have a median of 34 minutes. A histogram shows most people between 15 and 50 minutes, plus a group of about 60 people commuting over 3 hours. Those turn out to be people who typed hours and minutes the wrong way round (e.g. “130” for 1 h 30). Fixing them gives a mean of 37 minutes. Two lessons: report the median for skewed data, and always look at the histogram, because the outliers may be data errors rather than real.
6. Practice
- House prices in a town: median $420k, mean $610k. What does the gap tell you?
- A chart shows bars of 50 and 55 with the axis starting at 45. How many times longer does the second bar look?
- Find the median and IQR of 1, 3, 4, 7, 8, 9, 12, 20 (use the middle of each half for the quartiles).
Answers
- Prices are skewed to the right: a few very expensive homes pull the mean up.
- Lengths 5 and 10: it looks twice as long, though it is only 10 % bigger.
- Median (7 + 8)/2 = 7.5. Lower half 1, 3, 4, 7: . Upper half 8, 9, 12, 20: . IQR = 7.
Try it on your own data: try different bin widths in the histogram builder, compare groups in the box plot lab, or get every column’s summary at once in the CSV profiler.
New words: skewed, quartile, interquartile range, box plot, histogram, bin width, robust, breakdown point.
- Use the median and IQR for skewed data; the mean and standard deviation for roughly symmetric data or when totals matter.
- Tukey’s rule flags values beyond Q1 − 1.5 IQR or Q3 + 1.5 IQR; check whether each is an error or real.
- Histograms show shape (try several bin widths); box plots compare groups; scatter plots show relationships.
- Bars start at zero, compare charts on the same scale, show whole periods, and avoid area and 3-D tricks.
Check yourself
Sources
- Tukey, J. W. (1977), Exploratory Data Analysis, Addison-Wesley (box plots and the 1.5 IQR rule)
- Tufte, E. R. (2001), The Visual Display of Quantitative Information, 2nd ed., Graphics Press (the lie factor and honest scales)
- Huff, D. (1954), How to Lie with Statistics, W. W. Norton
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.