Home / Learn / Data & AI / ML foundations / Lesson 2.2
All lessons

Module 2: Looking at data before modelling · Lesson 2.2 · about 20 min

Summaries and charts that tell the truth

Mean or median, histogram or box plot, and how a cut-off axis can turn a 4 % gap into a landslide. Drag an outlier and move an axis to see how summaries and charts mislead, and how to make honest ones.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 2 · about 20 minutes · read first: 1.6. You’ll be able to choose between the mean and median, read a histogram and a box plot, and spot four common ways charts mislead.

Look before you model, but look honestly: pick summaries that a few wild values cannot drag around, and draw charts whose shapes match the numbers.

1. Why this matters

Before any model, you look. A mechanic listens to the engine before taking it apart; an analyst draws the data before modelling it. Looking catches broken data, shows which features might matter, and hints at which model shapes make sense. But summaries and charts can also hide things or exaggerate them, sometimes by accident. A good analyst knows the tricks so as not to use them, and to catch them in other people’s reports.

2. Mean or median?

Try it: drag the biggest pay to the right
mean (average)
median (middle value)
people earning less than the mean
The mean adds everything up and shares it out equally, so one huge value drags it along. The median is the middle person once everyone is lined up, so it ignores how extreme the ends are. For pay, house prices and anything with a long tail, the median describes a typical person better. Made-up pay.
  • Use the median for skewed data with a long tail: pay, house prices, time on a website, hospital stays.
  • Use the mean when values are roughly symmetric, or when totals matter (total revenue = mean × number of sales).
  • Report both when they differ a lot. The gap itself tells you the data is skewed.

The same goes for spread: the standard deviation is pulled by outliers too. The interquartile range (IQR), the distance from the first quartile Q1 (25 % of values below) to the third Q3 (75 % below), is the robust partner of the median.

Here is how a box plot is built from eight waiting times, in minutes:

Sort. 4, 6, 7, 9, 11, 12, 15, 41. The median is the middle: (9 + 11) ÷ 2 = 10 minutes.
Quartiles. The lower half 4, 6, 7, 9 has middle Q1 = 6.5. The upper half 11, 12, 15, 41 has middle Q3 = 13.5.
The box. It runs from 6.5 to 13.5, with a line at 10. Its width, the IQR, is 7 minutes: the middle half of all callers waited within this range.
The fences. 1.5 × IQR = 10.5. Upper fence 13.5 + 10.5 = 24; lower fence 6.5 − 10.5 = −4.
The outlier. 41 is beyond 24, so it is drawn as a dot. Check it: a real long wait, or a call left on hold by mistake? The mean (13.1) is pulled up by it; the median (10) is not.

3. Four charts worth knowing

Chart Shows Use it for
Histogram How one number column is spread: bars count values in equal bins Shape: symmetric, skewed, two peaks, odd spikes
Box plot Median, quartiles, whiskers and outliers Comparing a number across groups
Scatter plot Two number columns, one dot per row Relationships, clusters, outliers (Lesson 2.3)
Bar chart A number per category Counts or totals by group

Histograms deserve a warning: the bin width changes the picture. Too wide and two peaks merge into one; too narrow and noise looks like structure. Try two or three widths.

4. How charts mislead

Try it: move where the axis starts

Store A sold 102 units and store B sold 98: a difference of about 4 %. Bars show size by their length, so a bar chart must start at zero. Cut the axis and B looks a fraction of A. (Line charts of things like temperature may start elsewhere, because there you read the position, not the length.)
  1. Cut-off bar axis. As above. Bars must start at zero.
  2. Stretched or squashed scale. The same line rises steeply in a tall narrow chart and gently in a wide one. Compare charts only on the same scale.
  3. Cherry-picked range. Showing sales from the dip in March to the peak in July makes “growth” out of a normal season. Show whole years.
  4. Area and 3-D tricks. A picture of a coin twice as tall is four times the area and looks four times bigger. 3-D pie charts make the front slice look larger. Use flat bars.

“A chart can’t lie: it just shows the numbers.” The numbers can be right while the picture is wrong. Bars of 50 and 55 on an axis starting at 45 are drawn 5 and 10 units long, so a 10 % gap looks like 100 %. Edward Tufte called the ratio between the size of an effect in the picture and in the data the “lie factor”; an honest chart keeps it close to 1.

Maths and principle

Principle: robust summaries (median, IQR) barely move when a few values go wild; the mean and standard deviation can move without limit.

Equations: the quartiles split the sorted data into four equal parts, and

IQR=Q3−Q1

and a value is an outlier if

x<Q1−1.5IQRorx>Q3+1.5IQR

This is Tukey’s rule, used to draw the dots beyond a box plot’s whiskers.

Breakdown point: the share of values you can make arbitrarily wrong before a summary breaks. For the mean it is 0 (one value can drag it anywhere); for the median it is 50 %.

Analogy: the median is the middle runner in a race. It doesn’t matter how far ahead the winner finishes.

Try it: the values 2, 3, 3, 4, 5, 6, 30 have Q1=3, Q3=5.5. Is 30 an outlier? (IQR = 2.5, upper fence 5.5 + 3.75 = 9.25, so yes.)

Median
The middle value once sorted: half the values are below, half above.
Q1, Q3
The values with a quarter and three quarters of the data below them.
IQR = Q3 − Q1
The width of the middle half of the data.
Q3 + 1.5 IQR
Tukey's upper fence: values beyond it are flagged for a closer look.
Physical meaning
Robust summaries describe the typical case, not the loudest one. When one billionaire walks into a café, the mean wealth of the customers jumps to millions, but the median customer is exactly as rich as before. The median and IQR tell you about the people actually in the room.

5. Worked example

Made-up numbers. A city report says “average commute: 52 minutes”. The underlying 1,000 survey answers have a median of 34 minutes. A histogram shows most people between 15 and 50 minutes, plus a group of about 60 people commuting over 3 hours. Those turn out to be people who typed hours and minutes the wrong way round (e.g. “130” for 1 h 30). Fixing them gives a mean of 37 minutes. Two lessons: report the median for skewed data, and always look at the histogram, because the outliers may be data errors rather than real.

6. Practice

  1. House prices in a town: median $420k, mean $610k. What does the gap tell you?
  2. A chart shows bars of 50 and 55 with the axis starting at 45. How many times longer does the second bar look?
  3. Find the median and IQR of 1, 3, 4, 7, 8, 9, 12, 20 (use the middle of each half for the quartiles).
Answers
  1. Prices are skewed to the right: a few very expensive homes pull the mean up.
  2. Lengths 5 and 10: it looks twice as long, though it is only 10 % bigger.
  3. Median (7 + 8)/2 = 7.5. Lower half 1, 3, 4, 7: Q1=3.5. Upper half 8, 9, 12, 20: Q3=10.5. IQR = 7.

Try it on your own data: try different bin widths in the histogram builder, compare groups in the box plot lab, or get every column’s summary at once in the CSV profiler.

New words: skewed, quartile, interquartile range, box plot, histogram, bin width, robust, breakdown point.

  • Use the median and IQR for skewed data; the mean and standard deviation for roughly symmetric data or when totals matter.
  • Tukey’s rule flags values beyond Q1 − 1.5 IQR or Q3 + 1.5 IQR; check whether each is an error or real.
  • Histograms show shape (try several bin widths); box plots compare groups; scatter plots show relationships.
  • Bars start at zero, compare charts on the same scale, show whole periods, and avoid area and 3-D tricks.

Check yourself

1Ten people earn about $4,000 a month and one earns $40,000. Which describes a typical person better?
2Why must a bar chart start at zero?
3What does the box in a box plot show?

Sources

  • Tukey, J. W. (1977), Exploratory Data Analysis, Addison-Wesley (box plots and the 1.5 IQR rule)
  • Tufte, E. R. (2001), The Visual Display of Quantitative Information, 2nd ed., Graphics Press (the lie factor and honest scales)
  • Huff, D. (1954), How to Lie with Statistics, W. W. Norton

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.