Home / Learn / Data & AI / ML foundations / Lesson 11.1
All lessons

Module 11: Capstone and next steps · Lesson 11.1 · about 35 min

Capstone: predict shop sales from a CSV

The whole course in one project: frame the question, clean and split the data, set a baseline, fit linear regression, a tree and a random forest, compare them fairly and explain the winner. Run it live and change the data.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 11 · about 35 minutes · read first: 2.5, 3.5, 6.3, 7.2. You’ll be able to run a small prediction project from question to explanation, compare models fairly on held-out data, and report the result in plain words.

A good project is not the fanciest model: it is a clear question, an honest test on days the model never saw, a baseline to beat, and an answer someone can act on.

1. Why this matters

Real projects are rarely about one model. They are about asking the right question, keeping the test honest, beating a simple guess and explaining the answer to someone who will act on it.

This lesson puts the whole course together on one realistic job. A shop owner wants to plan staff and stock from a forecast of tomorrow’s sales.

2. The project, step by step

  1. Frame it. Predict one day’s sales (a number, so regression) from what is known the day before: the weather forecast, whether it is a weekend, whether a promotion runs. Judge it by mean absolute error (MAE), the average miss: “we miss by about 30 sales a day” is something an owner understands.
  2. Look and clean (Lessons 2.2 and 2.4). Check each column’s range, missing values and outliers. Plot sales against temperature: here, sales rise with warmth, then drop on very hot days.
  3. Split first (Lesson 2.5). Hold back the last 100 days as a test set, and don’t touch them until the end. For dates, keep the test days after the training days, as they will be in real use.
  4. Baseline. Predict the average of the training days for every day. This is the “do nothing clever” guess every model must beat.
  5. Fit three models: linear regression (Lesson 3.4), a decision tree of depth 4 (Lesson 5.4), a random forest (Lesson 6.3). Tune settings with cross-validation on the training days only (Lesson 7.2).
  6. Compare on the test days, once.
  7. Explain with the linear weights and the forest’s feature importance (Lesson 6.4), and say how sure you are.
Try it: four models, one fair test
best model
linear regression says a promotion adds
Made-up data: 400 days, of which the last 100 are never used for training. Error is the mean absolute error in sales on those 100 days: the typical miss, in the same units as sales. No model can beat the noise, so even a perfect one misses by about 0.8 × the noise. The average of the training days is the baseline every model must beat. Linear regression cannot bend, so it misses the drop in sales on very hot days and the extra boost of a promotion at a weekend; trees can, and a forest of them is steadier than one tree. "day of month" does nothing, and its importance stays near zero.

Everyday picture: a driving test. You practise on any roads you like, but the examiner picks a route you have never driven. Passing on your practice street proves nothing.

Watch the scoring step on five test days, with a forecast and the baseline side by side:

The test days. Actual sales on 5 days the model never saw: 320, 410, 290, 450, 380. The training-day average, our baseline guess, is 340 every day.

The forecast. The model predicted 300, 430, 300, 420, 360.

The misses. Actual minus forecast, sign dropped: 20, 20, 10, 30, 20. Total 100, so the model's MAE = 100 ÷ 5 = 20 sales a day.

The baseline's misses. |320 − 340| = 20, then 70, 50, 110, 40. Total 290, so the baseline's MAE = 58.

The verdict. The model cuts the error from 58 to 20: (58 − 20) ÷ 58 = 66 % less. Worth using, and easy to explain to the owner.

Maths and principle

Principle: the only fair score is on data the model and you never used to make choices.

Mean absolute error over the m test days:

MAE=1m∑i=1m|yi−yˆi|

The floor: if daily sales vary randomly around their true level by a normal spread σ, even a perfect model misses by

𝔼|ε|=σ2/π≈0.80σ

so with σ=30, no model can average much under 24.

Analogy: a cooking contest judged on a dish none of the cooks has tasted. Practising on the judges’ plate would win the contest and teach nothing.

Try it: a model’s errors on 4 test days are +10, −30, +20, −20. MAE? ((10+30+20+20)/4=20.)

yi
what really happened on test day i (actual sales)
ŷi
what the model forecast for that day ("y-hat")
|yi − ŷi|
the miss on that day, ignoring whether it was too high or too low
m
how many test days there are
σ√(2/π) ≈ 0.8σ
the floor: the smallest average miss any model can reach when daily sales wobble randomly by σ
Physical meaning
MAE is the number of loaves you will bake too many or too few on a typical day. It turns model quality into stock and staff: an MAE of 28 means planning with a buffer of about 30 sales, and no model can push it below the random wobble of the days themselves.

3. Worked example

The simulation at its starting settings: 150 training days, daily noise ±30, 100 test days.

model test MAE
Average of training days 83.8
Linear regression 34.2
Decision tree (depth 4) 34.7
Random forest (40 trees) 28.3

Every model beats the baseline easily. The forest wins, close to the floor of about 24, because it can bend where the data bends: the drop above 26 °C, and the extra lift when a promotion falls on a weekend. Linear regression is not far behind and is the easiest to explain: each degree adds about 6 sales, a weekend about 141, a promotion about 84.

The report: “Tomorrow’s sales can be forecast to within about 28 a day, on days the model never saw. Weekends and temperature matter most; a promotion adds about 80 sales; the day of the month does not matter. Recommended: the forest for planning, with the linear model’s weights as the explanation.”

Now slide the training days down to 20. The forest’s error rises to about 51 and the single tree’s to about 87, worse than the baseline, while linear regression holds at about 36. With little data, simple wins.

“The model with the lowest error on the training days is the best.” Training error rewards memorising. A deep tree can score almost 0 on the days it learned from and still miss badly tomorrow. Only the error on days held back from every choice you made tells you how the model will do in real use.

4. Practice

  1. In the simulation, raise the noise to 80. What happens to every model’s error, and to the gap between them?
  2. With 20 training days, why might “day of month” look important to the forest?
  3. Why should the test days come after the training days for a sales forecast?
Answers
  1. All errors rise, to between about 63 and 70 for the three models (the floor is now about 64), and they bunch together: noise hides the patterns the flexible models could use.
  2. With few rows, chance patterns look real, and trees split on them. More data, or permutation importance on held-out rows, shows it is noise.
  3. In real use the model always predicts the future from the past. Mixing dates lets it learn from days after the ones it predicts: a form of leakage (Lesson 7.4).

Try it on your own data: load your own CSV into the CSV profiler for step 2, the regression workbench to fit a linear model to a number column, and the small model trainer when the answer is a category.

New words: baseline, mean absolute error, held-out test set, time-based split, model comparison, irreducible error.

  • Frame the question first: what to predict, from what is known in advance, judged by an error the owner understands (MAE).
  • Split before you look closely, and for dates keep the test days after the training days.
  • Always beat a baseline, such as the training average; a low error means little until you compare.
  • Compare models once, on the held-out days; with little data, the simple model often wins.
  • Report in plain words: how far off, what drives the answer, and what to do with it.

Check yourself

1Why fit "the average of the training days" as a model?
2With only 20 training days, linear regression beat the random forest. Why?
3The forest ranks "day of month" at 24 % importance with 20 rows, and 4 % with 300. What does that suggest?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.