Home / Learn / Data & AI / ML foundations / Lesson 6.4
All lessons

Module 6: Ensembles: random forests and boosting · Lesson 6.4 · about 25 min

Feature importance, and what it does not tell you

Forests report how much each column helped. Learn the two common measures, impurity and permutation importance, and their traps: noise columns that score, copies that split the credit, and importance that is not cause.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 6 · about 25 minutes · read first: 2.3, 6.3. You’ll be able to read a feature-importance chart, compute permutation importance by hand, and list the traps before you quote one.

Feature importance tells you which columns the model leaned on, not which ones change the real world, and the cheap version can be fooled by noise.

1. Why this matters

“Which factors drive churn?” is often the real question behind a model, and the importance chart is the first thing managers ask for. It is useful, and easy to misread. Knowing the two measures and their traps keeps you from telling a board that the customer ID drives sales.

2. Impurity importance

Every split in every tree lowers impurity by some amount (Lesson 5.3). Add up those drops for each column, over all splits and all trees, and divide by the grand total. Each column gets a share, and the shares add to 100 %. It comes free (worked out while training) and is what most libraries show by default.

Try it: what does the forest lean on?

Made-up data: price depends on floor area most, then distance to the centre, a little on age. The forest's importance is each column's share of all the impurity it removed. Add the house number: pure noise, yet it gets a sliver, because a deep tree can always find some split that helps on training data. Add area in square feet, which says the same thing as area: the two share the credit, and area suddenly looks half as important.

Its traps, all visible in the simulation:

  • Noise gets a sliver. A column of random numbers still offers thresholds, and deep trees use some of them on training data. Columns with many different values (IDs, timestamps) get the most of this undeserved credit.
  • Copies split the credit. Two columns that carry the same information share its importance, so each looks half as important.
  • It is measured on training data, so it reflects what the model memorised as well as what carries over to new data.

3. Permutation importance

A more honest measure, for any model, measured on data the model has not seen:

  1. Score the trained model on validation data.
  2. Shuffle one column’s values (which breaks its link with the target), keep everything else, and score again.
  3. The drop in score is that column’s importance.

Score the model. A churn forest gets 0.880 accuracy on validation data it never trained on.

Shuffle one column. Mix up "contract type" between customers, like dealing the values out again at random. The column still looks normal, but it no longer matches anyone.

Score again. Accuracy falls to 0.792. The drop, 0.088, is contract type's permutation importance.

Repeat for each column. Shuffling "customer ID" gives 0.879: a drop of 0.001. The model was not really using it on new data.

Average over a few shuffles to smooth out luck, then rank the columns by their drop.

S
The model's score (say accuracy) on validation data.
S(j)
The score after column j's values are shuffled.
PI(j) = S − S(j)
Permutation importance: how much worse the model does without the real information in column j.
Physical meaning
Finding out which players a team relies on by benching them one at a time and watching the score. A player whose absence costs nothing wasn't carrying the team, however busy they looked in training.

A column the model really relies on causes a big drop; a noise column, almost none. Permutation importance still shares the copies problem (shuffle one copy and the model uses the other), so drop or group near-duplicate columns first.

4. Importance is not cause

Importance says what the model used, not what changes the outcome. Ice-cream sales are an important predictor of drownings (both rise in summer, Lesson 2.3); banning ice cream saves nobody. To ask “what if we changed X?”, you need an experiment or a causal analysis.

“The top feature is the lever to pull.” A model can lean heavily on a column that changes nothing. If big discounts are given to customers who were already about to leave, “discount size” will predict churn well, and giving everyone bigger discounts may do nothing, or make churn look worse. Importance points at where to look; an experiment tells you what works.

Maths and principle

Principle: credit a column with the improvement it brings, either inside the trees (impurity) or on held-out data (permutation).

Impurity importance of column j over a forest of B trees, where each split s on column j has ns rows and gain ΔIs:

Imp(j)=1B∑b=1B∑s∈Tb,s splits on jnsnΔIs

then scaled so all columns add to 1.

Permutation importance with score S (say accuracy) on validation data, and S(j) the score after shuffling column j:

PI(j)=S−S(j)

usually averaged over several shuffles.

Try it: a model scores 0.86 on validation. Shuffling “age” gives 0.84, shuffling “income” 0.71, shuffling “customer ID” 0.86. Rank them. (Income 0.15, age 0.02, ID 0.)

5. Worked example

Made-up numbers. A churn forest on validation data (accuracy 0.880):

column impurity importance accuracy after shuffling permutation importance
contract type 21 % 0.792 0.088
monthly charge 30 % 0.851 0.029
tenure (months) 26 % 0.836 0.044
customer ID 23 % 0.879 0.001

Impurity importance ranks monthly charge first and gives the customer ID nearly a quarter of the credit: it has thousands of different values, so the trees found plenty of lucky splits in it. Permutation importance, measured on new data, puts contract type first and shows the ID is worthless. Drop the ID and retrain.

6. Practice

  1. In the simulation, how much importance does the house number get, and why is it not zero?
  2. Why might “area in square metres” look less important once “area in square feet” is added?
  3. A manager reads “discount size is the top feature” as “bigger discounts reduce churn”. What would you say?
Answers
  1. A few per cent. Deep trees find some splits on it that happen to help on training data.
  2. The two carry the same information, so the trees split their use (and the credit) between them.
  3. Importance shows the model relies on it, not that it causes anything. Maybe big discounts go to customers who were already about to leave. Test it with an experiment.
  • Impurity importance adds up each column’s impurity drops inside the trees; it is free but measured on training data.
  • It over-credits noisy, many-valued columns such as IDs, and splits credit between near-copies.
  • Permutation importance shuffles one column on validation data and measures the drop in score.
  • Importance describes the model, not cause and effect; use an experiment to learn what a change would do.

Try it on your own data: the small model trainer trains models on your CSV; shuffle one column in a copy of your file and see how much the test score drops, which is permutation importance by hand.

New words: feature importance, impurity importance, permutation importance, high-cardinality column, correlated features.

Check yourself

1Impurity-based importance of a column is:
2Two columns are near copies of each other. Their importances
3"Square footage has the highest importance" means:

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.