All lessons

Module 2: Probability for decisions · Lesson 2.4 · about 15 min

Counting: combinations and permutations

How many ways can things happen? Factorials, permutations (order matters) and combinations (order does not), with analyst uses: how many test variants, how many pairs to compare, and the binomial coefficient behind the next module.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 2 · about 15 minutes · school maths. You’ll be able to count outcomes with the multiplication rule, factorials, permutations and combinations, and see why the number of tests or variants grows so fast.

Choices multiply: a few options each give a huge number of combinations, which is why testing everything at once rarely works.

In everyday terms: 3 shirts and 4 pairs of trousers make 3 × 4 = 12 outfits. Add 5 pairs of shoes and it is 60. Every new option multiplies, it does not add.

1. Why analysts count

Counting tells you how big a problem is before you start: how many versions a test needs, how many pairs of groups you might compare, how many ways k of n customers can respond. It also supplies the “n choose k” in the binomial distribution (Lesson 3.2).

2. The multiplication rule

If one choice has a options and a second, independent choice has b, together there are a×b options. Three subject lines × two images × four send times = 24 email versions. This is why “test everything at once” quickly needs enormous traffic.

3. Factorials and permutations

The number of ways to put n different things in order is n! (“n factorial”): n×(n−1)×⋯×1. Five products can be lined up on a shelf in 5! = 120 ways.

If you only fill k ordered slots from n things, that’s a permutation:

nPk=n!(n−k)!=n×(n−1)×⋯×(n−k+1)

4. Combinations

When order does not matter (a team, a set of pairs, which customers said yes), divide out the k! orderings of each group. That is a combination, “n choose k”:

(nk)=n!k!(n−k)!
  • Pairs among m groups: (m2)=m(m−1)/2. With 10 regions, 45 pairs.
  • Which 3 of 10 customers responded: (103)=120 possible sets.
Multiplication rule
k choices of one thing and m of another make k × m combinations.
n! (factorial)
n × (n − 1) × ... × 1: the number of ways to put n things in order.
Permutations nPr
Ordered picks of r from n: n! ÷ (n − r)!. Order matters (1st, 2nd, 3rd prize).
Combinations nCr
Unordered picks: n! ÷ (r! (n − r)!). Order does not matter (a team of 3).
Physical meaning
Counting tells you how fast possibilities explode. Ten on/off features make 1,024 versions of a page; comparing every pair of 10 versions makes 45 comparisons, and each comparison is another chance of a lucky false "win".

“Combinations and permutations are the same count.” Choosing 3 of 10 products for a shelf (a combination) gives 120 ways; choosing which 3 go 1st, 2nd and 3rd on it (a permutation) gives 720, six times as many, because each group of 3 can be ordered 3! = 6 ways.

Watch it happen, one step at a time:

The plan. Test 4 new checkout features, each on or off.
Multiply. 2 × 2 × 2 × 2 = 16 versions of the page.
Share the traffic. 1,600 buyers a week split 16 ways is 100 buyers per version: far too few to see small effects.
Simplify. Test each feature against the current page instead: 5 arms in one test.
Count the comparisons. Every pair of 5 arms: 5C2 = 10 comparisons. More comparisons, more chances of a false win (Lesson 6.8).
Maths and principle

Principle: count ordered arrangements first, then divide by the number of orderings you do not care about.

Equations: n!; nPk=n!/(n−k)!; (nk)=n!/(k!(n−k)!). Note (nk)=(nn−k): choosing 3 to include is the same as choosing 7 to leave out.

In code: Excel COMBIN(10,3) and PERMUT(5,3); Python math.comb(10, 3) and math.perm(5, 3).

Analogy: a pizza with 3 toppings from 10: pepperoni-mushroom-olive is the same pizza in any order (a combination). A podium of 3 runners from 10: gold-silver-bronze order matters (a permutation).

5. Worked example

Made-up numbers. A product manager wants to test 4 new checkout features. Each can be on or off.

  • Full combination: 2 × 2 × 2 × 2 = 16 versions.
  • With 1,600 buyers a week split 16 ways, each version gets 100 buyers a week: far too few to see small effects (Lesson 7.3).
  • Testing each feature on its own against the current page: 4 tests, or one test with 5 arms.
  • Comparing every pair among those 5 arms afterwards: (52)=10 comparisons, which raises the chance of a false “win” (Lesson 6.8).

Counting first turned an impossible plan into a workable one.

6. Practice

  1. How many ways can 4 managers be ranked 1st to 4th?
  2. How many ways can you choose 2 stores to visit from 12?
  3. A menu test has 3 prices × 2 layouts × 2 photos. How many versions?
Answers
  1. 4! = 24.
  2. (122) = 12 × 11 ÷ 2 = 66.

Try it on your own data: the combinatorics calculator works out factorials, permutations and combinations for any numbers, and the counting machine lists every arrangement for small cases.

New words: multiplication rule, factorial, permutation, combination, n choose k.

  • Independent choices multiply: k × m × …
  • n! counts the orders of n things.
  • Permutations count ordered picks; combinations count unordered ones and are smaller by r!.
  • nC2 = n(n − 1)/2 is the number of pairs: it grows fast and inflates false positives.

Check yourself

1You want to compare every pair of 6 store formats. How many comparisons is that?
2How many ways can 3 banners fill the top, middle and bottom slots, chosen from 5 designs?
3An email test has 3 subject lines, 2 images and 4 send times. How many versions is a full test?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.