Home / Learn / Data & AI / ML foundations / Lesson 4.2
All lessons

Module 4: Classification and logistic regression · Lesson 4.2 · about 25 min

The sigmoid and odds: from a line to a probability

Logistic regression keeps the straight-line score of a linear model and squeezes it through the S-shaped sigmoid into a probability between 0 and 1. Shape the curve with two sliders and learn to read weights as changes in odds.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 4 · about 25 minutes · read first: 1.5, 3.1, 4.1. You’ll be able to compute the sigmoid, convert between probability, odds and log-odds, and interpret a logistic regression weight.

Logistic regression adds up evidence as a score, then squeezes that score through an S-shaped curve so it always lands between 0 and 1.

1. Why this matters

Logistic regression is the everyday workhorse of yes/no prediction in medicine, finance and marketing. It is also the simplest neural network: one “neuron” with an S-shaped curve. Its output is a real probability, its weights can be explained in plain words, and the same S-curve comes back in Lessons 9.2 and 10.4.

2. The model

Start with the straight-line score from Module 3. Each input gets a weight, and a bias is added:

z=𝐰⋅𝐱+b

Here 𝐱 is the list of inputs, 𝐰 their weights (how much each counts), b the bias (the starting point), and z the score. z can be any number at all, from very negative to very positive. To turn it into a probability, squeeze it through the sigmoid (also called the logistic function):

p=σ(z)=11+e−z

e≈2.718 is the base of natural logarithms. You don’t need to love it: just watch what the formula does.

  • Big positive z: e−z is tiny, so p is close to 1.
  • z=0: e0=1, so p=1/2=0.5.
  • Big negative z: e−z is huge, so p is close to 0.
Try it: shape the S-curve
score z = wx + b
probability p = σ(z)
odds p ÷ (1 − p)
log-loss on all students
Passes sit at p = 1 and fails at p = 0. The S-curve is the model's probability of passing. w sets how steep it is (how quickly study time changes the chance), b slides it left or right, and the point where p = 0.5 is at x = −b ÷ w. Try to make the log-loss small: the curve should be high over the passes and low over the fails.

The weight w sets how steep the S is: a big weight means a small change in x swings the probability fast. The bias slides the S sideways. The 50 % point, where z=0, sits at x=−b/w.

Inputs to a score. A pass/fail model uses hours studied: z = 0.7 × hours − 4.2. A student who studied 8 hours gets z = 5.6 − 4.2 = 1.4.

Flip the sign and raise e to it. e−1.4 ≈ 0.247.

Add 1 and take one over it. 1 ÷ (1 + 0.247) ≈ 0.80. So the model gives this student an 80 % chance of passing.

Read it as odds. 0.80 ÷ 0.20 = 4: "4 to 1 on". That is just ez = e1.4 ≈ 4.06, rounded.

Two more hours. z rises by 1.4 to 2.8, so the odds are multiplied by e1.4 ≈ 4 again: about 16 to 1, a 94 % chance.

3. Odds and log-odds

Gamblers use odds: the chance something happens divided by the chance it doesn’t.

odds=p1−p

p=0.5 is odds 1 (“evens”). p=0.8 is odds 4 (“4 to 1 on”). p=0.2 is odds 0.25 (“4 to 1 against”). Take the natural log (ln) of the odds and you get the log-odds, also called the logit. It runs from minus infinity to plus infinity, just like the score z. That is no accident. Solving p=σ(z) for z gives

z=ln⁡p1−p
z
The score: the weighted sum of the evidence. It is also the log-odds of a "yes".
σ(z)
The sigmoid: turns any score into a probability between 0 and 1.
p ÷ (1 − p)
The odds: how many times more likely "yes" is than "no".
ew
The odds ratio: what one more unit of an input multiplies the odds by.
Physical meaning
Evidence adds up on the log-odds scale, like weights on a balance. The sigmoid is the pointer that shows how far the balance has tipped, and it can never go past fully "yes" or fully "no".

So logistic regression is simply a straight-line model for the log-odds. That gives its weights a clear reading: “one more unit of xj adds wj to the log-odds”, which is the same as “multiplies the odds by ewj”, whatever the other inputs are. That number ewj is the odds ratio, the way doctors and economists report these models.

“A weight of 0.7 means each extra hour adds 0.7, or 70 %, to the chance of passing.” No. The weight adds 0.7 to the log-odds, which multiplies the odds by about 2. How much the probability moves depends on where you start: from 50 % it jumps to about 67 %, but from 95 % it can only creep up to about 97 %.

Maths and principle

Principle: the sigmoid maps any score to a probability, and its inverse, the logit, maps a probability back to a score. The model is a straight line on the logit scale.

Equations:

σ(z)=11+e−z,σ−1(p)=ln⁡p1−p,σ′(z)=σ(z)(1−σ(z))

The last one is the sigmoid’s slope (how fast it rises). It is largest, 0.25, at z=0 and tiny far out. That matters in Lesson 4.3 and again for neural networks (Lesson 9.2).

Also σ(−z)=1−σ(z): the curve is symmetric about the 50 % point.

Analogy: a dimmer switch with soft ends. In the middle, a small turn changes the light a lot; near fully on or fully off, turning further barely matters.

Try it: w=0.7, b=−4.2. Probability at x=6? At x=8? (z=0, so 0.5. z=1.4, so 1/(1+e−1.4)≈1/(1+0.247)≈0.80.)

4. Worked example

Made-up numbers. A model of whether a student passes, from hours studied x: z=0.7x−4.2.

hours z odds ez p
2 −2.8 0.061 0.06
4 −1.4 0.247 0.20
6 0 1 0.50
8 1.4 4.06 0.80
10 2.8 16.4 0.94

Every 2 extra hours adds 1.4 to z and multiplies the odds by e1.4≈4.06. But the probability gain is uneven: from 4 to 6 hours it goes up 30 points, from 8 to 10 only 14. Effects are steady on the odds scale, not on the probability scale.

5. Practice

  1. Convert p=0.25 to odds and log-odds.
  2. z=−1. What is p? (e≈2.718.)
  3. A medical model has an odds ratio of 1.5 for smokers. What does that mean?
Answers
  1. Odds 0.25 / 0.75 = 1/3; log-odds ln(1/3) ≈ −1.10.
  2. 1/(1+e)≈1/3.718≈0.27.
  3. With everything else the same, smokers’ odds of the outcome are 1.5 times non-smokers’ odds (not 1.5 times the probability).
  • Logistic regression computes a straight-line score z = w·x + b, then p = σ(z) = 1 / (1 + e−z).
  • σ(0) = 0.5; big positive scores give p near 1, big negative scores p near 0.
  • Odds are p ÷ (1 − p); the score z is the log of the odds.
  • Each unit of an input multiplies the odds by ew, the odds ratio; the change in probability depends on the starting point.

Try it on your own data: the regression workbench fits logistic regression with odds ratios and their confidence intervals.

New words: logistic regression, sigmoid, odds, log-odds or logit, odds ratio.

Check yourself

1What is σ(0)?
2A probability of 0.8 is what odds?
3In a logistic model, the weight on "hours studied" is 0.7. What does one extra hour do?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Numbers in the examples are made up to show the idea unless a source is given. Simulations run in your browser; nothing you type is sent anywhere.