Logistic regression: from a yes-or-no verdict to a probability

What logistic regression is

Logistic regression is a classic model for yes-or-no questions. Is this part defective? Is this email spam? Instead of replying with a flat "yes" or "no", it replies with a probability: a number between 0 and 1 that says how likely "yes" is. Despite the word "regression" in its name, its job here is to sort things into two groups.

What the video shows

The video's main point: instead of answering yes or no, this classic model gives a probability, and that probability tells a factory which parts to double-check first. It also shows the other half of the idea: the model learns from its own mistakes.

A plain "reject" could not do that: it treats a part that is almost certainly broken and a part that barely crossed the line in exactly the same way.

An everyday example

Imagine a friend helping you clear out the fridge who can only say "bin it" or "keep it". A yogurt covered in mould and a yogurt one day past its date get exactly the same verdict. Now imagine the same friend saying "I'm 99% sure this one has gone off" about the first and "maybe 55%" about the second. You bin the first without a thought and give the second a sniff. Same friend, far more useful answer.

How it works

The model does three simple things:

1. Add up the evidence. It multiplies each measurement of a part by a weight, a number that says how much that measurement matters, and adds everything up. The total can be any number, from very negative to very positive.
2. Squash it into a probability. An S-shaped curve called the sigmoid turns that total into a value between 0 and 1. A total of 0 becomes 0.5, a coin toss.
3. Learn from the miss. During training, the model compares its probability with the truth: 1 for defective, 0 for fine. The correction it applies to each weight is simply the difference, prediction minus truth, multiplied by that measurement. Said 0.9 for a part that was fine? Big miss, big correction. Said 0.1? Small miss, small correction.

The way it scores its mistakes is not arbitrary. The course treats each label like a weighted coin flip: there is some chance the part is defective. Take how likely the real labels were under the model's probabilities, turn that into a penalty that grows as the likelihood shrinks, and you get the scoring rule, called cross-entropy. Because the pieces fit together so neatly, the training loop the course built in Chapter 3 (gradient descent, which nudges the weights downhill step by step) works after changing a single line.

A common misconception

People hear "linear model" and expect straight-line behaviour in the probability: double a measurement, double the chance. That is not how it works. The model is a straight line in the log-odds, a way of writing how the chance of "yes" compares with the chance of "no". Raising a measurement shifts the log-odds by a fixed amount, but how far the probability moves depends on where it already was. Near 50%, a small push moves it a lot. Near 99%, there is almost no room left to move.

Why it matters

A probability is something you can act on. You can rank parts, messages or requests by how likely they are to need attention, and decide separately where to draw the line between "check" and "ignore". A plain yes or no hides that information for good. This small model is also where several ideas of the chapter first appear together: the scoring rule (cross-entropy), the S-shaped curve (the sigmoid) and the raw scores before the curve (logits).

Learn it step by step in Chapter 4 of our free course AI From Scratch: Classification, Cross-Entropy, and How Not to Fool Yourself.