Logits: the raw bet a model places before it gives you a percentage

What a logit is

A logit is the raw score a model produces before anything turns it into a probability. It is one plain number for each possible answer, and it has no limits: it can be -12, 0, 3.7 or 250. Only afterwards does a function, such as the S-shaped sigmoid, convert it into a percentage you can read.

What the video shows

The video shows that this first output, which can be any number, is really a statement about odds. A logit of 0 means even odds: "yes" and "no" are equally likely. At +4, the odds are roughly 55 to 1 in favour of "yes".

Odds, in one minute

Odds compare the chance that something happens with the chance that it does not. A 50% chance is odds of 1 to 1, often called "evens". A chance of about 98% is odds of about 55 to 1: for every 55 times it happens, it fails to happen once.

The logit is the log-odds: the logarithm of those odds. A logarithm is a way of compressing huge ranges into manageable ones. Odds that run from "almost impossible" to "almost certain" become a single number that runs from very negative to very positive, with 0 sitting exactly in the middle. Negative logits lean towards "no" in the same way: a logit of -4 gives "yes" a chance of only about 2%.

An everyday example

Imagine a bookmaker who never talks in percentages, only in odds. "Evens" means a coin toss. Long odds in favour mean a near-certainty. A model's logit works the same way. The model places its bet first, in the language of odds, and only afterwards does someone translate that bet into a percentage for you.

Why the scale matters

A logistic model, the classic yes-or-no model, is a straight line in the log-odds, not in the probability. Raising one of its inputs adds a fixed amount to the logit. But what that does to the percentage depends on where you started:

• moving the logit from -1 to 0 takes the probability from about 27% to 50%, a jump of 23 points;
• moving it from 0 to +4, four times as far, takes it from 50% to about 98%, only 48 points: four times the step for about twice the gain.

The closer the model gets to certainty, the less each extra step buys, because there is hardly any room left. That is why "double the input, double the probability" is wrong. An equal step in the logit is not an equal step in percent.

Where you meet logits again

The idea travels a long way. A model that chooses between many answers produces one logit per answer, and those raw scores are then turned into probabilities all at once. Later in the course, in Chapter 17, the temperature setting of a language model, the kind of model behind chatbots, works by dividing its logits before they become probabilities. Same raw scores, one more knob.

Why it matters

Once you know what a logit is, a model's raw output stops being a mystery. The sign tells you which way it leans. The size tells you how strongly. Zero means it cannot decide. And you know not to treat equal jumps in the score as equal jumps in the percentage, which is an easy way to misread how these models respond to their inputs.

Learn it step by step in Chapter 4 of our free course AI From Scratch: Classification, Cross-Entropy, and How Not to Fool Yourself.