Bayes' rule: reasoning backwards from a clue to its most likely cause

What Bayes' rule is

Bayes' rule is a way to turn a question around. It is often easy to say how likely a clue is if some explanation is true: if it rained, the street will be wet. What you usually want is the reverse: the street is wet, so how likely is rain? Bayes' rule converts the first kind of chance into the second. The extra ingredient it needs is how common each explanation was before you saw the clue.

What the video shows

The video opens with a question: the street is wet, so did it rain, or was it the sprinkler? Knowing that rain wets streets is the easy direction. Knowing whether a wet street means rain is harder. Bayes' rule flips the question and weighs how common each possible cause is.

An everyday example

Imagine 100 mornings in a small town, with made-up numbers to keep the arithmetic easy:

• It rained on 10 of them, and every one of those left the street wet.
• The other 90 were dry, but a neighbour's sprinkler ran on 30 of them, and it always wets the street too.

So the street was wet on 40 mornings, and only 10 of those 40 were rain. "If it rains, the street gets wet" is true every time, yet "if the street is wet, it rained" is true just 1 time in 4.

Rain and the sprinkler explain a wet street equally well, since both always wet it, so the clue alone cannot choose. What decides is how often each cause happens, and in this made-up town the sprinkler is behind three times as many wet mornings as the rain.

How it works

In words, the rule reads:

Chance of the cause given the clue = chance of the clue given the cause × how common the cause is ÷ how common the clue is.

Check it on the town: 1 (rain always wets the street) × 10/100 (rain on 10 mornings) ÷ 40/100 (wet on 40) = 1/4. Same answer as the count.

It is not a separate law: it comes from the product rule (the chance of two things together is the chance of one times the chance of the other once the first is known), written once in each direction and set equal.

Where it shows up in AI

A model can usually compute the easy direction: how probable the data you collected would be if a particular answer were true. The question you care about runs the other way: given this data, how probable is this answer? Bayes' rule is the bridge between the two.

The course uses it to explain an important shortcut. When you have no reason to favour one answer over another before looking at the data, the answer that makes your data most probable is also the most probable answer. That is the idea behind maximum likelihood, the next concept in this series. The town shows why that condition matters: the clue fits both causes equally well, so how common each cause is settles the answer.

A common misconception

The classic mistake is treating the two directions as the same claim. "Rain makes streets wet" and "wet streets mean rain" sound alike, but the gap between them is exactly how common each cause is. Forget that, and a rare cause starts to look like the obvious explanation just because it fits the clue.

Why it matters

Whenever you reason from a clue back to its cause, you are doing what Bayes' rule describes, written down or not. Knowing its parts helps you spot the question you are really asking, and the information you might be leaving out.

Learn it step by step in Chapter 2 of our free course AI From Scratch: Where a Loss Function Comes From: Likelihood, Not Convention.