Learning from mistakes: the perceptron rule that only moves when wrong

What the perceptron learning rule is

The perceptron learning rule is how a perceptron teaches itself. A perceptron is one of the simplest learning models: it multiplies each input by a weight, adds everything up and answers yes or no depending on whether the total is positive or negative. Its learning rule fits in one sentence. If you got the example right, change nothing; if you got it wrong, nudge yourself toward the right answer. Then take the next example, round and round, until the mistakes stop.

What the video shows

The video presents this as the single rule behind the first learning machine: right means no change, wrong means a nudge toward the fix, repeated until nothing is wrong any more. The hook puts it plainly: this AI only learns when it gets something wrong. Whether the mistakes are guaranteed to stop is its own story, told in the Convergence Theorem entry of this series.

An everyday example

Imagine a homemade spam filter that gives every word a score. An email's total is the sum of the scores of its words, and above zero it goes to the spam folder.

When the filter gets an email right, you leave the scores alone. When it lets a spam email through, you add a little to the score of every word in that email. When it wrongly blocks a real message, you take a little away. Words that keep turning up in missed spam drift upward, and the mistakes thin out. That is the perceptron rule, applied to words.

How it works

In our course, a perceptron learns to sort factory parts into accept and reject:

1. Start from nothing. All the weights (how much each measurement counts) and the bias (one extra number added to the total) begin at zero. Every score is zero, and zero counts as accept, so at first it accepts everything.
2. Label the answers. Parts that should be accepted count as +1; parts that should be rejected count as −1.
3. Check one example. If the sign of the score matches the label, do nothing.
4. Fix a mistake. If not, add the example's measurements, multiplied by its label, to the weights, and add the label to the bias.

That nudge is not a guess. Say a part should have been accepted but scored below zero. Adding its measurements to the weights raises that same part's score by its measurements squared and added together, which is always a positive number. So the score moves exactly the way it needed to. The fix can break a different example, so the rule loops over the data again.

A common misconception

Many people assume machine learning always means calculus: derivatives, which are slopes that tell you which way is downhill. The perceptron needs none.

What you really want to shrink is the number of mistakes, and that count behaves like a staircase. While you move the line a little, it stays flat. The moment the line crosses a point, it jumps by a whole step. A flat stair has no slope to follow, so calculus has nothing to hold on to. The perceptron sidesteps this by asking only "right or wrong?" and moving in a direction simple geometry can justify.

Why it matters

This rule shows the basic loop behind much of machine learning in its barest form: predict, compare with the truth, adjust. It also shows its own limit. A rule that only knows "wrong" cannot train models built from several stacked layers, and the next chapters of the course exist to win back a slope that learning can follow. Still, the perceptron keeps one trick none of its successors share: it learns without any calculus at all.

Learn it step by step in Chapter 1 of our free course AI From Scratch: The Perceptron From Scratch: What a Neuron Computes.