Multilayer perceptron: how a few neurons solve what one never can

What a multilayer perceptron is

A perceptron is the simplest artificial neuron. It takes a few numbers in, multiplies each one by a weight (how much that input matters), adds everything up together with one extra number called the bias, and passes the total through a small function that shapes the final output.

A multilayer perceptron, or MLP, is several of those neurons arranged in layers. The first layer reads the inputs, the next layer reads what the first layer produced, and so on until the final answer. The layers in the middle are called hidden layers, because you never see their outputs directly. It is the plainest kind of neural network there is.

What the video shows

The video sets up a tiny puzzle called XOR. A single artificial neuron can never solve it, however you tune it. Stack a few neurons in layers, let training adjust just nine numbers, and the network gets all four cases right: one neuron failed, three working together succeeded.

An everyday example

Imagine a staircase light with one switch at the bottom and another at the top. The light is on when exactly one switch is flipped up, and off when both are down or both are up. That is XOR: one or the other, but not both.

A single neuron can only split its inputs with one straight cut. Draw the four switch combinations as the corners of a square: the two "on" cases sit on opposite corners, and the two "off" cases sit on the other diagonal. No single straight line puts both "on" corners on one side and both "off" corners on the other. The course's first chapter proves it formally: the conditions for a solution would need the bias to be positive and negative at the same time.

How it works

The network that cracks the puzzle is small enough to count on your fingers:

• 2 inputs: the two switches.
• 2 hidden neurons: each has 2 weights and a bias, so 6 numbers.
• 1 output neuron: 2 weights and a bias, so 3 more numbers.

That makes nine adjustable numbers. The weights start as random guesses, and training nudges all nine, step by step, to shrink the error. For the first 50 steps the error barely moves (from about 4.16 to about 4.00). By step 100 it has started to fall, and by step 200 it has collapsed to about 0.04. Every answer is now right: the network outputs close to +1 for the two "on" cases and close to -1 for the two "off" cases.

Here is the part that surprises people. In the course's code, the network is built from three tiny pieces: a neuron, a layer (a list of neurons) and the network itself (a list of layers). None of those pieces contains a single line for working out slopes (the backward pass that tells each of the nine numbers which way to move). Every slope comes from the automatic slope engine built earlier in the chapter, which has no idea it is being used for a neural network.

Why it matters

XOR is the classic proof that one neuron has hard limits. The multilayer perceptron answers it with layers: several simple units, stacked, can compute things none of them can compute alone. And the answer was not designed by hand. Nine numbers were found automatically by training.

How the hidden layer actually pulls this off, by rearranging the four cases, is a story of its own: the learned representation.

Learn it step by step in Chapter 5 of our free course AI From Scratch: Backpropagation From Scratch: The Engine, Then the Network.