Learned representation: how a hidden layer reshapes a problem

What a learned representation is

A learned representation is a new way of describing the data that a neural network works out for itself. The hidden layer (the layer between the input and the final answer) takes each input and gives it new coordinates: a new position on a new map. Nobody draws that map by hand: training shapes it, with a single purpose, to make the next layer's task easy.

The course is careful with the term, because it gets used loosely all over AI. A learned representation is not magic, not a summary and not some compressed essence of the data. It is simply a change of coordinates, one that training finds instead of a person.

What the video shows

The video looks inside a small trained network. Its hidden layer learns to fold the data into a new space, and two opposite corners of the problem end up almost touching. Once they sit together, a single straight line can finally separate the two groups.

An everyday example

Imagine labelling room temperatures as comfortable or not. A chilly 15°C and a stuffy 27°C sit at opposite ends of the thermometer, yet both get the same label, while 21°C in the middle is fine. No single cut-off on the thermometer works: "above this number" or "below this number" always gets one end wrong.

Now describe each reading a different way: how many degrees it is away from 21°C. The chilly room and the stuffy room both become "6 degrees off". The two opposite ends have landed on the same spot, and one simple rule now works: more than 3 degrees off is uncomfortable.

Here a person chose the new description. A hidden layer does the same by itself: training finds a description in which the answer is easy to read off.

How it works

The course uses the XOR puzzle: two inputs, each 0 or 1, and the answer is "yes" when exactly one of them is 1. Picture the four cases as the corners of a square. The two "no" cases, (0,0) and (1,1), sit on opposite corners, as far apart as two points get in this problem. No straight line can put both "no" corners on one side and both "yes" corners on the other.

With only two hidden units, the new position of each case is just a point on a flat page, so you can print it and look. After training, this is where the four cases land:

| Input | Position after the hidden layer | Answer | |---|---|---| | (0,0) | (0.82, -0.85) | no | | (1,1) | (0.84, -0.86) | no | | (0,1) | (1.00, 0.86) | yes | | (1,0) | (-0.84, -1.00) | yes |

The two far-apart "no" corners now land almost on the same spot, as if the square had been folded like a sheet of paper. With both in one place, a single straight line separates them from the two "yes" cases.

The output neuron draws exactly that line. It is still the same kind of simple straight-line classifier (a perceptron) that fails on XOR when it looks at the raw inputs, and it has gained no new power. What changed is the space it is looking at.

Why it matters

This tiny network gives a concrete picture of what "a network learned a representation" means. It also shows where a hidden layer's power comes from: the last step can stay a plain straight line, as long as the layer before it hands over a good description. The phrase shows up constantly in AI talk, and remembering its plain meaning (new coordinates, found by training, that make the next step easy) makes that talk far less mysterious.

Learn it step by step in Chapter 5 of our free course AI From Scratch: Backpropagation From Scratch: The Engine, Then the Network.