Layer normalization: why chatbots steady each input on its own

What layer normalization is

A neural network passes long lists of numbers from one layer to the next. Left alone, those numbers drift in size as training goes on, and a network whose numbers drift too far stops learning well. A normalization layer resets them to a steady scale: it subtracts their average, divides by their spread, and then applies a scale and shift the network learns for itself, in case a different level works better.

Layer normalization, or LayerNorm, computes that average and spread from a single example: all the numbers that one input produces at that layer. It never looks at any other example. That one design choice decides almost everything else about it.

What the video shows

The short calls it the quiet layer inside every chatbot. Layer normalization keeps each example's signals at a steady level on its own, without looking at any other example. Because of that, it behaves exactly the same while the model is learning and while it is answering, and that is why chatbots use it.

An everyday example

Imagine the auto-fix button in a photo app that balances each picture using only that picture's own light and dark areas. It does not matter whether you edit one photo or a whole album, or whether you are experimenting or printing the final copy: the same photo always comes out the same.

Compare that with a teacher who grades on a curve, where your mark depends on the rest of the class. That is its sibling, batch normalization, which measures each example against the group it arrived with.

What measuring on its own buys you

Because LayerNorm only needs the one example in front of it, it gets several things for free:

• No dependence on other examples. A result does not change with the company it keeps.
• Nothing to remember. There are no averages to collect during training and switch to later.
• One behaviour. Training and real use run the same calculation, so a whole family of mode-switching bugs disappears.
• No sensitivity to size or length. It works the same on one example or a thousand, and on a short sentence or a long one.

Why chatbots need exactly this

A chatbot writes its reply one token at a time (a token is a word or a piece of a word), often for a single person. There is no crowd of examples to average over, and the length of the text keeps changing. In that setting, the properties above stop being nice extras and become requirements. That is why the transformer, the design behind today's large language models, normalizes each example on its own. The transformer our course builds later reuses this exact layer, unchanged.

A common misconception

It is tempting to think LayerNorm won because it is simply better at steadying signals. Our course makes a subtler point: the transformer uses it for those practical reasons, not because it is superior in some general sense. It does do the core job well, though. In the course's experiment, adding LayerNorm on its own was enough to bring a stuck six-layer network back to life.

Why it matters

If you ever look inside a transformer, you will meet a layer like this in every block. Knowing why it is there, rather than just that it is there, tells you something useful about how these systems are built: the choices are often driven by how the model will be used, one person and one word at a time, and not only by what scores best in a lab.

Learn it step by step in Chapter 6 of our free course AI From Scratch: Getting It to Train, and Getting It to Generalise.