Ablation: change one thing at a time to find what really works

What ablation is

Ablation means testing the parts of a system one at a time to find out what each part is really responsible for. You add or remove a single piece, keep everything else exactly the same, and see what changes. It is the opposite of throwing every fix at a problem and hoping.

In AI, where a training run has dozens of knobs, it is how people separate the ingredient that matters from the ones that just came along for the ride.

What the video shows

A stuck AI starts working after four fixes are applied at once. But which one mattered? Testing them one by one shows that two of the fixes work on their own, and the other two were aimed at a different problem.

An everyday example

Imagine your car won't start. In one afternoon you replace the battery, the spark plugs, the fuel filter and the starter motor, and it starts. Success, but you have learned nothing. The next time it fails, your only plan is to replace all four again. Swap one part at a time and you find out which one was actually broken.

The experiment behind it

Our free course starts with a six-layer network that is dead: it is stuck at 50% accuracy, no better than a coin flip. There are four candidate fixes:

• Xavier initialization: better-sized starting values for the network's settings.
• LayerNorm: a layer that keeps the signals inside the network at a steady size.
• Residual connections: shortcuts that let corrections skip past a layer.
• Adam: a smarter rule for how big each learning step should be.

Each one is tried alone, with the same random starting point, the same data and the same design.

| What was added | Accuracy after 800 steps | After 5,000 steps | |---|---|---| | Nothing | 50.0% | 50.0% | | Xavier initialization | 60.4% | 100% | | LayerNorm | 61.5% | 100% | | Residual connections | 56.6% | 56.7% | | Adam | 58.7% | 53.4% | | All four together | 100% | not shown |

Read only the first column and the conclusion seems to be that no fix works alone and they only work as a team, which makes deep learning look like magic. The second column tells a different story. Xavier alone and LayerNorm alone each reach 100%. Both treat the real illness: the signal travelling forward through the network was shrinking to zero. At 800 steps they had already fixed it and were simply still climbing.

Residual connections and Adam never fix it, however long they run. They are good tools aimed at a different problem. A shortcut that routes around a dead layer just delivers the same dead value. And adjusting the size of each step cannot help when the network gives the same output whatever you feed it.

How to use it

• Change one thing and hold everything else fixed: same seed (the random starting point), same data, same design.
• Give each test a fair budget. A fix can look like partial credit only because it has not finished working.
• Keep a do-nothing row. The untouched network's error was exactly the same after 5,000 steps as after 800. Set next to the rows where a fix works, that shows it was dead, not slow.

Why it matters

Changing everything at once can give you a working model today and leave you helpless tomorrow, with a ritual instead of an explanation. Testing one change at a time turns that ritual into a diagnosis. It is what separates a method from superstition.

Learn it step by step in Chapter 6 of our free course AI From Scratch: Getting It to Train, and Getting It to Generalise.