F1 score: the one number that won't let a model hide its weak spot

What the F1 score is

The F1 score is a single number that combines two others: precision (when the model raises an alarm, how often is it right?) and recall (of all the real cases, how many did it find?). It runs from 0 to 1, and higher is better.

The trick is in how it combines them. F1 does not take the ordinary average. It uses the harmonic mean, a kind of average that gets pulled down hard by whichever number is smaller. So a model cannot earn a good F1 by being excellent at one thing and hopeless at the other.

What the video shows

The video makes two points. First, F1 blends two scores into one number that stays close to the weaker of the two. Second, when a model sorts things into several categories instead of just two, there are two different ways to average F1, and they can give very different answers. So whenever someone reports one F1 figure, ask which one they mean.

An everyday example

Imagine a road trip. You drive to a town at 30 km/h and come back along the same road at 90 km/h. Your average speed is not 60. You spent three times longer on the slow leg, so the real average is 45 km/h, much closer to the slower number. That is the harmonic mean at work, and it is exactly how F1 treats precision and recall.

A chain, which is only as strong as its weakest link, is the extreme version of the same idea. F1 is not quite that strict, but it leans the same way.

How it works on a real model

In the course, a model checks factory parts for defects. At its default setting, 8 of its 10 alarms are real (precision 0.80), but it finds only 8 of the 74 defective parts (recall about 0.11). A plain average of those two would be about 0.45, which sounds half decent. F1 gives 0.19, which is the honest verdict: this model misses most defects.

It works in the other direction too. Lower the model's alarm line until it flags parts very readily and it finds 71 of the 74 defects, but only about 5 in every 100 alarms are real. F1 drops to 0.09. Neither extreme can hide.

Two ways to average with many categories

When a model can give several possible answers, you get one F1 per category, and you have to combine them:

• Macro F1 takes the plain average across categories. A rare category counts exactly as much as a common one.
• Micro F1 pools every single prediction together first. It comes out identical to accuracy, the share of answers that are right.

In the course's example with several categories, macro F1 is 0.798 while micro F1 is 0.94. Same model, same predictions, a gap of about 14 points that depends only on how you average. A macro score that far below accuracy tells you some categories are handled much worse than the headline suggests.

Why it matters

F1 is useful precisely because it is hard to flatter. A report that shows only precision, or only recall, can make a weak model look strong. F1 will not, as long as you also know which version was used. Two questions to ask whenever you see one: is it macro or micro? And what were the precision and recall behind it?

Learn it step by step in Chapter 4 of our free course AI From Scratch: Classification, Cross-Entropy, and How Not to Fool Yourself.