Precision and recall: is the alarm right, and does it catch enough?

What precision and recall are

When an AI model raises alarms, for example flagging emails as spam or factory parts as defective, two separate questions matter:

• Precision: of everything the model flagged, how much was really a problem? It tells you how many alarms were wasted.
• Recall: of all the real problems out there, how many did the model catch? It tells you how many slipped through.

A model can score well on one and terribly on the other. That is why a single headline number, such as accuracy (the share of all answers that were right), can hide so much.

What the video shows

The video looks at a model that flags defective parts. When it raises an alarm, it is usually right: 8 of its 10 flags were real defects, so its precision is 0.80. But there were 74 defective parts in total, and it caught only 8 of them. Its recall is 8 out of 74, about 0.108. Right when it speaks, silent most of the time.

An everyday example

Imagine fishing with a small hand net that you only dip in when you can clearly see a fish. Almost everything you pull up is a fish, not an old boot, so your precision is excellent. But you leave most of the fish in the lake, so your recall is poor. Now drag a huge net through the whole lake and the picture flips: you catch nearly every fish, along with boots, weeds and plastic bags.

Neither net is simply correct. Each one answers a different question.

Which one matters more

Here is the key point: choosing between the two is a business decision, not a mathematical one. You have to ask what each kind of mistake costs.

In the course's factory example, a wasted inspection costs a few seconds of someone's time. A defective part that slips through reaches a customer and can lead to a recall. So recall matters most here, and catching 8 of 74 defects is a failure, even with 80% precision.

Change the setting and the answer can change too. Imagine a filter that moves emails into a spam folder. There, a false alarm hides a message you needed, so you might care more about precision.

How to use them

• Always ask for both numbers, never just one.
• Before you look at any scores, write down which mistake is more expensive: a false alarm or a miss.
• Expect a trade-off. Making the model flag things more readily usually catches more real problems but also raises more false alarms. That setting is the decision threshold, the cut-off at which a probability turns into a yes, and it deserves its own explanation.

The counts behind both numbers (right flags, false alarms and misses) come from a confusion matrix, a simple table of hits and mistakes.

Why it matters

In the course, this same model reaches 98.3% accuracy, which sounds excellent. Yet it lets 66 of the 74 defective parts through. Precision and recall are how you find that out. Whenever someone quotes one impressive score for a system that raises alarms, ask two questions: when it warns, is it right? And how much does it miss?

Learn it step by step in Chapter 4 of our free course AI From Scratch: Classification, Cross-Entropy, and How Not to Fool Yourself.