Precision and recall

Precision and recall measure different kinds of mistakes. Precision asks: of everything the system labeled urgent, flagged, or retrieved, what share was correct? Recall asks: of everything that should have been labeled, flagged, or retrieved, what share did the system catch? A classifier that flags only the most obvious cases has high precision and low recall; one that flags everything has perfect recall and poor precision.

Which matters more is a product decision. A moderation filter that blocks legitimate users has a precision problem; one that misses abuse has a recall problem. An urgent-ticket detector probably needs high recall, because a missed outage is costly and a false alarm costs a glance. Report both per class instead of one accuracy number, which hides failures on rare but important classes, and use a confusion matrix to see which labels get mixed up.

Retrieval uses the same ideas. Recall@k asks whether the right document appears in the top k results at all, which matters a lot for RAG, because the model can ignore an irrelevant chunk but can't use one it never saw. Thresholds trade one metric for the other, so tune them on labeled data with the real cost of each kind of error in mind.