Precision, Recall and F1
Move the threshold and watch the confusion matrix rearrange itself. One model, and every score in the table changes.
Overview
One model, many scores
A classifier does not output a class. It outputs a score — how positive it thinks each case is — and a class only appears once you compare that score against a threshold.
The histogram shows both distributions: actual negatives in grey, actual positives in orange, overlapping in the middle because the model is good and not perfect. The dashed line is the threshold.
Move it. Every number below the chart changes, and the model has not been retrained. The scores are properties of the threshold as much as of the model, which is the single most useful thing to understand here.
Precision, Recall and F1
Worth knowing
Precision, Recall and F1
Four counts in a table, three ratios drawn from them, and one slider that moves all of it.
The four counts
| Predicted positive | Predicted negative | |
|---|---|---|
| Actually positive | True positive | False negative |
| Actually negative | False positive | True negative |
Everything else is a ratio of these. The two that matter most divide by different denominators, which is exactly why they disagree.
Precision = TP / (TP + FP) — of everything you flagged, what fraction was right. The denominator is what *you* predicted, so precision is damaged by false alarms.
Recall = TP / (TP + FN) — of everything that was actually there, what fraction you caught. The denominator is what *reality* contained, so recall is damaged by misses.
Why they trade off
Drag the threshold to 0.10. Almost everything is flagged, so almost every real positive is caught and recall approaches 100%. But the flagged set is now full of negatives, and precision collapses.
Drag it to 0.90. Only the most confident cases are flagged, nearly all of them correct, and precision approaches 100%. But most real positives are below the line, and recall collapses.
You cannot maximise both by moving the threshold. The threshold only chooses where on the trade-off you sit. Improving both at once requires a better model — one whose two distributions overlap less.
Which one you want depends on the cost
The question is never "which metric is best". It is which mistake is more expensive.
Recall matters more when a miss is costly: screening for a serious disease, detecting fraud, finding safety defects. A false alarm costs a second look. A miss costs the thing you were trying to prevent.
Precision matters more when a false alarm is costly: flagging accounts for suspension, recommending content, sending an alert that wakes someone. Missing one is a shame. Being wrong repeatedly destroys trust in the system.
F1, and why it is a harmonic mean
F1 combines them:
F1 = 2 * precision * recall / (precision + recall)That is the harmonic mean, and the choice of mean is the point. Take precision of 1.0 and recall of 0.01 — a model that flags exactly one case and gets it right. The arithmetic mean is 0.505, which sounds respectable. F1 is 0.0198.
The harmonic mean is pulled toward the smaller number, so it stays low unless both are high. That makes it hard to game with a degenerate model, which is precisely what a single summary number needs to be.
F1 weights the two equally, which is a decision and often the wrong one. F-beta lets you weight recall β times as much as precision; F2 favours recall, F0.5 favours precision. If you have a reason to prefer one, use it.
Why accuracy is on the readout and not in the headline
Accuracy is (TP + TN) / everything. It is the most intuitive metric and it is close to useless when the classes are unbalanced.
If 1% of transactions are fraudulent, predicting "never fraud" scores 99% accuracy while catching nothing at all. Precision and recall are both zero, which is the honest description. [Imbalanced data](precision_recall_vs_roc.html) is where this matters most.
The same model at two thresholds
Precision and recall are properties of the threshold as much as the model, which a worked confusion matrix makes concrete. The same classifier, the same scores, two cuts:
| threshold | caught (of 100 real) | false alarms | precision | recall |
|---|---|---|---|---|
| 0.10 (permissive) | 98 | 300 | 0.246 | 0.98 |
| 0.90 (strict) | 40 | 4 | 0.909 | 0.40 |
Drop the threshold and recall climbs toward 100% as almost everything is flagged, but the flagged set fills with negatives and precision collapses; raise it and precision climbs as only confident cases survive, but most real positives fall below the line and recall collapses. The threshold only chooses *where on the trade-off you sit* — improving both at once needs a better model, one whose positive and negative score distributions overlap less. F1 summarises a single point as the harmonic mean, 2PR/(P+R), and the choice of *harmonic* is the point: precision 1.0 with recall 0.01 averages to a respectable-looking 0.505 arithmetically but an F1 of just 0.0198, because the harmonic mean is dragged to the smaller number and refuses to reward a model that flags one case and hides.
Which error to minimise is a question about costs, never about which metric is "best". Recall matters more when a miss is expensive — screening for a serious disease, catching fraud — where a false alarm costs only a second look. Precision matters more when a false alarm is expensive — suspending an account, paging a human at 3am — where a miss is merely dealt with later. And accuracy, the most intuitive metric, is close to useless under imbalance: if 1% of transactions are fraud, predicting "never fraud" scores 99% while catching nothing, with precision and recall both zero telling the honest story the single accuracy number hides.
Where it goes wrong
Reporting one number without the threshold. "Precision 0.9" is uninterpretable alone; at some threshold nearly every model achieves it.
Optimising F1 when the costs are not equal. F1 assumes they are.
Comparing models at 0.5. Two models can rank cases equally well and differ only in calibration. Compare across thresholds, or tune each one.
Accuracy on imbalanced data. It measures the class balance more than the model.
The trade you cannot avoid
Precision and recall pull in opposite directions. Sweeping the decision threshold shows exactly how much of one each unit of the other costs.
Check yourself
0 of …Answer without scrolling back up.
What happens to precision and recall as the threshold falls?
Why is F1 a harmonic rather than arithmetic mean?
Why is accuracy misleading when 1% of cases are positive?
Precision, Recall and F1
A classifier does not output a class. It outputs a score — how positive it thinks each case is — and a class only appears once you compare that score against a threshold.