Precision-Recall against ROC
The same model on the same scores. Drag the positive rate down and watch ROC stay flattering while the PR curve tells the truth.
Overview
Start balanced
Leave the slider at 0.5. Half the rows are positive, and the two curves broadly agree: ROC bows toward the top left, PR bows toward the top right, and both say the model is decent.
Now drag the slider down toward 0.02, which is a realistic rate for fraud, disease screening or ad clicks. Watch the two charts diverge.
ROC barely moves. The AUC in the readout stays high.
PR collapses. The precision that was achievable at any useful recall falls away, and the best F1 falls with it.
The scores did not change. The model did not change. Only the proportion of positives changed, and the two curves disagree about whether that matters.
Precision-Recall against ROC
Worth knowing
Precision-Recall against ROC
Two curves over identical scores, and the reason one of them flatters a model that cannot be deployed.
Why ROC does not notice
ROC plots true positive rate against false positive rate:
TPR = TP / (TP + FN) denominator: all actual positives
FPR = FP / (FP + TN) denominator: all actual negativesEach is computed entirely within one class. TPR asks what fraction of positives were caught; FPR asks what fraction of negatives were wrongly flagged. Adding a million more negatives leaves TPR untouched and changes FPR only through its own denominator, which grows in proportion.
That class-independence is often described as a virtue, and for some purposes it is. It also means ROC cannot see the thing that makes rare-positive problems hard.
Why PR does notice
Precision is different:
precision = TP / (TP + FP) denominator: everything you flaggedThe denominator mixes both classes. When negatives outnumber positives a hundred to one, even a small false positive *rate* produces a large *number* of false positives, and those go straight into precision's denominator.
Concretely: 10,000 rows, 100 positive. A model at 90% recall and 5% FPR catches 90 real positives and flags 495 negatives. FPR of 5% sounds excellent and ROC records it as such. Precision is 90 / 585 = 15%. Five out of six flagged cases are wrong, and every one costs somebody an investigation.
Reading the baselines
Each chart has a dashed no-skill line, and they differ.
For ROC it is the diagonal, always. A random classifier gets AUC 0.5 regardless of balance.
For PR it is a horizontal line at the positive rate. At 50% positives, random scores 0.5 precision. At 2%, random scores 0.02. Drag the slider and watch it drop.
This is what makes PR honest and slightly harder to read: there is no fixed scale. A PR AUC of 0.4 is poor on balanced data and outstanding at a 2% base rate. The number must always be compared against the baseline, never quoted alone.
Which to use
| Situation | Curve | Reason |
|---|---|---|
| Roughly balanced | either | they broadly agree |
| Rare positives | PR | ROC hides the false-positive volume |
| You care about ranking overall | ROC | it is what AUC measures |
| Someone acts on each flag | PR | precision is that person's experience |
| Comparing across datasets | ROC, carefully | PR baselines differ by base rate |
The practical rule: if a human or a process has to act on every positive prediction, precision is the thing they experience, and the PR curve is the one to look at.
Add a million negatives, and watch only one curve move
The divergence is arithmetic. Hold the model's rates fixed — 90% recall, 5% false-positive rate — and change only the class balance in 10,000 rows:
| positives | TP | FP | precision | TPR / FPR |
|---|---|---|---|---|
| 5,000 (balanced) | 4,500 | 250 | 0.95 | 0.90 / 0.05 |
| 100 (1%) | 90 | 495 | 0.15 | 0.90 / 0.05 |
The ROC coordinates are identical in both rows, because TPR divides by actual positives and FPR by actual negatives — each lives inside one class, so the balance cannot touch them. Precision divides by everything flagged, which mixes the classes, so when negatives outnumber positives a 5% false-positive rate turns into 495 false alarms drowning 90 real ones and precision falls to 0.15. Same model, same scores, and ROC records it as excellent while five of every six flagged cases are wrong. That is why ROC AUC alone on rare-positive data is the standard way to make an undeployable model look ready, and why the PR curve — whose no-skill baseline is the positive rate itself, 0.01 here, not a fixed 0.5 — is the honest one to quote.
The practical rule follows from who acts on the output. If a human or a process has to deal with every positive prediction — an analyst investigating each flag, a queue of cases to review — then precision is the experience that person actually has, and the PR curve is the one to read. ROC still has its place: it is the right tool when you care about ranking quality overall, and it is more comparable across datasets because its baseline is a fixed 0.5 rather than a base-rate that shifts. Neither curve, though, tells you which threshold to deploy; that is a separate decision driven by [costs](threshold_tuning.html).
One more asymmetry is worth stating: because the PR baseline is the positive rate, a PR AUC has no fixed scale. A PR AUC of 0.4 is poor on balanced data and outstanding at a 2% base rate, so the number means nothing quoted alone — it has to be read against the baseline for that dataset, which is exactly why it cannot be compared across datasets with different balances the way ROC can.
A last comparison point: the two curves also treat the negatives differently. ROC rewards a model for correctly ranking the vast pool of easy negatives, which is where its optimism on rare-positive data comes from; PR ignores true negatives entirely — they appear in neither precision nor recall — so it reports only on the positives you care about and the false alarms among them.
Where it goes wrong
Reporting ROC AUC alone on imbalanced data. It is the standard way to make a model that will drown a team in false positives look ready to ship.
Comparing PR AUC across datasets with different base rates. The baselines are different, so the numbers are not comparable.
Forgetting that both curves span all thresholds. Neither tells you which threshold to deploy — that is a separate decision, made with [costs](threshold_tuning.html).
The same model, two curves, two different stories
ROC and precision-recall are computed from the same predictions. On balanced data they agree; on imbalanced data one of them is misleading, and it is always the same one.
Check yourself
0 of …Answer without scrolling back up.
Why does ROC stay flattering as positives become rare?
10,000 rows, 100 positive, 90% recall and 5% false positive rate. What is precision?
What is the no-skill baseline on a PR curve?
Precision-Recall against ROC
Leave the slider at 0.5. Half the rows are positive, and the two curves broadly agree: ROC bows toward the top left, PR bows toward the top right, and both say the model is decent.