Threshold Tuning
0.5 is a default, not a decision. Set what each kind of mistake costs and watch the cheapest threshold move away from it.
Overview
Where 0.5 comes from
It comes from the library. predict() calls predict_proba() and cuts at 0.5, because a default has to be something and 0.5 is the least arbitrary-looking choice available.
It is optimal under two assumptions that are rarely true together: the classes are balanced, and the two kinds of mistake cost the same. Change either and 0.5 stops being the right cut.
Threshold Tuning
Worth knowing
predict() happens to cut. It is not derived from your problem.Threshold Tuning
Where 0.5 came from, why it is almost never right, and how to find the number that is.
Put a price on each mistake
The two sliders set what a false negative and a false positive cost. The curve is the total cost across every threshold, computed over 1,500 cases with 20% positives. The grey line marks 0.5; the orange line marks the cheapest threshold.
Start with a miss costing 20 and a false alarm costing 1 — a screening problem, where investigating a healthy person is cheap and letting a sick one through is not. The minimum sits well below 0.5. The model should flag generously, because the flags are cheap and the misses are not.
Now reverse them: a miss at 1 and a false alarm at 50 — an automated account suspension, where a wrong suspension is expensive and a missed offender is dealt with later. The minimum moves above 0.5.
Set them equal and, with these class proportions, the minimum lands near 0.5 but not exactly on it, because the balance is 20/80 rather than 50/50.
The arithmetic
The expected cost of predicting positive on a case with predicted probability *p* is (1 - p) x cost_FP. Predicting negative costs p x cost_FN. Flagging is worth it when
(1 - p) * cost_FP < p * cost_FNwhich rearranges to
p > cost_FP / (cost_FP + cost_FN)With equal costs that is 0.5, which is where the default comes from. With a miss ten times as expensive as a false alarm it is 1/11, about 0.09 — and a default of 0.5 is then five times too strict.
That formula is worth carrying around. It gives a defensible starting threshold from two numbers a domain expert can usually supply, without any tuning at all.
When you cannot price the errors
Often nobody can put a number on a miss. Three usable fallbacks:
Maximise F1, or F-beta if one error matters more. This is the least opinionated option and it is what most people do.
Fix an operating constraint. "The team can investigate 200 cases a day" sets the threshold directly: take the top 200 scores. "Precision must be at least 80%" does the same from the other side. These are often more honest than a metric, because they describe what will actually happen.
Choose from the curve. Plot precision and recall against threshold and pick the point where the trade-off turns. It is a judgement, but an informed one.
Two things that must be right
Tune on validation data. Choosing the threshold on the test set and then reporting test performance is the same category of error as tuning hyperparameters there — the reported number includes the choice you made to maximise it.
Do not confuse this with calibration. Threshold tuning takes the scores as given and picks a cut. Calibration changes the scores so that a score of 0.8 really means 80% of such cases are positive.
They are independent. A badly calibrated model can rank perfectly, and if the ranking is right, a tuned threshold works fine. But if the threshold is being derived from the cost formula above, the *p* in it must mean an actual probability — and then calibration matters. Platt scaling and isotonic regression are the usual tools.
The threshold the costs imply
The optimal cut is not a matter of taste; it drops out of the two error costs. Flagging a case with predicted probability p is worth it when (1 - p) · cost_FP < p · cost_FN, which rearranges to a threshold of cost_FP / (cost_FP + cost_FN):
| a miss costs | a false alarm costs | optimal threshold |
|---|---|---|
| 20 | 1 (screening) | 0.048 |
| 1 | 1 (symmetric) | 0.500 |
| 1 | 50 (auto-suspension) | 0.980 |
The familiar 0.5 is just the middle row — the special case where the two mistakes cost the same. Move to a screening problem, where missing a sick patient is twenty times worse than a needless second look, and the right cut falls to 0.048: flag almost anything, because flags are cheap and misses are not, and the default of 0.5 is then more than ten times too strict. Reverse the costs and it climbs to 0.98. The formula needs only two numbers a domain expert can usually supply, and it gives a defensible threshold with no tuning at all — but the p in it must be an actual probability, which is why a cost-derived threshold is exactly the case where [calibration](probability_calibration.html) stops being optional.
When nobody can price the errors, three fallbacks are honest. Maximise F1 (or F-beta if one error matters more), which is the least opinionated option. Fix an operating constraint — "the team can investigate 200 cases a day" sets the threshold directly as the top 200 scores, and often describes reality better than any metric. Or read the precision-recall trade-off off the curve and pick the knee. Whichever you use, tune it on validation data, never the test set: choosing the threshold to maximise a number and then reporting that number folds the choice into the result, exactly as tuning hyperparameters on the test set does.
Where it goes wrong
Reporting metrics at 0.5 and stopping. The most common evaluation mistake after using accuracy.
Tuning on the test set. Optimistic by exactly the amount you gained.
Retuning per fold and averaging. The thresholds are not comparable across folds. Tune once on a held-out validation set.
Forgetting it drifts. The optimal threshold depends on the class balance, and that moves in production. It needs rechecking, not setting once.
0.5 is a default, not a decision
The threshold is the one hyperparameter you can change after training, in production, in a config file. Here is what it costs and what it buys.
Check yourself
0 of …Answer without scrolling back up.
Where does the 0.5 default threshold come from?
If a miss costs ten times a false alarm, roughly what threshold does the cost formula give?
How does threshold tuning differ from calibration?
Threshold Tuning
It comes from the library. predict() calls predict_proba() and cuts at 0.5, because a default has to be something and 0.5 is the least arbitrary-looking choice available.