Choosing the Decision Threshold

0.5 is only a default

Changing the probability threshold moves the balance between precision and recall. Lowering it flags more cases: recall rises, precision falls. Pick the threshold from the business trade-off, for example the lowest threshold that keeps precision above 0.8 so investigators are not flooded, and choose it on validation data, not the test set. Report the threshold with the metrics, because "the model" is really the model plus its threshold.

Precision and recall at four thresholds, run

I ran this with Python 3, numpy 2.5.3 and scikit-learn 1.9.1, using fixed random seeds. On the same imbalanced data, threshold 0.5 gives precision 0.978 and recall 0.830; 0.15 gives 0.825 and 0.887; 0.05 catches 0.906 of positives but precision falls to 0.432.

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score, recall_score
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=4000, weights=[0.95], random_state=1)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=0, stratify=y)
prob = LogisticRegression().fit(X_tr, y_tr).predict_proba(X_te)[:, 1]
print("threshold | precision | recall")
for t in [0.5, 0.3, 0.15, 0.05]:
    pred = (prob >= t).astype(int)
    print(f"{t:>9} | {precision_score(y_te, pred):>9.3f} | {recall_score(y_te, pred):.3f}")

Output:

threshold | precision | recall
      0.5 |     0.978 | 0.830
      0.3 |     0.939 | 0.868
     0.15 |     0.825 | 0.887
     0.05 |     0.432 | 0.906

Tune thresholds on validation data

Choosing the threshold on the test set leaks information and inflates the final result.

त्वरित जाँच: What usually happens when you lower the classification threshold?

• Both always rise
• Recall rises and precision falls
• Precision rises and recall falls
• Nothing changes