Always Beat a Baseline

Dummy models and simple rules

Before celebrating a metric, compare it with a baseline: predicting the mean (regression), the most frequent class (classification), last week's value (forecasting), or the current business rule. scikit-learn provides DummyRegressor and DummyClassifier. If your model barely beats the baseline, it may not be worth deploying; if it beats it massively, check for leakage. The baseline also gives non-specialists a reference point: 20% lower error than the current rule is a clear message.

Baselines by problem type

Pick one before modelling.

regression       DummyRegressor(strategy="mean")      or the current business estimate
classification   DummyClassifier(strategy="most_frequent")
forecasting      "same as last period" / seasonal naive
ranking          popularity order
any problem      the existing rule the business uses today

Report the lift

State results as improvement over the baseline, for example MAE 45 versus 58 for the mean.

Quick check: Why compare a model with a baseline?

• It trains the model faster
• Baselines are always better
• To know whether it actually adds value over something simple
• It removes the need for a test set