Evaluation: train vs eval mode and metrics

eval() and no_grad() do different jobs

model.train() and model.eval() set a flag that changes how certain layers behave. Dropout is active in training and does nothing in eval. BatchNorm uses the current batch's statistics in training and its stored running averages in eval. Forgetting eval() makes validation scores noisy and usually worse; forgetting to switch back with train() makes the next epoch train without dropout.

model.eval() does not turn off gradient tracking. Combine it with torch.no_grad() or torch.inference_mode() to save memory and time. They are separate switches for separate problems.

Split your data three ways. Train on the training set, choose hyperparameters and stopping points on the validation set, and measure the test set once at the end; if you tune on the test set, its score no longer estimates real-world performance. Accuracy is argmax of logits compared with labels. For imbalanced classes, also report precision, recall and F1, available in libraries such as scikit-learn and torchmetrics.

A reusable evaluate function

Note the weighting by batch size, so a smaller final batch does not distort the average, and the switch back to training mode at the end.

import torch

def evaluate(model, loader, loss_fn, device):
    model.eval()
    total_loss, correct, count = 0.0, 0, 0
    with torch.inference_mode():
        for x, y in loader:
            x, y = x.to(device), y.to(device)
            logits = model(x)
            total_loss += loss_fn(logits, y).item() * y.size(0)
            correct += (logits.argmax(dim=1) == y).sum().item()
            count += y.size(0)
    model.train()
    return total_loss / count, correct / count

Practice sessions and the board exam

In practice sessions a teacher may randomly cover parts of your notes to make you stronger (dropout) and grade you against today's class average (batch statistics). In the board exam nothing is hidden and you are judged against a fixed standard. model.eval() is walking into the exam hall.

Do not tune on the test set

If you change the architecture or learning rate because the test score went down, the test set has quietly become a validation set. Keep a separate validation split, or use cross-validation for small datasets, and touch the test set only for the final number you report.

त्वरित जाँच: What does calling `model.eval()` change?

• The behaviour of layers such as Dropout and BatchNorm
• It disables gradient tracking for all operations
• It moves the model to the CPU
• It freezes the parameters so the optimizer cannot change them