# Evaluation: train vs eval mode and metrics — PyTorch

Source: https://www.skillbyai.com/en/pytorch/pt-evaluation

> Write an evaluation function that uses eval mode and inference_mode together, and use train, validation and test splits correctly.

## eval() and no_grad() do different jobs

`model.train()` and `model.eval()` set a flag that changes how certain layers behave. **Dropout** is active in training and does nothing in eval. **BatchNorm** uses the current batch's statistics in training and its stored running averages in eval. Forgetting `eval()` makes validation scores noisy and usually worse; forgetting to switch back with `train()` makes the next epoch train without dropout.

`model.eval()` does **not** turn off gradient tracking. Combine it with `torch.no_grad()` or `torch.inference_mode()` to save memory and time. They are separate switches for separate problems.

Split your data three ways. Train on the **training** set, choose hyperparameters and stopping points on the **validation** set, and measure the **test** set once at the end; if you tune on the test set, its score no longer estimates real-world performance. Accuracy is `argmax` of logits compared with labels. For imbalanced classes, also report precision, recall and F1, available in libraries such as scikit-learn and torchmetrics.

## A reusable evaluate function

Note the weighting by batch size, so a smaller final batch does not distort the average, and the switch back to training mode at the end.

```python
import torch

def evaluate(model, loader, loss_fn, device):
    model.eval()
    total_loss, correct, count = 0.0, 0, 0
    with torch.inference_mode():
        for x, y in loader:
            x, y = x.to(device), y.to(device)
            logits = model(x)
            total_loss += loss_fn(logits, y).item() * y.size(0)
            correct += (logits.argmax(dim=1) == y).sum().item()
            count += y.size(0)
    model.train()
    return total_loss / count, correct / count
```

## Practice sessions and the board exam

In practice sessions a teacher may randomly cover parts of your notes to make you stronger (dropout) and grade you against today's class average (batch statistics). In the board exam nothing is hidden and you are judged against a fixed standard. `model.eval()` is walking into the exam hall.

## Do not tune on the test set

If you change the architecture or learning rate because the test score went down, the test set has quietly become a validation set. Keep a separate validation split, or use cross-validation for small datasets, and touch the test set only for the final number you report.

**Quiz:** What does calling `model.eval()` change?

- [x] The behaviour of layers such as Dropout and BatchNorm
- [ ] It disables gradient tracking for all operations
- [ ] It moves the model to the CPU
- [ ] It freezes the parameters so the optimizer cannot change them

*Answer:* The behaviour of layers such as Dropout and BatchNorm. `eval()` flips the training flag that Dropout and BatchNorm read. Gradient tracking needs `no_grad()` or `inference_mode()` separately.