# Saving, Loading and Serving Models — Deep Learning & Neural Networks

Source: https://www.skillbyai.com/en/deep-learning/p-serve

> state_dict, eval mode and no_grad.

## Inference is different from training

Save model weights with `torch.save(model.state_dict(), path)` and load them into the same architecture with `load_state_dict`. For inference, always call `model.eval()` (turns off dropout and makes batch norm use its stored statistics) and wrap predictions in `torch.no_grad()` (or `torch.inference_mode()`) to save memory and time. Record the preprocessing, library versions and class labels with the weights. For production, consider exporting (for example to ONNX or TorchScript-style formats) or using a serving framework, and monitor inputs for drift.

## Why eval mode matters, run

I ran this on CPU with Python 3, PyTorch 2.14.1, numpy 2.5.3 and scikit-learn 1.9.1, with fixed seeds and one thread. After saving and reloading a model with dropout, two calls on the same input give different outputs in training mode, and identical outputs after model.eval().

```python
import torch, os, tempfile
torch.manual_seed(0)
model = torch.nn.Sequential(torch.nn.Linear(4, 8), torch.nn.ReLU(), torch.nn.Dropout(0.5), torch.nn.Linear(8, 2))
path = os.path.join(tempfile.mkdtemp(), "model.pt")
torch.save(model.state_dict(), path)
restored = torch.nn.Sequential(torch.nn.Linear(4, 8), torch.nn.ReLU(), torch.nn.Dropout(0.5), torch.nn.Linear(8, 2))
restored.load_state_dict(torch.load(path))
x = torch.randn(1, 4)
restored.train()
a, b = restored(x), restored(x)
print("train mode, two calls equal:", torch.allclose(a, b))
restored.eval()
with torch.no_grad():
    a, b = restored(x), restored(x)
print("eval mode,  two calls equal:", torch.allclose(a, b))
```

Output:

```
train mode, two calls equal: False
eval mode,  two calls equal: True
```

## Save a checkpoint bundle

Store weights, optimiser state, epoch, metrics, preprocessing settings and the code version together so training can resume and results can be reproduced.

**Quiz:** Why call model.eval() before inference?

- [ ] It saves the model to disk
- [ ] It trains the model faster
- [x] It disables dropout and uses stored batch-norm statistics for consistent outputs
- [ ] It increases the learning rate

*Answer:* It disables dropout and uses stored batch-norm statistics for consistent outputs. Training-mode behaviour makes predictions noisy.