PyTorch

What you'll learn

• Create, index, reshape and broadcast tensors while keeping track of shape, dtype and device.
• Explain how autograd records operations and computes gradients, and control it with no_grad, inference_mode and detach.
• Build models with nn.Module, choose the right layers and loss functions, and write a correct training and evaluation loop.
• Feed models with Dataset, DataLoader and torchvision transforms, and apply transfer learning to real tasks.
• Speed up training with GPUs, mixed precision and torch.compile, and make runs reproducible.
• Save, load, debug, export and serve models safely, and answer common PyTorch interview questions.

Syllabus

Mental model and setup

1. What PyTorch is and why people use it
2. Installing PyTorch and choosing a device
3. The training loop idea, without PyTorch

Tensor foundations

1. Creating tensors: shape, dtype and device
2. Tensor operations and reductions
3. Indexing, views and reshaping
4. Broadcasting rules and the silent shape bug

Autograd: automatic gradients

1. requires_grad, backward and .grad
2. The dynamic computation graph
3. no_grad, inference_mode, detach and zeroing gradients

Building models with torch.nn

1. nn.Module: the shape of every model
2. Common layers: Linear, Conv2d, activations and Sequential
3. Loss functions and the logits rule

Training: optimizers, the loop and evaluation

1. Optimizers and learning-rate schedules
2. The canonical training loop
3. Evaluation: train vs eval mode and metrics

Data pipelines: Dataset, DataLoader and transforms

1. Writing a custom Dataset
2. DataLoader: batching, shuffling and workers
3. Image transforms and augmentation

Real-world models

1. A convolutional image classifier
2. Transfer learning with pretrained models
3. Embeddings and sequence models

GPUs and performance

1. Devices: moving models and data to the GPU
2. Mixed precision with autocast and GradScaler
3. torch.compile and a faster input pipeline

Saving, debugging, security and deployment

1. Saving and loading with state_dict
2. Debugging training: shapes, NaNs and silent failures
3. Reproducibility and randomness
4. Exporting and serving models

Revision and interview prep

1. PyTorch cheat sheet
2. Common interview and exam questions
3. Self-check: spot the bugs
4. Mixed self-check

Related courses

• Deep Learning (25 lessons)
• AI (25 lessons)
• Airflow (25 lessons)
• Fine-tuning (27 lessons)