What PyTorch is and why people use it

Three pieces, one library

PyTorch is an open-source Python library for numerical computing and deep learning. It gives you three things that fit together. Tensors are n-dimensional arrays, like NumPy arrays, that can live on a CPU or a GPU. Autograd records the operations you perform on tensors and can then compute gradients automatically, which is what training needs. torch.nn and torch.optim provide ready-made layers, loss functions and update rules so you do not write matrix calculus by hand.

PyTorch runs in eager mode: each line executes immediately when Python reaches it, exactly like ordinary Python. You can print a tensor in the middle of a model, step through it with a debugger, and use normal if and for statements. The trade-off is some Python overhead per operation; torch.compile (covered later) can recover much of that speed. Libraries such as torchvision, torchaudio and Hugging Face Transformers are built on top of PyTorch, so learning it unlocks a large ecosystem.

The whole course in one picture

Every PyTorch project follows the same cycle. Data arrives as tensors, the model turns them into predictions, the loss scores those predictions, autograd computes gradients, and the optimizer nudges the weights. The sections of this course walk through these stages in order.

A first look at a tensor

Tensors behave like NumPy arrays. The comments show what each line produces. @ is matrix multiplication, and .item() turns a one-element tensor into a plain Python number.

import torch

x = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
print(x.shape)        # torch.Size([2, 2])
print(x.dtype)        # torch.float32
print(x @ x)          # matrix product: [[7, 10], [15, 22]]
print(x.sum().item()) # 10.0, a plain Python float

A ledger that remembers

Think of a tensor with gradient tracking as a CA's ledger. Every entry records which earlier entries it was computed from. When the final balance looks wrong, the CA can trace backwards through the ledger to see how much each original entry contributed. Autograd does the same backward trace for your loss.

Quick check: What does it mean that PyTorch runs in eager mode by default?

• You must compile the whole model into a static graph before running it
• Operations are queued and run only when you call `torch.run()`
• Each operation executes immediately as Python reaches it, so you can print and debug normally
• Gradients are computed eagerly before the forward pass