# Common interview and exam questions — PyTorch

Source: https://www.skillbyai.com/en/pytorch/pt-interview-questions

> Answer the PyTorch questions that come up most often in interviews and viva exams, with the reasoning behind each answer.

## Questions about tensors and autograd

**What is the difference between `view` and `reshape`?** `view` returns a tensor sharing memory and fails if the layout does not allow it; `reshape` returns a view when it can and copies otherwise.

**Why call `optimizer.zero_grad()`?** Because `backward()` adds to `.grad`. Without clearing, each step would use the sum of all past gradients.

**What is a leaf tensor?** One created directly by the user (or a layer parameter), not as the result of an operation. Only leaves with `requires_grad=True` keep `.grad` by default.

**`detach()` versus `no_grad()`?** `detach()` cuts one tensor out of the graph; `no_grad()` stops recording for a whole block of code.

**What does "dynamic graph" mean?** The graph is built during each forward pass as Python runs, so models can use ordinary control flow and standard debuggers.

**Why `float32` and not `float64`?** It is faster on GPUs, uses half the memory, and deep learning rarely needs the extra precision.

## How to answer well

Give the definition, then the *why*, then a one-line example or a pitfall. "Gradients accumulate, so we zero them; accumulation is also used on purpose to simulate larger batches" scores better than the bare rule. If you do not know something, say how you would find out, for example by checking the shapes or the documentation.

## Questions about models and training

**`model.eval()` versus `torch.no_grad()`?** `eval()` changes Dropout and BatchNorm behaviour; `no_grad()` stops gradient tracking. Evaluation uses both.

**Why no softmax before `CrossEntropyLoss`?** It applies `log_softmax` internally; adding softmax double-applies it and weakens gradients.

**What does BatchNorm do?** It normalises activations using batch statistics while training and stored running averages at evaluation, which stabilises and speeds up training.

**How do you handle overfitting?** More data or augmentation, dropout, weight decay, early stopping on validation loss, or a smaller model or pretrained backbone.

**How do you resume training?** Save and restore the model, optimizer and scheduler state dicts plus the epoch number.

**What is transfer learning?** Starting from pretrained weights, replacing the head, then training only the head or fine-tuning with a small learning rate.

**How do you use multiple GPUs?** `DistributedDataParallel` runs one process per GPU and averages gradients; it is generally preferred over the older `DataParallel`.

## The viva examiner's follow-up

Interviewers behave like a viva examiner in a college lab: your first answer is just the opening. If you say "we zero gradients", expect "what if we don't?" and "when would you not?". Preparing the follow-up is what turns a memorised line into understanding.

**Quiz:** An interviewer asks why `CrossEntropyLoss` should receive raw logits. Which answer is correct?

- [x] It applies log_softmax internally, so an extra softmax distorts the loss and weakens gradients
- [ ] Logits are always positive, which the loss requires
- [ ] Softmax outputs cannot be moved to the GPU
- [ ] Raw logits make the loss independent of the batch size

*Answer:* It applies log_softmax internally, so an extra softmax distorts the loss and weakens gradients. CrossEntropyLoss combines log_softmax and negative log-likelihood. Feeding probabilities applies softmax twice.