Common interview and exam questions

Questions about tensors and autograd

What is the difference between view and reshape? view returns a tensor sharing memory and fails if the layout does not allow it; reshape returns a view when it can and copies otherwise.

Why call optimizer.zero_grad()? Because backward() adds to .grad. Without clearing, each step would use the sum of all past gradients.

What is a leaf tensor? One created directly by the user (or a layer parameter), not as the result of an operation. Only leaves with requires_grad=True keep .grad by default.

detach() versus no_grad()? detach() cuts one tensor out of the graph; no_grad() stops recording for a whole block of code.

What does "dynamic graph" mean? The graph is built during each forward pass as Python runs, so models can use ordinary control flow and standard debuggers.

Why float32 and not float64? It is faster on GPUs, uses half the memory, and deep learning rarely needs the extra precision.

How to answer well

Give the definition, then the why, then a one-line example or a pitfall. "Gradients accumulate, so we zero them; accumulation is also used on purpose to simulate larger batches" scores better than the bare rule. If you do not know something, say how you would find out, for example by checking the shapes or the documentation.

Questions about models and training

model.eval() versus torch.no_grad()? eval() changes Dropout and BatchNorm behaviour; no_grad() stops gradient tracking. Evaluation uses both.

Why no softmax before CrossEntropyLoss? It applies log_softmax internally; adding softmax double-applies it and weakens gradients.

What does BatchNorm do? It normalises activations using batch statistics while training and stored running averages at evaluation, which stabilises and speeds up training.

How do you handle overfitting? More data or augmentation, dropout, weight decay, early stopping on validation loss, or a smaller model or pretrained backbone.

How do you resume training? Save and restore the model, optimizer and scheduler state dicts plus the epoch number.

What is transfer learning? Starting from pretrained weights, replacing the head, then training only the head or fine-tuning with a small learning rate.

How do you use multiple GPUs? DistributedDataParallel runs one process per GPU and averages gradients; it is generally preferred over the older DataParallel.

The viva examiner's follow-up

Interviewers behave like a viva examiner in a college lab: your first answer is just the opening. If you say "we zero gradients", expect "what if we don't?" and "when would you not?". Preparing the follow-up is what turns a memorised line into understanding.

त्वरित जाँच: An interviewer asks why `CrossEntropyLoss` should receive raw logits. Which answer is correct?

• It applies log_softmax internally, so an extra softmax distorts the loss and weakens gradients
• Logits are always positive, which the loss requires
• Softmax outputs cannot be moved to the GPU
• Raw logits make the loss independent of the batch size