Deep Learning & Neural Networks interview questions

25 questions from the Deep Learning & Neural Networks course, each with the correct answer and an explanation. Try to answer before you open the answer.

Neural Network Foundations

1. What is a key advantage of deep learning over classic ML?

  1. It is always cheaper
  2. It never needs data
  3. It learns useful features directly from raw data
  4. It works best on tiny tables
Show answer

Answer: It learns useful features directly from raw data

Representation learning is the core idea.

Lesson: What Deep Learning Is

2. Why are activation functions needed?

  1. They make the network non-linear; without them stacked layers equal one linear layer
  2. They store the training data
  3. They speed up disk access
  4. They replace the weights
Show answer

Answer: They make the network non-linear; without them stacked layers equal one linear layer

Non-linearity gives networks their power.

Lesson: Neurons and Activation Functions

3. Why can't a single linear layer solve XOR?

  1. XOR needs images
  2. XOR has too much data
  3. Linear layers have no weights
  4. XOR's classes cannot be separated by a straight line
Show answer

Answer: XOR's classes cannot be separated by a straight line

Hidden non-linear layers bend the boundary.

Lesson: Why Hidden Layers Matter: The XOR Problem

How Networks Learn

4. Which loss is standard for multi-class classification?

  1. Mean squared error on class ids
  2. Cross-entropy on the logits
  3. Accuracy
  4. Number of parameters
Show answer

Answer: Cross-entropy on the logits

Cross-entropy penalises low probability on the true class.

Lesson: Loss Functions

5. What does backpropagation compute?

  1. The final accuracy
  2. The gradient of the loss with respect to every weight, via the chain rule
  3. The dataset size
  4. The best learning rate
Show answer

Answer: The gradient of the loss with respect to every weight, via the chain rule

Gradients tell each weight which way to move.

Lesson: Gradient Descent and Backpropagation

6. Why call optimizer.zero_grad() each step?

  1. It loads new data
  2. It saves the model
  3. It sets the learning rate
  4. PyTorch accumulates gradients by default
Show answer

Answer: PyTorch accumulates gradients by default

Clear old gradients before computing new ones.

Lesson: Automatic Differentiation in PyTorch

7. What does momentum add to SGD?

  1. A running direction that smooths noisy gradients
  2. A larger dataset
  3. Random weight resets
  4. A second loss function
Show answer

Answer: A running direction that smooths noisy gradients

Momentum accelerates along consistent directions.

Lesson: Optimisers: SGD, Momentum and Adam

The PyTorch Workflow

8. What shape convention do PyTorch image layers expect?

  1. (channels, batch)
  2. (height, width) only
  3. (batch, channels, height, width)
  4. (pixels, labels)
Show answer

Answer: (batch, channels, height, width)

NCHW is the default layout.

Lesson: Tensors and Shapes

9. What is the correct order inside a training step?

  1. backward, zero_grad, step, forward
  2. step, backward, forward, loss
  3. zero_grad, forward, loss, backward, step
  4. forward, step, zero_grad, backward
Show answer

Answer: zero_grad, forward, loss, backward, step

Clear, compute, differentiate, update.

Lesson: Modules and the Training Loop

10. What is an epoch?

  1. One full pass over the training data
  2. One weight update
  3. One layer
  4. One GPU
Show answer

Answer: One full pass over the training data

Many mini-batch steps make up an epoch.

Lesson: Mini-Batches, Epochs and Devices

Making Training Work

11. How do residual connections help deep networks train?

  1. They freeze the first layer
  2. They remove all activations
  3. They reduce the dataset
  4. They give gradients a direct path around each block
Show answer

Answer: They give gradients a direct path around each block

Skip connections keep gradients flowing.

Lesson: Vanishing Gradients, Normalisation and Residual Connections

12. When is dropout active?

  1. Only during evaluation
  2. During training only; it is disabled in eval mode
  3. Never
  4. Only when saving the model
Show answer

Answer: During training only; it is disabled in eval mode

model.eval() turns dropout off.

Lesson: Regularisation: Dropout, Weight Decay and Augmentation

13. What does early stopping keep?

  1. The last checkpoint always
  2. The checkpoint with the best validation score
  3. The first checkpoint
  4. Only the optimiser state
Show answer

Answer: The checkpoint with the best validation score

Stop when validation stops improving and keep the best.

Lesson: Learning-Rate Schedules and Early Stopping

Convolutional Neural Networks

14. What does a convolutional filter produce?

  1. A new dataset
  2. A single class label
  3. A loss value
  4. A feature map showing where its pattern appears
Show answer

Answer: A feature map showing where its pattern appears

Filters respond to local patterns.

Lesson: The Convolution Operation

15. What does max pooling do?

  1. Keeps the strongest value in each window, shrinking the feature map
  2. Adds more pixels
  3. Normalises the labels
  4. Computes the loss
Show answer

Answer: Keeps the strongest value in each window, shrinking the feature map

Pooling downsamples and adds shift tolerance.

Lesson: Building a Small CNN

16. Why does a convolution layer have few parameters?

  1. It only processes one pixel
  2. It has no weights
  3. The same small filter is shared across all image positions
  4. It stores the dataset
Show answer

Answer: The same small filter is shared across all image positions

Weight sharing is the key.

Lesson: Why Convolutions Are Efficient

Sequences, Embeddings and Attention

17. What is an embedding layer?

  1. A data loader
  2. A loss function
  3. A learnable lookup table from ids to dense vectors
  4. A type of pooling
Show answer

Answer: A learnable lookup table from ids to dense vectors

Embeddings turn discrete ids into learnable vectors.

Lesson: Embeddings: Tokens as Vectors

18. Why did transformers replace RNNs for most language tasks?

  1. They process tokens in parallel and handle long-range context better
  2. RNNs cannot learn at all
  3. Transformers need no data
  4. RNNs only work on images
Show answer

Answer: They process tokens in parallel and handle long-range context better

Parallel attention scales better.

Lesson: Recurrent Networks and Their Limits

19. What do attention weights in each row sum to?

  1. The model size
  2. 0
  3. The sequence length
  4. 1, because of the softmax
Show answer

Answer: 1, because of the softmax

They form a weighted average over values.

Lesson: Self-Attention

Transformers, Transfer Learning and Generative Models

20. Why do transformers need positional information?

  1. Attention only works on images
  2. Self-attention by itself does not know token order
  3. To reduce parameters
  4. Because embeddings are random
Show answer

Answer: Self-attention by itself does not know token order

Positions are added to the token vectors.

Lesson: The Transformer Block

21. Why is transfer learning usually preferred over training from scratch?

  1. It never needs a validation set
  2. It needs much less data and compute because the model already has useful features
  3. It makes models smaller automatically
  4. Pretrained models cannot be changed
Show answer

Answer: It needs much less data and compute because the model already has useful features

Start from what the model already knows.

Lesson: Transfer Learning and Fine-Tuning

22. How do autoregressive language models generate text?

  1. By sorting the vocabulary
  2. By denoising an image
  3. By clustering documents
  4. By predicting the next token repeatedly
Show answer

Answer: By predicting the next token repeatedly

Next-token prediction, one step at a time.

Lesson: Generative Models at a Glance

Debugging, Deploying and a Checklist

23. What does failing to overfit a single small batch suggest?

  1. A bug in the model, loss or training step
  2. The dataset is too large
  3. The model is perfect
  4. You need more epochs on all data
Show answer

Answer: A bug in the model, loss or training step

A working setup can memorise a few examples.

Lesson: Debugging Neural Networks

24. Why call model.eval() before inference?

  1. It saves the model to disk
  2. It trains the model faster
  3. It disables dropout and uses stored batch-norm statistics for consistent outputs
  4. It increases the learning rate
Show answer

Answer: It disables dropout and uses stored batch-norm statistics for consistent outputs

Training-mode behaviour makes predictions noisy.

Lesson: Saving, Loading and Serving Models

25. Which item belongs on a deep learning checklist?

  1. Run inference in training mode
  2. Skip validation curves
  3. Verify the model can overfit a single batch
  4. Tune on the test set
Show answer

Answer: Verify the model can overfit a single batch

Sanity checks catch bugs early.

Lesson: A Deep Learning Project Checklist

← Back to the course