MLOps interview questions

25 questions from the MLOps course, each with the correct answer and an explanation. Try to answer before you open the answer.

Why MLOps Exists

1. Which is a failure mode specific to ML systems?

  1. A missing semicolon
  2. A syntax error in Python
  3. Input data changes so the model becomes less accurate with no code error
  4. A server running out of disk
Show answer

Answer: Input data changes so the model becomes less accurate with no code error

ML systems can fail silently through data.

Lesson: How ML Systems Fail in Production

2. What closes the MLOps loop after deployment?

  1. Monitoring that triggers retraining or rollback
  2. Deleting the training data
  3. Writing a slide deck
  4. Turning off logging
Show answer

Answer: Monitoring that triggers retraining or rollback

Monitoring feeds the next iteration.

Lesson: The MLOps Lifecycle

3. What characterises level 0 (manual) MLOps?

  1. A model registry with gates
  2. Fully automated retraining
  3. Canary releases with alerts
  4. Notebooks and hand-copied model files with no tracking
Show answer

Answer: Notebooks and hand-copied model files with no tracking

Manual processes are fragile and hard to reproduce.

Lesson: Maturity Levels and Roles

Experiment Tracking and Reproducibility

4. What should an experiment tracker record for each run?

  1. Only the final accuracy
  2. Parameters, metrics, artefacts, and code and data versions
  3. Only the model file name
  4. Nothing until deployment
Show answer

Answer: Parameters, metrics, artefacts, and code and data versions

Complete records make results reproducible.

Lesson: Experiment Tracking With MLflow

5. Which is NOT needed to reproduce a training run?

  1. The data version
  2. The name of the person who watched the dashboard
  3. The code commit
  4. The random seed
Show answer

Answer: The name of the person who watched the dashboard

Data, code, config, environment and seeds define a run.

Lesson: Reproducible Training

6. Why include preprocessing in the packaged model?

  1. To hide the model
  2. To make the file smaller
  3. Because metrics require it
  4. So serving applies exactly the same transformations as training
Show answer

Answer: So serving applies exactly the same transformations as training

Packaging the whole pipeline prevents skew.

Lesson: Packaging Models With Metadata

Data in Production

7. Which is a typical data validation check?

  1. Values within allowed ranges and categories
  2. Model accuracy on the test set
  3. Number of engineers on call
  4. GPU temperature
Show answer

Answer: Values within allowed ranges and categories

Check the data before trusting predictions.

Lesson: Data Validation

8. What is the most robust way to prevent skew in preprocessing?

  1. Recompute statistics on each request batch
  2. Re-implement it separately in the serving language
  3. Package the preprocessing with the model so serving uses the same code
  4. Skip preprocessing in production
Show answer

Answer: Package the preprocessing with the model so serving uses the same code

One implementation, used in both places.

Lesson: Training-Serving Skew

9. What does a point-in-time join prevent?

  1. Slow queries
  2. Using too few features
  3. Using feature values that were not yet known at the label's time
  4. Duplicate model versions
Show answer

Answer: Using feature values that were not yet known at the label's time

Time-travel correctness avoids leakage.

Lesson: Feature Stores and Point-in-Time Correctness

Model Registry and Continuous Integration

10. Why load production models by alias rather than a file path?

  1. Promotion and rollback become pointer changes without code changes
  2. Aliases make models more accurate
  3. File paths are illegal
  4. Aliases compress the model
Show answer

Answer: Promotion and rollback become pointer changes without code changes

The registry decides which version is live.

Lesson: Model Registry, Versions and Aliases

11. What is an invariance test for a model?

  1. Checking the number of rows
  2. Checking that training finishes
  3. Checking the GPU type
  4. Checking that an irrelevant change in input does not change the prediction
Show answer

Answer: Checking that an irrelevant change in input does not change the prediction

Behavioural tests catch unwanted dependencies.

Lesson: Testing ML Code, Data and Models

12. Why compare candidate and champion on the same data?

  1. It doubles the dataset
  2. So differences reflect the models, not the data
  3. It makes training faster
  4. It is required by Python
Show answer

Answer: So differences reflect the models, not the data

Fair comparisons use identical evaluation sets.

Lesson: Automated Evaluation Gates

Deploying Models Safely

13. Which serving pattern fits nightly churn scores loaded into a CRM?

  1. Online API with 10 ms latency
  2. Batch scoring
  3. On-device inference
  4. Reinforcement learning
Show answer

Answer: Batch scoring

Freshness of a day is enough here.

Lesson: Batch, Online and Streaming Serving

14. Why report p95 or p99 latency instead of the average?

  1. p99 measures accuracy
  2. Averages are always higher
  3. Percentiles are faster to compute
  4. Users experience the slow tail, which averages hide
Show answer

Answer: Users experience the slow tail, which averages hide

Tail latency defines user experience.

Lesson: Serving Performance: Latency and Batching

15. What can a shadow deployment NOT measure?

  1. How users react to the new model's predictions
  2. Latency on real traffic
  3. Disagreement with the live model
  4. Errors from missing features
Show answer

Answer: How users react to the new model's predictions

Users never see shadow predictions.

Lesson: Shadow Deployment

16. What is a canary release?

  1. Running models only offline
  2. Deploying to all users at once
  3. Sending a small share of traffic to a new version and expanding only if metrics hold
  4. Deleting the old version first
Show answer

Answer: Sending a small share of traffic to a new version and expanding only if metrics hold

Gradual exposure limits the blast radius.

Lesson: Canary Releases and A/B Tests

Monitoring Models in Production

17. Which is a model-layer signal?

  1. Disk space
  2. CPU usage
  3. The share of positive predictions over time
  4. Number of deploys
Show answer

Answer: The share of positive predictions over time

Prediction distributions reveal model changes early.

Lesson: What to Monitor

18. Why not alert on KS p-values alone with large samples?

  1. Tiny, harmless shifts become statistically significant
  2. p-values cannot be computed on large data
  3. KS only works on images
  4. Large samples hide all drift
Show answer

Answer: Tiny, harmless shifts become statistically significant

Use effect sizes and model impact as well.

Lesson: Detecting Data Drift

19. What can you monitor before true labels arrive?

  1. The training loss
  2. Final accuracy only
  3. Nothing at all
  4. Input drift, prediction distributions and early business signals
Show answer

Answer: Input drift, prediction distributions and early business signals

Proxies bridge the label delay.

Lesson: Delayed Labels and Proxy Metrics

Retraining, Rollback and Incidents

20. What is concept drift?

  1. A bug in the logging code
  2. A change in the relationship between inputs and the target
  3. A faster GPU
  4. A new model registry
Show answer

Answer: A change in the relationship between inputs and the target

The rules the model learned no longer hold.

Lesson: Concept Drift and Model Decay

21. Why must automated retraining still pass an evaluation gate?

  1. Gates make training faster
  2. Training on bad or shifted data can produce a worse model
  3. Retrained models are always better
  4. It is required by the GPU
Show answer

Answer: Training on bad or shifted data can produce a worse model

Automation needs guardrails.

Lesson: Retraining Pipelines and Triggers

22. What is the first priority in a model incident?

  1. Wait for the weekly review
  2. Retrain a bigger model immediately
  3. Delete the logs
  4. Mitigate impact, for example by rolling back or using a fallback
Show answer

Answer: Mitigate impact, for example by rolling back or using a fallback

Stop the harm first, then diagnose.

Lesson: Rollback and Incident Response

Governance, LLMOps and a Checklist

23. What does lineage let you answer?

  1. Which data, code and approvals produced the model behind a prediction
  2. Which colour the dashboard uses
  3. How many GPUs exist
  4. The CEO's schedule
Show answer

Answer: Which data, code and approvals produced the model behind a prediction

Traceability supports audits and debugging.

Lesson: Governance: Lineage, Model Cards and Access

24. What is often the main versioned artefact in LLM applications?

  1. The training dataset of the provider
  2. Only GPU drivers
  3. Prompts, retrieval settings and the chosen model id
  4. The office floor plan
Show answer

Answer: Prompts, retrieval settings and the chosen model id

Version what you control.

Lesson: How LLMOps Differs

25. Which item belongs on an MLOps readiness checklist?

  1. Recomputing scaling statistics on each request
  2. Emailing model files to the ops team
  3. Alias-based serving from a registry with an evaluation gate
  4. Skipping monitoring for small models
Show answer

Answer: Alias-based serving from a registry with an evaluation gate

Repeatable, gated, monitored releases.

Lesson: An MLOps Readiness Checklist

← Back to the course