Common Interview Questions

Questions and model answers

"How would you debug a RAG bot that gives wrong answers?" Open traces for failing cases; check whether the right chunk was retrieved. If not, fix retrieval (chunking, embeddings, filters, top_k); if yes, fix the prompt or model. Add the cases to a dataset. "How do you know a prompt change helped?" Run both versions on the same dataset with evaluators, compare per example, then canary with variant tags and compare feedback, scores, latency and cost. "How do you control LLM costs?" Attribute cost per feature and user from traces, then trim prompts and history, cache, route easy requests to cheaper models, and cap agent steps. "What are the risks of logging prompts?" Personal data and secrets; mask in the SDK, minimise, restrict access, set retention. "Why is the trace for my background job split?" Context was not propagated. Answer with a method, not a list of tool names.

Structure beats buzzwords

Interviewers listen for a process: observe (traces), hypothesise (which stage), verify (dataset experiment), ship safely (canary), monitor (dashboards and alerts). Mentioning Langfuse or LangSmith is fine, but explaining the loop is what scores.

A detective's method

A good detective does not name a suspect first. They collect evidence at the scene, form a theory, test it, and only then make an arrest. Your interview answer should follow the same order: evidence from traces, a hypothesis, a test, then the fix.

त्वरित जाँच: An interviewer asks how you would reduce the cost of a chat feature. What is the best first step?

• Switch every request to the smallest model available
• Use traces to attribute cost by feature, step and token type to find where money actually goes
• Turn off tracing to save money
• Shorten every answer to one sentence