RAG Retrieval & Evaluation Interview Questions Hindi mein

RAG Retrieval & Evaluation कोर्स के 25 सवाल, हर सवाल के सही जवाब और explanation के साथ। जवाब देखने से पहले ख़ुद सोचें।

Why Evaluate Retrieval Separately

1. An answer is wrong and the relevant passage never appeared in the top-k. Which stage failed?

  1. The content was missing
  2. Generation
  3. Retrieval
  4. Output formatting
जवाब देखें

जवाब: Retrieval

The model could not use text it never received.

पाठ: Retrieval Errors Versus Generation Errors

2. Why write relevance rules before labelling?

  1. Without shared rules, labels disagree and metrics become noisy
  2. Rules make the retriever faster
  3. Metrics do not use labels
  4. Rules replace the evaluation set
जवाब देखें

जवाब: Without shared rules, labels disagree and metrics become noisy

Metrics are only as consistent as the labels.

पाठ: What Counts as Relevant

3. What is a known weakness of model-generated synthetic questions?

  1. They require no review because they are perfect
  2. They are always wrong
  3. They cannot be stored
  4. They often reuse passage wording, making retrieval look easier than with real queries
जवाब देखें

जवाब: They often reuse passage wording, making retrieval look easier than with real queries

Mix in and prefer real queries.

पाठ: Building an Evaluation Set

Retrieval Metrics

4. Which metric tells you how early the first relevant passage appears?

  1. Recall@k
  2. Mean reciprocal rank (MRR)
  3. Precision@k
  4. Total tokens
जवाब देखें

जवाब: Mean reciprocal rank (MRR)

MRR rewards a relevant result at rank 1.

पाठ: Recall@k, Precision@k, Hit Rate and MRR

5. What does nDCG capture that recall@k does not?

  1. Whether the answer is fluent
  2. Whether the best passages are ranked near the top
  3. The latency of the retriever
  4. The number of documents in the index
जवाब देखें

जवाब: Whether the best passages are ranked near the top

nDCG discounts gains lower in the list.

पाठ: Graded Relevance and nDCG

6. In a retrieve-then-rerank pipeline, what should the first stage be measured on?

  1. Precision@1 of the first stage
  2. Only nDCG@1
  3. Answer fluency
  4. Recall over the full candidate pool passed to the reranker
जवाब देखें

जवाब: Recall over the full candidate pool passed to the reranker

The reranker can only reorder what stage one found.

पाठ: Choosing k and the Primary Metric

7. Why track context precision alongside recall?

  1. Recall can be bought by sending many irrelevant passages that add cost and distraction
  2. Precision measures latency
  3. Recall is always 1
  4. Precision is required by the API
जवाब देखें

जवाब: Recall can be bought by sending many irrelevant passages that add cost and distraction

Watch both sides of the trade-off.

पाठ: Context Precision and Noise

Is the Difference Real?

8. Roughly how much does quadrupling the number of queries shrink the interval?

  1. It does not change it
  2. It divides it by four
  3. It halves it
  4. It doubles it
जवाब देखें

जवाब: It halves it

Width scales with one over the square root of n.

पाठ: How Many Queries Do You Need?

9. Why compare systems on the same queries (paired)?

  1. It doubles the dataset size
  2. It makes both systems faster
  3. Shared difficulty cancels out, so smaller real differences become detectable
  4. It removes the need for labels
जवाब देखें

जवाब: Shared difficulty cancels out, so smaller real differences become detectable

Pairing reduces noise.

पाठ: Paired Comparisons and the Bootstrap

10. Why report metrics per slice?

  1. An average can hide a segment that performs badly
  2. Slices always have more queries
  3. It makes the overall score higher
  4. Slices remove sampling error
जवाब देखें

जवाब: An average can hide a segment that performs badly

Look at the distribution, not only the mean.

पाठ: Slices: Averages Hide Weak Spots

Experiments on Retrieval Design

11. Why compare retrievers on your own evaluation set rather than a public leaderboard?

  1. Retrievers cannot be compared
  2. Leaderboards are always fake
  3. Your set has no labels
  4. Your queries and documents differ from benchmark data
जवाब देखें

जवाब: Your queries and documents differ from benchmark data

Measure on the distribution you serve.

पाठ: Comparing Retrievers on Your Queries

12. What can go wrong with very small chunks?

  1. They always send too many tokens
  2. The matched text can be separated from the sentence holding the answer
  3. They make indexing impossible
  4. They remove the need for embeddings
जवाब देखें

जवाब: The matched text can be separated from the sentence holding the answer

Retrieval matches one chunk; the answer must be inside it.

पाठ: Chunk Size Experiments

13. When should an added retrieval stage be kept?

  1. Always, more stages are better
  2. When it improves the primary metric beyond noise at acceptable latency and cost
  3. Only if it is the newest technique
  4. When it lowers recall
जवाब देखें

जवाब: When it improves the primary metric beyond noise at acceptable latency and cost

Ablations show what each stage contributes.

पाठ: Evaluating Rerankers, Hybrid Search and Query Rewriting

Evaluating Generated Answers

14. Where does token F1 work poorly?

  1. Single names
  2. Short dates
  3. Exact numbers
  4. Long, freely worded answers where correct paraphrases share few words with the reference
जवाब देखें

जवाब: Long, freely worded answers where correct paraphrases share few words with the reference

Overlap is a proxy, not understanding.

पाठ: Reference-Based Metrics: Exact Match and Token F1

15. What does faithfulness measure?

  1. Whether the answer's claims are supported by the retrieved context
  2. Whether the answer is long enough
  3. Whether retrieval was fast
  4. Whether the user liked the tone
जवाब देखें

जवाब: Whether the answer's claims are supported by the retrieved context

Groundedness is about the context, not the world.

पाठ: Faithfulness: Is Every Claim Supported?

16. Low citation precision means what?

  1. The answer was too short
  2. Too few sources were cited
  3. Many cited sources do not actually support the answer
  4. Retrieval recall is perfect
जवाब देखें

जवाब: Many cited sources do not actually support the answer

Precision is about the citations given.

पाठ: Citation Precision and Recall

LLM Judges You Can Trust

17. Which judge design is more reliable?

  1. Letting the generator grade itself unchecked
  2. A single 1-10 overall quality score with no rubric
  3. One narrow criterion per call with a rubric and structured verdict
  4. No evidence given to the judge
जवाब देखें

जवाब: One narrow criterion per call with a rubric and structured verdict

Narrow, evidence-based questions give consistent grades.

पाठ: Designing a Judge Prompt

18. Why can 84% raw agreement be unimpressive?

  1. When most answers are good, a lenient judge agrees often by chance
  2. 84% is always perfect
  3. Agreement cannot be measured
  4. Kappa is always higher than agreement
जवाब देखें

जवाब: When most answers are good, a lenient judge agrees often by chance

Kappa and failure recall tell the real story.

पाठ: Calibrating a Judge Against Humans

19. How do you reduce position bias in pairwise judging?

  1. Remove the rubric
  2. Always put your system first
  3. Use longer answers
  4. Judge both orders and count only consistent verdicts
जवाब देखें

जवाब: Judge both orders and count only consistent verdicts

Order-swapping exposes and cancels position effects.

पाठ: Judge Biases and Mitigations

Evaluation in the Release Cycle and in Production

20. Why did the example gate block the release despite better recall?

  1. Recall improved too much
  2. Faithfulness regressed beyond its allowed tolerance
  3. Latency got worse
  4. Cost fell
जवाब देखें

जवाब: Faithfulness regressed beyond its allowed tolerance

Guardrail metrics matter as much as the headline.

पाठ: Evaluation as a Release Gate

21. Why are thumbs-up/down ratings not a precise quality score?

  1. Ratings are always positive
  2. Few users rate and those who do are not representative
  3. Ratings cannot be stored
  4. They measure latency
जवाब देखें

जवाब: Few users rate and those who do are not representative

Use them for trends and failure discovery.

पाठ: Online Signals and A/B Tests

22. What can cause retrieval quality to drop with no code change?

  1. Adding more tests
  2. Nothing, quality is fixed
  3. Reading the logs
  4. New or changed documents and new kinds of user queries
जवाब देखें

जवाब: New or changed documents and new kinds of user queries

Monitor the data, not only the code.

पाठ: Drift and Index Freshness

Putting Evaluation Into Practice

23. What does failure analysis add to metrics?

  1. The causes of failures, so you can fix the most common one first
  2. A higher score automatically
  3. Faster retrieval
  4. More documents
जवाब देखें

जवाब: The causes of failures, so you can fix the most common one first

Counts by cause turn numbers into actions.

पाठ: Failure Analysis With a Taxonomy

24. In the case study, what came right after measuring the baseline?

  1. Deleting the Hindi queries
  2. Switching to the newest model immediately
  3. Failure analysis to find the biggest cause
  4. Skipping evaluation
जवाब देखें

जवाब: Failure analysis to find the biggest cause

Diagnose before changing.

पाठ: Case Study: Evaluating a Policy Assistant

25. Which item belongs on a RAG evaluation checklist?

  1. Reporting averages without sample size
  2. Only reading the final answer by eye
  3. Measuring retrieval separately at the k actually sent to the model
  4. Letting the generator grade itself unchecked
जवाब देखें

जवाब: Measuring retrieval separately at the k actually sent to the model

Separate, quantified, repeatable measurement.

पाठ: A RAG Evaluation Checklist

← कोर्स पर वापस जाएँ