1. An answer is wrong and the relevant passage never appeared in the top-k. Which stage failed?
- The content was missing
- Generation
- Retrieval
- Output formatting
जवाब देखें
जवाब: Retrieval
The model could not use text it never received.
RAG Retrieval & Evaluation कोर्स के 25 सवाल, हर सवाल के सही जवाब और explanation के साथ। जवाब देखने से पहले ख़ुद सोचें।
जवाब: Retrieval
The model could not use text it never received.
जवाब: Without shared rules, labels disagree and metrics become noisy
Metrics are only as consistent as the labels.
जवाब: They often reuse passage wording, making retrieval look easier than with real queries
Mix in and prefer real queries.
जवाब: Mean reciprocal rank (MRR)
MRR rewards a relevant result at rank 1.
जवाब: Whether the best passages are ranked near the top
nDCG discounts gains lower in the list.
जवाब: Recall over the full candidate pool passed to the reranker
The reranker can only reorder what stage one found.
जवाब: Recall can be bought by sending many irrelevant passages that add cost and distraction
Watch both sides of the trade-off.
जवाब: It halves it
Width scales with one over the square root of n.
जवाब: Shared difficulty cancels out, so smaller real differences become detectable
Pairing reduces noise.
जवाब: An average can hide a segment that performs badly
Look at the distribution, not only the mean.
जवाब: Your queries and documents differ from benchmark data
Measure on the distribution you serve.
जवाब: The matched text can be separated from the sentence holding the answer
Retrieval matches one chunk; the answer must be inside it.
जवाब: When it improves the primary metric beyond noise at acceptable latency and cost
Ablations show what each stage contributes.
पाठ: Evaluating Rerankers, Hybrid Search and Query Rewriting
जवाब: Long, freely worded answers where correct paraphrases share few words with the reference
Overlap is a proxy, not understanding.
जवाब: Whether the answer's claims are supported by the retrieved context
Groundedness is about the context, not the world.
जवाब: Many cited sources do not actually support the answer
Precision is about the citations given.
जवाब: One narrow criterion per call with a rubric and structured verdict
Narrow, evidence-based questions give consistent grades.
जवाब: When most answers are good, a lenient judge agrees often by chance
Kappa and failure recall tell the real story.
जवाब: Judge both orders and count only consistent verdicts
Order-swapping exposes and cancels position effects.
जवाब: Faithfulness regressed beyond its allowed tolerance
Guardrail metrics matter as much as the headline.
जवाब: Few users rate and those who do are not representative
Use them for trends and failure discovery.
जवाब: New or changed documents and new kinds of user queries
Monitor the data, not only the code.
जवाब: The causes of failures, so you can fix the most common one first
Counts by cause turn numbers into actions.
जवाब: Failure analysis to find the biggest cause
Diagnose before changing.
जवाब: Measuring retrieval separately at the k actually sent to the model
Separate, quantified, repeatable measurement.