Context Precision and Noise

Distractors cost tokens and accuracy

High recall achieved by sending many passages has a price: irrelevant passages add tokens, latency and cost, and can distract the model into using wrong but plausible text. Track context precision (share of sent passages that are relevant) or simply the number of tokens per query beside recall. For questions without an answer in the corpus, high-scoring irrelevant passages are especially dangerous because they invite a confident wrong answer; a relevance threshold and a not-found path help.

Searching a stack of papers

Handing someone the right page together with twenty wrong ones makes them slower and occasionally makes them quote the wrong page.

Plot recall against tokens

For each setting of k or chunk size, plot recall against average context tokens and choose the knee of the curve.

त्वरित जाँच: Why track context precision alongside recall?

• Recall can be bought by sending many irrelevant passages that add cost and distraction
• Precision measures latency
• Recall is always 1
• Precision is required by the API