# Context Precision and Noise — RAG Retrieval & Evaluation

Source: https://www.skillbyai.com/en/rag-evaluation/m-noise

> Irrelevant passages are not free.

## Distractors cost tokens and accuracy

High recall achieved by sending many passages has a price: **irrelevant passages** add tokens, latency and cost, and can **distract** the model into using wrong but plausible text. Track **context precision** (share of sent passages that are relevant) or simply the number of tokens per query beside recall. For questions without an answer in the corpus, high-scoring irrelevant passages are especially dangerous because they invite a confident wrong answer; a relevance threshold and a not-found path help.

## Searching a stack of papers

Handing someone the right page together with twenty wrong ones makes them slower and occasionally makes them quote the wrong page.

## Plot recall against tokens

For each setting of k or chunk size, plot recall against average context tokens and choose the knee of the curve.

**Quiz:** Why track context precision alongside recall?

- [x] Recall can be bought by sending many irrelevant passages that add cost and distraction
- [ ] Precision measures latency
- [ ] Recall is always 1
- [ ] Precision is required by the API

*Answer:* Recall can be bought by sending many irrelevant passages that add cost and distraction. Watch both sides of the trade-off.