# Citation Precision and Recall — RAG Retrieval & Evaluation

Source: https://www.skillbyai.com/en/rag-evaluation/a-cite

> Do the cited sources really support the answer, and are the supporting sources cited?

## Two sides of citation quality

When answers cite passage ids, measure **citation precision** (of the cited passages, how many really support the claims) and **citation recall** (of the passages needed to support the answer, how many were cited). Low precision means decorative citations users cannot trust; low recall means claims without visible support. Check basic validity in code too: every cited id must exist in the context that was sent. Answers to unanswerable questions should cite nothing and say not found.

## Citation precision and recall over four answers, run

I ran this with Python 3 (numpy 2.5.3 and scikit-learn 1.9.1 where imported) on small made-up data, with fixed seeds where random. Across four answers, 3 of 4 cited documents truly support the answer (precision 0.75) and 3 of 5 needed documents were cited (recall 0.60). The last answer cited nothing.

```python
cases = [  # (cited doc ids, gold supporting doc ids)
    ({"d1"}, {"d1"}),
    ({"d1", "d4"}, {"d1"}),
    ({"d2"}, {"d2", "d3"}),
    (set(), {"d5"}),
]
tp = fp = fn = 0
for cited, gold in cases:
    tp += len(cited & gold); fp += len(cited - gold); fn += len(gold - cited)
print(f"citation precision = {tp / (tp + fp):.2f}  (cited docs that really support the answer)")
print(f"citation recall    = {tp / (tp + fn):.2f}  (supporting docs that were cited)")
```

Output:

```
citation precision = 0.75  (cited docs that really support the answer)
citation recall    = 0.60  (supporting docs that were cited)
```

## Validate ids in code

Reject or flag any answer citing an id that was not in the context; it is a fabricated citation.

**Quiz:** Low citation precision means what?

- [ ] The answer was too short
- [ ] Too few sources were cited
- [x] Many cited sources do not actually support the answer
- [ ] Retrieval recall is perfect

*Answer:* Many cited sources do not actually support the answer. Precision is about the citations given.