Lesson 25 / 25

A Context Engineering Checklist

Review any LLM feature with these questions.

Ten questions before shipping

Can you see the exact rendered prompt? Is there a token budget with reserved output and a trimming order? Are retrieved chunks thresholded, de-duplicated, filtered for freshness and access rights? Are the most relevant items placed where they are used reliably, with the question last? Are content types delimited and escaped? Is history bounded and are key facts pinned? Are large tool outputs truncated or paged? Is the prefix stable for caching? Is structured output validated? Are versions logged and changes tested with ablations?

The checklist

Use it in design reviews.

[ ] rendered prompt is logged and viewable
[ ] token budget: output reserved, trim order defined, overflow tested
[ ] retrieval: min score, dedupe, current versions, ACL enforced
[ ] order: best content at the edges, question last, rules reminder
[ ] delimiters + escaping for every untrusted block
[ ] history: window + summary, pinned constraints
[ ] tool output: truncated / paged, marked untrusted
[ ] cache-friendly: stable prefix, cache hits monitored
[ ] structured output validated, stop reason checked
[ ] versions logged; changes tested with ablations

Revisit after model upgrades

A new model can change how well it uses long context; rerun your ablations when you switch.

Quick check: When should you rerun context ablations?

  • Only when the UI changes
  • Never, results are permanent
  • After switching or upgrading the model, and after major prompt changes
  • Only once a year regardless of changes
Answer

After switching or upgrading the model, and after major prompt changes — Context behaviour depends on the model.