Anatomy of a Model Request

Six building blocks

A typical request has: (1) system instructions (role, rules, format); (2) tool definitions (names, descriptions, parameter schemas, which also cost tokens); (3) examples (few-shot demonstrations); (4) conversation history; (5) retrieved or injected content (documents, database rows, tool results); and (6) the current user request. Each block has a different owner, change rate and trust level: instructions are yours and stable, documents change per request, user input and tool output are untrusted. Treat them as separate components with their own budget, versioning and tests.

The blocks at a glance

Owner, change rate and trust per block.

block              owner        changes          trust
system rules       you          per release      trusted
tool definitions   you          per release      trusted
examples           you          per release / per request (selected)   trusted
history            users+model  every turn       mixed
documents/results  data, tools  every request    UNTRUSTED content
user request       user         every request    UNTRUSTED

Count tool schemas too

Twenty verbose tool definitions can take thousands of tokens on every call; include only the tools the current step needs.

Quick check: Which block should be treated as untrusted?

• Your fixed examples
• Your own reviewed system rules
• Your tool definitions
• Retrieved documents and tool results