Prompts and Models as Versioned Configuration

Treat prompts like code

Prompts, model names, temperatures, retrieval settings and safety thresholds change behaviour as much as code does. Store them as versioned configuration with an id, review changes, run the evaluation set on every change, and roll them out through the same flags and stages as code. Log the config version with every request so you can tie complaints to a specific prompt or model. Pin provider model versions where possible so behaviour does not shift without your knowledge.

A versioned AI config record

Every field that changes behaviour is in one reviewed file.

feature: ai-summary
config_version: 2026-09-28.3
model: <provider model id, pinned version>
temperature: 0.2
max_output_tokens: 400
prompt_template: prompts/summary_v7.txt
retrieval: {top_k: 5, min_score: 0.35}
safety: {pii_redaction: on, refusal_check: on}
eval_result: pass 0.874, slices ok, red-team blockers 0 (run 2026-09-28)

Log the config version per request

When a user reports a bad answer, the config version tells you exactly which prompt and model produced it.

Quick check: Why version prompts and model settings like code?

• Versioning makes models free
• Prompts never affect output
• They change behaviour, so they need review, evaluation and rollback
• Providers require it