A Prompt Structure Checklist

Task, format, constraints, examples, edge cases

Well-performing prompts usually state the task clearly, define the output format, give constraints (what to avoid, length limits, sources to use), include an example where the format or style is subtle, say what to do when information is missing, and delimit inserted input so it is not mistaken for instructions. A simple lint checklist cannot judge quality, but it flags prompts that are missing obvious parts, and it runs in milliseconds on every edit.

Lint before you run

Cheap checks on the prompt text catch missing structure and ambiguity before any model call.

Linting three prompt versions, run

I ran this with Python 3 (scipy 1.18.1 where imported) on example or seeded simulated data, not results from a real product. A regex-based checklist scores "Summarize this." 1 out of 6, a version with bullet format, a constraint and delimiters 4 out of 6, and a version with a JSON schema, a missing-information rule and an example 6 out of 6. The checklist flags gaps; output tests still decide quality.

import re
CHECKS = {
    "states the task":        lambda p: bool(re.search(r"\b(summari[sz]e|classify|extract|write|answer|translate)\b", p, re.I)),
    "defines output format":  lambda p: bool(re.search(r"\b(json|bullet|table|format|schema|one word|label)\b", p, re.I)),
    "gives constraints":      lambda p: bool(re.search(r"\b(only|must|do not|never|at most|under \d+)\b", p, re.I)),
    "has an example":         lambda p: "example" in p.lower() or "input:" in p.lower(),
    "handles missing info":   lambda p: bool(re.search(r"(not (found|in the)|unknown|if .* (missing|absent))", p, re.I)),
    "marks input delimiters": lambda p: bool(re.search(r"<\w+>|\"\"\"|```", p)),
}
prompts = {
    "v1": "Summarize this.",
    "v2": "Summarize the support ticket below in 3 bullet points for an agent. Do not invent details.\n<ticket>{ticket}</ticket>",
    "v3": ("Summarize the support ticket below for an agent. Output JSON: {\"issue\": str, \"customer_ask\": str, "
           "\"urgency\": \"low|medium|high\"}. Use only facts in the ticket; if the ask is unknown write \"unknown\".\n"
           "Example input: <ticket>App crashes on login since update</ticket>\n<ticket>{ticket}</ticket>"),
}
for name, p in prompts.items():
    passed = [c for c, f in CHECKS.items() if f(p)]
    print(f"{name}: {len(passed)}/{len(CHECKS)}  missing: {[c for c in CHECKS if c not in passed]}")

Output:

v1: 1/6  missing: ['defines output format', 'gives constraints', 'has an example', 'handles missing info', 'marks input delimiters']
v2: 4/6  missing: ['has an example', 'handles missing info']
v3: 6/6  missing: []

Run lint in the editor or CI

Add the checklist to pre-commit or CI for prompt files so gaps are caught before anyone runs an evaluation.

त्वरित जाँच: What can a static prompt checklist NOT tell you?

• Whether an output format is specified
• Whether the outputs are actually correct
• Whether input is delimited
• Whether constraints are stated