Chat Templates and Special Tokens

Messages are flattened into special-token text

A model does not natively understand a list of {role, content} messages. During instruct fine-tuning it saw conversations written in one specific text layout with special tokens marking where each turn starts and ends, for example tokens like <|start_header_id|> in Llama 3 or <|im_start|> in ChatML-style models. The chat template is the code that converts your message list into that exact layout, then appends the header for the assistant's turn so the model knows to reply. GGUF files usually embed the original template (written in Jinja) in their metadata. Ollama ships its own equivalent written in Go's template language, visible with ollama show --template, and llama.cpp can apply the embedded Jinja template when you pass --jinja. If the template is wrong, the model sees an unfamiliar format: it may ignore the system prompt, talk to itself, or never emit its end-of-turn token.

Seeing the template and a few-shot MESSAGE

MESSAGE lines add example turns that are sent before the real conversation.

ollama show llama3.2 --template     # Go template using .System, .Messages, etc.

cat > Modelfile <<'EOF'
FROM llama3.2
SYSTEM Reply with only the sentiment: positive, negative or neutral.
MESSAGE user The canteen food was amazing today
MESSAGE assistant positive
MESSAGE user The bus was late again
MESSAGE assistant negative
EOF
ollama create sentiment -f Modelfile

Symptom checklist for a template bug

Output that continues with fake "User:" lines, endless generation, or visible tokens like <|eot_id|> in the text almost always means a template or stop-token mismatch, not a bad model. Compare against the template published with the original model.

त्वरित जाँच: A freshly imported GGUF keeps generating fake "User:" turns after its answer. What is the most likely cause?

• The quantization level is too high
• The GPU is overheating
• The model has too many parameters
• A wrong chat template or missing stop token