# Chat Models and Chat Templates — Hugging Face Transformers

Source: https://www.skillbyai.com/en/hugging-face/hf-chat-templates

> Format multi-turn conversations correctly with apply_chat_template and the chat-aware pipeline.

## Each chat model has its own prompt format

Instruction-tuned chat models were trained on conversations wrapped in special markers that separate the **system**, **user** and **assistant** turns. The exact markers differ between model families, and using the wrong format quietly degrades answers. Instead of hand-writing these strings, store the conversation as a list of messages, each a dictionary with a `role` and `content`, and call `tokenizer.apply_chat_template()`. It uses a Jinja template saved with the tokenizer to render the correct string. Pass `add_generation_prompt=True` so the template ends with the marker that tells the model "now the assistant speaks"; without it the model may continue the user's message instead. The text-generation pipeline also accepts a list of messages directly, applies the template for you and returns the conversation with the new assistant turn appended. Base (non-chat) checkpoints usually have no chat template, so they should be prompted with plain text.

## Rendering a conversation with the model's template

`return_dict=True` gives input_ids and attention_mask together, ready for generate().

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Qwen/Qwen2.5-0.5B-Instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto")

messages = [
    {"role": "system", "content": "You are a concise tutor for engineering students."},
    {"role": "user", "content": "Explain overfitting in one sentence."},
]
print(tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True))

inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
                                 return_dict=True, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=60)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

## The chat-aware pipeline

Give the pipeline messages; it returns the whole conversation with the assistant's reply as the last item.

```python
from transformers import pipeline

chat = pipeline("text-generation", model="Qwen/Qwen2.5-0.5B-Instruct")
messages = [{"role": "user", "content": "Give me two tips for a viva exam."}]
result = chat(messages, max_new_tokens=80)
print(result[0]["generated_text"][-1]["content"])
```

**Quiz:** What does `add_generation_prompt=True` add to a rendered chat prompt?

- [ ] A system message with safety rules
- [ ] The model's end-of-sequence token
- [x] The marker that starts the assistant's turn, so the model replies instead of continuing the user
- [ ] A random seed for sampling

*Answer:* The marker that starts the assistant's turn, so the model replies instead of continuing the user. It appends the assistant-turn header defined by the template, cueing the model to answer.