# Common Pitfalls and How to Spot Them — Hugging Face Transformers

Source: https://www.skillbyai.com/en/hugging-face/hf-pitfalls

> Recognise the classic transformers bugs from their symptoms and fix them quickly.

## Bugs that produce no error

The worst transformers bugs run without errors and just give bad results. **Mismatched tokenizer**: loading a tokenizer from one checkpoint and the model from another scrambles every id; always load both from the same id or folder. **Silent truncation**: with `truncation=True`, text beyond the limit is dropped, so a label that depends on the end of a long document becomes unlearnable; check how many examples exceed `max_length`. **Wrong padding side** for decoder generation leads to odd outputs after pad tokens; use left padding. **Missing pad token** on many decoder tokenizers raises an error when batching; setting `pad_token = eos_token` is a common fix. **Label mapping drift**: if `id2label` does not match your dataset's integers, metrics look fine but predictions are named wrongly. **Data leakage**: near-duplicate rows shared between train and test inflate scores. **Ignoring the chat template** for instruct models degrades answers. A habit that catches most of these: decode a few tokenized examples back to text and eyeball them before training.

## A five-line sanity check before training

Look at real model inputs, not your assumptions about them.

```python
sample = train_ds[0]
print(tok.decode(sample["input_ids"]))                  # matches the raw text?
print(model.config.id2label, sample["labels"])          # label names line up?
lengths = [len(x) for x in train_ds["input_ids"]]
print(sum(l >= 256 for l in lengths), "of", len(lengths), "examples hit max_length")
```

## Checking the question paper set

Before an exam, invigilators check that every student got the right paper set. Decoding your tokenized inputs is the same check: a few seconds of looking prevents hours of grading the wrong answers.

**Quiz:** A fine-tuned classifier scores 99% on the test set but performs poorly on new user data. Which pitfall is the most likely cause?

- [ ] Using safetensors instead of pickle
- [ ] Setting padding_side to left
- [ ] Running inference in bfloat16
- [x] Duplicate or near-duplicate examples shared between train and test splits

*Answer:* Duplicate or near-duplicate examples shared between train and test splits. Leakage between splits makes the test set measure memorisation. Deduplicate before splitting.