# Padding, Truncation and Batches — Hugging Face Transformers

Source: https://www.skillbyai.com/en/hugging-face/hf-padding

> Build fixed-shape batches with padding and truncation and pick the right padding side.

## Why batches need padding and truncation

A tensor must be rectangular, but sentences have different lengths. **Padding** appends a special pad id to shorter sequences so every row in the batch has the same length, and sets the matching `attention_mask` entries to 0. `padding=True` (or `"longest"`) pads to the longest sequence in the current batch, which is usually the most efficient. `padding="max_length"` pads everything to `max_length`, which wastes compute but gives fixed shapes that some hardware prefers. **Truncation** cuts sequences longer than a limit: `truncation=True` cuts to `max_length` or to the model's maximum input length. Without truncation, a long document can exceed the model's context and raise an error or produce silent problems. Pass `return_tensors="pt"` to get PyTorch tensors that can go straight into the model. For tasks that need the whole long document, split it into overlapping windows instead of truncating, so no evidence is thrown away.

## Padding by hand

This is exactly what `padding=True` does: pad to the longest row and record the mask.

```python
# What padding=True does to a batch of token-id lists
batch = [[101, 7592, 102], [101, 2129, 2024, 2017, 102]]
pad_id = 0
longest = max(len(ids) for ids in batch)

input_ids = [ids + [pad_id] * (longest - len(ids)) for ids in batch]
attention_mask = [[1] * len(ids) + [0] * (longest - len(ids)) for ids in batch]

for ids, mask in zip(input_ids, attention_mask):
    print(ids, mask)
```

Output:

```
[101, 7592, 102, 0, 0] [1, 1, 1, 0, 0]
[101, 2129, 2024, 2017, 102] [1, 1, 1, 1, 1]
```

## The same thing with a real tokenizer

One call pads, truncates and returns PyTorch tensors ready for the model.

```python
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("google-bert/bert-base-uncased")
batch = tok(
    ["Hi!", "How are you doing this semester?"],
    padding=True,          # pad to the longest in this batch
    truncation=True,       # never exceed the model's max length
    max_length=32,
    return_tensors="pt",
)
print(batch["input_ids"].shape)       # (2, length of the longer sentence)
print(batch["attention_mask"])
```

## Decoder models generate with left padding

For batched *generation* with decoder-only models, set `tokenizer.padding_side = "left"` so every prompt ends at the last position, where the next token is appended. Many decoder tokenizers also have no pad token; a common fix is `tokenizer.pad_token = tokenizer.eos_token`.

**Quiz:** You tokenize a batch with `padding=True`. What length does each row end up with?

- [ ] The model's maximum context length
- [ ] A fixed 512 tokens
- [ ] The length of the shortest sequence in the batch
- [x] The length of the longest sequence in the batch

*Answer:* The length of the longest sequence in the batch. padding=True means padding="longest": rows are padded to the longest sequence in that batch. Use padding="max_length" for a fixed size.