Cheat Sheet: The Whole Course on One Page

Everything in one place

Ecosystem: the Hub hosts versioned repos; transformers runs models; tokenizers, datasets, accelerate and peft handle text, data, devices and adapters. Pipelines: pipeline(task, model=...) = preprocess, forward, postprocess; add device, dtype and batch_size for speed. Tokenizers: subword algorithms (BPE, WordPiece, Unigram); outputs input_ids and attention_mask; padding=True, truncation=True, return_tensors="pt"; left padding for decoder generation. Models: config + body + head; pick AutoModelFor... by task; logits -> softmax -> argmax -> id2label. Generation: autoregressive loop with KV cache; max_new_tokens; greedy, beam, or sampling with temperature, top-k, top-p; apply_chat_template(..., add_generation_prompt=True) for chat models. Data: load_dataset, batched map, train_test_split(seed=...), collator pads dynamically. Training: Trainer + TrainingArguments, labels give loss, compute_metrics on validation, test once. Efficiency: LoRA adapters, 4-bit NF4, QLoRA, merge for serving. Production: load once, batch, smaller models, safetensors, pinned revisions, no hard-coded tokens.

How to revise

Revise in three passes: recall the ideas, practise with code, then test yourself without notes.

The ten lines worth memorising

If you can write this from memory and explain each line, you understand the core of the library.

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "distilbert/distilbert-base-uncased-finetuned-sst-2-english"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
inputs = tok(["Placement season starts next week!"], padding=True,
             truncation=True, return_tensors="pt")
with torch.inference_mode():
    probs = model(**inputs).logits.softmax(dim=-1)
print(model.config.id2label[int(probs.argmax(dim=-1)[0])])

Explain, do not recite

Interviewers rarely want argument names. They want the why: why subwords, why a causal mask, why LoRA saves memory, why pin revisions. For each line of the cheat sheet, practise one sentence of reasoning.

Quick check: Which pair correctly matches a task to its Auto class?

• Translation with T5 -> AutoModelForCausalLM
• Chat with a decoder-only model -> AutoModelForCausalLM
• Sentence classification -> AutoModelForTokenClassification
• Named-entity recognition -> AutoModelForSeq2SeqLM