Supervised Fine-Tuning (SFT) in Plain Terms

Show examples, nudge weights, repeat

Supervised fine-tuning (SFT) trains a pretrained model on pairs of input and desired output. For each example the model predicts the answer tokens, a loss measures the distance from the target, backpropagation computes each weight's contribution, and an optimiser (such as Adam) nudges weights to reduce the loss. This repeats over batches for one or a few epochs. Because the model starts with strong general ability it needs far fewer examples than training from scratch, but big steps or poor data can distort what it knew. The loss is usually computed only on the assistant answer tokens.

Small steps on a pretrained model

Fine-tuning continues gradient descent on your examples, ideally changing only a small part of the weights.

The training loop in one picture

Conceptual, not tied to one library.

for epoch in 1..E:
    for batch in shuffled(training_examples):
        predictions = model(batch.input)                        # forward pass
        loss = cross_entropy(predictions, batch.answer_tokens)  # only the answer part counts
        gradients = backward(loss)                              # how each weight affects the loss
        weights -= learning_rate * adjust(gradients)            # small step (Adam etc.)
    check validation loss on held-out examples                  # stop early if it stops improving

Mask the prompt

Train on the answer tokens only; otherwise the model also spends capacity learning to imitate user prompts.

Quick check: What does SFT optimise?

• The price per token
• The length of the user prompt
• The weights, to reduce loss on desired example outputs
• The GPU fan speed