# Sampling Parameters: Temperature, top_p and Friends — Local LLMs with Ollama & llama.cpp

Source: https://www.skillbyai.com/en/local-llms/ll-params-sampling

> Tune temperature, top_k, top_p, repeat penalty, seed and num_predict for predictable or creative output.

## How the next token is chosen

After each forward pass the model outputs a score (a **logit**) for every token. The **sampler** turns those into a choice. **Temperature** divides the logits before softmax: low values (0.1 to 0.3) sharpen the distribution so the top token almost always wins, which suits extraction and code; higher values (0.8 to 1.2) flatten it for varied, creative text. At 0 most runtimes simply pick the top token (greedy). **top_k** keeps only the k most likely tokens; **top_p** (nucleus) keeps the smallest set whose probabilities add up to p; **min_p** drops tokens far less likely than the best one. **repeat_penalty** discourages recently used tokens to reduce loops. **seed** fixes the random generator so the same prompt and settings repeat output on the same build and hardware. **num_predict** caps generated tokens, and **stop** ends generation at given strings.

## What temperature does to probabilities

Three candidate next tokens after "The capital of India is New", with made-up logits.

```python
import math

def softmax(logits, temperature):
    scaled = [x / temperature for x in logits]
    m = max(scaled)
    exps = [math.exp(x - m) for x in scaled]
    total = sum(exps)
    return [e / total for e in exps]

tokens = ["Delhi", "Mumbai", "Pune"]
logits = [4.0, 3.0, 1.0]
for t in (0.2, 0.8, 1.5):
    probs = softmax(logits, t)
    print(f"T={t}: " + ", ".join(f"{w}={p:.2f}" for w, p in zip(tokens, probs)))
```

Output:

```
T=0.2: Delhi=0.99, Mumbai=0.01, Pune=0.00
T=0.8: Delhi=0.76, Mumbai=0.22, Pune=0.02
T=1.5: Delhi=0.61, Mumbai=0.31, Pune=0.08
```

## A quiz-show contestant

Low temperature is a contestant who always says the answer they are most sure of. High temperature is one who sometimes gambles on the second or third guess. For a factual MCQ you want the first; for writing a poem the second is more fun.

## Change one knob at a time

Beginners set temperature, top_k, top_p and min_p all at once and cannot tell which helped. Start from the model's defaults, change temperature first, and fix a seed while comparing prompts so differences come from your change, not randomness.

**Quiz:** For extracting fields from invoices as JSON, which setting is most appropriate?

- [ ] Temperature around 1.5 for variety
- [ ] A very high repeat_penalty
- [x] A low temperature such as 0 to 0.2
- [ ] top_k of 1000

*Answer:* A low temperature such as 0 to 0.2. Extraction wants deterministic, most-likely tokens. Low temperature (or greedy decoding) reduces random variation and format errors.