# Running a Model by Hand — Hugging Face Transformers

Source: https://www.skillbyai.com/en/hugging-face/hf-model-outputs

> Call a model directly, read its ModelOutput and convert logits into probabilities.

## From tokens to probabilities without a pipeline

Running a model yourself means repeating the pipeline's three steps explicitly. First tokenize with `return_tensors="pt"`. Then call `model(**inputs)`; the double star unpacks `input_ids` and `attention_mask` as keyword arguments. The result is a **ModelOutput**, an object you can read by attribute (`outputs.logits`) or by key. **Logits** are raw, unnormalised scores, one per class; they can be any real number. To get probabilities, apply **softmax** along the last dimension, and to get a prediction take the **argmax** and look it up in `model.config.id2label`. Two habits matter at inference time. Call `model.eval()` so layers such as dropout behave deterministically (models loaded with `from_pretrained` already start in eval mode, but training code may switch it). And wrap the call in `torch.no_grad()` or `torch.inference_mode()` so PyTorch does not store the activations needed for gradients, which saves memory and time. Pass `output_hidden_states=True` when you need every layer's vectors.

## Manual inference with PyTorch

The same result a pipeline gives, with every step visible.

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "distilbert/distilbert-base-uncased-finetuned-sst-2-english"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

inputs = tok(["Loved the new syllabus", "The lab was a waste of time"],
             padding=True, return_tensors="pt")
with torch.inference_mode():
    outputs = model(**inputs)

probs = outputs.logits.softmax(dim=-1)   # shape: (batch, num_labels)
for p in probs:
    label_id = int(p.argmax())
    print(model.config.id2label[label_id], round(float(p[label_id]), 3))
```

## Softmax and argmax, no library needed

The postprocessing step in plain Python, so you can see that logits are just scores.

```python
import math

# Logits a sentiment classifier might return for one review
logits = [-1.2, 2.3]
id2label = {0: "NEGATIVE", 1: "POSITIVE"}

exps = [math.exp(x) for x in logits]
probs = [e / sum(exps) for e in exps]
best = max(range(len(probs)), key=probs.__getitem__)

print([round(p, 3) for p in probs])
print(id2label[best])
```

Output:

```
[0.029, 0.971]
POSITIVE
```

## Logits are not probabilities

A logit of 2.3 is not "230 percent" or "2.3 times more likely". Only after softmax do the values sum to 1. When comparing models, compare probabilities or metrics, never raw logits from different models.

**Quiz:** Why wrap inference in `torch.inference_mode()` or `torch.no_grad()`?

- [ ] It turns logits into probabilities
- [ ] It moves the model to the GPU
- [x] It stops PyTorch from recording operations for gradients, saving memory and time
- [ ] It switches dropout off

*Answer:* It stops PyTorch from recording operations for gradients, saving memory and time. These context managers disable gradient tracking. Dropout is controlled by model.eval(), and softmax is a separate step.