# Auto Classes, Configs and Heads — Hugging Face Transformers

Source: https://www.skillbyai.com/en/hugging-face/hf-auto-classes

> Load any checkpoint with the right Auto class and explain what a config and a task head are.

## Body plus head

Every checkpoint on the Hub has a `config.json` that records its **architecture** (for example `BertForSequenceClassification`) and its hyperparameters: number of layers, hidden size, vocabulary size and label names. **Auto classes** read that config and build the right Python class for you, so `AutoModel.from_pretrained(id)` works for BERT, RoBERTa or Llama without you naming the class. It helps to see a model as a **body** plus a **head**. `AutoModel` loads only the body, which returns *hidden states*: one vector per token. Task-specific classes such as `AutoModelForSequenceClassification`, `AutoModelForTokenClassification`, `AutoModelForQuestionAnswering`, `AutoModelForCausalLM` and `AutoModelForSeq2SeqLM` add a small **head** on top that turns hidden states into task logits. If a checkpoint has no trained weights for the head you ask for, the library initialises the head randomly and warns you that some weights are newly initialised. That warning is expected before fine-tuning and a red flag at inference time.

## Config, body and head

The config decides the architecture, the body produces hidden states and the head turns them into task-specific logits.

![Three connected stages: the model config, the base model producing hidden states, and the task head producing logits.](assets/figures/hugging-face/section-4-map.svg) — Figure 4.1: config -> base model (hidden states) -> task head (logits)

## Loading a body and a body-with-head

`num_labels` sizes a new classification head. The warning about newly initialised weights is expected here because bert-base-uncased was never trained for this task.

```python
from transformers import AutoConfig, AutoModel, AutoModelForSequenceClassification

model_id = "google-bert/bert-base-uncased"
config = AutoConfig.from_pretrained(model_id)
print(config.model_type, config.num_hidden_layers, config.hidden_size)  # bert 12 768

body = AutoModel.from_pretrained(model_id)  # hidden states only
clf = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    num_labels=3,
    id2label={0: "negative", 1: "neutral", 2: "positive"},
    label2id={"negative": 0, "neutral": 1, "positive": 2},
)
print(type(clf).__name__)  # BertForSequenceClassification
```

## Read the "newly initialized" warning

If you see "Some weights of ... were not initialized from the model checkpoint" while you expect a ready-made classifier, you loaded the wrong checkpoint or the wrong Auto class. The predictions will be random until you fine-tune.

**Quiz:** You want a model that outputs one score per class for a whole sentence. Which class do you load?

- [x] AutoModelForSequenceClassification
- [ ] AutoModel
- [ ] AutoModelForTokenClassification
- [ ] AutoTokenizer

*Answer:* AutoModelForSequenceClassification. Sequence classification puts a head on the pooled sentence representation. Token classification produces one prediction per token, and AutoModel has no head at all.