# Generations: Recording Model Calls — LLM Observability (Langfuse / LangSmith)

Source: https://www.skillbyai.com/en/llm-observability/lo-generations

> Record a model call with its model name, parameters, messages, token usage and cost.

## The special span for a model call

Most steps in a trace are generic spans, but a call to a language model deserves a richer record. Langfuse names this observation type a **generation**; LangSmith uses a run with `run_type="llm"`. A generation stores the **model** identifier, **model parameters** (temperature, max tokens, top_p), the **input** as a list of chat messages, the **output** message, and **usage**: input tokens, output tokens and sometimes cached or reasoning tokens. From usage and a price table the tool computes **cost**. For streaming calls a generation may also record when the first token arrived, which gives you time to first token. Why separate it from a plain span? Because analytics depend on it: cost per user, tokens per model and latency per model all aggregate over generations only. If you log a model call as a plain span with the text in a string, the tool cannot add up tokens or money.

## A generation record in Langfuse (manual style)

Uses the Langfuse Python SDK (v3 style). Replace the placeholder reply with your real model call; the drop-in OpenAI wrapper shown later fills these fields automatically.

```python
from langfuse import get_client

langfuse = get_client()  # reads LANGFUSE_PUBLIC_KEY / SECRET_KEY / HOST from env

with langfuse.start_as_current_generation(
    name="draft-reply",
    model="my-chat-model",
    model_parameters={"temperature": 0.2},
    input=[{"role": "user", "content": "Namaste! Where is my order?"}],
) as gen:
    reply = "Your order is out for delivery."   # call your model here
    gen.update(output=reply, usage_details={"input": 14, "output": 8})

langfuse.flush()
```

## Let the tool count tokens when it can

Prefer the usage numbers returned by the provider's API over your own estimate; tokenizers differ by model and your count will drift. If you self-host a model that returns no usage, record your best estimate and label it as such in metadata.

**Quiz:** Why should model calls be recorded as generations (or llm runs) instead of plain spans?

- [ ] Plain spans cannot have inputs or outputs
- [ ] Only generations can be nested inside a trace
- [ ] Generations are stored for free while spans are billed
- [x] Token usage and cost analytics aggregate over model-call records with model and usage fields

*Answer:* Token usage and cost analytics aggregate over model-call records with model and usage fields. Cost and token dashboards rely on the model and usage fields that the generation type carries. Plain spans still support input and output, but lack those semantics.