# Structured Output with JSON Schemas — Local LLMs with Ollama & llama.cpp

Source: https://www.skillbyai.com/en/local-llms/ll-structured-output

> Force a local model to return JSON that matches a schema and validate it with Pydantic.

## Constrained decoding, not polite requests

Asking a model "please reply in JSON" works most of the time, which is not good enough for a program. Local runtimes can do better with **constrained decoding**: at each step, the sampler masks out every token that would make the output invalid, so only tokens that keep the text on a valid path can be chosen. llama.cpp implements this with **GBNF grammars**, a grammar notation it can also generate from a JSON Schema (`--json-schema` or the `json_schema` field in requests). Ollama exposes it through the `format` field: `"json"` for any valid JSON, or a full **JSON Schema** object to fix field names and types. The guarantee is about *shape*, not truth: the model can still put a wrong value in a correctly typed field. So validate the result in code, and keep the schema description in the prompt too, because the model writes better values when it knows what each field means.

## Schema from a Pydantic model

Requires `pip install ollama pydantic` and a running server.

```python
from ollama import chat
from pydantic import BaseModel

class Course(BaseModel):
    name: str
    credits: int
    has_lab: bool

text = "Operating Systems is a 4-credit course with a weekly lab."
resp = chat(
    model="llama3.2",
    messages=[{"role": "user", "content": f"Extract the course details: {text}"}],
    format=Course.model_json_schema(),
    options={"temperature": 0},
)
course = Course.model_validate_json(resp.message.content)
print(course)
```

## Leave room for the closing brace

If `num_predict` is too small, generation can stop mid-object and you get truncated JSON even with a schema. Set a generous token limit and always wrap parsing in error handling.

**Quiz:** What does constrained decoding with a JSON schema guarantee?

- [ ] That every value in the JSON is factually correct
- [x] That the output's structure is valid for the schema, as long as generation completes
- [ ] That the model runs faster than unconstrained decoding in all cases
- [ ] That no tokens are sampled

*Answer:* That the output's structure is valid for the schema, as long as generation completes. Masking invalid tokens guarantees the shape, not the truth of the values. Truncation can still cut the output short.