# The REST API: generate, chat and streaming — Local LLMs with Ollama & llama.cpp

Source: https://www.skillbyai.com/en/local-llms/ll-rest-api

> Call /api/chat and /api/generate with curl, with and without streaming, and pass options per request.

## Two text endpoints and a streaming default

The Ollama server exposes JSON endpoints on port 11434. `POST /api/generate` takes a single `prompt` (plus optional `system`) and is suited to one-shot completions. `POST /api/chat` takes a `messages` array with `system`, `user`, `assistant` and `tool` roles; you send the whole history each time because the server keeps no conversation state. Both **stream by default**: the response is **newline-delimited JSON** (NDJSON), one object per chunk, with the final object carrying `"done": true` and timing statistics such as `eval_count` (generated tokens) and durations in nanoseconds. Send `"stream": false` to receive one JSON object instead. Per-request `options` override Modelfile parameters, for example `temperature` or `num_ctx`, and `keep_alive` controls how long the model stays loaded afterwards. Other useful endpoints include `/api/tags` (local models), `/api/ps` (loaded models), `/api/show` and `/api/embed`.

## A non-streaming chat request

The reply text is in `message.content` of the response.

```bash
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    {"role": "system", "content": "Answer in one sentence."},
    {"role": "user", "content": "What is a mutex?"}
  ],
  "stream": false,
  "options": {"temperature": 0.2}
}'
```

## One request, start to finish

Every client, whether curl, the Python library or an OpenAI SDK, follows the same path. The stages, in order: client sends JSON to /api/chat -> server loads the model and generates -> NDJSON chunks stream back until done is true.

![3 connected stages, left to right: client sends JSON to /api/chat, server loads the model and generates, NDJSON chunks stream back until done is true.](assets/figures/local-llms/section-5-map.svg) — Figure 5.1: client sends JSON to /api/chat -> server loads the model and generates -> NDJSON chunks stream back until done is true

## Tracing a streamed response

This snippet replays four NDJSON lines like those /api/generate streams. Follow how each chunk is parsed and appended.

```python
import json

# What /api/generate streams back: one JSON object per line
raw = """{"model":"llama3.2","response":"Namaste","done":false}
{"model":"llama3.2","response":", ","done":false}
{"model":"llama3.2","response":"world!","done":false}
{"model":"llama3.2","response":"","done":true,"eval_count":3}"""

text = ""
for line in raw.splitlines():
    chunk = json.loads(line)
    text += chunk["response"]
    if chunk["done"]:
        print("tokens generated:", chunk["eval_count"])
print(text)
```

- Line 4: The raw stream is several complete JSON objects separated by newlines, not one JSON array. That is why json.loads on the whole body would fail.
- Line 9: We accumulate the answer ourselves because each chunk carries only a small piece of text.
- Line 11: First line parsed on its own. Each line is a valid JSON document.
- Line 12: The piece is appended. Real UIs print it here too, which is what makes text appear word by word.
- Line 12: Lines two and three add ', ' and 'world!'. Chunks can split anywhere, even mid-word.
- Line 14: The final chunk has an empty response but done=true and statistics such as eval_count, the number of generated tokens.
- Line 15: After the loop the full answer is assembled and printed.

## The server is stateless

Students often send only the newest user message and wonder why the model forgot the previous turn. With /api/chat you must resend the full message history each time; trimming old turns is your job when it approaches the context limit.

**Quiz:** By default, what format does /api/chat return?

- [x] A stream of newline-delimited JSON objects
- [ ] A single JSON array of tokens
- [ ] Plain text with no metadata
- [ ] An XML document

*Answer:* A stream of newline-delimited JSON objects. Streaming is on by default and arrives as NDJSON chunks; set "stream": false for one JSON response.