The REST API: generate, chat and streaming

Two text endpoints and a streaming default

The Ollama server exposes JSON endpoints on port 11434. POST /api/generate takes a single prompt (plus optional system) and is suited to one-shot completions. POST /api/chat takes a messages array with system, user, assistant and tool roles; you send the whole history each time because the server keeps no conversation state. Both stream by default: the response is newline-delimited JSON (NDJSON), one object per chunk, with the final object carrying "done": true and timing statistics such as eval_count (generated tokens) and durations in nanoseconds. Send "stream": false to receive one JSON object instead. Per-request options override Modelfile parameters, for example temperature or num_ctx, and keep_alive controls how long the model stays loaded afterwards. Other useful endpoints include /api/tags (local models), /api/ps (loaded models), /api/show and /api/embed.

A non-streaming chat request

The reply text is in message.content of the response.

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    {"role": "system", "content": "Answer in one sentence."},
    {"role": "user", "content": "What is a mutex?"}
  ],
  "stream": false,
  "options": {"temperature": 0.2}
}'

One request, start to finish

Every client, whether curl, the Python library or an OpenAI SDK, follows the same path. The stages, in order: client sends JSON to /api/chat -> server loads the model and generates -> NDJSON chunks stream back until done is true.

Tracing a streamed response

This snippet replays four NDJSON lines like those /api/generate streams. Follow how each chunk is parsed and appended.

import json

# What /api/generate streams back: one JSON object per line
raw = """{"model":"llama3.2","response":"Namaste","done":false}
{"model":"llama3.2","response":", ","done":false}
{"model":"llama3.2","response":"world!","done":false}
{"model":"llama3.2","response":"","done":true,"eval_count":3}"""

text = ""
for line in raw.splitlines():
    chunk = json.loads(line)
    text += chunk["response"]
    if chunk["done"]:
        print("tokens generated:", chunk["eval_count"])
print(text)

1. Line 4: The raw stream is several complete JSON objects separated by newlines, not one JSON array. That is why json.loads on the whole body would fail.
2. Line 9: We accumulate the answer ourselves because each chunk carries only a small piece of text.
3. Line 11: First line parsed on its own. Each line is a valid JSON document.
4. Line 12: The piece is appended. Real UIs print it here too, which is what makes text appear word by word.
5. Line 12: Lines two and three add ', ' and 'world!'. Chunks can split anywhere, even mid-word.
6. Line 14: The final chunk has an empty response but done=true and statistics such as eval_count, the number of generated tokens.
7. Line 15: After the loop the full answer is assembled and printed.

The server is stateless

Students often send only the newest user message and wonder why the model forgot the previous turn. With /api/chat you must resend the full message history each time; trimming old turns is your job when it approaches the context limit.

Quick check: By default, what format does /api/chat return?

• A stream of newline-delimited JSON objects
• A single JSON array of tokens
• Plain text with no metadata
• An XML document