Context, Keep-Alive and Concurrency Settings

The knobs that shape a shared server

Ollama is tuned mostly through environment variables on the server process. OLLAMA_CONTEXT_LENGTH sets the default context window; the built-in default has changed between versions, and a request's num_ctx option overrides it. If a conversation exceeds the window, older tokens are dropped, so a too-small context makes the model silently "forget" the start of long chats or documents. OLLAMA_KEEP_ALIVE (or the keep_alive request field) controls how long an idle model stays loaded: longer avoids reload delays, shorter frees memory; -1 keeps it loaded indefinitely and 0 unloads immediately. OLLAMA_NUM_PARALLEL sets how many requests one loaded model processes at once, and multiplies the KV cache needed. OLLAMA_MAX_LOADED_MODELS caps how many different models can be resident together, and OLLAMA_MAX_QUEUE limits waiting requests before the server returns busy errors. Change one variable, restart the server, and measure.

Setting variables for the Linux service

On systemd installs, add overrides to the service, then restart it.

sudo systemctl edit ollama.service
# In the editor, add:
# [Service]
# Environment="OLLAMA_CONTEXT_LENGTH=8192"
# Environment="OLLAMA_KEEP_ALIVE=30m"
# Environment="OLLAMA_NUM_PARALLEL=2"

sudo systemctl daemon-reload
sudo systemctl restart ollama

# Per request instead:
curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"hi","keep_alive":"1h","options":{"num_ctx":8192}}'

Keeping the pressure cooker on the stove

If you will cook dal again in ten minutes, leave the cooker on the stove (keep-alive). If the kitchen is tiny and you need the burner for rice, put it away (unload). Keep-alive trades memory for faster next requests.

त्वरित जाँच: Users report that a chatbot forgets details from the start of long conversations. What is the most likely cause?

• OLLAMA_KEEP_ALIVE is too long
• The temperature is too low
• The context window is too small, so older tokens are truncated
• The model is quantized to Q8_0