# Context, Keep-Alive and Concurrency Settings — Local LLMs with Ollama & llama.cpp

Source: https://www.skillbyai.com/en/local-llms/ll-server-tuning

> Configure Ollama's context length, keep-alive, parallel requests and loaded-model limits for your workload.

## The knobs that shape a shared server

Ollama is tuned mostly through **environment variables** on the server process. `OLLAMA_CONTEXT_LENGTH` sets the default context window; the built-in default has changed between versions, and a request's `num_ctx` option overrides it. If a conversation exceeds the window, older tokens are dropped, so a too-small context makes the model silently "forget" the start of long chats or documents. `OLLAMA_KEEP_ALIVE` (or the `keep_alive` request field) controls how long an idle model stays loaded: longer avoids reload delays, shorter frees memory; `-1` keeps it loaded indefinitely and `0` unloads immediately. `OLLAMA_NUM_PARALLEL` sets how many requests one loaded model processes at once, and multiplies the KV cache needed. `OLLAMA_MAX_LOADED_MODELS` caps how many different models can be resident together, and `OLLAMA_MAX_QUEUE` limits waiting requests before the server returns busy errors. Change one variable, restart the server, and measure.

## Setting variables for the Linux service

On systemd installs, add overrides to the service, then restart it.

```bash
sudo systemctl edit ollama.service
# In the editor, add:
# [Service]
# Environment="OLLAMA_CONTEXT_LENGTH=8192"
# Environment="OLLAMA_KEEP_ALIVE=30m"
# Environment="OLLAMA_NUM_PARALLEL=2"

sudo systemctl daemon-reload
sudo systemctl restart ollama

# Per request instead:
curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"hi","keep_alive":"1h","options":{"num_ctx":8192}}'
```

## Keeping the pressure cooker on the stove

If you will cook dal again in ten minutes, leave the cooker on the stove (keep-alive). If the kitchen is tiny and you need the burner for rice, put it away (unload). Keep-alive trades memory for faster next requests.

**Quiz:** Users report that a chatbot forgets details from the start of long conversations. What is the most likely cause?

- [ ] OLLAMA_KEEP_ALIVE is too long
- [ ] The temperature is too low
- [x] The context window is too small, so older tokens are truncated
- [ ] The model is quantized to Q8_0

*Answer:* The context window is too small, so older tokens are truncated. When the conversation exceeds num_ctx, the oldest tokens are dropped. Increase the context (memory permitting) or summarize older turns.