llama-server: HTTP, Web UI and Parallel Slots

A lightweight production-style server

llama-server loads one model and serves it over HTTP, by default on 127.0.0.1:8080. Opening that address in a browser gives a built-in chat web UI. For programs it offers OpenAI-compatible routes (/v1/chat/completions, /v1/embeddings when started in embedding mode, /v1/models), its own /completion endpoint with llama.cpp-specific options, and /health for readiness checks. The server handles concurrency with slots: -np 4 (--parallel) creates four slots that are decoded together in batches (continuous batching), which raises total throughput when several users are active. The context size given with -c is shared among slots, so -c 16384 -np 4 gives each slot about 4096 tokens (newer builds also offer --kv-unified, a shared pool that any slot can fill, so check --help on your build). Other useful flags: --host to choose the bind address, --port, --api-key to require a bearer token, and --jinja to use the model's embedded chat template, which tool calling needs.

Start the server and call it

The server must be running before the curl request.

./build/bin/llama-server -m models/Qwen2.5-7B-Instruct-Q4_K_M.gguf \
  -c 16384 -np 4 -ngl 99 --jinja --port 8080

# In another terminal
curl http://localhost:8080/health
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Define deadlock in one line."}]}'

Billing counters at a supermarket

Opening four billing counters (slots) lets four customers check out at once, but the store's total floor space (context) is split between the queues. More counters serve more people; each counter has less room for a giant trolley.

Ollama or llama-server?

Choose Ollama when you want easy model management, automatic loading of many models and a friendly CLI. Choose llama-server when you want one model, explicit control over every flag, the latest llama.cpp features, and a single binary with few moving parts.

त्वरित जाँच: You start llama-server with `-c 8192 -np 4`. Roughly how much context does each slot get?

• 8192 tokens each
• 32768 tokens each
• About 2048 tokens each
• Unlimited