Installing Ollama and the Server Model

A CLI in front of a local server

Ollama is an open-source tool that packages model download, storage and inference behind a simple command line and an HTTP API. Internally it builds on llama.cpp's ggml library for running GGUF models and handles GPU detection, memory estimation and chat templates for you. The important mental model: Ollama is a client-server program. A background server (started by the desktop app, by a system service on Linux, or manually with ollama serve) listens on 127.0.0.1:11434. Every ollama CLI command, and every app you write, is just a client sending HTTP requests to that server. The server loads models into memory on first use, keeps them loaded for a while (five minutes by default) so the next request is fast, and unloads them when idle. Models are stored under a models directory in your home folder, which you can relocate with the OLLAMA_MODELS environment variable.

Install on Linux and check the server

macOS and Windows use the installer from the official site; Linux has an install script. Always read a script before piping it to a shell.

# Linux install (review the script first if you prefer)
curl -fsSL https://ollama.com/install.sh | sh

ollama --version

# Is the server up? It answers with a short status message
curl http://localhost:11434

# If not running as a service, start it in a terminal
ollama serve

Who talks to whom

The CLI and your own programs are both clients of the same local server. The stages, in order: ollama CLI or your app -> HTTP on localhost:11434 -> Ollama server -> loaded model on CPU/GPU.

Port already in use

If ollama serve complains that the address is already in use, the server is probably already running as a service or desktop app. You do not need a second one; just use the CLI.

त्वरित जाँच: What happens when you run an ollama CLI command such as `ollama run`?

• The CLI loads the model directly without any server
• The CLI uploads your prompt to Ollama's cloud
• The CLI recompiles llama.cpp each time
• The CLI sends HTTP requests to the local Ollama server, which loads and runs the model