What llama.cpp Is and How to Build It

The engine under many local tools

llama.cpp is an open-source C/C++ inference engine for LLMs, built on the ggml tensor library. It runs GGUF models on CPUs (using SIMD instructions such as AVX2 and NEON) and on GPUs through several backends: CUDA for NVIDIA, Metal for Apple Silicon (enabled by default on macOS), Vulkan, HIP for AMD, SYCL for Intel, and more. Ollama, LM Studio and many other apps build on it or on ggml. Using llama.cpp directly gives you the newest features first and fine-grained control over every flag, at the cost of more manual setup. You can install prebuilt binaries from the project's releases or package managers such as Homebrew, or build from source with CMake. A build produces several tools in build/bin, including llama-cli (terminal chat), llama-server (HTTP server with web UI), llama-quantize, llama-bench and llama-perplexity.

Building from source

The CUDA flag is only needed on machines with an NVIDIA GPU and the CUDA toolkit.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

# CPU build (Metal is enabled automatically on macOS)
cmake -B build
cmake --build build --config Release -j

# NVIDIA GPU build instead
# cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release -j

ls build/bin | grep llama-

Source to running server

The usual path from a fresh checkout to a model answering requests. The stages, in order: clone repository -> configure with CMake (choose backend) -> build binaries -> llama-cli or llama-server loads a GGUF.

Old Makefile instructions are outdated

Older blog posts say to run make. The project moved to CMake, and the binaries were renamed with a llama- prefix (for example main became llama-cli). Prefer the README in the repository over tutorials.

Quick check: Which llama.cpp binary would you use to expose a model over HTTP with an OpenAI-compatible API?

• llama-quantize
• llama-bench
• llama-perplexity
• llama-server