# What llama.cpp Is and How to Build It — Local LLMs with Ollama & llama.cpp

Source: https://www.skillbyai.com/en/local-llms/ll-llamacpp-build

> Build llama.cpp from source with CMake for CPU, CUDA or Metal and know which binaries it produces.

## The engine under many local tools

**llama.cpp** is an open-source C/C++ inference engine for LLMs, built on the **ggml** tensor library. It runs GGUF models on CPUs (using SIMD instructions such as AVX2 and NEON) and on GPUs through several backends: **CUDA** for NVIDIA, **Metal** for Apple Silicon (enabled by default on macOS), **Vulkan**, **HIP** for AMD, **SYCL** for Intel, and more. Ollama, LM Studio and many other apps build on it or on ggml. Using llama.cpp directly gives you the newest features first and fine-grained control over every flag, at the cost of more manual setup. You can install prebuilt binaries from the project's releases or package managers such as Homebrew, or build from source with **CMake**. A build produces several tools in `build/bin`, including `llama-cli` (terminal chat), `llama-server` (HTTP server with web UI), `llama-quantize`, `llama-bench` and `llama-perplexity`.

## Building from source

The CUDA flag is only needed on machines with an NVIDIA GPU and the CUDA toolkit.

```bash
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

# CPU build (Metal is enabled automatically on macOS)
cmake -B build
cmake --build build --config Release -j

# NVIDIA GPU build instead
# cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release -j

ls build/bin | grep llama-
```

## Source to running server

The usual path from a fresh checkout to a model answering requests. The stages, in order: clone repository -> configure with CMake (choose backend) -> build binaries -> llama-cli or llama-server loads a GGUF.

![4 connected stages, left to right: clone repository, configure with CMake (choose backend), build binaries, llama-cli or llama-server loads a GGUF.](assets/figures/local-llms/section-6-map.svg) — Figure 6.1: clone repository -> configure with CMake (choose backend) -> build binaries -> llama-cli or llama-server loads a GGUF

## Old Makefile instructions are outdated

Older blog posts say to run `make`. The project moved to CMake, and the binaries were renamed with a `llama-` prefix (for example `main` became `llama-cli`). Prefer the README in the repository over tutorials.

**Quiz:** Which llama.cpp binary would you use to expose a model over HTTP with an OpenAI-compatible API?

- [ ] llama-quantize
- [ ] llama-bench
- [ ] llama-perplexity
- [x] llama-server

*Answer:* llama-server. llama-server is the HTTP server with /v1 endpoints and a web UI. The others quantize, benchmark and measure quality.