llama-cli and the Core Flags

Flags you will set every time

llama-cli loads a model and generates text in the terminal. -m points to a GGUF file; alternatively -hf user/repo downloads a GGUF from Hugging Face into a local cache (optionally with :Q4_K_M to pick a quant). -p gives a prompt and -n caps the number of tokens to generate. -c (--ctx-size) sets the context window; larger values cost KV-cache memory. -ngl (--n-gpu-layers) says how many transformer layers to put on the GPU; a large number such as 99 means "all of them" and recent builds may already offload by default when a GPU backend is present. -t sets CPU threads, which is best kept near the number of physical (performance) cores. Sampling flags mirror Ollama's parameters: --temp, --top-k, --top-p, --min-p, --repeat-penalty, --seed. When the model has a chat template, recent builds start an interactive conversation; --help on your build is the source of truth.

One run, every important flag

Paths and the model repo are examples; substitute your own.

./build/bin/llama-cli \
  -m models/Llama-3.2-3B-Instruct-Q4_K_M.gguf \
  -c 4096 -ngl 99 -t 8 \
  --temp 0.3 --seed 42 -n 256 \
  -p "Explain the difference between a process and a thread."

# Or fetch from Hugging Face directly
./build/bin/llama-cli -hf ggml-org/gemma-3-1b-it-GGUF

More threads is not always faster

Decode is limited by memory bandwidth, so beyond the physical core count extra threads mostly fight each other. On laptops with performance and efficiency cores, setting -t to the number of performance cores is often faster than using every core. Measure with llama-bench.

Quick check: What does `-ngl 99` mean in a llama.cpp command?

• Offload up to 99 layers (effectively all) to the GPU
• Use 99 CPU threads
• Generate 99 tokens
• Set temperature to 0.99