# Mixed Self-Check — Local LLMs with Ollama & llama.cpp

Source: https://www.skillbyai.com/en/local-llms/ll-self-check

> Test yourself across hardware, formats, Ollama, llama.cpp, applications and security with mixed scenarios.

## How to use this self-check

Work through each scenario before reading its answer. First, write down which layer the problem belongs to: **hardware and memory** (does it fit, where does it run), **model and format** (variant, quantization, template), **runtime configuration** (context, keep-alive, parallelism, offload), **application logic** (history, schemas, retrieval, tools), or **security** (exposure, injection, trust). Then name the one command or setting you would check first, such as `ollama ps`, `ollama show --modelfile`, a `num_ctx` value, or the retrieved chunks. Debugging in this order, from hardware upwards, prevents the most common waste of time: tweaking prompts and temperature when the real problem is a model running on the CPU or a context that is quietly truncating your input. If you miss a scenario, revisit the matching section; each one maps to a part of this course.

## Debug from the bottom up

Check each layer in this order before touching prompts. The stages, in order: hardware and memory -> model and format -> runtime configuration -> application logic -> security.

![5 connected stages, left to right: hardware and memory, model and format, runtime configuration, application logic, security.](assets/figures/local-llms/section-10-map.svg) — Figure 10.1: hardware and memory -> model and format -> runtime configuration -> application logic -> security

## Scenario: slow on a gaming laptop

An 8 GB NVIDIA laptop runs a 14B Q4_K_M model at a few tokens per second. *Answer*: the model (about 9 GB) does not fit in VRAM, so `ollama ps` will show a CPU/GPU split. Use a 7B or 8B model at Q4_K_M, which fits fully on the GPU.

## Scenario: the bot forgets the system prompt

A custom GGUF imported with a minimal Modelfile ignores its SYSTEM instruction and prints odd tokens. *Answer*: the TEMPLATE is missing or wrong, so the system message is not placed where the model expects. Copy the template from the original model or use `--jinja` in llama-server.

## Scenario: JSON parse errors in production

Your extractor uses `format: "json"` but occasionally fails to parse. *Answer*: check `num_predict` for truncation, pass a full JSON Schema instead of plain JSON mode, set temperature low, and validate with Pydantic, retrying on failure.

**Quiz:** A teammate set OLLAMA_NUM_PARALLEL=8 on a GPU that was already nearly full, and the model now runs partly on the CPU. Why?

- [ ] Parallel requests require a different GGUF file
- [x] Each parallel slot needs its own KV cache, so total memory grew and layers were pushed off the GPU
- [ ] OLLAMA_NUM_PARALLEL disables the GPU backend
- [ ] It forces the model to F16 precision

*Answer:* Each parallel slot needs its own KV cache, so total memory grew and layers were pushed off the GPU. More parallel slots multiply KV-cache memory. With less VRAM left for weights, Ollama offloads fewer layers to the GPU.