पाठ 30 / 30
Mixed Self-Check
यह पाठ अभी हिंदी में अनुवादित नहीं है, इसलिए अंग्रेज़ी में दिखाया जा रहा है।
Test yourself across hardware, formats, Ollama, llama.cpp, applications and security with mixed scenarios.
लेखक: Rahul Agarwal · 15 साल से वेब डेवलपर
How to use this self-check
Work through each scenario before reading its answer. First, write down which layer the problem belongs to: hardware and memory (does it fit, where does it run), model and format (variant, quantization, template), runtime configuration (context, keep-alive, parallelism, offload), application logic (history, schemas, retrieval, tools), or security (exposure, injection, trust). Then name the one command or setting you would check first, such as ollama ps, ollama show --modelfile, a num_ctx value, or the retrieved chunks. Debugging in this order, from hardware upwards, prevents the most common waste of time: tweaking prompts and temperature when the real problem is a model running on the CPU or a context that is quietly truncating your input. If you miss a scenario, revisit the matching section; each one maps to a part of this course.
Debug from the bottom up
Check each layer in this order before touching prompts. The stages, in order: hardware and memory -> model and format -> runtime configuration -> application logic -> security.
Scenario: slow on a gaming laptop
An 8 GB NVIDIA laptop runs a 14B Q4_K_M model at a few tokens per second. Answer: the model (about 9 GB) does not fit in VRAM, so ollama ps will show a CPU/GPU split. Use a 7B or 8B model at Q4_K_M, which fits fully on the GPU.
Scenario: the bot forgets the system prompt
A custom GGUF imported with a minimal Modelfile ignores its SYSTEM instruction and prints odd tokens. Answer: the TEMPLATE is missing or wrong, so the system message is not placed where the model expects. Copy the template from the original model or use --jinja in llama-server.
Scenario: JSON parse errors in production
Your extractor uses format: "json" but occasionally fails to parse. Answer: check num_predict for truncation, pass a full JSON Schema instead of plain JSON mode, set temperature low, and validate with Pydantic, retrying on failure.
त्वरित जाँच: A teammate set OLLAMA_NUM_PARALLEL=8 on a GPU that was already nearly full, and the model now runs partly on the CPU. Why?
- Parallel requests require a different GGUF file
- Each parallel slot needs its own KV cache, so total memory grew and layers were pushed off the GPU
- OLLAMA_NUM_PARALLEL disables the GPU backend
- It forces the model to F16 precision
Answer
Each parallel slot needs its own KV cache, so total memory grew and layers were pushed off the GPU — More parallel slots multiply KV-cache memory. With less VRAM left for weights, Ollama offloads fewer layers to the GPU.