</>SkillByAI
🇬🇧 English
Test our app

Local LLMs with Ollama & llama.cpp interview questions

30 questions from the Local LLMs with Ollama & llama.cpp course, each with the correct answer and an explanation. Try to answer before you open the answer.

Why Run LLMs Locally

1. Which is a genuine advantage of running an LLM locally rather than through a hosted API?

  1. Local models are always more accurate than hosted frontier models
  2. Prompts and data never have to leave your machine
  3. You no longer need any RAM or VRAM planning
  4. Local models never need updating
Show answer

Answer: Prompts and data never have to leave your machine

Privacy and control are the core advantages. Local models are typically smaller and less capable than frontier hosted models, and hardware planning becomes your job.

Lesson: What Local LLMs Are and Why They Matter

2. What is the main purpose of the KV cache during generation?

  1. To store the downloaded model file on disk
  2. To cache HTTP responses from the server
  3. To keep attention keys and values for earlier tokens so each new token does not reprocess the whole sequence
  4. To save the user's chat history between sessions
Show answer

Answer: To keep attention keys and values for earlier tokens so each new token does not reprocess the whole sequence

The KV cache holds per-token attention keys and values, so decode only computes the new token. It grows with context length, which is why long contexts need more memory.

Lesson: The Mental Model: Tokens, Weights and Inference

3. Why does a 4-bit quantized model usually generate tokens faster than the same model at 16-bit on the same hardware?

  1. Fewer bytes must be read from memory for each generated token
  2. 4-bit models have fewer layers
  3. 4-bit models skip the tokenizer
  4. The GPU clock speed increases for smaller files
Show answer

Answer: Fewer bytes must be read from memory for each generated token

Decode is mostly memory-bandwidth bound. A 4-bit model has the same architecture but roughly a quarter of the bytes to stream per token.

Lesson: Hardware Basics: RAM, VRAM and Bandwidth

Model Files, GGUF and Quantization

4. You want a local assistant that answers questions in a chat. Which variant should you choose?

  1. The base model, because it is the original
  2. Any variant; they behave identically
  3. The largest model regardless of RAM
  4. The instruct (chat) fine-tuned variant
Show answer

Answer: The instruct (chat) fine-tuned variant

Instruct models are trained to follow instructions in a chat format. Base models only continue text and often do not answer questions directly.

Lesson: Weights, Parameters and Model Families

5. Besides the weights, what does a GGUF file typically contain?

  1. The original training dataset
  2. Metadata such as architecture, tokenizer and chat template
  3. A Python virtual environment
  4. Compiled CUDA kernels for every GPU
Show answer

Answer: Metadata such as architecture, tokenizer and chat template

GGUF stores tensors plus metadata (architecture, hyperparameters, tokenizer, special tokens, often the chat template), so one file is enough to run the model.

Lesson: The GGUF File Format

6. What does the "_M" in Q4_K_M indicate?

  1. The model was trained on medical data
  2. It only runs on Mac
  3. A medium mix that keeps some sensitive tensors at higher precision
  4. It uses mixed CPU and GPU automatically
Show answer

Answer: A medium mix that keeps some sensitive tensors at higher precision

K-quant variants come in S, M and L mixes. M is a medium mix where certain tensors get more bits, balancing size and quality.

Lesson: Quantization Levels and Trade-offs

Getting Started with Ollama

7. What happens when you run an ollama CLI command such as `ollama run`?

  1. The CLI loads the model directly without any server
  2. The CLI uploads your prompt to Ollama's cloud
  3. The CLI recompiles llama.cpp each time
  4. The CLI sends HTTP requests to the local Ollama server, which loads and runs the model
Show answer

Answer: The CLI sends HTTP requests to the local Ollama server, which loads and runs the model

Ollama is client-server. The CLI talks to the server on port 11434, which manages model loading and inference.

Lesson: Installing Ollama and the Server Model

8. Which command shows the models currently loaded into memory and their CPU/GPU placement?

  1. ollama ps
  2. ollama list
  3. ollama show
  4. ollama pull
Show answer

Answer: ollama ps

`ollama ps` lists running (loaded) models with memory use and processor split. `ollama list` only shows what is downloaded.

Lesson: Everyday Ollama CLI Commands

9. You need vectors for semantic search, not chat replies. What kind of model should you pull?

  1. A coder model
  2. An embedding model such as nomic-embed-text
  3. The largest instruct model available
  4. A base model
Show answer

Answer: An embedding model such as nomic-embed-text

Embedding models map text to vectors for similarity search. Chat and coder models generate text.

Lesson: Choosing Models from the Library

Customizing Models with Modelfiles

10. What does `ollama create dsa-tutor -f Modelfile` do with the base model's weights?

  1. It copies and re-quantizes all the weights
  2. It reuses the existing weight layers and adds only the new configuration
  3. It fine-tunes the weights on the SYSTEM prompt
  4. It uploads them to the Ollama library
Show answer

Answer: It reuses the existing weight layers and adds only the new configuration

Ollama stores models as layers. A derived model points to the same weight blob and adds small layers for system prompt, parameters and template. No training happens.

Lesson: Modelfile Basics: FROM, SYSTEM and PARAMETER

11. For extracting fields from invoices as JSON, which setting is most appropriate?

  1. Temperature around 1.5 for variety
  2. A very high repeat_penalty
  3. A low temperature such as 0 to 0.2
  4. top_k of 1000
Show answer

Answer: A low temperature such as 0 to 0.2

Extraction wants deterministic, most-likely tokens. Low temperature (or greedy decoding) reduces random variation and format errors.

Lesson: Sampling Parameters: Temperature, top_p and Friends

12. A freshly imported GGUF keeps generating fake "User:" turns after its answer. What is the most likely cause?

  1. The quantization level is too high
  2. The GPU is overheating
  3. The model has too many parameters
  4. A wrong chat template or missing stop token
Show answer

Answer: A wrong chat template or missing stop token

If the template or stop tokens do not match training, the model never sees its end-of-turn marker and keeps writing the conversation.

Lesson: Chat Templates and Special Tokens

Programming Against the Ollama API

13. By default, what format does /api/chat return?

  1. A stream of newline-delimited JSON objects
  2. A single JSON array of tokens
  3. Plain text with no metadata
  4. An XML document
Show answer

Answer: A stream of newline-delimited JSON objects

Streaming is on by default and arrives as NDJSON chunks; set "stream": false for one JSON response.

Lesson: The REST API: generate, chat and streaming

14. How do you make the ollama Python library talk to a server on another machine?

  1. Pass remote=True to chat()
  2. Create Client(host="http://other-machine:11434") and call its methods
  3. Edit the model's Modelfile
  4. It is impossible; the library only works locally
Show answer

Answer: Create Client(host="http://other-machine:11434") and call its methods

A Client takes a host argument; the default client also honours the OLLAMA_HOST environment variable.

Lesson: The Official Python Library

15. You have working code using the openai SDK. What is the minimum change to run it against local Ollama?

  1. Rewrite it with raw sockets
  2. Convert the model to safetensors
  3. Set base_url to http://localhost:11434/v1, use a local model name, and pass any placeholder api_key
  4. Install CUDA on the client machine
Show answer

Answer: Set base_url to http://localhost:11434/v1, use a local model name, and pass any placeholder api_key

Ollama serves an OpenAI-compatible API under /v1. The SDK needs some api_key value, which Ollama ignores.

Lesson: OpenAI-Compatible Endpoints

Working Directly with llama.cpp

16. Which llama.cpp binary would you use to expose a model over HTTP with an OpenAI-compatible API?

  1. llama-quantize
  2. llama-bench
  3. llama-perplexity
  4. llama-server
Show answer

Answer: llama-server

llama-server is the HTTP server with /v1 endpoints and a web UI. The others quantize, benchmark and measure quality.

Lesson: What llama.cpp Is and How to Build It

17. What does `-ngl 99` mean in a llama.cpp command?

  1. Offload up to 99 layers (effectively all) to the GPU
  2. Use 99 CPU threads
  3. Generate 99 tokens
  4. Set temperature to 0.99
Show answer

Answer: Offload up to 99 layers (effectively all) to the GPU

-ngl / --n-gpu-layers controls how many layers live on the GPU. A number larger than the layer count means the whole model is offloaded.

Lesson: llama-cli and the Core Flags

18. You start llama-server with `-c 8192 -np 4`. Roughly how much context does each slot get?

  1. 8192 tokens each
  2. 32768 tokens each
  3. About 2048 tokens each
  4. Unlimited
Show answer

Answer: About 2048 tokens each

With the classic per-slot split, the total context is divided among parallel slots: 8192 / 4 = 2048 tokens per slot.

Lesson: llama-server: HTTP, Web UI and Parallel Slots

Building Applications on Local Models

19. What does constrained decoding with a JSON schema guarantee?

  1. That every value in the JSON is factually correct
  2. That the output's structure is valid for the schema, as long as generation completes
  3. That the model runs faster than unconstrained decoding in all cases
  4. That no tokens are sampled
Show answer

Answer: That the output's structure is valid for the schema, as long as generation completes

Masking invalid tokens guarantees the shape, not the truth of the values. Truncation can still cut the output short.

Lesson: Structured Output with JSON Schemas

20. Your local RAG bot gives confident but wrong answers. Which check should usually come first?

  1. Raise the temperature
  2. Switch to a larger quantization of the chat model immediately
  3. Remove the system prompt
  4. Print the retrieved chunks to see whether the right passage was found
Show answer

Answer: Print the retrieved chunks to see whether the right passage was found

Most RAG failures are retrieval failures. If the correct chunk is not in the prompt, no chat model can answer from it.

Lesson: Embeddings and Local RAG

21. In tool calling, who actually executes the function?

  1. The LLM runs it inside the GPU
  2. Ollama runs any Python function it finds on the system
  3. Your application code, after reading and validating the model's tool call
  4. The tokenizer
Show answer

Answer: Your application code, after reading and validating the model's tool call

The model only emits a structured request. Your code decides whether and how to execute it, then returns the result as a tool message.

Lesson: Tool Calling with Local Models

Performance Tuning and Memory

22. A model fits at 4K context but runs out of memory at 32K. Which component grew?

  1. The KV cache, which scales linearly with context length
  2. The model weights
  3. The tokenizer vocabulary
  4. The GGUF metadata
Show answer

Answer: The KV cache, which scales linearly with context length

Weights are fixed by the file. The KV cache stores keys and values for every token of context, so 8 times the context means about 8 times the cache.

Lesson: Memory Math: Weights plus KV Cache

23. Why does partially offloading a model to the GPU often give disappointing speed?

  1. The GPU refuses to compute quantized layers
  2. Layers left in system RAM run at much lower bandwidth and bottleneck every token
  3. Partial offload disables the KV cache
  4. Ollama re-downloads the model each time
Show answer

Answer: Layers left in system RAM run at much lower bandwidth and bottleneck every token

Each token must pass through every layer. The CPU-resident layers are slow, so they dominate the time per token.

Lesson: GPU Offload and CPU/GPU Splits

24. Users report that a chatbot forgets details from the start of long conversations. What is the most likely cause?

  1. OLLAMA_KEEP_ALIVE is too long
  2. The temperature is too low
  3. The context window is too small, so older tokens are truncated
  4. The model is quantized to Q8_0
Show answer

Answer: The context window is too small, so older tokens are truncated

When the conversation exceeds num_ctx, the oldest tokens are dropped. Increase the context (memory permitting) or summarize older turns.

Lesson: Context, Keep-Alive and Concurrency Settings

Deployment, Security and Pitfalls

25. Why mount a volume at /root/.ollama when running Ollama in Docker?

  1. It enables the GPU
  2. It is required for the OpenAI-compatible API
  3. It encrypts the prompts
  4. So downloaded models persist across container restarts instead of being re-downloaded
Show answer

Answer: So downloaded models persist across container restarts instead of being re-downloaded

Models live in that directory. Without a volume they vanish with the container and must be pulled again.

Lesson: Running Local LLMs in Docker and on a Team Server

26. What is the main risk of starting Ollama with OLLAMA_HOST=0.0.0.0 on a cloud VM with an open port?

  1. Anyone who can reach the port can use the API, because Ollama has no built-in authentication
  2. The model will run only on CPU
  3. Generation becomes deterministic
  4. Models are automatically uploaded to the public library
Show answer

Answer: Anyone who can reach the port can use the API, because Ollama has no built-in authentication

Binding to all interfaces exposes an unauthenticated API. Use a firewall, SSH tunnel or authenticating reverse proxy.

Lesson: Securing Local Inference

27. You want to know whether Q4_K_M is good enough compared to Q8_0 for your support-ticket classifier. What is the best approach?

  1. Ask the model which quantization it prefers
  2. Pick whichever downloads faster
  3. Assume Q8_0 is always required
  4. Run both on the same labelled set of real tickets with fixed settings and compare accuracy and speed
Show answer

Answer: Run both on the same labelled set of real tickets with fixed settings and compare accuracy and speed

A task-specific evaluation with fixed settings answers the question directly; perplexity is a useful secondary signal.

Lesson: Common Pitfalls and Evaluating Quality

Revision and Interview Prep

28. Which pair correctly matches tool and default address?

  1. Ollama at 127.0.0.1:11434, llama-server at 127.0.0.1:8080
  2. Ollama at 0.0.0.0:8080, llama-server at 127.0.0.1:11434
  3. Both on 0.0.0.0:80
  4. Ollama at 127.0.0.1:5000, llama-server at 127.0.0.1:3000
Show answer

Answer: Ollama at 127.0.0.1:11434, llama-server at 127.0.0.1:8080

Ollama listens on localhost port 11434 and llama-server on localhost port 8080 by default; both bind to loopback unless configured otherwise.

Lesson: Cheat Sheet: Local LLMs in One Page

29. An interviewer asks why a 70B model at 4-bit runs slowly on a 24 GB GPU even though inference "works". What is the best explanation?

  1. 70B models cannot be quantized
  2. The tokenizer is larger for big models
  3. At roughly 40 GB the weights do not fit in 24 GB of VRAM, so many layers run from system RAM
  4. The GPU automatically lowers its clock for large models
Show answer

Answer: At roughly 40 GB the weights do not fit in 24 GB of VRAM, so many layers run from system RAM

70B x about 4.8 bits / 8 is roughly 40 GB. The overflow lives in system RAM, and those layers bottleneck every token.

Lesson: Common Interview and Exam Questions

30. A teammate set OLLAMA_NUM_PARALLEL=8 on a GPU that was already nearly full, and the model now runs partly on the CPU. Why?

  1. Parallel requests require a different GGUF file
  2. Each parallel slot needs its own KV cache, so total memory grew and layers were pushed off the GPU
  3. OLLAMA_NUM_PARALLEL disables the GPU backend
  4. It forces the model to F16 precision
Show answer

Answer: Each parallel slot needs its own KV cache, so total memory grew and layers were pushed off the GPU

More parallel slots multiply KV-cache memory. With less VRAM left for weights, Ollama offloads fewer layers to the GPU.

Lesson: Mixed Self-Check

← Back to the course