# The Official Python Library — Local LLMs with Ollama & llama.cpp

Source: https://www.skillbyai.com/en/local-llms/ll-python-sdk

> Use the ollama Python package for chat, streaming and options, and point it at a remote host.

## A thin, typed wrapper over the REST API

The `ollama` package on PyPI (installed with `pip install ollama`) wraps the same REST endpoints in Python functions: `chat`, `generate`, `embed`, `list`, `pull`, `show` and others. Calling the top-level functions uses a default client pointed at `http://localhost:11434`, or at whatever `OLLAMA_HOST` is set to. For a different server, create `Client(host="http://gpu-box:11434")`; for asyncio code use `AsyncClient`. Responses are typed objects that also allow dictionary-style access, so both `response.message.content` and `response["message"]["content"]` work. Passing `stream=True` returns an iterator of partial responses. Parameters go in an `options` dictionary with the same names as Modelfile parameters. Errors from the server, such as a model that is not pulled, raise `ollama.ResponseError`, whose `status_code` you can inspect. A JavaScript library with a near-identical shape exists for Node and browsers.

## Chat, then stream

Requires a running Ollama server and the model pulled locally.

```python
from ollama import chat, ResponseError

messages = [{"role": "user", "content": "Give one tip for learning recursion."}]

try:
    reply = chat(model="llama3.2", messages=messages, options={"temperature": 0.3})
    print(reply.message.content)
except ResponseError as e:
    print("server error:", e.status_code, e.error)

# Streaming: print pieces as they arrive
for part in chat(model="llama3.2", messages=messages, stream=True):
    print(part.message.content, end="", flush=True)
print()
```

## Keep history in a list you own

Append the assistant's reply to `messages` after each call (`messages.append({"role": "assistant", "content": reply.message.content})`) before adding the next user turn. That list is the whole memory of the conversation.

**Quiz:** How do you make the ollama Python library talk to a server on another machine?

- [ ] Pass remote=True to chat()
- [x] Create Client(host="http://other-machine:11434") and call its methods
- [ ] Edit the model's Modelfile
- [ ] It is impossible; the library only works locally

*Answer:* Create Client(host="http://other-machine:11434") and call its methods. A Client takes a host argument; the default client also honours the OLLAMA_HOST environment variable.