Tool Calling with Local Models

The model asks, your code acts

Tool calling (function calling) lets a model request that your program run a function, such as looking up a timetable or doing arithmetic it would otherwise get wrong. You describe each tool with a name, a description and a JSON Schema for its arguments, and pass the list in the tools field of a chat request. A model trained for tool use may then reply with tool_calls instead of text: each call names a function and gives arguments as JSON. Crucially, the model never executes anything; your code reads the call, validates the arguments, runs the real function, and appends a message with role tool containing the result. You then call the model again so it can write a final answer using that result. This loop is the basis of local agents. Only some models are trained for tools, and small models pick wrong tools or arguments more often, so validate everything.

One round trip with the Python library

The library builds the tool schema from the function signature and docstring.

from ollama import chat

def add_marks(a: int, b: int) -> int:
    """Add two exam marks.

    Args:
      a: first mark
      b: second mark
    """
    return a + b

TOOLS = {"add_marks": add_marks}
messages = [{"role": "user", "content": "What is 47 plus 38 marks?"}]
resp = chat(model="qwen2.5:7b", messages=messages, tools=[add_marks])

messages.append(resp.message)
for call in resp.message.tool_calls or []:
    fn = TOOLS.get(call.function.name)        # only allow-listed tools
    if fn:
        result = fn(**call.function.arguments)
        messages.append({"role": "tool", "content": str(result), "tool_name": call.function.name})

final = chat(model="qwen2.5:7b", messages=messages)
print(final.message.content)

A manager writing a request slip

The model is like a manager who cannot touch the office computer. It fills a slip: "run fee_status for roll number 21CS042". The clerk (your code) checks the slip is allowed, runs it, and hands back the printout. The manager then writes the reply to the student.

त्वरित जाँच: In tool calling, who actually executes the function?

• The LLM runs it inside the GPU
• Ollama runs any Python function it finds on the system
• Your application code, after reading and validating the model's tool call
• The tokenizer