LLM Tool Calling (1): The Model Doesn't Call Functions, It Asks for Calls

Version note: The examples in this post were tested with Ollama 0.34 on local models (qwen3:4b, gemma3:4b), using .NET 10 and Node 22.

Guess First

You gave the Bookstore assistant a tool called get_order_status, and a user asks "Where is my order 1042?". Does the model connect to your order table and read the record? Does it make something up? Or is there a third option?

In this post we will find out with raw HTTP requests and no abstractions at all. In the next post we will hand the same work to Microsoft.Extensions.AI, but first you need to know what happens under the hood.

The Model Doesn't Run Functions, It Asks for Calls

The name tool calling (function calling) is misleading. The model doesn't call anything. It is a text generator; with tool calling, what it generates instead of a plain answer is a structured request that says "I want you to run this function with these arguments". Running the function, getting the result and handing it back to the model is always your code's job.

The flow has four steps. First you put a tools list in the request: each tool's name, description and a JSON schema for its arguments. The model sees the question and the tools; if it needs one, it returns tool_calls instead of answer text. You run that call in your own code and append the result to the message history with the tool role. The model reads the result and writes the final answer. If the result isn't enough, it asks for another tool; that repetition is called the agent loop.

A restaurant analogy: the waiter (the model) never goes into the kitchen, it writes an order ticket. The kitchen (your code) reads the ticket, cooks the dish and hands it to the waiter, who then tells the table what it brought. Whoever leaves the kitchen door open is responsible for what walks in; we will come back to that, from the security side, in the next post.

Which Models Can Use Tools?

Not every model is trained for tool calling. Send tools to gemma3:4b, the series' default model, and Ollama rejects the request before it even reaches the model:

{"error":"registry.ollama.ai/library/gemma3:4b does not support tools"}

You can read what a model supports from the capabilities field of the /api/show response. In this post we use qwen3:4b for tool calling:

$ curl -s http://localhost:11434/api/show -d '{"model":"qwen3:4b"}'
...
"capabilities": ["completion", "tools", "thinking"]

For gemma3:4b the same list only contains completion and vision. On ollama.com you can also look for the "tools" badge on the model page. If your app reads the model name from configuration, doing this check once at startup saves you from surprising errors.

The First Request: the tools Field

Let's put the request body in a file. messages is as usual; the new part is the tools array. The argument definition is exactly the JSON schema we saw in the structured output post:

{
  "model": "qwen3:4b",
  "stream": false,
  "options": {
    "temperature": 0,
    "seed": 42
  },
  "messages": [
    {
      "role": "user",
      "content": "Where is my order 1042?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_order_status",
        "description": "Returns the shipping status of a bookstore order.",
        "parameters": {
          "type": "object",
          "properties": {
            "order_id": {
              "type": "integer",
              "description": "The order number"
            }
          },
          "required": [
            "order_id"
          ]
        }
      }
    }
  ]
}

$ curl -s http://localhost:11434/api/chat -d @request.json

I shortened the response: I cut the middle of the thinking text, the created_at field and some of the durations.

{
  "model": "qwen3:4b",
  "message": {
    "role": "assistant",
    "content": "",
    "thinking": "Okay, the user is asking where their order 1042 is. Let me check the tools provided. [...] I'll generate the tool call with the order_id as 1042.\n",
    "tool_calls": [
      {
        "id": "call_0oq5nk8o",
        "function": {
          "index": 0,
          "name": "get_order_status",
          "arguments": {
            "order_id": 1042
          }
        }
      }
    ]
  },
  "done": true,
  "done_reason": "stop",
  "total_duration": 3281668583,
  "prompt_eval_count": 152,
  "eval_count": 129
}

Notice three things. content is empty: the model said nothing to the user, it only asked for a call. arguments is not a string but a real JSON object, and order_id is an integer, just as the schema asked. The third is the thinking field: qwen3:4b is a thinking model and writes a short line of reasoning before it decides. Most of those 129 output tokens are that reasoning text.

The id value changes on every run; my first attempt returned call_9cohdcuh, this one call_0oq5nk8o. Don't write a test that depends on it.

Sending the Result Back: the tool Role

Now we are in the kitchen. We ran get_order_status(1042) ourselves and found the result. The second request sends the whole history: the user's question, the model's call request and our result. The tools list stays the same:

"messages": [
  { "role": "user", "content": "Where is my order 1042?" },
  { "role": "assistant", "content": "",
    "tool_calls": [{ "function": { "name": "get_order_status", "arguments": { "order_id": 1042 } } }] },
  { "role": "tool", "tool_name": "get_order_status",
    "content": "{\"order_id\":1042,\"status\":\"shipped\",\"carrier\":\"UPS\",\"eta\":\"2026-11-09\"}" }
]

tool_name on the tool message says which tool the result belongs to; content is always a string, so we serialized the JSON into one. The model's answer:

"content": "Your order #1042 is currently **shipped** with UPS and is estimated to arrive by **November 9, 2026**. Let me know if you need further assistance! 📦"
"prompt_eval_count": 219,
"eval_count": 271

The model never saw the order table; it only turned the four fields we gave it into a sentence. There is a small detail in the thinking field too: the model thought "Wait, that's a bit far in the future. Maybe a typo?". Its training data ends before 2026, so it takes the date for the distant future, but it decides "I have to go with the data given" and uses our date. A tool result being a stronger source than what the model "knows" is exactly what we want.

Building the Loop in TypeScript

Sending two requests by hand is instructive, but in a real app you can't know how many rounds you will need. The model may want to look up two orders at once, or see one result and ask for another tool. So we turn it into a loop: while the model asks for tools, run them; when it doesn't, return the answer. With the ollama package:

import ollama, { type Message, type Tool } from 'ollama';

// Our fake order table; in a real app this is the database
const orders: Record<number, { status: string; carrier: string; eta: string }> = {
  1042: { status: 'shipped', carrier: 'UPS', eta: '2026-11-09' },
  1043: { status: 'preparing', carrier: 'DHL', eta: '2026-11-12' },
};

// The real function lives in our code; the model only asks for it to be called
function getOrderStatus(args: { order_id: number }): string {
  const order = orders[args.order_id];
  return JSON.stringify(order ? { order_id: args.order_id, ...order } : { error: 'order not found' });
}

// What the model sees: name, description and a JSON schema for the arguments
const tools: Tool[] = [
  {
    type: 'function',
    function: {
      name: 'get_order_status',
      description: 'Returns the shipping status of a bookstore order.',
      parameters: {
        type: 'object',
        properties: {
          order_id: { type: 'integer', description: 'The order number' },
        },
        required: ['order_id'],
      },
    },
  },
];

// Tool name -> function that actually runs
const handlers: Record<string, (args: any) => string> = {
  get_order_status: getOrderStatus,
};

async function ask(question: string): Promise<string> {
  const messages: Message[] = [{ role: 'user', content: question }];

  // Upper bound so a confused model cannot loop forever
  for (let round = 1; round <= 5; round++) {
    const response = await ollama.chat({
      model: 'qwen3:4b',
      messages,
      tools,
      options: { temperature: 0, seed: 42 },
    });
    // The assistant message with tool_calls must stay in the history
    messages.push(response.message);

    const calls = response.message.tool_calls ?? [];
    if (calls.length === 0) {
      // No more tool requests: this is the final answer
      return response.message.content;
    }

    for (const call of calls) {
      const handler = handlers[call.function.name];
      const result = handler
        ? handler(call.function.arguments)
        : JSON.stringify({ error: `unknown tool ${call.function.name}` });
      console.log(`round ${round}: ${call.function.name}(${JSON.stringify(call.function.arguments)}) -> ${result}`);
      // The result goes back as a 'tool' message
      messages.push({ role: 'tool', tool_name: call.function.name, content: result });
    }
  }
  throw new Error('Too many tool rounds');
}

const question = process.argv[2] ?? 'Where is my order 1042?';
console.log(await ask(question));

Let's run it with three questions:

$ npx tsx tools-loop.ts "Where are my orders 1042 and 1043?"
round 1: get_order_status({"order_id":1042}) -> {"order_id":1042,"status":"shipped","carrier":"UPS","eta":"2026-11-09"}
round 1: get_order_status({"order_id":1043}) -> {"order_id":1043,"status":"preparing","carrier":"DHL","eta":"2026-11-12"}
Here are the current statuses for your orders:

- **Order #1042**: Shipped via UPS (Expected delivery: November 9, 2026)
- **Order #1043**: Preparing with DHL (Expected delivery: November 12, 2026)

Let me know if you need further assistance!

$ npx tsx tools-loop.ts "Where is my order 9999?"
round 1: get_order_status({"order_id":9999}) -> {"error":"order not found"}
Your order 9999 was not found. Please verify the order number or contact customer support for assistance.

$ npx tsx tools-loop.ts "Recommend a short science fiction novel."
I don't have access to book recommendation functionality or a science fiction novel database. My capabilities are limited to checking bookstore order statuses using the provided `get_order_status` tool. Would you like me to check the status of a specific order instead?

For the first question the model asked for two calls in a single round (parallel tool calls). Because our loop walks the whole tool_calls array, it ran both and appended two tool messages. For the second question the tool returned an error; handing the error to the model as JSON instead of throwing let the model write a proper sentence for the user. For the third question the model asked for no tool at all. It noticed it had no tool for book recommendations and stated its limit instead of making something up. That doesn't always happen; it depends on the tool list and the system prompt.

The Same Loop in C#, No Packages

To make the concept stick, let's write the C# side without any AI package too, just HttpClient and System.Text.Json. When you see in the next post how much of this Microsoft.Extensions.AI takes over, the difference will be clearer:

using System.Net.Http.Json;
using System.Text.Json;
using System.Text.Json.Nodes;

// Fake order table, same data as the TypeScript version
Dictionary<int, object> orders = new()
{
    [1042] = new { status = "shipped", carrier = "UPS", eta = "2026-11-09" },
    [1043] = new { status = "preparing", carrier = "DHL", eta = "2026-11-12" },
};

string GetOrderStatus(JsonNode? args)
{
    int orderId = args?["order_id"]?.GetValue<int>() ?? 0;
    return orders.TryGetValue(orderId, out object? order)
        ? JsonSerializer.Serialize(order)
        : """{"error":"order not found"}""";
}

// Same tool definition we sent with curl
JsonArray tools = JsonNode.Parse("""
[{
  "type": "function",
  "function": {
    "name": "get_order_status",
    "description": "Returns the shipping status of a bookstore order.",
    "parameters": {
      "type": "object",
      "properties": { "order_id": { "type": "integer", "description": "The order number" } },
      "required": ["order_id"]
    }
  }
}]
""")!.AsArray();

using HttpClient http = new() { BaseAddress = new Uri("http://localhost:11434"), Timeout = TimeSpan.FromMinutes(5) };

string question = args.Length > 0 ? args[0] : "Where is my order 1043?";
JsonArray messages = [new JsonObject { ["role"] = "user", ["content"] = question }];

// At most 5 rounds, then give up
for (int round = 1; round <= 5; round++)
{
    JsonObject request = new()
    {
        ["model"] = "qwen3:4b",
        ["stream"] = false,
        ["messages"] = messages.DeepClone(),
        ["tools"] = tools.DeepClone(),
        ["options"] = new JsonObject { ["temperature"] = 0, ["seed"] = 42 },
    };

    HttpResponseMessage httpResponse = await http.PostAsJsonAsync("/api/chat", request);
    httpResponse.EnsureSuccessStatusCode();
    JsonNode body = (await httpResponse.Content.ReadFromJsonAsync<JsonNode>())!;
    JsonObject message = body["message"]!.AsObject();
    // Keep the assistant message (with its tool_calls) in the history
    messages.Add(message.DeepClone());

    JsonArray? calls = message["tool_calls"]?.AsArray();
    if (calls is null || calls.Count == 0)
    {
        // No tool call: the model has given its final answer
        Console.WriteLine(message["content"]?.GetValue<string>());
        Console.WriteLine($"prompt tokens: {body["prompt_eval_count"]}, output tokens: {body["eval_count"]}");
        return;
    }

    foreach (JsonNode? call in calls)
    {
        string name = call!["function"]!["name"]!.GetValue<string>();
        JsonNode? arguments = call["function"]!["arguments"];
        string result = name == "get_order_status"
            ? GetOrderStatus(arguments)
            : $$"""{"error":"unknown tool {{name}}"}""";
        Console.WriteLine($"round {round}: {name}({arguments?.ToJsonString()}) -> {result}");
        // Send the result back as a 'tool' message
        messages.Add(new JsonObject { ["role"] = "tool", ["tool_name"] = name, ["content"] = result });
    }
}

Console.WriteLine("Too many tool rounds");

$ dotnet run
round 1: get_order_status({"order_id":1043}) -> {"status":"preparing","carrier":"DHL","eta":"2026-11-12"}
Your order #1043 is currently **preparing** with DHL. The estimated delivery date is **November 12, 2026**. Let me know if you need further assistance!
prompt tokens: 317, output tokens: 273

I raised the HttpClient Timeout to five minutes on purpose. The default is 100 seconds, and a thinking model on a slow machine or a shared Ollama easily goes past that. Also note the DeepClone calls: a JsonNode can belong to only one parent at a time, so putting the same message in both the history and the request needs a copy.

Tool Definitions Cost Tokens Too

Tools aren't free. The question "Where is my order 1042?" on its own took 20 prompt tokens. The same question with one tool definition took 152. So that small JSON schema adds about 130 tokens to every request, and it is sent again on every round of the loop. With a list of ten tools that is a noticeable share of the context window and of the cost. Keep descriptions short and clear, and leave out tools that a given request doesn't really need.

The answer qwen3:4b gave to the same question without tools is instructive as well: for 890 tokens it explained that it can't access orders and linked to the help pages of Amazon, Shopify and Walmart. Without a tool, the model uses the only thing it has, its general knowledge.

Common Pitfalls

The first pitfall is forgetting to add the assistant message to the history. If you only add the tool message, the model can't tell which call the result belongs to. The order is always: assistant (tool_calls), then one tool message per call.

The second is not putting an upper bound on the loop. The model may ask for the same tool again and again, especially when the tool keeps returning an error. Our examples stop after five rounds. When the limit is hit, give the user a proper error.

The third is not being ready for unknown tool names. The model can invent a name that isn't in the list. The loop should catch that without crashing and tell the model "there is no such tool"; our code returns the unknown tool error exactly for that.

The fourth is trying to switch thinking off on qwen3:4b. According to ollama show, this tag is the Qwen3 4B Thinking 2507 release (general.finetune: Thinking), a model that always thinks. With this model version, sending "think": false didn't make the reasoning go away; it leaked into content and ended with a </think> tag. When I left the think field out entirely, the reasoning went to the separate thinking field and content stayed clean. The details are in the reasoning post; what matters here is that you look at content once before printing it to the user as is.

The fifth, and the most important, is trusting the arguments the model produces. order_id is a number the model wrote; it could be another customer's order or a number that doesn't exist. The tool code must validate every argument as if it came from the user. Authorization, approval and validation are the main topic of the next post.

A Small Experiment

★☆☆ Add a second tool to tools-loop.ts for searching books (search_books, with a genre argument). Then ask "Recommend a short science fiction novel." again and see whether the model calls the tool this time.

★★☆ Leave the description of get_order_status empty, or write an unrelated sentence. Ask the same questions and note whether the model still picks the tool correctly and how prompt_eval_count changes.

★★☆ Lower the loop limit in the C# program to 1 and ask "Where are my orders 1042 and 1043?". Since the two parallel calls arrive in the same round, is the limit still enough, or do you exit before getting the final answer?

★★★ Write a version where the tool deliberately returns {"error":"database timeout"} every time. How many rounds does the model retry, and what does it finally tell the user? What would happen without the loop limit?

You now know the raw flow: definition, call request, you run it, result, final answer. In the next post we will hand this loop to Microsoft.Extensions.AI's UseFunctionInvocation middleware and AIFunctionFactory, and then move to the hard question: should we run every call the model asks for?