Writing Prompts for LLMs (2): Step-by-Step Thinking and Reasoning Models

Version note: The examples in this post were tested with Ollama 0.34 on local models (gemma3:4b, qwen3:4b), using .NET 10 and Node 22.

Guess First

You ask the model a small order calculation and get a wrong answer. You add "think step by step" to the end of the prompt. Does the answer get fixed? And if you switch to a reasoning model trained to think, how many tokens and how many seconds does that correctness cost you?

The answer to the first question is "sometimes", and we will see in which case it gets fixed. The answer to the second one is a much bigger number than most people guess.

Questions That Can't Be Answered in One Step

At the Bookstore, a customer buys 3 books at $12.50 and 2 bookmarks at $1.75. A 15% coupon applies to the books only. Shipping is $4.99, but free if the order total after the discount is over $35. The correct answer: the books are $37.50, $31.875 after the discount; together with the bookmarks, $35.375. That is over 35, so shipping is free, and the customer pays $35.38. It is a deliberately tricky question that lands just above the threshold.

What makes this hard for the model: remember the loop from the first post of the series, the model only produces the next token at each step. When you say "reply with the amount only", the first token of the answer, the first digit of the amount, has to be produced without any intermediate calculation. The model has no scratch paper; the only place it can think is the text it produces. Having it write out intermediate steps means giving it that scratch paper.

Making the Model Think Step by Step

Let's ask gemma3:4b the same question in three ways: directly, by saying "work through it step by step", and with the steps described by us:

import ollama from 'ollama';

const problem = `A customer buys 3 books at $12.50 each and 2 bookmarks at $1.75 each.
A 15% coupon applies to the books only. Shipping costs $4.99, but it is free
when the order total after the discount is over $35.
What does the customer pay?`;

const variants: Record<string, string> = {
  direct: `${problem} Reply with the amount only.`,
  'step by step': `${problem} Work through it step by step, then give the final line as "Total: $X".`,
  'given steps': `${problem} Solve it in these steps:
1. Book subtotal, then apply the coupon to the books only.
2. Add the bookmarks to get the order total after the discount.
3. Compare that total with $35 to decide on shipping.
4. Give the final line as "Total: $X".`,
};

for (const [name, prompt] of Object.entries(variants)) {
  const res = await ollama.chat({
    model: 'gemma3:4b',
    messages: [{ role: 'user', content: prompt }],
    options: { temperature: 0, seed: 42 },
  });
  console.log(`--- ${name}: ${res.eval_count} output tokens, ${(res.total_duration / 1e6).toFixed(0)} ms`);
  // Only the lines with an amount, to keep the output short
  for (const line of res.message.content.split('\n')) {
    if (line.includes('$')) console.log(line.trim());
  }
}

--- direct: 8 output tokens, 287 ms
$30.75
--- step by step: 206 output tokens, 4664 ms
1. **Cost of books:** 3 books * $12.50/book = $37.50
2. **Cost of bookmarks:** 2 bookmarks * $1.75/bookmark = $3.50
3. **Total cost before discount:** $37.50 + $3.50 = $41.00
4. **Discount amount:** 15% of $41.00 = 0.15 * $41.00 = $6.15
5. **Total cost after discount:** $41.00 - $6.15 = $34.85
6. **Shipping cost:** $4.99
7. **Final cost:** $34.85 + $4.99 = $39.84
Total: $39.84
--- given steps: 325 output tokens, 7490 ms
*   Cost per book: $12.50
*   Book subtotal: $12.50 * 3 = $37.50
*   Coupon discount (15% of $37.50): $37.50 * 0.15 = $5.625
*   Book total after coupon: $37.50 - $5.625 = $31.875
*   Cost per bookmark: $1.75
*   Bookmark subtotal: $1.75 * 2 = $3.50
*   Order total after discount: $31.875 + $3.50 = $35.375
**3. Compare Total with $35 to Decide on Shipping:**
*   Order total after discount: $35.375
*   Shipping cost: $4.99
*   Since $35.375 > $35, shipping is free.
Total: $35.375
Total: $35.38

The direct answer came in 8 tokens and 287 ms: $30.75. Wrong, but confident. When we said "work through it step by step", the model spent 206 tokens and was wrong again: $39.84. The difference is that this time we can see where the mistake is. In the fourth step it applied the coupon to the whole $41 total; it missed the "books only" constraint. Every step after that built a correct calculation on top of that mistake.

In the third variant we gave the steps: first the book subtotal and the coupon, then the bookmarks, then the comparison with $35. The model reached the correct result in 325 tokens. This technique is called chain-of-thought prompting; it became widespread with the 2022 paper by Wei et al. The lesson from our experiment is more concrete: saying "think step by step" gives the model scratch paper, but if it doesn't know which steps to take, it walks down the wrong path step by step. Breaking down the problem yourself is usually more reliable with a small model.

It has a price too: the variant that reached the correct answer produced 40 times more tokens than the direct answer and took 26 times longer.

Reasoning Models

Instead of asking for chain-of-thought in the prompt, you can also train the model to think before it answers. Reasoning models like OpenAI's o1 and DeepSeek-R1 are built on this idea. These models first produce a long thinking text and then write the answer.

Ollama controls this with the think parameter and returns the thinking text in a field separate from the answer, message.thinking. Not every model supports it: the capabilities list in the /api/show response must contain thinking. gemma3:4b doesn't have it, and when you send think: true Ollama returns an error right away: "gemma3:4b" does not support thinking. In this post we use qwen3:4b:

import ollama from 'ollama';

const info = await ollama.show({ model: 'qwen3:4b' });
const meta = info.model_info as unknown as Record<string, unknown>;

console.log('capabilities:', info.capabilities);
console.log('finetune:', meta['general.finetune'], '| version:', meta['general.version']);
// The last lines of the chat template: what is written right before the answer starts
console.log(info.template.trimEnd().split('\n').slice(-4).join('\n'));

capabilities: [ 'completion', 'tools', 'thinking' ]
finetune: Thinking | version: 2507
{{- if and (ne .Role "assistant") $last }}<|im_start|>assistant
<think>
{{ end }}
{{- end }}

We will discuss this output again in the pitfalls section; for now, note two lines. finetune: Thinking and version: 2507 say that the qwen3:4b tag on this machine is Qwen3's 2507 release dedicated to thinking. The last lines of the template also show that the answer is always started with a <think> tag.

Separating the Thinking from the Answer

Now let's ask qwen3:4b the same order question. We use streaming, because the thinking will take a while and in a real interface you will want to show the user something during that time:

import ollama from 'ollama';

const problem = `A customer buys 3 books at $12.50 each and 2 bookmarks at $1.75 each.
A 15% coupon applies to the books only. Shipping costs $4.99, but it is free
when the order total after the discount is over $35.
What does the customer pay? Reply with the amount only.`;

const started = performance.now();
let thinking = '';
let answer = '';
let firstAnswerMs = -1;

const stream = await ollama.chat({
  model: 'qwen3:4b',
  messages: [{ role: 'user', content: problem }],
  think: true,
  options: { temperature: 0, seed: 42 },
  stream: true,
});

for await (const chunk of stream) {
  // Thinking and answer come in separate fields of the same chunk
  thinking += chunk.message.thinking ?? '';
  if (chunk.message.content) {
    if (firstAnswerMs < 0) firstAnswerMs = performance.now() - started;
    answer += chunk.message.content;
  }
  if (chunk.done) {
    console.log(`thinking: ${thinking.length} chars, starts: ${JSON.stringify(thinking.slice(0, 70))}`);
    console.log(`answer starts after ${(firstAnswerMs / 1000).toFixed(1)} s`);
    console.log(`answer: ${JSON.stringify(answer.trim().slice(-60))}`);
    console.log(`output tokens: ${chunk.eval_count}, total: ${(chunk.total_duration / 1e9).toFixed(1)} s`);
  }
}

thinking: 7573 chars, starts: "Okay, let's try to figure out how much the customer pays. Hmm, first, "
answer starts after 68.3 s
answer: "rest cent:\n- $35.375$ rounds to $35.38$\n\n$$\n\\boxed{35.38}\n$$"
output tokens: 2984, total: 76.7 s

The model found the correct answer: $35.38. But to get there, it wrote a thinking text of 7573 characters and produced 2984 tokens in total. The first character of the answer arrived after 68 seconds; during that time, the only thing to show the user is the text flowing in the thinking field. The "thinking..." indicator in chat interfaces relies on exactly this field.

A second observation: even though we said "reply with the amount only", the answer ended with a summary of the calculation and a LaTeX \boxed{35.38}. A model that thinks correctly can still bend a format instruction; correctness and obedience are different things. If you are going to use the answer in code, you need structured output, the topic of the next post.

Reasoning in C#: ReasoningOptions

In Microsoft.Extensions.AI, thinking is requested with a provider-independent option: ChatOptions.Reasoning. Below we ask two models the same question; the question is deliberately easy:

using System.Diagnostics;
using Microsoft.Extensions.AI;
using OllamaSharp;

Uri ollama = new("http://localhost:11434");
string question = "Which is larger, 9.11 or 9.9? Reply with the number only.";

// A regular model: answers directly
IChatClient gemma = new OllamaApiClient(ollama, "gemma3:4b");
await Ask(gemma, new ChatOptions { Temperature = 0, Seed = 42 });

// A reasoning model: thinks first, then answers
IChatClient qwen = new OllamaApiClient(ollama, "qwen3:4b");
await Ask(qwen, new ChatOptions
{
    Temperature = 0,
    Seed = 42,
    Reasoning = new ReasoningOptions { Effort = ReasoningEffort.Medium },
});

async Task Ask(IChatClient client, ChatOptions options)
{
    Stopwatch sw = Stopwatch.StartNew();
    ChatResponse response = await client.GetResponseAsync(question, options);

    // The thinking text arrives as a separate content type, not inside response.Text
    string thinking = string.Concat(response.Messages
        .SelectMany(m => m.Contents)
        .OfType<TextReasoningContent>()
        .Select(c => c.Text));

    Console.WriteLine($"--- {response.ModelId}");
    Console.WriteLine($"answer: {response.Text.Trim()}");
    Console.WriteLine($"thinking: {thinking.Length} chars");
    Console.WriteLine($"output tokens: {response.Usage?.OutputTokenCount}, time: {sw.ElapsedMilliseconds} ms");
}

--- gemma3:4b
answer: 9.9
thinking: 0 chars
output tokens: 5, time: 296 ms
--- qwen3:4b
answer: 9.9
thinking: 4680 chars
output tokens: 1566, time: 42025 ms

When I captured the outgoing request with a DelegatingHandler, I saw that OllamaSharp sends ReasoningEffort.Medium to Ollama as "think": "medium". Besides true/false, Ollama accepts the levels "low", "medium" and "high" for think; but on this model "medium" and true produced exactly the same thinking, 4680 characters, for the same question. How much a model respects the levels depends on the model. The thinking text does not leak into response.Text either; it arrives as a separate TextReasoningContent, and you decide what to write to the history or the log.

The Price: Tokens and Time

Let's put this post's measurements side by side. On the order question, gemma3:4b answered directly: 8 tokens, 0.3 seconds, wrong. When we gave the steps: 325 tokens, 7.5 seconds, correct. qwen3:4b, thinking: 2984 tokens, 77 seconds, correct. On "9.11 or 9.9?" both models answered correctly; gemma3:4b with 5 tokens and 296 ms, qwen3:4b with 1566 tokens and 42 seconds. On an easy question, thinking spent 300 times the tokens without improving the answer.

In the cloud, this ratio goes straight to the bill, because thinking tokens are generated tokens too. The eval_count we measured locally includes the thinking; cloud providers also usually bill thinking tokens as output tokens, so check your provider's pricing page for the details. On the latency side, the time the user waits before seeing the answer grows with the length of the thinking text.

The practical split I use: for single-step tasks such as classification, extraction, summaries and translation, reasoning is unnecessary; a fast model and a good prompt are enough. For calculations where several constraints interact, multi-step planning and "where is the bug in this code" kinds of analysis, reasoning can really make a difference. But for calculations that need an exact answer, like money, the most reliable way is not to leave the math to the model at all: the model's job is to understand the rules and call the right function, and the code computes the total. We will do that in the tool calling posts.

Common Pitfalls

The first pitfall is assuming that think: false turns thinking off on every model. Remember the show.ts output above: this model is a release dedicated to thinking, and its template always starts the answer with <think>. Let's see what think: false does:

import ollama from 'ollama';

const res = await ollama.chat({
  model: 'qwen3:4b',
  messages: [{ role: 'user', content: 'Which is larger, 9.11 or 9.9? Reply with the number only.' }],
  think: false,
  options: { temperature: 0, seed: 42 },
});

const content = res.message.content;
console.log('thinking field:', res.message.thinking);
console.log('content length:', content.length);
console.log('contains </think>:', content.includes('</think>'));
console.log('after </think>:', JSON.stringify(content.split('</think>').pop()!.trim()));
console.log('output tokens:', res.eval_count);

thinking field: undefined
content length: 4693
contains </think>: true
after </think>: "9.9"
output tokens: 1566

The thinking field came back empty, but the model did not stop thinking: it still produced 1566 tokens, and the thinking text got mixed into content together with the </think> tag. On this model, think: false turned off not the thinking but the parsing of the thinking. An application that prints content shows the user four thousand characters of inner monologue. When you change models, or when a tag starts pointing to a new release, check what you are working with using /api/show.

The second is setting the output limit without accounting for the thinking. Thinking tokens eat from the num_predict budget too:

import ollama from 'ollama';

const res = await ollama.chat({
  model: 'qwen3:4b',
  messages: [{ role: 'user', content: 'Which is larger, 9.11 or 9.9? Reply with the number only.' }],
  think: true,
  options: { temperature: 0, seed: 42, num_predict: 200 },
});

console.log('done_reason:', res.done_reason, '| output tokens:', res.eval_count);
console.log('thinking chars:', res.message.thinking?.length ?? 0);
console.log('answer:', JSON.stringify(res.message.content));

done_reason: length | output tokens: 200
thinking chars: 569
answer: ""

The 200-token limit ran out in the middle of the thinking, and the answer came back as an empty string. With a reasoning model, set the limit high enough for the thinking and the answer together, and always check done_reason.

The third is taking the first "Total:" line you find in chain-of-thought output. The output of the third variant has two: first 35.375, then the rounded 35.38. If you extract a result from free text, take the last line; better yet, use structured output that asks for the result in a separate field.

The fourth is treating the thinking text as proof of correctness. Thinking is a useful trace of how the model reached the answer, but as we saw in the second variant, tidy-looking steps can be built on a wrong assumption. Read the thinking text to debug, and validate the answer separately.

A Small Experiment

★☆☆ In the order question in cot.ts, set the shipping threshold to $36 instead of $35. What should the correct answer be? Which of the three variants apply the new threshold correctly?

★★☆ Run think-stream.ts with an easy question such as "What is the capital of France?". How many tokens did qwen3:4b think, and after how many seconds did the answer start? Ask gemma3:4b the same question without thinking and compare the two results.

★★★ Add another Stopwatch to the C# example and read the thinking as a stream with GetStreamingResponseAsync: print the TextReasoningContent pieces in gray and the answer in the normal color, and measure when the first character of the answer arrives.

In this post we tried to read the model's answer in code twice, and both times we got stuck on free text: two "Total:" lines and a \boxed{}. In the next post we fix this problem at the root: with structured output we will ask the model for an answer that matches a JSON schema, take the answer straight into a record with GetResponseAsync<T> in C#, and see what the schema guarantees and what it does not.