Unbounded Consumption: Denial of Wallet, Context Bombs and Limiting Agent Loops
In an LLM the unit of cost is the token, not the request, so input, output, time and budget are limited before every call. We measure a 120,000-character document being silently truncated by Ollama into a wrong answer and qwen3:4b thinking for 8,000 tokens on a seven-word question before returning nothing, then build reserved user and tenant budgets, timeouts and a state-hash loop guard.
Version note: The code in this post was built and run with .NET 10 and ASP.NET Core 10, Microsoft.Extensions.AI 11.0.0, OllamaSharp 5.5.2, and Ollama 0.34.4 for the local models (gemma3:4b, qwen3:4b). Cost figures are examples based on Anthropic's pricing page (14 November 2026); this post was not run against a cloud provider.
Guess First
Bookstore.Assistant has a new feature: users can paste a document into the chat and ask questions about it. Someone pastes a 120,000-character text; the first line states the return window, the rest is a shipping FAQ. The question: "Within how many days of delivery can I return a book?" The assistant handles the request with a local gemma3:4b and there are no limits. What happens: an error, a long wait, or a wrong answer?
The second guess: on the same assistant, qwen3:4b is asked a seven-word question, "Is 1000003 a prime number? Answer only yes or no." The answer is one word. How many output tokens does this question cost?
This post covers LLM06 Unbounded Consumption. In the OWASP Top 10 for LLM Applications 2026 the entry rose four places, from tenth to sixth. At the center of the document's definition is an asymmetry: with a request that costs the attacker almost nothing, they can trigger disproportionately expensive computation on your side. The target is sometimes the availability of the service, sometimes the bill (denial of wallet), sometimes the model itself.
What LLM06 Covers in 2026
The 2026 edition takes the entry beyond the classic "send lots of requests" scenario. Of the risks the document lists, these concern Bookstore directly: floods of variable-length input and output explosion, denial of wallet, large-context abuse with requests close to the context window limit, short and innocent-looking prompts that push reasoning models into long or non-terminating thinking loops, tool calls multiplying by triggering each other, and model extraction. The emphasis changes on the mitigation side too: according to the document, limiting the number of requests is no longer enough; you need token-aware cost controls, spending caps that cannot be exceeded, agent-level circuit breakers and continuous cost monitoring.
In post 7 of the AI with .NET series we built the three basic layers of cost: a cache with DistributedCachingChatClient, per-user rate limiting, and a daily token budget with UsageDetails. This post sits on top of them and looks at how an attacker or a careless client can slip between those layers. The general API counterpart, API4 Unrestricted Resource Consumption, was in post 5 of the API Security series; the ideas of page size, timeouts and quotas apply here too, but in an LLM the unit is the token, not the request.
Measure First: Large Context Gets Truncated Silently
I measured the guess from the opening. The return window is on the first line of the document, followed by the shipping FAQ until the target size is reached; the question comes last:
IChatClient gemma = new OllamaApiClient(http, "gemma3:4b");
foreach (int size in args[1].Split(',').Select(int.Parse))
{
string document = PolicyDocument.Build(size);
ChatOptions options = new() { MaxOutputTokens = 60, Temperature = 0 };
if (numCtx is int ctx)
{
options.AddOllamaOption(OllamaOption.NumCtx, ctx);
}
Stopwatch sw = Stopwatch.StartNew();
ChatResponse response = await gemma.GetResponseAsync($"{document}\n\n{PolicyDocument.Question}", options);
bool correct = response.Text.Contains("14");
Console.WriteLine(
$"chars {document.Length,7} | input tokens {response.Usage?.InputTokenCount,6} | {sw.Elapsed.TotalSeconds,5:F1} s | correct {(correct ? "yes" : "no "),-3} | {response.Text.ReplaceLineEndings(" ").Trim()[..Math.Min(60, response.Text.Trim().Length)]}");
}$ dotnet run -- context 2000,10000,40000,120000
chars 2067 | input tokens 666 | 3.0 s | correct yes | According to the document, books can be returned within **14
chars 10066 | input tokens 3204 | 3.4 s | correct yes | According to the document, books can be returned within **14
chars 40082 | input tokens 2051 | 2.5 s | correct no | The document doesn't provide information about return delive
chars 120051 | input tokens 2051 | 2.5 s | correct no | The document doesn't provide information about returns or de
# the Ollama server's log
level=WARN source=llama_server.go:320 msg="truncating input prompt" limit=2051 prompt=12920 keep=5 new=2051
level=WARN source=llama_server.go:320 msg="truncating input prompt" limit=2051 prompt=38980 keep=5 new=2051
$ dotnet run -- context 40000,120000 40960
chars 40082 | input tokens 12920 | 14.2 s | correct yes | A book can be returned within 14 days of delivery.
chars 120051 | input tokens 38980 | 34.1 s | correct yes | A book can be returned within 14 days of delivery.No error. The 40,000- and 120,000-character requests return successfully in 2.5 seconds, and both give the wrong answer. The usage field says 2,051 input tokens, while Ollama's log says the prompt was 12,920 and 38,980 tokens and was truncated to 2,051. On this machine and this Ollama version gemma3:4b was loaded with a 4,096-token context (the default can vary by version and hardware); the beginning of the prompt that does not fit is dropped, and the return window sits exactly in that part. All that remains is a WARN-level server log line; the HTTP response the client gets is 200. That makes context size an accuracy setting as much as a cost setting. Your application needs to know about it, not just someone reading the server log.
With the context raised to 40,960 tokens the answer is correct, but the price becomes visible: 120,000 characters are 38,980 input tokens and 34 seconds. For this text the measured ratio is about 3.1 characters per token. Now picture the same request in the cloud. As of 14 November 2026 Anthropic's pricing page lists 2 dollars per million input tokens for Claude Sonnet 5.5; 38,980 tokens cost about 0.08 dollars in one request. That looks small. But if the document stays in the conversation history it is sent again on every new turn: a 20-turn conversation is about 780,000 input tokens and 1.56 dollars. A bot that opens a thousand conversations a day produces a bill of more than forty thousand dollars a month, without any single request hitting a rate limit. The 2026 document describes exactly this scenario: in agent sessions whose context grows gradually the cost of each turn climbs, while each request stays within budget on its own.
Input Limits: Bytes, Characters and Conversation Size
The first defense is to reject before anything expensive happens. I collected the limits in a single record; the token estimate is a bit more pessimistic than the ratio we measured:
public sealed record AssistantLimits
{
public int MaxMessageChars { get; init; } = 8_000;
public int MaxConversationTokens { get; init; } = 6_000;
public int MaxOutputTokens { get; init; } = 400;
public TimeSpan RequestTimeout { get; init; } = TimeSpan.FromSeconds(30);
// Measured ratio ~3.1 chars/token; the estimate is deliberately pessimistic
public static int EstimateTokens(string text) => (int)Math.Ceiling(text.Length / 3.0);
}The endpoint applies three checks in order: body size in Kestrel (before model binding even starts), message length, and the estimated number of tokens the conversation would send to the model on every turn. The model call is made with a CancellationToken tied to the client's connection and to a wall-clock time limit:
public static class AssistantEndpoint
{
private static readonly ConcurrentDictionary<string, List<ChatMessage>> Conversations = new();
public static void MapAssistant(this WebApplication app)
{
app.MapPost("/api/assistant/conversations/{id}/messages", HandleAsync)
.WithMetadata(new RequestSizeLimitAttribute(32_768)); // body over 32 KB: 413 without calling the model
}
private static async Task<IResult> HandleAsync(
string id, MessageRequest body, HttpContext http, IChatClient model, TokenLedger ledger, AssistantLimits limits, ILogger<MessageRequest> logger)
{
// Demo: identity from headers; in a real application it comes from token claims
Caller caller = new(http.Request.Headers["X-Demo-Tenant"].ToString(), http.Request.Headers["X-Demo-User"].ToString());
if (body.Message.Length > limits.MaxMessageChars)
{
return Results.Problem(statusCode: 400, title: "Message too long",
detail: $"Messages are limited to {limits.MaxMessageChars} characters. Upload long documents as files instead.");
}
List<ChatMessage> history = Conversations.GetOrAdd(id, _ => [new(ChatRole.System, "You are the Bookstore assistant. Answer briefly.")]);
ChatMessage user = new(ChatRole.User, body.Message);
int conversationTokens = history.Sum(m => AssistantLimits.EstimateTokens(m.Text)) + AssistantLimits.EstimateTokens(user.Text);
if (conversationTokens > limits.MaxConversationTokens)
{
return Results.Problem(statusCode: 422, title: "Conversation too long",
detail: $"This conversation would send about {conversationTokens} tokens per turn. Start a new conversation.");
}
// If the client disconnects or the time runs out, the model call is cancelled too
using CancellationTokenSource cts = CancellationTokenSource.CreateLinkedTokenSource(http.RequestAborted);
cts.CancelAfter(limits.RequestTimeout);
Stopwatch sw = Stopwatch.StartNew();
try
{
IChatClient chat = new BudgetChatClient(model, ledger, caller);
ChatResponse response = await chat.GetResponseAsync(
[.. history, user], new ChatOptions { MaxOutputTokens = limits.MaxOutputTokens }, cts.Token);
history.Add(user);
history.AddRange(response.Messages);
return Results.Ok(new
{
answer = response.Text,
inputTokens = response.Usage?.InputTokenCount,
outputTokens = response.Usage?.OutputTokenCount,
finishReason = response.FinishReason?.Value,
});
}
catch (BudgetExceededException ex)
{
http.Response.Headers.RetryAfter = ((int)ex.RetryAfter.TotalSeconds).ToString();
return Results.Problem(statusCode: 429, title: "Assistant budget exhausted", detail: ex.Message);
}
catch (OperationCanceledException) when (http.RequestAborted.IsCancellationRequested)
{
logger.LogInformation("Client disconnected; model call cancelled after {Seconds:F1} s", sw.Elapsed.TotalSeconds);
return Results.Empty;
}
catch (OperationCanceledException)
{
logger.LogWarning("Model call timed out after {Seconds:F1} s", sw.Elapsed.TotalSeconds);
return Results.Problem(statusCode: 504, title: "The assistant took too long", detail: "Please try again with a shorter question.");
}
}
}# a 40,000-character message
$ curl -i -X POST .../conversations/c2/messages --data-binary @big.json
HTTP/1.1 413 Payload Too Large
# a 9,000-character message
$ curl -i -X POST .../conversations/c2/messages --data-binary @mid.json
HTTP/1.1 400 Bad Request
{"type":"https://tools.ietf.org/html/rfc9110#section-15.5.1","title":"Message too long","status":400,"detail":"Messages are limited to 8000 characters. Upload long documents as files instead."}
# a 6,052-character message, twice in the same conversation
$ curl -X POST .../conversations/long/messages --data-binary @faq.json
{"answer":"We ship nationwide! ...","inputTokens":1930,"outputTokens":29,"finishReason":"stop"}
$ curl -X POST .../conversations/long/messages --data-binary @faq.json
{"type":"https://tools.ietf.org/html/rfc4918#section-11.2","title":"Conversation too long","status":422,"detail":"This conversation would send about 4094 tokens per turn. Start a new conversation."}The 40,000-character body stopped in Kestrel with a 413, and the 9,000-character message got a 400; neither reached the model. The interesting case is the third: a 6,000-character message, reasonable on its own, got a 422 when it came a second time in the same conversation, because together with the history every turn would have sent about 4,094 tokens. The conversation size limit is the cure for the "document resent on every turn" bill above. Summarizing long conversations instead of rejecting them is another option; we saw chat reducers in post 5 of the AI with .NET series. And the right place for long documents is not a chat message but a file upload flow with size limits, page limits and a scanning pipeline.
Error messages are a design decision too. Saying "Message too long" is enough; stating the exact limit helps the user but also tells an attacker the size of the largest valid request. In Bookstore the character limit is visible in the UI anyway, so it is fine in the message; for the conversation limit we give an approximate number.
Time Limits: Timeouts and Dropped Connections
A token limit bounds cost but not time; a slow model, a queued GPU or a never-ending reasoning can keep a request busy for minutes. The linked token in the endpoint catches two cases:
# the client closes the connection after 2 seconds
$ curl --max-time 2 -X POST .../conversations/d1/messages -d '{"message":"Write a detailed 2000-word essay about the history of the printing press."}'
curl exit 28
info: Bookstore.Assistant.Consumption.MessageRequest[0]
Client disconnected; model call cancelled after 2.0 s
# the Ollama side
srv stop: cancel task, id_task = 368
slot release: id 0 | task 368 | stop processing: n_tokens = 119, truncated = 0
# the same request with Assistant__RequestTimeout=00:00:03
$ curl -X POST .../conversations/t1/messages -d '{"message":"Write a detailed 2000-word essay about the history of the printing press."}'
{"type":"https://tools.ietf.org/html/rfc9110#section-15.6.5","title":"The assistant took too long","status":504,"detail":"Please try again with a shorter question."}
warn: Bookstore.Assistant.Consumption.MessageRequest[0]
Model call timed out after 3.0 sWhen the client gave up after two seconds, RequestAborted fired, the model call was cancelled, and generation stopped on the Ollama side too: only 75 tokens were generated for the 44-token prompt before the task was closed. Without that link the model would have kept producing, for up to 400 tokens, an answer nobody would read; in the cloud those output tokens go on the bill. When the wall-clock limit was exceeded the client got a 504. The budget reservation of a cancelled request is not returned; that is a deliberate choice, because a cloud provider charges for the tokens generated up to the moment of cancellation anyway.
Output Limits and Reasoning Tokens
On to the second guess. I asked qwen3:4b the same question with three settings:
IChatClient qwen = new OllamaApiClient(http, "qwen3:4b");
string question = "Is 1000003 a prime number? Answer only yes or no.";
(string Label, ChatOptions Options)[] variants =
[
("thinking off", new() { Reasoning = new ReasoningOptions { Effort = ReasoningEffort.None }, MaxOutputTokens = 2000 }),
("thinking on", new() { Reasoning = new ReasoningOptions { Effort = ReasoningEffort.Medium }, MaxOutputTokens = 8000 }),
("thinking on, cap 256", new() { Reasoning = new ReasoningOptions { Effort = ReasoningEffort.Medium }, MaxOutputTokens = 256 }),
];
foreach ((string label, ChatOptions options) in variants)
{
Stopwatch sw = Stopwatch.StartNew();
ChatResponse response = await qwen.GetResponseAsync(question, options);
int reasoningChars = response.Messages.SelectMany(m => m.Contents).OfType<TextReasoningContent>().Sum(c => c.Text.Length);
Console.WriteLine(
$"{label,-21} | output tokens {response.Usage?.OutputTokenCount,5} | reasoning chars {reasoningChars,6} | {sw.Elapsed.TotalSeconds,5:F1} s | finish {response.FinishReason?.Value,-6} | answer: \"{response.Text.Trim()}\"");
}$ dotnet run -- think
thinking off | output tokens 2000 | reasoning chars 0 | 58.4 s | finish length | answer: "Okay, let's see. I need to figure out if 1000003 is a ..."
thinking on | output tokens 8000 | reasoning chars 11060 | 216.8 s | finish length | answer: ""
thinking on, cap 256 | output tokens 256 | reasoning chars 626 | 5.9 s | finish length | answer: ""All three lines teach something. With thinking on, the model spent all 8,000 tokens thinking, took 217 seconds and returned an empty answer; for a seven-word question with a one-word answer. This is exactly the 2026 document's "reasoning-loop" risk: a short, innocent-looking prompt consumes a large thinking budget without tripping any input filter. At Sonnet 5.5's output price (10 dollars per million tokens) this single question costs 0.08 dollars in the cloud.
The "thinking off" line is even more surprising. Thinking did not switch off with ReasoningEffort.None; when I sent the request directly to the Ollama API with think: false the result did not change either. The model wrote its reasoning into the answer text and hit the 2,000-token limit. So relying on a "reasoning off" setting and loosening the output limit only moves the cost somewhere else. The only reliable control is MaxOutputTokens: with a limit of 256 the same request finished in 5.9 seconds. The answer is still empty, but this time its price is known and small. The practical conclusion: use a reasoning model only in workflows that really need it, put an output ceiling on every request, and show responses whose FinishReason is length with a meaningful message instead of an empty bubble.
Budgets: Reserve First, Settle Later
The daily budget in post 7 of the AI with .NET series added up actual usage after the response. In an attack scenario that has a gap: a hundred requests arriving at once do not appear to exceed the budget, because none of them has finished yet. The 2026 document also recommends that spending caps be enforcement that halts inference rather than an alert, and that requests be rejected by pre-flight estimation before inference starts. In Bookstore I did this with two-level accounting: a daily limit per user and a monthly limit per tenant. Before the call the worst case (input estimate plus the maximum allowed output) is reserved, and when the response arrives it is replaced with actual usage:
// Hard daily limit per user, monthly per tenant; reserve BEFORE the call
public sealed class TokenLedger(TimeProvider time, ILogger<TokenLedger> logger, long userDailyLimit, long tenantMonthlyLimit)
{
private static readonly Meter Meter = new("Bookstore.Assistant");
private static readonly Counter<long> Tokens = Meter.CreateCounter<long>("bookstore.assistant.tokens", "{token}");
private static readonly Counter<long> Rejections = Meter.CreateCounter<long>("bookstore.assistant.budget.rejections", "{request}");
private readonly ConcurrentDictionary<string, long> _used = new();
private readonly ConcurrentDictionary<string, bool> _alerted = new();
private readonly Lock _gate = new();
public long Reserve(Caller caller, long estimate)
{
DateTimeOffset now = time.GetUtcNow();
string userKey = $"user:{caller.UserId}:{now:yyyy-MM-dd}";
string tenantKey = $"tenant:{caller.TenantId}:{now:yyyy-MM}";
lock (_gate)
{
if (_used.GetValueOrDefault(userKey) + estimate > userDailyLimit)
{
Reject(caller, "user");
throw new BudgetExceededException($"user {caller.UserId}", now.UtcDateTime.Date.AddDays(1) - now.UtcDateTime);
}
if (_used.GetValueOrDefault(tenantKey) + estimate > tenantMonthlyLimit)
{
Reject(caller, "tenant");
throw new BudgetExceededException($"tenant {caller.TenantId}", TimeSpan.FromHours(1));
}
_used[userKey] = _used.GetValueOrDefault(userKey) + estimate;
_used[tenantKey] = _used.GetValueOrDefault(tenantKey) + estimate;
}
return estimate;
}
// When the response arrives, the reservation is replaced with actual usage
public void Settle(Caller caller, long reserved, long input, long output)
{
DateTimeOffset now = time.GetUtcNow();
string tenantKey = $"tenant:{caller.TenantId}:{now:yyyy-MM}";
long delta = input + output - reserved;
lock (_gate)
{
_used.AddOrUpdate($"user:{caller.UserId}:{now:yyyy-MM-dd}", delta, (_, v) => v + delta);
_used.AddOrUpdate(tenantKey, delta, (_, v) => v + delta);
}
Tokens.Add(input, new("tenant", caller.TenantId), new("user", caller.UserId), new("direction", "input"));
Tokens.Add(output, new("tenant", caller.TenantId), new("user", caller.UserId), new("direction", "output"));
// Alert threshold: warn once at 80%, the limit itself is hard
long tenantUsed = _used[tenantKey];
if (tenantUsed >= tenantMonthlyLimit * 0.8 && _alerted.TryAdd(tenantKey, true))
{
logger.LogWarning("Tenant {TenantId} used {Used} of {Limit} tokens this month", caller.TenantId, tenantUsed, tenantMonthlyLimit);
}
}
public long Used(string key) => _used.GetValueOrDefault(key);
private void Reject(Caller caller, string scope)
{
Rejections.Add(1, new("tenant", caller.TenantId), new("scope", scope));
logger.LogWarning("Budget rejection for {Scope}: tenant {TenantId}, user {UserId}", scope, caller.TenantId, caller.UserId);
}
}// Every model call is charged to the budget first; estimate = input estimate + maximum allowed output
public sealed class BudgetChatClient(IChatClient inner, TokenLedger ledger, Caller caller) : DelegatingChatClient(inner)
{
public override async Task<ChatResponse> GetResponseAsync(
IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken cancellationToken = default)
{
List<ChatMessage> list = messages as List<ChatMessage> ?? [.. messages];
int maxOutput = options?.MaxOutputTokens
?? throw new InvalidOperationException("MaxOutputTokens must be set; an unbounded output cannot be budgeted.");
long estimate = list.Sum(m => AssistantLimits.EstimateTokens(m.Text)) + maxOutput;
long reserved = ledger.Reserve(caller, estimate);
ChatResponse response = await base.GetResponseAsync(list, options, cancellationToken);
ledger.Settle(caller, reserved, response.Usage?.InputTokenCount ?? estimate, response.Usage?.OutputTokenCount ?? maxOutput);
return response;
}
public override IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken cancellationToken = default) =>
throw new NotSupportedException("Streaming is not budgeted in this sample.");
}A call without MaxOutputTokens cannot be budgeted; the client treats it as a configuration error and never makes the call. Three users from the same tenant each send a 6,000-character request:
# Budget__UserDailyTokens=6000, Budget__TenantMonthlyTokens=9000; each request 6,052 characters
ada r1 -> 200 retry-after= {"inputTokens": 1930, "outputTokens": 39}
ada r2 -> 200 retry-after= {"inputTokens": 1930, "outputTokens": 38}
ada r3 -> 429 retry-after=4558 {"detail": "Token budget exhausted for user ada."}
grace r4 -> 200 retry-after= {"inputTokens": 1930, "outputTokens": 38}
grace r5 -> 200 retry-after= {"inputTokens": 1930, "outputTokens": 42}
linus r6 -> 429 retry-after=3600 {"detail": "Token budget exhausted for tenant acme."}
# server console
metric bookstore.assistant.tokens +1930 tenant=acme user=ada direction=input
metric bookstore.assistant.tokens +39 tenant=acme user=ada direction=output
metric bookstore.assistant.tokens +1930 tenant=acme user=ada direction=input
metric bookstore.assistant.tokens +38 tenant=acme user=ada direction=output
metric bookstore.assistant.budget.rejections +1 tenant=acme scope=user
warn: Bookstore.Assistant.Consumption.TokenLedger[0]
Budget rejection for user: tenant acme, user ada
metric bookstore.assistant.tokens +1930 tenant=acme user=grace direction=input
metric bookstore.assistant.tokens +38 tenant=acme user=grace direction=output
metric bookstore.assistant.tokens +1930 tenant=acme user=grace direction=input
metric bookstore.assistant.tokens +42 tenant=acme user=grace direction=output
warn: Bookstore.Assistant.Consumption.TokenLedger[0]
Tenant acme used 7877 of 9000 tokens this month
metric bookstore.assistant.budget.rejections +1 tenant=acme scope=tenant
warn: Bookstore.Assistant.Consumption.TokenLedger[0]
Budget rejection for tenant: tenant acme, user linusAda's third request was rejected although she had used only 3,937 tokens so far, because the reservation counts the worst case and with an estimate of about 2,430 tokens the total exceeded the daily limit of 6,000. This pessimism is deliberate: because the reservation is larger than actual usage, concurrent requests cannot push past the limit. After Grace's second request the tenant passed 80 percent of its monthly limit and a warning fired once; Linus hit the tenant limit while his own daily budget was untouched. The Retry-After value differs by scope as well: for a user, the time left until UTC (Coordinated Universal Time) midnight; for a tenant, one hour. In production the _used dictionary moves to a shared store such as Redis; atomic increments and TTL exist exactly for this.
Metrics are the raw material of alerts. Once the bookstore.assistant.tokens counter leaves the process through OpenTelemetry with its tenant, user and direction tags (we set up GenAI telemetry in post 8 of the AI with .NET series), two simple rules catch a lot: a user's token consumption in the last hour rising to several times their own 30-day median, and the rate of rejected requests jumping. Watch the cardinality of the user tag: with tens of thousands of users, pulling the top N consumers from logs is cheaper than per-user metrics.
The Agent Loop: A Circuit Breaker with a State Hash
In post 4 we put an iteration limit, a per-request budget of total tool calls and a wall-clock limit on the agent loop. The 2026 document recommends one more thing: detecting loops with a state hash. The idea is simple; if the same tool is called with the same arguments and returns the same result, there is no progress. Bookstore's carrier integration is a realistic source of this: the carrier's API says "ask again" until the status updates.
// The carrier integration: always returns a result that says "try again"
public sealed class ShippingTools
{
[Description("Returns the shipment status of one of the current customer's orders.")]
public object GetShipmentStatus(int orderId) => new
{
orderId,
status = "pending",
message = "Status is not final yet. Call get_shipment_status again with the same orderId before answering the customer.",
};
}// Same tool, same arguments, same result means no progress; catch the loop with a state hash
public sealed class LoopGuard(int maxRepeats)
{
private readonly Dictionary<string, int> _seen = [];
public int Calls { get; private set; }
public async ValueTask<object?> InvokeAsync(FunctionInvocationContext context, CancellationToken ct)
{
Calls++;
object? result = await context.Function.InvokeAsync(context.Arguments, ct);
string state = Hash($"{context.Function.Name}|{JsonSerializer.Serialize(context.Arguments)}|{JsonSerializer.Serialize(result)}");
int count = _seen[state] = _seen.GetValueOrDefault(state) + 1;
Console.WriteLine($" call {Calls,2}: {context.Function.Name}({JsonSerializer.Serialize(context.Arguments)}) state {state[..8]} seen {count}x");
if (count > maxRepeats)
{
context.Terminate = true;
return new { error = "Stopped: the same call returned the same result repeatedly." };
}
return result;
}
private static string Hash(string text) => Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(text)));
}I tried the real model first. With two different message texts, and even though the tool result explicitly said "call again", qwen3:4b answered the customer after one call that the status was not final yet. Good behavior; but a model behaving well today does not mean it will tomorrow, especially when, as we saw in post 6, the model artifact can change quietly. To see the guard work I produced the loop with a fake model, as in post 10 of the AI with .NET series: an IChatClient that asks for the same tool call on every turn.
foreach (bool withGuard in new[] { false, true })
{
FakeLoopingChatClient fake = new();
LoopGuard loopGuard = new(maxRepeats: withGuard ? 2 : int.MaxValue);
ShippingTools tools = new();
IChatClient looping = fake.AsBuilder()
.UseFunctionInvocation(configure: f => f.FunctionInvoker = loopGuard.InvokeAsync)
.Build();
ChatResponse r = await looping.GetResponseAsync("Where is my order 1042?",
new ChatOptions { Tools = [AIFunctionFactory.Create(tools.GetShipmentStatus, "get_shipment_status")] });
Console.WriteLine($"guard {(withGuard ? "on " : "off")}: model calls {fake.ModelCalls}, tool calls {loopGuard.Calls}");
}$ dotnet run -- loop
call 1: get_shipment_status({"orderId":1042}) state fcc936a3 seen 1x
tool calls 1 | tokens in 602 out 1065 | 24.8 s
answer: Your order 1042 is currently **pending**. The status is not final yet, so please check again later for updates.
$ dotnet run -- loop-fake
guard off: model calls 41, tool calls 40
guard on : model calls 3, tool calls 3Without the guard, FunctionInvokingChatClient made 40 tool calls and 41 model calls. 40 is the default value of MaximumIterationsPerRequest; so in an application with no settings, a single user message can turn into up to 41 model calls. The guard cut the loop with Terminate when it saw the same state for the third time. I included the result in the hash because calling the same tool with the same arguments again is not always a mistake: if the shipment status really changed, the new result is a new state and the guard stays out of the way. In production this guard joins ToolBudget from post 4 in the same FunctionInvoker delegate.
The Expensive Path and Model Extraction
Two more risks, and the remedy for both is in design, not code. The first is routing to the expensive model. Some applications let the client choose which model to use, or route to a large model when they detect a "hard question". A model name coming from the client, ChatOptions.ModelId included, should be treated as user input: the server chooses from an allowlist according to the user's plan. The router itself is an attack surface too; text that makes it conclude "this question is very complex" can move a user from the cheap model to an expensive reasoning model. Tie the router's decision to the budget as well: give the expensive path its own, smaller quota.
The second is model extraction. An attacker can send your API many carefully chosen queries, collect the answers and train a model that functionally resembles your fine-tuned model. The document notes that exposing log-probabilities (logprobs) speeds this up significantly; that is why Bookstore's API does not return logprobs. The remaining measures are things this post already built: per-user quotas, anomaly metrics and per-account consumption monitoring. The other side of the coin is commercial terms: Anthropic's commercial terms, for example, prohibit using the service to build a competing product or to train competing AI models. If you expose your own API, putting a similar clause in your own terms of use gives you a legal basis where technical measures fall short (this is not legal advice).
Production Notes
Set the context size consciously and measure it. In Ollama that is num_ctx, in the cloud the model's context window. A check that compares the input token count in usage with your estimate of the text you sent catches silent truncation: if the estimate says 12,000 and usage says 2,000, the answer is most likely wrong.
Give every limit a metric. 413, 400 and 422 responses, timeouts, dropped connections, answers ending in length, budget rejections and requests cut by the loop guard should each be counted. If they all rise one morning, you know the system is being probed; if only one rises, it is most likely a client bug.
Tie limits to plans, not code. Free users, paying users and enterprise tenants get different token limits and different models; these values live in configuration and in the tenant record, and change without a redeploy.
Plan for graceful degradation. When a tenant's budget runs out, falling back to cached answers, a smaller model or a search-results-only mode, instead of shutting the assistant down entirely, protects both the user and the bill.
Common Pitfalls
First, limiting the number of requests and forgetting tokens. A limit of five requests per minute will happily accept five 120,000-character requests. The unit of cost in an LLM is the token; rate limits must be token-based as well.
Second, checking the budget after the response arrives. Concurrent requests all pass that check together. Reserve first, settle later.
Third, trusting the reasoning setting and not setting MaxOutputTokens. In the measurement above, "off" thinking spent 2,000 tokens and "on" thinking spent 8,000, and neither produced a useful answer.
Fourth, not passing cancellation to the model call. An endpoint that does not pass HttpContext.RequestAborted to the model call keeps paying for generation after the user has closed the page.
Fifth, turning a failure into a silent success. The truncated context returned 200 with a wrong answer; the empty reasoning answer returned 200 as well. Every answer that hits a limit should tell the user and the log that it hit a limit.
A Small Experiment
★☆☆ Repeat the context experiment with the return window moved to the end of the document. Is the answer correct with the truncated context? Does that make silent truncation less dangerous, or more?
★★☆ Add streaming support to BudgetChatClient: the reservation still happens before the call, and settlement uses the UsageContent in the last ChatResponseUpdate. If the client disconnects mid-stream, how much usage should be recorded?
★★★ Move TokenLedger to Redis: make the reservation atomic with a Lua script, put TTLs on the keys, and verify with a test that twenty concurrent requests from two application instances do not exceed the tenant limit by even one token.
We have looked at input, permissions, output, the supply chain and cost one by one. In the next post we bring them together: placing guardrails in the IChatClient pipeline in the right order, the fail-closed decision, safety evaluators, and turning the harmless canaries used throughout this series into a red-team regression set that runs in CI.