LLM Streaming in ASP.NET Core: Minimal API, IAsyncEnumerable and Server-Sent Events
The reliable way to stream an LLM answer to the browser is .NET 10's TypedResults.ServerSentEvents: delta, thinking, tool, usage, done and error events, a heartbeat during silent waits and a CancellationToken carried all the way to the model. With real measurements we show the limits of a plain IAsyncEnumerable, that the model really stops when the user closes the tab, and that an endpoint forgetting the token wastes 1024 tokens.
Version note: The code in this post was built and run with .NET 10, Microsoft.Extensions.AI 10.11.0, OllamaSharp 5.5.2, a local Ollama 0.34.4 (qwen3:4b) and, on the client side, Node 22, TypeScript 5.9 and tsx 4.23.
Guess First
You want to show the Bookstore assistant's answer to the user word by word. The shortest path that comes to mind: return an IAsyncEnumerable<string> from a minimal API (Application Programming Interface) endpoint. Does the browser see the tokens as they arrive, or the whole answer at the end?
Second question: the user closed the tab in the middle of the answer. Does the model keep generating on the server? The endpoint's signature has a CancellationToken, but you forgot to pass it to the model. What changes?
We'll answer both with real measurements. In the streaming post of LLM (Large Language Model) Fundamentals we talked about the NDJSON (Newline Delimited JSON; JSON: JavaScript Object Notation) stream and time to first token; the topic here is carrying the same stream reliably from an ASP.NET Core API to the browser.
The Shortest Path: Returning IAsyncEnumerable
// 1) Plain IAsyncEnumerable: minimal API writes it as a JSON array
app.MapPost("/chat/raw", (ChatStreamRequest request, IChatClient chat, CancellationToken ct) =>
chat.GetStreamingResponseAsync(request.Message, cancellationToken: ct)
.Where(u => u.Text.Length > 0)
.Select(u => u.Text));Where and Select on IAsyncEnumerable need no extra package; in .NET 10 System.Linq.AsyncEnumerable is part of the framework. Minimal API serializes a returned IAsyncEnumerable<T> as a JSON array. I printed the arrival time in front of every piece of curl output:
70005 ms ["Martin"
70027 ms ," Fowler"
70050 ms ,"’s"
70072 ms ," *"
70095 ms ,"Ref"
70118 ms ,"actoring"
70140 ms ,":"
70163 ms ," Impro"
70185 ms ,"ving"
70207 ms ," the"
70230 ms ," Design"
70252 ms ," of"
70275 ms ," Existing"
70297 ms ," Code"
70320 ms ,"*"
70342 ms ," is"
70365 ms ," the"
70387 ms ," definitive"
70410 ms ," guide"
70433 ms ," to"
70455 ms ," ref"
70478 ms ,"actoring"
70500 ms ,"."
70523 ms ]The answer to the first Guess First question is "yes, token by token". I read the output with a small Perl script that prints incoming bytes as soon as they arrive, with a timestamp, without waiting for a line: each element of the array was sent separately as soon as it came from the model, about 22 milliseconds apart. But this path has three problems. First, for the first 70 seconds the client received nothing; while the model waited in line and thought, the connection was completely silent. Second, what the client holds is a half-finished JSON array: JSON.parse only works once ] arrives, so to read piece by piece you need your own parser. Third, if an error happens mid-stream, the connection drops before the array is closed and the client is left with invalid JSON; there's no way to say "the model was cut off". We need a protocol that carries the state of the stream (thinking, calling a tool, done, error).
Server-Sent Events: TypedResults.ServerSentEvents in .NET 10
SSE (Server-Sent Events) is a simple protocol browsers have supported for years: an HTTP (Hypertext Transfer Protocol) response kept open with the text/event-stream content type, containing events separated by blank lines. Each event has a type (event:) and data (data:). It isn't bidirectional like WebSocket; but an LLM answer flows one way anyway, and since SSE is plain HTTP it works smoothly with proxies, load balancers and authentication infrastructure.
.NET 10 brought TypedResults.ServerSentEvents to ASP.NET Core. This result type, which we met in the ASP.NET Core Web API series post ".NET 10 Yenilikleri: ASP.NET Core'da AddValidation, OpenAPI 3.1 ve Server-Sent Events" (What's New in .NET 10), takes an IAsyncEnumerable<SseItem<T>> and writes and flushes each item in the right format. The Bookstore endpoint:
// 2) Server-Sent Events: typed events, heartbeat and cancellation
app.MapPost("/chat/sse", (ChatStreamRequest request, IServiceProvider services, ILogger<Program> logger, CancellationToken ct) =>
{
IChatClient chat = request.UseTools
? services.GetRequiredKeyedService<IChatClient>("tools")
: services.GetRequiredService<IChatClient>();
ChatOptions? options = request.UseTools ? new ChatOptions { Tools = [getStock] } : null;
IAsyncEnumerable<ChatResponseUpdate> updates = chat.GetStreamingResponseAsync(request.Message, options, ct);
return TypedResults.ServerSentEvents(
ChatStreaming.ToSseAsync(updates, logger, ct).WithHeartbeatAsync(TimeSpan.FromSeconds(5), ct));
});ASP.NET Core binds the CancellationToken in the endpoint's signature to HttpContext.RequestAborted: it's cancelled when the client closes the connection. We pass this token both to the model (GetStreamingResponseAsync) and to our own iterators. The same request with curl:
5005 ms HTTP/1.1 200 OK
5005 ms Content-Type: text/event-stream
5005 ms Date: Sun, 08 Nov 2026 08:06:16 GMT
5005 ms Server: Kestrel
5005 ms Cache-Control: no-cache,no-store
5005 ms Content-Encoding: identity
5005 ms Pragma: no-cache
5005 ms Transfer-Encoding: chunked
5005 ms event: ping
5005 ms data: {}
# ...
35014 ms event: ping
35014 ms data: {}
39868 ms event: thinking
39868 ms data: {}
44870 ms event: ping
44870 ms data: {}
49872 ms event: ping
49872 ms data: {}
51001 ms event: delta
51001 ms data: {"text":"\""}
51023 ms event: delta
51023 ms data: {"text":"Ref"}
51046 ms event: delta
51046 ms data: {"text":"actoring"}
# ...
51510 ms event: delta
51510 ms data: {"text":" code"}
51533 ms event: delta
51533 ms data: {"text":".\""}
51558 ms event: usage
51558 ms data: {"inputTokens":21,"outputTokens":520}
51558 ms event: done
51558 ms data: {}Let's read it line by line. The headers arrived after 5 seconds: ServerSentEventsResult starts the response when it writes the first event, and that first event is the heartbeat's ping. For the first 35 seconds the model waited in line in Ollama and the client got a ping every five seconds. At 39.9 seconds the model started thinking and a single thinking event went out; qwen3:4b is a reasoning model and produces no text while it thinks. At 51 seconds the answer tokens flowed as delta events about 23 milliseconds apart, and usage and done came last. 520 output tokens for a one-sentence answer: almost all of them thinking tokens; we'll come back to the cost side in post 7.
Event Types: Delta, Tool, Usage, Done, Error
The UI needs not only the text pieces but also the state of the stream: showing "checking stock" while the model calls a tool, enabling the send button when the stream ends, showing the token count. For that I turn the contents inside every ChatResponseUpdate into typed events:
namespace Bookstore.Assistant.Api;
public sealed record ChatStreamRequest(string Message, bool UseTools = false);
public sealed record DeltaEvent(string Text);
public sealed record ToolEvent(string Name, string Phase);
public sealed record UsageEvent(long? InputTokens, long? OutputTokens);
public sealed record ErrorEvent(string Message, bool Partial);using System.Diagnostics;
using System.Net.ServerSentEvents;
using System.Runtime.CompilerServices;
using Microsoft.Extensions.AI;
namespace Bookstore.Assistant.Api;
public static class ChatStreaming
{
// Turns model updates into typed SSE events; on failure reports the partial answer with an "error" event
public static async IAsyncEnumerable<SseItem<object>> ToSseAsync(
IAsyncEnumerable<ChatResponseUpdate> updates,
ILogger logger,
[EnumeratorCancellation] CancellationToken ct = default)
{
int deltas = 0;
bool thinking = false;
Stopwatch sw = Stopwatch.StartNew();
IAsyncEnumerator<ChatResponseUpdate> e = updates.GetAsyncEnumerator(ct);
try
{
while (true)
{
ChatResponseUpdate update;
string? error = null;
try
{
if (!await e.MoveNextAsync())
{
break;
}
update = e.Current;
}
catch (Exception ex) when (!ct.IsCancellationRequested)
{
logger.LogWarning(ex, "Model stream failed after {Deltas} deltas", deltas);
error = "The model stopped responding.";
update = null!;
}
// yield can't be used in a catch block; emit the error outside it
if (error is not null)
{
yield return new SseItem<object>(new ErrorEvent(error, Partial: deltas > 0), "error");
yield break;
}
foreach (AIContent content in update.Contents)
{
switch (content)
{
// A reasoning model sends no text while thinking; tell the UI once
case TextReasoningContent when !thinking:
thinking = true;
yield return new SseItem<object>(new { }, "thinking");
break;
case TextContent { Text.Length: > 0 } text:
deltas++;
yield return new SseItem<object>(new DeltaEvent(text.Text), "delta");
break;
case FunctionCallContent call:
yield return new SseItem<object>(new ToolEvent(call.Name, "started"), "tool");
break;
case FunctionResultContent:
yield return new SseItem<object>(new ToolEvent("", "completed"), "tool");
break;
case UsageContent usage:
yield return new SseItem<object>(
new UsageEvent(usage.Details.InputTokenCount, usage.Details.OutputTokenCount), "usage");
break;
}
}
}
yield return new SseItem<object>(new { }, "done");
}
finally
{
await e.DisposeAsync();
logger.LogInformation("Stream closed: deltas={Deltas}, elapsed={Elapsed} ms, aborted={Aborted}",
deltas, sw.ElapsedMilliseconds, ct.IsCancellationRequested);
}
}
}There's one more in the event list: thinking. In this post the model is qwen3:4b, a reasoning model. While it thinks, OllamaSharp puts TextReasoningContent into the updates instead of text; I don't show that text to the user, but I send the "model is thinking" information once. Otherwise the user stares at an empty screen for tens of seconds.
The odd-looking part of the code is walking the enumerator by hand. The reason is a C# rule: yield return can't be used inside a catch block. If we want to catch the error and send it to the client as an event, we have to capture the error in a variable and emit it outside the catch.
The first version of the catch filter is a lesson too. I had first written when (ex is not OperationCanceledException): "don't send an error event if the user cancelled". In one attempt while Ollama was crowded, the request never got an answer and the HttpClient timeout ran out. That exception is a TaskCanceledException, i.e. an OperationCanceledException; the filter took it for "user cancellation" and let it through, no error event reached the client, and the stream broke off in the middle. The right question isn't "what's the exception's type?" but "is it my token that was cancelled?": when (!ct.IsCancellationRequested).
To see the tool events I called the same endpoint with qwen3:4b, which supports tool calling, and the get_stock tool from post 2:
40761 ms event: thinking
40761 ms data: {}
50310 ms event: tool
50310 ms data: {"name":"get_stock","phase":"started"}
50310 ms event: usage
50310 ms data: {"inputTokens":145,"outputTokens":422}
50319 ms event: tool
50319 ms data: {"name":"","phase":"completed"}
102363 ms event: delta
102363 ms data: {"text":"There"}
102386 ms event: delta
102386 ms data: {"text":" are"}
# ...
102683 ms event: delta
102683 ms data: {"text":"."}
102706 ms event: usage
102706 ms data: {"inputTokens":604,"outputTokens":148}
102706 ms event: done
102706 ms data: {}This time there are two model rounds and the stream shows both. In the first round the model thought and decided to call get_stock (tool started); FunctionInvokingChatClient ran the tool (tool completed, 9 milliseconds later). The usage of the first round came as a separate usage event: 145 input, 422 output tokens. The second round started with a longer prompt that includes the tool result (604 input tokens) and the answer flowed at 102 seconds. With these events the UI can show "checking stock" and compute the total cost by adding up the two usage events. The tool name is empty in the completed event: FunctionResultContent carries only the CallId; if you want the name you have to match it with the CallId from the started event.
Cancellation: Stopping the Model When the User Leaves
Now let's answer the second opening question with a measurement. To simulate the user closing the tab, I ran the TypeScript client so that it cuts the connection after the 5th delta, and asked the model for a long story:
$ ./node_modules/.bin/tsx client.ts "Write a 300-word story about a cat who lives in a bookstore." 5
[ping]
[ping]
# ...
[aborted] AbortError, partial text: "Whisper wasn’t just"
# server log
Stream closed: deltas=6, elapsed=201753 ms, aborted=True
# Ollama log (whitespace trimmed)
[GIN] 2026/11/08 - 11:10:12 | 200 | 3m21s | 127.0.0.1 | POST "/api/chat"The client waited through the first 39 pings (while the model waited in line and thought), cut the connection at the fifth delta and was left with "Whisper wasn’t just". On the server, the finally block ran after the sixth delta: aborted=True. One more delta was written between the client's abort and the server noticing it. The endpoint's CancellationToken was cancelled, the iterator was disposed, and OllamaSharp closed the HTTP connection to Ollama. Ollama's log confirms it: the same request ended there at 3 minutes 21 seconds, the same moment as the 201.7-second server duration; the model didn't generate the rest of the 300-word story.
To measure the effect of forgetting to pass the token to the model, I wrote two non-streaming endpoints. Both ask for the same long story; one passes the token to the model, the other doesn't. The client (curl --max-time 3) cuts the connection after 3 seconds in both:
// 3) For measurement: two non-streaming endpoints, one not passing the cancellation token to the model and one passing it
app.MapPost("/chat/complete-leaky", async (ChatStreamRequest request, IChatClient chat, ILogger<Program> logger, HttpContext http) =>
{
Stopwatch sw = Stopwatch.StartNew();
ChatResponse response = await chat.GetResponseAsync(request.Message);
logger.LogInformation("Leaky: finished after {Elapsed} ms, outputTokens={Tokens}, clientGone={Gone}",
sw.ElapsedMilliseconds, response.Usage?.OutputTokenCount, http.RequestAborted.IsCancellationRequested);
return response.Text;
});
app.MapPost("/chat/complete", async (ChatStreamRequest request, IChatClient chat, ILogger<Program> logger, CancellationToken ct) =>
{
Stopwatch sw = Stopwatch.StartNew();
try
{
ChatResponse response = await chat.GetResponseAsync(request.Message, cancellationToken: ct);
return response.Text;
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
logger.LogInformation("Cancelled after {Elapsed} ms, model call stopped", sw.ElapsedMilliseconds);
throw;
}
});$ curl -s --max-time 3 -X POST localhost:19622/chat/complete-leaky ...
curl exit 28
$ curl -s --max-time 3 -X POST localhost:19622/chat/complete ...
curl exit 28
# server log
Cancelled after 3007 ms, model call stopped
Leaky: finished after 46586 ms, outputTokens=1024, clientGone=True
# Ollama log (whitespace trimmed)
[GIN] 2026/11/08 - 11:10:31 | 500 | 3.003718125s | 127.0.0.1 | POST "/api/chat"
[GIN] 2026/11/08 - 11:11:12 | 200 | 46.582353375s | 127.0.0.1 | POST "/api/chat"Both clients gave up after 3 seconds (curl exit code 28: timeout). The endpoint passing the token to the model was cancelled at 3007 milliseconds; Ollama closed that request at 3.0 seconds too (the 500 in the log is Ollama's record of the client closing the connection). The endpoint not passing the token kept working long after the user had gone: 46.6 seconds and 1024 output tokens for a story nobody will read. The clientGone=True line is the proof. On a local Ollama that's GPU (Graphics Processing Unit) time other users in the queue were waiting for; on a cloud provider it's 1024 tokens on the bill. That's the answer to the second opening question.
Heartbeat, Proxy Buffering and Compression
Something caught my attention in the SSE experiments: the response headers only arrive when the first event is written. While the model waits in line the client sees nothing; the browser, a proxy or a load balancer may treat that silence as a "dead connection" and cut the request. The fix is a classic: insert a small "ping" event while the source stays silent.
using System.Net.ServerSentEvents;
using System.Runtime.CompilerServices;
using System.Threading.Channels;
namespace Bookstore.Assistant.Api;
public static class Heartbeat
{
// Inserts a "ping" event every interval while the source stays silent
public static async IAsyncEnumerable<SseItem<object>> WithHeartbeatAsync(
this IAsyncEnumerable<SseItem<object>> source,
TimeSpan interval,
[EnumeratorCancellation] CancellationToken ct = default)
{
Channel<SseItem<object>> channel = Channel.CreateUnbounded<SseItem<object>>();
Task producer = Task.Run(async () =>
{
try
{
await foreach (SseItem<object> item in source.WithCancellation(ct))
{
await channel.Writer.WriteAsync(item, ct);
}
channel.Writer.Complete();
}
catch (Exception ex)
{
channel.Writer.Complete(ex);
}
}, CancellationToken.None);
Task<bool>? ready = null;
while (true)
{
ready ??= channel.Reader.WaitToReadAsync(ct).AsTask();
if (await Task.WhenAny(ready, Task.Delay(interval, ct)) != ready)
{
yield return new SseItem<object>(new { }, "ping");
continue;
}
if (!await ready)
{
break;
}
ready = null;
while (channel.Reader.TryRead(out SseItem<object> item))
{
yield return item;
}
}
await producer;
}
}We saw this helper's effect in the SSE output above: while the model waited in line for the first 35 seconds and then thought, the client got a ping every five seconds and the connection stayed alive.
The headers contain two things. Cache-Control: no-cache,no-store tells intermediaries not to store the response. Content-Encoding: identity turns response compression off for this response: a compressor such as gzip collects small pieces and sends them in blocks, which silently breaks streaming. ServerSentEventsResult adds these headers itself; if you write your own streaming response by hand, you have to do the same.
If nginx sits in front of the application, there's one more step. nginx buffers the upstream response by default; for SSE you either write proxy_buffering off; in the relevant location, or the application adds an X-Accel-Buffering: no header to the response. The header approach lets other endpoints behind the same nginx keep buffering.
Telling the Client About a Partial Answer
The least discussed problem of streaming is error handling. Once the headers and the first event have gone out, the HTTP status code is 200 and can't be changed; if the model is cut off midway, you can't tell the client "500". That's why we report the error inside the protocol, with an error event. The Partial field in ErrorEvent lets the UI tell two situations apart: if no text arrived, a "try again" button; if part of the text arrived, an "answer was cut off" warning. A separate done event says that the stream ended properly; the client shouldn't treat the connection closing as "done" on its own, because a proxy timeout closes the connection too.
A TypeScript Consumer
The browser's EventSource API can only send GET requests and can't send a body. Since we want to send the message with POST, I wrote a small SSE reader with fetch and ReadableStream. The Angular side is covered in the upcoming Angular ile AI Arayüzleri (AI Interfaces with Angular; AI: Artificial Intelligence) series; the code here shows the logic of the protocol:
type SseEvent = { event: string; data: string };
// Reads SSE with fetch + ReadableStream; we use fetch because EventSource can't send a POST body
async function* readSse(response: Response): AsyncGenerator<SseEvent> {
const reader = response.body!.pipeThrough(new TextDecoderStream()).getReader();
let buffer = '';
while (true) {
const { value, done } = await reader.read();
if (done) return;
buffer += value;
let boundary: number;
while ((boundary = buffer.indexOf('\n\n')) >= 0) {
const block = buffer.slice(0, boundary);
buffer = buffer.slice(boundary + 2);
let event = 'message';
const data: string[] = [];
for (const line of block.split('\n')) {
if (line.startsWith('event:')) event = line.slice(6).trim();
else if (line.startsWith('data:')) data.push(line.slice(5).trimStart());
}
yield { event, data: data.join('\n') };
}
}
}
const controller = new AbortController();
const message = process.argv[2] ?? 'Recommend one book about refactoring in one short sentence.';
const stopAfterDeltas = Number(process.argv[3] ?? 0);
const response = await fetch('http://localhost:19622/chat/sse', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ message }),
signal: controller.signal,
});
let text = '';
let deltas = 0;
try {
for await (const { event, data } of readSse(response)) {
switch (event) {
case 'delta':
text += JSON.parse(data).text;
// Simulate the user closing the tab
if (stopAfterDeltas > 0 && ++deltas === stopAfterDeltas) controller.abort();
break;
case 'ping': console.log('[ping]'); break;
case 'usage': console.log('[usage]', data); break;
case 'error': console.log('[error]', data); break;
case 'done': console.log('[done]', text); break;
}
}
} catch (err) {
console.log(`[aborted] ${(err as Error).name}, partial text: "${text}"`);
}AbortController.abort() closes the fetch connection; on the server this triggers the RequestAborted cancellation we measured in the previous section. In a real UI you wire the same call to a "stop" button and to the page-leave event.
In Production
Count your connections. Every open SSE connection holds a request, an iterator and an HTTP connection to the model on the server. With long answers and slow models the number of concurrent connections grows quickly; set Kestrel's connection limits and the timeouts of the proxy in front of it accordingly.
Put a limit on the total duration of the stream. As we saw in the previous post, the timeouts of the HttpClient resilience handler only cover the time until the first chunk in streaming. To stop a stream that never ends, take a copy in the endpoint with CancellationTokenSource.CreateLinkedTokenSource(ct) and set a deadline with CancelAfter.
Measure: time to first token, total duration, cancellation rate. The deltas, elapsed and aborted fields in the "Stream closed" log are the raw form of these three metrics; in post 8 we'll turn them into metrics with OpenTelemetry. If the cancellation rate is high, users aren't waiting for the answer; either the model is slow or the answers are longer than needed.
Common Pitfalls
First, not passing the CancellationToken to the model. We saw it in the measurement: generation goes on after the user has left, and you pay the bill.
Second, telling cancellation apart by exception type. An HttpClient timeout is an OperationCanceledException too. The reliable way to separate a user cancellation from a system error is to look at the state of your own token.
Third, treating a plain IAsyncEnumerable as a sufficient streaming protocol. The tokens do go out piece by piece, but the client holds a half-finished JSON array; there's no place for error, cancellation, tool and usage information, and if the stream breaks midway what's left is invalid JSON.
Fourth, forgetting response compression and proxy buffering. Even if the application writes correctly, if gzip or nginx collects the pieces and sends them in one go, the user still sees the whole answer at the end. TypedResults.ServerSentEvents turns compression off itself; the proxy is yours to configure.
A Small Experiment
★☆☆ Lower the heartbeat interval from 5 seconds to 1 second and ask for a long answer. How many ping events do you see on the client? Why do no pings arrive while the model is producing tokens?
★★☆ Add a "time to first token" measurement to ToSseAsync: log the elapsed time when the first delta is written and put it into the usage event as a timeToFirstTokenMs field.
★★★ Design a "resume" flow: the client keeps the sequence of each delta and, if the connection drops, reconnects with a Last-Event-ID header. Where do you keep pieces that were produced on the server but couldn't be sent? Can the model's stream continue from that point after it breaks, or does the whole answer have to be generated again?
The answer now flows to the user, and the model stops when the user leaves. But every request is still on its own; the assistant doesn't remember the previous message. In the next post we tackle chat history: who keeps the history with a stateless provider, storing ChatMessage instances together with their tool calls in EF (Entity Framework) Core, and trimming the history with chat reducers as the context window fills up.