Persisting Chat History: Stateless APIs, ConversationId, EF Core and Chat Reducers

Version note: The code in this post was built and run with .NET 10, Microsoft.Extensions.AI 10.11.0, OllamaSharp 5.5.2, Microsoft.EntityFrameworkCore.Sqlite 10.0.14 and a local Ollama 0.34.4 (qwen3:4b).

Guess First

Ada writes to the Bookstore assistant: "Hi, I'm Ada, I only read books about software design." A minute later, in the same chat window, she asks: "What's my name, and which kind of books do I like?" Behind it is qwen3:4b in Ollama. Does the model know the answer? If it does, who keeps that information: the model, Ollama or your code?

Second question: the history has grown and you enable Microsoft.Extensions.AI's MessageCountingChatReducer with a limit of 4 messages. The history has a system prompt, Ada's introduction, an order question, a tool call, a tool result and two assistant answers. Which ones reach the model?

The answer to the first is "your code"; the answer to the second is different from what most people guess. We'll see both with real output.

Who Keeps the History with a Stateless Provider?

As we saw in the context window post of LLM (Large Language Model) Fundamentals, the model doesn't remember the previous request; it knows exactly what you send it on each request. Ollama's /api/chat endpoint and OpenAI's Chat Completions API (Application Programming Interface) are stateless in this respect: to continue a conversation you send the whole history again on every request. IChatClient reflects this model directly: GetResponseAsync takes a list of messages, and the list is the whole conversation.

In a web API that list has to live between two requests. Keeping it in memory (in a Dictionary) works on the first try, but it's lost when the application restarts, when a second instance comes up or when the user connects from another device. This is data that must persist, so it goes to the database.

Storing ChatMessage: JSON with AIJsonUtilities

What we store isn't just text. An assistant message can carry a tool call (FunctionCallContent) instead of text, and its answer is a FunctionResultContent with the ChatRole.Tool role. If you lose them, in the next round the model can't tell where it got "order 1042 has shipped" from. The most robust way is to turn the ChatMessage into JSON (JavaScript Object Notation) as is. MEAI (Microsoft.Extensions.AI) has ready-made settings for this: AIJsonUtilities.DefaultOptions.

using System.Text.Json;
using Microsoft.Extensions.AI;

ChatMessage call = new(ChatRole.Assistant,
    [new FunctionCallContent("call_1", "get_order_status", new Dictionary<string, object?> { ["orderId"] = 1042 })]);
ChatMessage result = new(ChatRole.Tool,
    [new FunctionResultContent("call_1", new { status = "Shipped", carrier = "UPS" })]);

foreach (ChatMessage message in new[] { call, result })
{
    string json = JsonSerializer.Serialize(message, AIJsonUtilities.DefaultOptions);
    Console.WriteLine(json);

    ChatMessage back = JsonSerializer.Deserialize<ChatMessage>(json, AIJsonUtilities.DefaultOptions)!;
    Console.WriteLine($"  -> {back.Role}: {back.Contents[0].GetType().Name}");
}

// What happens with the default serializer settings?
string plain = JsonSerializer.Serialize(call);
Console.WriteLine(plain);
ChatMessage plainBack = JsonSerializer.Deserialize<ChatMessage>(plain)!;
Console.WriteLine($"  -> {plainBack.Role}: {plainBack.Contents[0].GetType().Name}");

{
  "role": "assistant",
  "contents": [
    {
      "$type": "functionCall",
      "name": "get_order_status",
      "arguments": {
        "orderId": 1042
      },
      "informationalOnly": false,
      "callId": "call_1"
    }
  ]
}
  -> assistant: FunctionCallContent
{
  "role": "tool",
  "contents": [
    {
      "$type": "functionResult",
      "result": {
        "status": "Shipped",
        "carrier": "UPS"
      },
      "callId": "call_1"
    }
  ]
}
  -> tool: FunctionResultContent
{"AuthorName":null,"CreatedAt":null,"Role":"assistant","Contents":[{"$type":"functionCall","Name":"get_order_status","Arguments":{"orderId":1042},"InformationalOnly":false,"CallId":"call_1","Annotations":null,"AdditionalProperties":null}],"MessageId":null,"AdditionalProperties":null}
  -> assistant: FunctionCallContent

The content's subtype is written in a $type field and maps back to the right class when read. The default settings also keep the type, because the polymorphism information sits on AIContent as attributes; but the field names are PascalCase and nulls are written too. DefaultOptions uses camelCase and skips nulls. One more detail: the output is indented. Nice to read on screen, but whitespace in every row of the database. That's why we'll use a non-indented copy for the database.

The EF Core Model: Conversation and StoredMessage

using Microsoft.EntityFrameworkCore;

namespace Bookstore.Assistant.Api.Data;

public sealed class Conversation
{
    public Guid Id { get; init; } = Guid.NewGuid();
    public required string UserId { get; init; }

    // With stateful providers (e.g. OpenAI Responses) the provider keeps the history; only its id lives here
    public string? ProviderConversationId { get; set; }

    public List<StoredMessage> Messages { get; init; } = [];
}

public sealed class StoredMessage
{
    public long Id { get; init; }
    public Guid ConversationId { get; init; }
    public required string Role { get; init; }

    // The whole ChatMessage: including text, tool call and tool result contents
    public required string Json { get; init; }
}

public sealed class ChatDbContext(DbContextOptions<ChatDbContext> options) : DbContext(options)
{
    public DbSet<Conversation> Conversations => Set<Conversation>();

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Conversation>().HasIndex(c => c.UserId);
        modelBuilder.Entity<Conversation>()
            .HasMany(c => c.Messages)
            .WithOne()
            .HasForeignKey(m => m.ConversationId);
    }
}

Each message is a row, its content a JSON column. It isn't worth splitting the structure of ChatMessage into relational tables (a table per content type and so on): MEAI keeps adding new content types and the history is always read as a whole. Keeping Role as a separate column is enough for queries and reporting. I order by Id; the SQLite provider can't translate ORDER BY on DateTimeOffset, and an increasing key already gives the insertion order. The JSON column mapping we covered in the EF (Entity Framework) Core series post "EF Core 7 Yenilikleri" (What's New in EF Core 7) is also an option, but here the JSON is produced by MEAI's serializer, not by EF Core, so a plain string column brings fewer surprises.

The JSON conversion lives in one place:

using System.Text.Json;
using Bookstore.Assistant.Api.Data;
using Microsoft.Extensions.AI;

namespace Bookstore.Assistant.Api;

public static class ChatHistory
{
    // Ready-made settings that know AIContent subtypes ($type); a non-indented copy for the database
    private static readonly JsonSerializerOptions Json = new(AIJsonUtilities.DefaultOptions) { WriteIndented = false };

    public static List<ChatMessage> Load(Conversation conversation) =>
        [.. conversation.Messages.OrderBy(m => m.Id).Select(m => JsonSerializer.Deserialize<ChatMessage>(m.Json, Json)!)];

    public static void Append(Conversation conversation, IEnumerable<ChatMessage> messages)
    {
        foreach (ChatMessage message in messages)
        {
            // Reasoning (thinking) metnini saklama: uzun, ve sonraki turda modele geri gitmesi gerekmiyor
            List<AIContent> contents = [.. message.Contents.Where(c => c is not TextReasoningContent)];
            if (contents.Count == 0)
            {
                continue;
            }

            ChatMessage stored = new(message.Role, contents) { AuthorName = message.AuthorName, MessageId = message.MessageId };
            conversation.Messages.Add(new StoredMessage
            {
                Role = stored.Role.Value,
                Json = JsonSerializer.Serialize(stored, Json),
            });
        }
    }
}

In this post the model is qwen3:4b, a reasoning model. OllamaSharp puts the model's thinking text into the response as a TextReasoningContent, and as we'll see in post 7, that text can take hundreds of tokens even for a one-sentence answer. I don't store it: it bloats the database, sending it back to the model in the next round brings no benefit, and checking whether a text the user never sees contains personal data is a burden of its own. A message made only of reasoning isn't written at all.

The Endpoint: Load, Send, Store

app.MapPost("/conversations/{id:guid}/messages", async (
    Guid id, SendMessageRequest request, HttpContext http, ChatDbContext db, IChatClient chat, CancellationToken ct) =>
{
    // The ownership check is inside the query: someone else's conversation counts as "not there"
    string userId = CurrentUser(http);
    Conversation? conversation = await db.Conversations
        .Include(c => c.Messages)
        .SingleOrDefaultAsync(c => c.Id == id && c.UserId == userId, ct);
    if (conversation is null)
    {
        return Results.NotFound();
    }

    ChatMessage userMessage = new(ChatRole.User, request.Text);
    ChatOptions options = new() { ConversationId = conversation.ProviderConversationId };

    // Stateless provider: the whole history is sent. Stateful provider: only the new message is sent, the provider keeps the history.
    List<ChatMessage> toSend = conversation.ProviderConversationId is null
        ? [new(ChatRole.System, SystemPrompt), .. ChatHistory.Load(conversation), userMessage]
        : [userMessage];

    ChatResponse response = await chat.GetResponseAsync(toSend, options, ct);

    conversation.ProviderConversationId = response.ConversationId;
    ChatHistory.Append(conversation, [userMessage, .. response.Messages]);
    await db.SaveChangesAsync(ct);

    return Results.Ok(new
    {
        answer = response.Text,
        sentMessages = toSend.Count,
        storedMessages = conversation.Messages.Count,
        providerConversationId = response.ConversationId,
    });
});

The system prompt isn't written to the database; it's added at the start of every request. That way, when you change the prompt, old conversations continue with the new one too. What gets stored is the user message and the whole of response.Messages, not response.Text. Had there been function invocation in the pipeline, response.Messages would also contain the tool call and tool result messages, and they'd all be stored with the same call.

The pipeline is set up like this:

IChatClient ollama = new OllamaApiClient(
    new HttpClient { BaseAddress = new Uri("http://localhost:11434"), Timeout = TimeSpan.FromMinutes(30) }, "qwen3:4b");

#pragma warning disable MEAI001 // chat reducers are experimental
builder.Services.AddChatClient(ollama)
    .UseChatReducer(new MessageCountingChatReducer(20));
#pragma warning restore MEAI001

The opening scenario, two rounds:

$ curl -s -X POST localhost:19623/conversations -H "X-Demo-User: ada"
{"id":"706332a3-37d8-4b1b-8873-c0eebe6d7e2a"}

$ POST /conversations/706332a3-37d8-4b1b-8873-c0eebe6d7e2a/messages (ada) Hi, I'm Ada...
{"answer":"Welcome, Ada! If you're into software design, I'd recommend *Clean Code* by Robert C. Martin for practical principles and *Design Patterns* by Gamma et al. for reusable solutions.","sentMessages":2,"storedMessages":2,"providerConversationId":null}
$ POST /conversations/706332a3-37d8-4b1b-8873-c0eebe6d7e2a/messages (ada) What is my name...
{"answer":"Your name is Ada, and you enjoy software design books.","sentMessages":4,"storedMessages":4,"providerConversationId":null}
$ GET /conversations/706332a3-37d8-4b1b-8873-c0eebe6d7e2a/messages (ada)
[{"role":"user","json":"{\"role\":\"user\",\"contents\":[{\"$type\":\"text\",\"text\":\"Hi, I am Ada. I only read books about software design.\"}]}"},{"role":"assistant","json":"{\"role\":\"assistant\",\"contents\":[{\"$type\":\"text\",\"text\":\"Welcome, Ada! If you're into software design, I'd recommend *Clean Code* by Robert C. Martin for practical principles and *Design Patterns* by Gamma et al. for reusable solutions.\"}],\"messageId\":\"2026-11-08T08:06:33.367066Z\"}"},{"role":"user","json":"{\"role\":\"user\",\"contents\":[{\"$type\":\"text\",\"text\":\"What is my name, and which kind of books do I like?\"}]}"},{"role":"assistant","json":"{\"role\":\"assistant\",\"contents\":[{\"$type\":\"text\",\"text\":\"Your name is Ada, and you enjoy software design books.\"}],\"messageId\":\"2026-11-08T08:07:22.39111Z\"}"}]
$ POST /conversations/706332a3-37d8-4b1b-8873-c0eebe6d7e2a/messages (grace)
HTTP/1.1 404 Not Found

$ sqlite
1|user|109
2|assistant|281
3|user|106
4|assistant|155

In the first round two messages went to the model (the system prompt and Ada's message), in the second round four: the system prompt, the two messages loaded from the database and the new question. The model knew Ada's name and interest because we sent them back in the second request. That's the answer to the first opening question: the memory is neither in the model nor in Ollama, it's in your database. There are four rows in the database and none of them contains thinking text; the filter in Append works. grace, coming with the same conversation id, got a 404. OllamaSharp fills the messageId field of assistant messages (Ollama's response time); we store it as is.

Authorization: Someone Else's Conversation

The conversation id travels in the URL (Uniform Resource Locator). A GUID (Globally Unique Identifier) is hard to guess, but it isn't a secret: it ends up in logs, browser history and screenshots sent to the support team. The counterpart of Broken Access Control from the Web Security (OWASP Top 10) series in this scenario is: not everyone who knows the id should be able to read the conversation. I put the ownership check inside the query (c.Id == id && c.UserId == userId); for another user's request the entry simply isn't found and the answer is 404, not 403. A 403 leaks "this conversation exists but isn't yours".

While writing this query I also hit a compiler error. In its first version I called the local function directly inside the lambda:

.SingleOrDefaultAsync(c => c.Id == id && c.UserId == CurrentUser(http), ct);

error CS8110: An expression tree may not contain a reference to a local function

An EF Core query is an expression tree; it can't contain a reference to a local function. Taking the user id into a variable first fixes the error and also leaves the query with a parameter that can be translated into SQL (Structured Query Language). In the example I read the user from a header; in a real application this value is the claim coming from authentication (the Authentication and Authorization series).

Stateful Providers: ConversationId

Some providers keep the history themselves; OpenAI's Responses API is the well-known example. Then you send only the new message and point at the conversation with an id. MEAI abstracts this with two properties: ChatOptions.ConversationId on the request and ChatResponse.ConversationId on the response. The endpoint above supports both: if an id comes back in the response it stores it and in the next round sends only the new message without the history; if not, it sends the whole history. That's also the approach the IChatClient documentation on Microsoft Learn recommends: if you don't know in advance whether the provider is stateful, look at the ConversationId in the response.

Ollama is stateless; providerConversationId in the output is null in both rounds. This code was not run against a stateful provider in this post. Even so, I recommend you keep storing the messages in your own database: how long the provider keeps the conversation is the provider's decision, while for auditing and support you need your own record.

When the Context Window Fills Up: Chat Reducers

Since every round sends the whole history, the request keeps growing. As we saw in LLM Fundamentals, when the context window fills up the model silently cuts the oldest part, and every extra token costs money. MEAI offers the IChatReducer abstraction and two ready-made implementations for this. Both are experimental (the MEAI001 warning); their APIs may change.

I tried the second opening question in a console application (Print is a small helper that writes each message with its role and contents on one line):

List<ChatMessage> history =
[
    new(ChatRole.System, "You are the Bookstore assistant."),
    new(ChatRole.User, "Hi, I'm Ada. I only read books about software design."),
    new(ChatRole.Assistant, "Nice to meet you, Ada! How can I help?"),
    new(ChatRole.User, "Where is my order 1042?"),
    new(ChatRole.Assistant, [new FunctionCallContent("call_1", "get_order_status", new Dictionary<string, object?> { ["orderId"] = 1042 })]),
    new(ChatRole.Tool, [new FunctionResultContent("call_1", new { status = "Shipped", carrier = "UPS" })]),
    new(ChatRole.Assistant, "Order 1042 has shipped with UPS."),
    new(ChatRole.User, "Recommend me a book for the weekend."),
];

#pragma warning disable MEAI001 // chat reducers are experimental
IChatReducer counting = new MessageCountingChatReducer(4);
Print("MessageCountingChatReducer(4):", await counting.ReduceAsync(history, CancellationToken.None));

IChatClient summarizer = new OllamaApiClient(new HttpClient { BaseAddress = new Uri("http://localhost:11434"), Timeout = TimeSpan.FromMinutes(30) }, "qwen3:4b");
IChatReducer summarizing = new SummarizingChatReducer(summarizer, targetCount: 2, threshold: 1);
Print("SummarizingChatReducer(target 2, threshold 1):", await summarizing.ReduceAsync(history, CancellationToken.None));
#pragma warning restore MEAI001

MessageCountingChatReducer(4):
  system    "You are the Bookstore assistant."
  assistant "Nice to meet you, Ada! How can I help?"
  user      "Where is my order 1042?"
  assistant "Order 1042 has shipped with UPS."
  user      "Recommend me a book for the weekend."
SummarizingChatReducer(target 2, threshold 1):
  system    "You are the Bookstore assistant."
  assistant "Ada mentioned she only reads software design books and then asked for the location of her order 1042. The assistant greeted her and asked how they could help."
  assistant [call call_1 get_order_status]
  tool      [result call_1]
  assistant "Order 1042 has shipped with UPS."
  user      "Recommend me a book for the weekend."

The two reducers behave in opposite ways. MessageCountingChatReducer(4) kept the system prompt and left the last four messages, but while counting it didn't take the tool call and tool result messages into account at all: both disappeared from the list entirely. Its documentation says so too: messages containing function call or function result content are excluded from the reduced output. Ada's "I only read books about software design" message is gone as well; the last question, asking for a weekend book, will now be answered without that information. SummarizingChatReducer, on the other hand, had the model summarize the old messages and put the summary at the start as a single assistant message; Ada's preference survived in the summary. It didn't touch the tool call and tool result messages and left them as they were, without summarizing. It has a cost too: summarization is a model call, which means extra latency and extra tokens.

When you put the reducer into the pipeline with UseChatReducer, the trimming applies only to the list sent to the model; the history in the database stays complete. That's the right split: full storage, trimmed sending. If you write a trimmed list back to the database, you can never recover what you lost.

In Production

Give conversations a retention policy. Chat history is personal data: the user's name, their orders, sometimes health or financial information. Regulations such as KVKK (Kişisel Verilerin Korunması Kanunu, Turkey's personal data protection law) and GDPR (General Data Protection Regulation) ask how long and for what purpose data is kept. A background job that deletes conversations older than a certain age and a "delete my history" option for the user should be designed in from the start.

Measure the growth. Log the number of messages sent and Usage.InputTokenCount for every round. Seeing how input tokens grow per round lets you set the reducer limit from data, not guesswork.

Think about concurrency. If messages arrive for the same conversation from two tabs at the same time, both read the same history and append their answers one after the other; two answers end up side by side in the history. A simple fix is to put a concurrency token on the conversation row ([ConcurrencyCheck] or a rowversion in EF Core) and reject the second request on conflict.

Common Pitfalls

First, reading history with another user's conversation id. Put the ownership check inside the query, not in a separate if, and answer with 404.

Second, storing only response.Text. The tool call and tool result messages are lost; in the next round the model ends up asking "where did I know this from?", calls the same tool again or makes things up. Store the whole of response.Messages.

Third, history that grows without limit. Every conversation without a reducer fills the context window one day. The fact that this doesn't show up as an error makes it harder: the model silently cuts old messages and the user says "I just told you that".

Fourth, not knowing that the reducer drops tool messages. MessageCountingChatReducer leaves out messages containing function calls and function results entirely, regardless of the count limit. Information based on a tool result only reaches the model if the assistant answer that summarized it stayed in the list.

A Small Experiment

★☆☆ Add inputTokens = response.Usage?.InputTokenCount to the endpoint's response and talk for five rounds in the same conversation. How does the input token count grow per round?

★★☆ Lower MessageCountingChatReducer(20) to 2 and run the two-round conversation from the opening again. Does the model still know Ada's name? If not, where would you keep lasting information such as the user's name, independent of the history?

★★★ Add a Summary column to Conversation: when the history passes 20 messages, summarize the old part with SummarizingChatReducer, store the summary and send it right after the system prompt on later requests. How do you run the summarization without making the user wait for their answer?

The assistant remembers now. In the next post we make it do work: taking tool calling, whose mechanics we saw in LLM Fundamentals, to production, FunctionInvokingChatClient settings, tools that use scoped services from DI (Dependency Injection), and an approval flow over HTTP (Hypertext Transfer Protocol) for irreversible actions such as cancelling an order.