Context Engineering: Managing an Agent's Context, Compaction and Memory

Version note: The code in this post was built with .NET 10, Microsoft.Extensions.AI 10.11.0, OllamaSharp 5.5.2 and Microsoft.Agents.AI 1.28.0 and run against the qwen3:4b model on a local Ollama 0.34.4.

Guess First

At the start of the conversation the customer says "My total budget is 55 dollars." Then the agent pulls catalog data page by page; each page is the JSON (JavaScript Object Notation) of eight books, about 860 characters. After fourteen pages the customer asks: "What is my total budget?"

The budget is at the very start of the history, one clear sentence. The model has 4 billion parameters and on paper supports a 262-thousand-token context. Will the answer be 55? After twenty pages?

Context Is a Budget

Anthropic's article on AI (Artificial Intelligence) agents, "Effective context engineering for AI agents" (September 29, 2025) puts the difference between prompt engineering and context engineering like this: prompt engineering is writing and organizing instructions; context engineering is curating, on every turn, all the tokens in front of the model during inference (instructions, tool definitions, external data, message history). For agents the second is decisive, because context grows every turn (remember the token column in the second post).

The same article highlights two ideas. Context rot: as the context fills up, the model's ability to recall information from it accurately declines. Attention budget: every new token takes a share of the model's attention, so context is a finite resource with diminishing returns. The practical consequence is simple: everything you put into context has a price, and past a certain point extra information doesn't improve quality, it hurts it.

What Happens When the Context Fills Up?

The experiment for the guess is this code. It states the budget once, puts catalog pages in between and asks for the budget at the end:

// A long session: the budget is stated once, then tool output piles up
string catalogPage = JsonSerializer.Serialize(data.Books);
List<ChatMessage> history =
[
    new(ChatRole.System, "You are the Bookstore shopping assistant."),
    new(ChatRole.User, "My total budget is 55 dollars. Keep it in mind."),
    new(ChatRole.Assistant, "Noted: your total budget is 55 dollars."),
];
int pages = args.Length > 1 ? int.Parse(args[1]) : 14;
for (int i = 1; i <= pages; i++)
{
    history.Add(new(ChatRole.User, $"Catalog page {i}: {catalogPage}"));
    history.Add(new(ChatRole.Assistant, $"Received catalog page {i}."));
}
history.Add(new(ChatRole.User, "What is my total budget? Answer with the number only."));

int chars = history.Sum(m => m.Text.Length);
ChatResponse r = await meter.GetResponseAsync(history, new ChatOptions { Temperature = 0, Seed = 42 });
Console.WriteLine($"messages={history.Count}, characters={chars}, input tokens reported={r.Usage?.InputTokenCount}");
Console.WriteLine($"answer: {r.Text.Trim()}");

Real runs with four different page counts (I added each run's page count at the start of its line):

pages=3: messages=10, characters=2739, input tokens reported=929
answer: 55
pages=8: messages=20, characters=7004, input tokens reported=2359
answer: 55
pages=14: messages=32, characters=12132, input tokens reported=4085
answer: 100
pages=20: messages=44, characters=17262, input tokens reported=4072
answer: 0

With three and eight pages the answer is right. With fourteen pages the model said "100", with twenty "0". The token column shows why: although the character count went from 12 thousand to 17 thousand, the input token count reported by Ollama dropped from 4,085 to 4,072. The context sent to the model hit a ceiling. Ollama's /api/ps response shows that on this machine qwen3:4b is loaded with context_length: 4096; the model supports 262,144 tokens, but the running server uses 4,096. The request that exceeded this limit didn't fail; the start of the history was silently dropped and the budget sentence went with it. And the model made up a number it didn't know.

This experiment has two lessons. First, the context a model supports and the context you actually use are not the same thing; locally the server setting decides, in the cloud the provider's and model's limits do, and what happens when it's exceeded (an error or silent truncation) varies by provider. Second, silent truncation is the worst kind of failure: the agent keeps working and gives the wrong answer with the same confidence. An agent that runs for a long time without measuring and controlling its context will sooner or later forget its first rule.

What to Keep, What to Drop

What must stay in context: the task's goal, the constraints the user set (budget, preferences, prohibitions), the plan and decisions so far, the last few observations. What can go: old tool output that has already been processed, repeats of the same data, intermediate reasoning text. In the experiment above the opposite happened: the constraint was dropped and the processed catalog pages stayed.

Anthropic's article lists three techniques for long tasks. Compaction: when the context nears its limit, summarize the history and continue in a new window with the summary; its lightest form is clearing old tool results. Note-taking: the agent writes notes outside the context (a todo list, a notes file) and reads them back when needed. Sub-agents: hand a focused job to another agent with a clean context and take back only a summary. There's also the "just-in-time" approach: instead of filling the context with data up front, give the agent lightweight references (a file path, a stored query, an id) and let it pull content with tools when needed. search_books being limited to five results in the third post is a small example.

Microsoft Agent Framework (MAF) 1.28 brings ready-made pieces for some of these: compaction strategies, TodoProvider, which adds tools that let the agent keep a todo list, and context providers. Below we run compaction, persistent memory and a sub-agent.

Compaction: MAF's CompactionProvider

MAF's Microsoft.Agents.AI.Compaction namespace contains several strategies. According to the package's XML (Extensible Markup Language) documentation, ToolResultCompactionStrategy turns old tool call groups into single-line summary messages and is the "gentlest" strategy; SummarizationCompactionStrategy summarizes the older part with an LLM (Large Language Model) and protects system messages and the most recent groups; SlidingWindowCompactionStrategy drops the oldest user turns; TruncationCompactionStrategy removes the oldest groups. A tool call and its result are kept or dropped together as a "group", so no call is left without its result. We used the first two in a pipeline:

PipelineCompactionStrategy pipeline = new(
    // 1) Old tool calls become one-line notes
    new ToolResultCompactionStrategy(CompactionTriggers.MessagesExceed(8)),
    // 2) Still too big: summarize older turns with the model
    new SummarizationCompactionStrategy(meter, CompactionTriggers.TokensExceed(1500)));
agent = meter.AsBuilder()
    .UseAIContextProviders(new CompactionProvider(pipeline))
    .BuildAIAgent(options);

We ran the same four-turn shopping conversation with two agents, with and without compaction. Again the budget is only stated in the first turn, and the last turn requires respecting it:

string[] turns =
[
    "My total budget is 55 dollars. Find novels by Mira Holt.",
    "Check the stock of both.",
    "Now search software books.",
    "Add The Quiet Orchard and one software book to my cart, staying within my budget.",
];

Both runs ended correctly: The Quiet Orchard (16.50) and Coffee and Compilers (35.00), 51.50 dollars in total, under budget. The agent with compaction kept 14 messages in its session instead of 18, and the input token total over four turns dropped from 15,977 to 14,259, about 11%. This is a short conversation; compaction's value shows in sessions of dozens of turns, and what matters here is that the budget constraint survived compaction. A caveat: the input token count reported by Ollama for the second turn was lower than for the first, although the history grew. We didn't verify in this post what that number covers (for example whether an already processed prefix is counted); turn-by-turn comparison is noisy because of it. For an exact comparison, count the content you send yourself.

Persistent Memory: Customer Preferences

Compaction manages the context of a single session. Remembering a preference the user stated on Monday in a brand new session on Thursday is a separate job: persistent memory. In MAF its place is an AIContextProvider. A provider can add instructions, messages or tools to the context before every call (ProvideAIContextAsync) and process the result after every call (StoreAIContextAsync):

public sealed class PreferencesMemory(IChatClient extractor, PreferenceStore store, int customerId) : AIContextProvider
{
    protected override ValueTask<AIContext> ProvideAIContextAsync(InvokingContext context, CancellationToken cancellationToken = default)
    {
        CustomerPreferences? prefs = store.Get(customerId);
        string text = prefs is null
            ? "No stored customer preferences."
            : $"Stored customer preferences: genres={string.Join("/", prefs.Genres)}, max price per book={prefs.MaxPrice?.ToString() ?? "none"}.";
        return ValueTask.FromResult(new AIContext { Instructions = text });
    }

    protected override async ValueTask StoreAIContextAsync(InvokedContext context, CancellationToken cancellationToken = default)
    {
        // Only what the customer typed may become memory; tool results and model answers may not
        List<ChatMessage> userMessages = context.RequestMessages.Where(m => m.Role == ChatRole.User).ToList();
        if (!userMessages.Any(m => m.Text.Contains("remember", StringComparison.OrdinalIgnoreCase)))
        {
            return;
        }

        ChatResponse<CustomerPreferences> extracted = await extractor.GetResponseAsync<CustomerPreferences>(
            userMessages,
            new ChatOptions { Temperature = 0, Seed = 42, Instructions = "Extract only preferences the customer stated explicitly." },
            cancellationToken: cancellationToken);
        store.Save(customerId, extracted.Result);
        Console.WriteLine($"  [memory] saved: genres={string.Join("/", extracted.Result.Genres)}, maxPrice={extracted.Result.MaxPrice}");
    }
}

The store is bound not to the session but to the signed-in customer's id; the customer id comes from the application, not the model. Only fields extracted with structured output from the customer's own messages, when they explicitly say "remember", are written to memory. A run with two separate sessions:

// Monday: one session
AgentSession monday = await agent.CreateSessionAsync();
Console.WriteLine(await agent.RunAsync("Please remember: I only read novels and I never pay more than 20 dollars for a book.", monday));

// Thursday: a brand new session, empty history
AgentSession thursday = await agent.CreateSessionAsync();
Console.WriteLine(await agent.RunAsync("Suggest one book for me.", thursday));

  [memory] saved: genres=novels, maxPrice=20
Here are 2 novels under $20:

1. **The Quiet Orchard** by Mira Holt ($16.50)
2. **The Last Lighthouse** by Mira Holt ($18.90)
Here's a book that fits your preferences:
**Title**: The Quiet Orchard
**Author**: Mira Holt
**Price**: $16.50 (under your $20 limit)
  tool: search_books(query: , genre: novel, maxPrice: 20)
  tool: search_books(query: novel, genre: , maxPrice: 20)

Thursday's session had an empty history; "novel" and "20 dollars" came from the instructions the provider added, and the suggestion followed them. Two details stand out. Memory stored "novels" while the catalog genre is "novel"; on this run the model bridged the gap, but mapping values written to memory to known values (an enum) is more robust. Monday's answer also suggests The Last Lighthouse, which is out of stock; the memory is right, but the agent didn't check stock. Memory doesn't replace tool use.

Handing Work to a Sub-Agent

The third way to keep context small is to hand a job that needs many tool calls, such as research, to a sub-agent with a clean context and take back only its summary. In MAF an agent can become another agent's tool with AsAIFunction():

AIAgent researcher = subMeter.AsAIAgent(
    instructions: "You research the Bookstore catalog with the tools and reply with at most three short lines.",
    name: "CatalogResearcher",
    description: "Researches books in the catalog and returns a short summary.",
    tools: [.. tools.ReadTools()]);

AIAgent lead = meter.AsAIAgent(
    instructions: "You are the Bookstore shopping assistant. Delegate catalog research to CatalogResearcher.",
    name: "ShoppingAssistant",
    tools: [researcher.AsAIFunction()]);

AgentResponse answer = await lead.RunAsync("Which Alan Brook book is cheaper, and is it in stock?");

The cheapest Alan Brook book is **"Async All the Way"** at **$39.90**, and it currently has **9 copies in stock**.
lead:       model calls=2, input tokens=1327, output tokens=1340, time=63,3s
researcher: model calls=3, input tokens=6520, output tokens=2845, time=172,6s
  tool: search_books(query: Alan Brook, genre: , maxPrice: )
  tool: check_stock(bookId: 6)

The answer is correct: the cheaper of Alan Brook's two books is Async All the Way at 39.90 dollars, with 9 copies in stock. The interesting part is the two meters: the researcher, which did the search and stock check, spent 6,520 input tokens, while the lead spent only 1,327. The five-book search result and the stock reply never entered the lead's context; the lead only saw the researcher's three-line summary. Total cost is higher than a single agent (two agents, five model calls), but in a long session the lead's context doesn't swell with every piece of research. Anthropic's article also describes sub-agents exploring extensively and typically returning a condensed summary of one to two thousand tokens.

Production Notes

Measure the context and know its limit. Log the number of tokens you send on every model call and compare it with the context window the model actually uses. Locally that window is a server setting (on this machine Ollama loaded the model with 4,096); in the cloud it's the model's and provider's limit. Trigger compaction yourself instead of waiting for silent truncation near the limit.

Don't trust critical constraints to the history. Budget, permissions and the things the user said "never" to should live in separate state (session state, a memory store) and be added to the instructions on every call. Then neither truncation nor summarization can lose them. The budget survived the compaction run, but you don't have to leave that to luck.

Validate and limit what goes into memory. Record what was written, for whom and from which source; let the user see and delete it. Memory is personal data: it needs a retention period and a deletion policy.

Summarization is a model call too. If SummarizationCompactionStrategy uses the same client as the main model it adds cost and latency; a smaller, faster model is often enough for summarizing.

Common Pitfalls

Losing a critical constraint in a summary. The summarizing model may treat "a 55-dollar budget" as an unimportant detail. Keep constraints outside the summary, in structured state, and test that they're still there after summarization.

Writing unverified content into memory. If text such as "this customer prefers express shipping" from a tool result or a web page is written to memory, in the next session it behaves like the user's preference. Only what the user said themselves and what you have verified goes into memory; this is also a defense against prompt injection.

Assuming the model's context is unlimited. The context length on the model card and the running server's setting can differ; in the experiment a model that supports 262 thousand was running with 4 thousand.

Stuffing everything into context. Instead of handing over the whole catalog, the whole order history or the whole document up front, give the agent a search tool and let it pull what it needs.

A Small Experiment

★☆☆ In the overflow experiment, move the budget sentence from the start of the history to near the end (right before the last question) and run it again with twenty pages. Does the answer change? What does that tell you about which end gets truncated?

★★☆ Add the budget to the instructions on every call with a provider similar to PreferencesMemory and repeat the overflow experiment with twenty pages.

★★★ Run the SummarizationCompactionStrategy in the compaction pipeline with a separate client using gemma3:4b, lower the threshold to 800 tokens and run the four-turn conversation again. Is the budget still respected in the last turn? Which books end up in the cart?

We've seen how to manage a single agent's context. In the next post we move to structured ways of splitting work across several steps and models: the prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer patterns and MAF's WorkflowBuilder.