Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
The .NET AI Stack: Where Microsoft.Extensions.AI, VectorData, Evaluation and Agent Framework Fit
An application takes Microsoft.Extensions.AI plus a provider package, while a library depends only on Abstractions; VectorData, DataIngestion, Evaluation and Agent Framework sit on top of or next to that IChatClient layer. We map the packages, run the aichatweb template against a local Ollama and fix three real errors that appear at the layer boundaries.
The IChatClient Pipeline: ChatClientBuilder, Middleware Order and Your Own DelegatingChatClient
In ChatClientBuilder the first middleware added is outermost, and order changes behavior: with the cache outside function invocation the tool never runs, inside it the tool reads fresh data on every request. We write two DelegatingChatClient classes for Bookstore, a token budget and personal data masking, and show with a real run how forgetting the streaming override silently leaks data.
Wiring IChatClient into ASP.NET Core: AddChatClient, Keyed Clients, Options and Secrets
We register IChatClient in ASP.NET Core as a singleton with AddKeyedChatClient, use keyed clients for a "fast" and a "smart" model, and choose provider and model through options validated at startup. With real logs we show the standard resilience handler's 10 s/30 s defaults regenerating an LLM call three times and ending in a 500, then tune timeouts and retries for LLMs.
LLM Streaming in ASP.NET Core: Minimal API, IAsyncEnumerable and Server-Sent Events
The reliable way to stream an LLM answer to the browser is .NET 10's TypedResults.ServerSentEvents: delta, thinking, tool, usage, done and error events, a heartbeat during silent waits and a CancellationToken carried all the way to the model. With real measurements we show the limits of a plain IAsyncEnumerable, that the model really stops when the user closes the tab, and that an endpoint forgetting the token wastes 1024 tokens.
Persisting Chat History: Stateless APIs, ConversationId, EF Core and Chat Reducers
With a stateless provider your code keeps the chat history: we turn ChatMessage instances, including tool call and tool result contents, into JSON with AIJsonUtilities and store them in EF Core per user and conversation. We put the ownership check into the query, also support stateful providers through ConversationId, and compare the real output of MessageCountingChatReducer and SummarizingChatReducer.
Taking Tool Calling to Production: FunctionInvokingChatClient Settings, DI Services and Approvals
In production, on whose behalf a tool runs comes from the session, not from the model's arguments, and irreversible actions wait for user approval through ApprovalRequiredAIFunction. With real runs we show an OrderTools class using scoped services, FunctionInvokingChatClient settings, separating tool errors from user errors, and an approval flow over two HTTP requests.
Controlling LLM Cost: DistributedCachingChatClient, Rate Limiting and Token Usage
We control LLM cost in three layers: DistributedCachingChatClient with Redis answers repeated questions with zero tokens, per-user rate limiting limits abuse, and a daily token budget kept with UsageDetails limits real spending. With real output we show what goes into the cache key, that entries are written without expiry by default, and why token accounting must sit inside the cache.
Observing LLM Calls: UseOpenTelemetry, GenAI Semantic Conventions and the Aspire Dashboard
To observe LLM calls, put UseOpenTelemetry inside UseFunctionInvocation, listen to the same source name with AddSource and read the trace in the Aspire Dashboard. In a real trace we find that a two-minute delay came from the Ollama queue rather than the model, and see how EnableSensitiveData carries an email address into telemetry.
Microsoft.Extensions.VectorData: Your First Collection with the Vector Store Abstraction
Microsoft.Extensions.VectorData lets you define a record with attributes and move from an in-memory store to PostgreSQL + pgvector with the same code; the generator you give the store produces the embeddings. With real outputs we look at filtered search, ScoreThreshold, how the distance function flips the meaning of the score, and a dimension bug the InMemory provider never catches.
Testing Code That Uses IChatClient: Fake Clients, Deterministic Tests and Microsoft.Extensions.AI.Evaluation
You test the correctness of code that uses IChatClient with a fake client, without a model and in milliseconds; you measure the model's answer quality with Microsoft.Extensions.AI.Evaluation, in separate, cached evaluation runs. In real runs we see judge timeouts, reasoning hitting an 800-token cap, a cache returning stale answers, and a Relevance score that catches a prompt bug.
Getting Started with Microsoft Agent Framework: From IChatClient to AIAgent
AsAIAgent turns an IChatClient into a ChatClientAgent; the agent takes over instructions, function invocation and chat history through AgentSession, but it doesn't make the model smarter. We write the same Bookstore assistant first with a plain IChatClient and then as an agent, and compare RunAsync, RunStreamingAsync and AgentResponse with real outputs.
Middleware, Context Providers and Session Persistence in Agent Framework
In Agent Framework, agent run middleware handles inputs like card numbers, function invocation middleware blocks access to other customers' orders, and a context provider writes the user's preference into the session, which is saved and restored as JSON. In a real run we see the memory work, the model still invent an author, and reasoning bloat the session.