Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
Microsoft.Extensions.VectorData: Your First Collection with the Vector Store Abstraction
Microsoft.Extensions.VectorData lets you define a record with attributes and move from an in-memory store to PostgreSQL + pgvector with the same code; the generator you give the store produces the embeddings. With real outputs we look at filtered search, ScoreThreshold, how the distance function flips the meaning of the score, and a dimension bug the InMemory provider never catches.
Observing LLM Calls: UseOpenTelemetry, GenAI Semantic Conventions and the Aspire Dashboard
To observe LLM calls, put UseOpenTelemetry inside UseFunctionInvocation, listen to the same source name with AddSource and read the trace in the Aspire Dashboard. In a real trace we find that a two-minute delay came from the Ollama queue rather than the model, and see how EnableSensitiveData carries an email address into telemetry.
Controlling LLM Cost: DistributedCachingChatClient, Rate Limiting and Token Usage
We control LLM cost in three layers: DistributedCachingChatClient with Redis answers repeated questions with zero tokens, per-user rate limiting limits abuse, and a daily token budget kept with UsageDetails limits real spending. With real output we show what goes into the cache key, that entries are written without expiry by default, and why token accounting must sit inside the cache.
Taking Tool Calling to Production: FunctionInvokingChatClient Settings, DI Services and Approvals
In production, on whose behalf a tool runs comes from the session, not from the model's arguments, and irreversible actions wait for user approval through ApprovalRequiredAIFunction. With real runs we show an OrderTools class using scoped services, FunctionInvokingChatClient settings, separating tool errors from user errors, and an approval flow over two HTTP requests.
Persisting Chat History: Stateless APIs, ConversationId, EF Core and Chat Reducers
With a stateless provider your code keeps the chat history: we turn ChatMessage instances, including tool call and tool result contents, into JSON with AIJsonUtilities and store them in EF Core per user and conversation. We put the ownership check into the query, also support stateful providers through ConversationId, and compare the real output of MessageCountingChatReducer and SummarizingChatReducer.
LLM Streaming in ASP.NET Core: Minimal API, IAsyncEnumerable and Server-Sent Events
The reliable way to stream an LLM answer to the browser is .NET 10's TypedResults.ServerSentEvents: delta, thinking, tool, usage, done and error events, a heartbeat during silent waits and a CancellationToken carried all the way to the model. With real measurements we show the limits of a plain IAsyncEnumerable, that the model really stops when the user closes the tab, and that an endpoint forgetting the token wastes 1024 tokens.
Wiring IChatClient into ASP.NET Core: AddChatClient, Keyed Clients, Options and Secrets
We register IChatClient in ASP.NET Core as a singleton with AddKeyedChatClient, use keyed clients for a "fast" and a "smart" model, and choose provider and model through options validated at startup. With real logs we show the standard resilience handler's 10 s/30 s defaults regenerating an LLM call three times and ending in a 500, then tune timeouts and retries for LLMs.
The IChatClient Pipeline: ChatClientBuilder, Middleware Order and Your Own DelegatingChatClient
In ChatClientBuilder the first middleware added is outermost, and order changes behavior: with the cache outside function invocation the tool never runs, inside it the tool reads fresh data on every request. We write two DelegatingChatClient classes for Bookstore, a token budget and personal data masking, and show with a real run how forgetting the streaming override silently leaks data.
The .NET AI Stack: Where Microsoft.Extensions.AI, VectorData, Evaluation and Agent Framework Fit
An application takes Microsoft.Extensions.AI plus a provider package, while a library depends only on Abstractions; VectorData, DataIngestion, Evaluation and Agent Framework sit on top of or next to that IChatClient layer. We map the packages, run the aichatweb template against a local Ollama and fix three real errors that appear at the layer boundaries.
The Evolution of LLM APIs: From Completions to Tool Calling, Structured Outputs and MCP
The evolution of LLM APIs with verified dates, from the first text-continuing API to the chat format, from function calling to JSON mode and structured outputs, from reasoning parameters to the Model Context Protocol and agents. We see which example in the series each step maps to, and at the end of the series we build a bridge to the next AI series.
LLMs from 2023 to Today: Open-Weight Models, Long Context and Reasoning
A summary of the LLM world from GPT-4 to today with verified dates: the rise of open-weight models with Llama, Mistral, Qwen and Gemma, the context window growing from 2,048 tokens to a million, models that read images, and the reasoning models that came with o1 and DeepSeek-R1. We also discuss the difference between open-weight and open source, and why we give no model ranking.
From the Transformer to ChatGPT: The Birth of LLMs (2017–2022)
The steps that made today's LLMs possible, from the 2017 "Attention Is All You Need" paper to the launch of ChatGPT: the Transformer, pre-training with GPT-1 and BERT, GPT-2's staged release, scaling laws and GPT-3, InstructGPT and RLHF. We see which concept in the series each step is the root of and what it changed for developers.