Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
The Evolution of the OWASP LLM Top 10: From 2023 to 2025 and 2026, the Agentic Top 10 and the Agent Control Standard
In the OWASP LLM Top 10 the same ID can point to three different risks in three editions: 2023's LLM06 was sensitive information disclosure, 2026's is Unbounded Consumption. We align the 2023, 2025 and 2026 lists by topic, cover dissolved and added entries, the 2026 rank moves, the Agentic Top 10 and the Agent Control Standard, and write a C# tool that moves an old penetration test report to 2026 IDs.
Guardrail Architecture and Security Testing: IChatClient Middleware, Safety Evaluators and Red-Team Regression in CI
Guardrails are middleware placed in the right order in the IChatClient pipeline, with a deliberately chosen fail mode, re-measured with the same harmless red-team set on every model change. We build a kill switch and input and output guards, look at safety evaluators and our own IEvaluator, and show the xUnit set catching our canary-token approach breaking in two different directions when moving to gemma3:12b.
Unbounded Consumption: Denial of Wallet, Context Bombs and Limiting Agent Loops
In an LLM the unit of cost is the token, not the request, so input, output, time and budget are limited before every call. We measure a 120,000-character document being silently truncated by Ollama into a wrong answer and qwen3:4b thinking for 8,000 tokens on a seven-word question before returning nothing, then build reserved user and tenant budgets, timeouts and a state-hash loop guard.
Supply Chain, Poisoning and Vector Weaknesses: Protecting Models, Data and the Embedding Store
Models, prompts and tool definitions are dependencies too: pin them by digest, keep them in an inventory with their hashes and review every change. We catch the gemma3:4b tag moving to a different artifact in the registry, write an AIBOM with CycloneDX 1.7, measure five campaign reviews misleading the answer 5/5 times and an ingestion gate stopping it, and show that a note's content can be guessed from its leaked embedding alone.
LLM Output Is Untrusted Input: Improper Output Handling, Markdown Rendering and Misinformation
Model output is untrusted input like any external input: Angular's sanitizer stops scripts but not a Markdown image pointing to an external address. We write a safe message component with an allowlisting marked renderer and Vitest tests, show CSP img-src cutting off the leak in Chromium, validate structured output against the catalog, limit the model to query template parameters instead of SQL, and measure how a correctly numbered citation can carry a wrong claim.
Excessive Agency: Narrowing Tool Permissions, Acting on Behalf of the User and Limiting Autonomy
The cure for Excessive Agency is not a smarter model but narrower tools, permissions and autonomy. In the Bookstore assistant we split the cancel tool into a preview and a user confirmation, add a SELECT-only PostgreSQL role, downstream calls with the user's token, an amount threshold, a rate limit, a call budget via FunctionInvoker and an audit trail, and watch qwen3:4b's real behavior on "cancel all my orders".
Sensitive Data and Hidden Context Exposure: System Prompts, RAG Context and Data Sent to Providers
Everything given to the model, system prompts and tool schemas included, should be treated as given to the user; secrets and authorization decisions live in code, not in the prompt. We show a discount code in a system prompt leaking 10/10 times and hiding in reasoning output, mask personal data sent to the provider with an HMAC-redacting DelegatingChatClient, and look at KVKK and content recording in telemetry.
Layered Defense Against Prompt Injection: Direct, Indirect and Multimodal Injection, Spotlighting and Privilege Separation
No single measure stops prompt injection; the real protection is limiting in code what the model can do once it is fooled, while spotlighting and classifiers are only extra layers. With a harmless canary we measure delimiters, datamarking, encoding, a local classifier, image scanning and a quarantined-model architecture on gemma3, and show with real numbers what each layer misses.
Threat Modeling an LLM Application: Trust Boundaries and the OWASP LLM Top 10 2026 Map
In an LLM application the model is not a security boundary: the system prompt, the user's message and retrieved documents meet in one token stream, so authorization and validation decisions belong in code. We map Bookstore.Assistant's data flow with its trust boundaries, show a harmless canary review being followed 10/10 times on two local models, and map the OWASP LLM Top 10 2026 and Agentic Top 10 entries onto the parts of the system.
Agentic RAG and GraphRAG: When Classic RAG Is Not Enough
Classic RAG handles most questions whose answer sits in one place; on multi-hop questions an agent that uses retrieval as a tool found the right source but spent 3 calls, 11 times the tokens and 30 times the time. GraphRAG's first step produced fragmented entities with a small model, and long context with the whole corpus in the prompt tied classic RAG at 7/10.
Measuring RAG: Recall@k, MRR, nDCG, Faithfulness and Groundedness
Measuring RAG means measuring retrieval against labelled sources (recall@k, MRR, nDCG) and generation with an LLM judge (groundedness, relevance) separately. We compare four retrieval methods on 30 questions in C#, show Microsoft.Extensions.AI.Evaluation giving 5 to an answer that misses the question, and cut a 1 min 42 s test to 1.1 s with response caching.
Authorization and Security in RAG: Document-Level Permissions, Multi-Tenant Filters and Indirect Prompt Injection
In RAG, authorization isn't a request in the prompt but the WHERE clause of the retrieval query: told "don't reveal this", gemma3:4b told a customer the staff discount. We take tenant and ACL from the token into the query with EF Core 10 named query filters, and measure an HNSW filter returning 1 result instead of 10 and a poisoned reader review asking for card numbers.