Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
Document Ingestion: Clean Text, Metadata and Incremental Updates from PDF, HTML and Markdown
RAG quality starts at ingestion: a "fi" ligature from a PDF made Contains("fiyat") return false, a page number landed in the middle of a section, and an HTML menu leaked into a chunk. Clean text, metadata on every chunk, a pipeline with partial success using Microsoft.Extensions.DataIngestion (preview), incremental updates with content hashes, and removing deleted documents from the index.
Choosing a Vector Database: pgvector, SQL Server 2025, Qdrant, Azure AI Search and Elasticsearch
Try your existing database's vector support first, and tune the ANN index by measuring recall on your own data. On a 100,000-vector pgvector table exact search took 237 ms and HNSW 1 ms, but recall stayed at 0.83 with the default ef_search; with a filter, HNSW forced onto the index returned zero rows while an iterative scan found 10. HNSW, IVFFlat, DiskANN, choosing a metric, and five options compared.
Vector Search with EF Core (2): PostgreSQL pgvector, HNSW Index and Dapper
With Pgvector.EntityFrameworkCore a Vector property, an HNSW index and CosineDistance ordering take a few lines; the real risk is silent failures: an index built with vector_l2_ops was never used by a cosine query, and without ReloadTypes Npgsql could not write the vector. Search with a permission filter, EXPLAIN checks, halving storage with halfvec (identical top-5 for 30 of 30 questions) and the same query with Dapper.
Hybrid Search and Reciprocal Rank Fusion: Combining Keyword and Vector Search
Vector search misses SKUs, ISBNs and rare terms; adding keyword search with PostgreSQL tsvector (turkish) and merging the two lists by rank rather than score, with Reciprocal Rank Fusion, closes that gap. We build it with EF Core, a single SQL query and VectorData HybridSearchAsync, and measure vector, keyword and hybrid recall on 30 questions.
Authorization and Security in RAG: Document-Level Permissions, Multi-Tenant Filters and Indirect Prompt Injection
In RAG, authorization isn't a request in the prompt but the WHERE clause of the retrieval query: told "don't reveal this", gemma3:4b told a customer the staff discount. We take tenant and ACL from the token into the query with EF Core 10 named query filters, and measure an HNSW filter returning 1 result instead of 10 and a poisoned reader review asking for card numbers.