Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
How Search Engines Work: Crawling, Rendering, Indexing and Serving
Google processes a page in four steps: it discovers and crawls the URL, renders the JavaScript, indexes the content through duplicate clusters and canonical selection, and serves it by relevance and quality when a query arrives. We show Googlebot's 2 MB limit with a real measurement and follow a post on mertozen.com step by step.
"Why Isn't My Page Indexed?" Diagnosing Indexing Problems Step by Step
For a page that isn't indexed, ask in order: does Google know the URL, does robots.txt allow it, does it return 200, is there a noindex, does the canonical point to itself, is the content real; the first no is where the problem is. We cover the diagnosis tree, a .NET 10 check tool that runs locally, the robots.txt and noindex conflict, and the Crawled/Discovered – currently not indexed states.