Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
Handling Text Correctly: Normalization, Diagnosing Mojibake and Base64
Two strings that look the same may not be equal, because "ş" can be written as one code point (NFC) or as s plus a combining mark (NFD); the fix is to bring text to NFC at the edge of the system. Search keys with NFKC, culture-dependent search, diagnosing and repairing mojibake like "ÅŸ" with a hex dump, the CSV/Excel BOM, HTTP charset, the N prefix and UTF-8 collations in SQL Server, Base64/Base64Url and why a JWT isn't encrypted.
Unicode and UTF-8/UTF-16: Code Points, Surrogate Pairs, Rune and Why string.Length Breaks on Emoji
string.Length breaks on emoji because .NET and JavaScript count strings in UTF-16 code units; the family emoji is one grapheme but 5 code points, 8 code units and 18 UTF-8 bytes. From ASCII to Unicode, code points, planes and scalar values, encoding UTF-8 and surrogate pairs by hand, Rune, u8 literals and the BOM, and counting a 280-character review limit correctly with StringInfo and Intl.Segmenter.