Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
Troubleshooting Terraform: Reading the Plan, TF_LOG, Provider Errors, State Locks, "Already Exists" and Drift
The first step in troubleshooting Terraform is to check which stage the error came from (init, plan or apply) and to read the notes in the plan. We read the plan as text and as JSON, look inside the provider with TF_LOG, and go through a lock file mismatch, an "already exists" error, a locked state, drift the provider cannot see, a cycle error and -target with real output; on Azure we solve the case of a Key Vault waiting in soft-delete.
Terraform Built-in Functions and Expression Patterns: String, Collection, Encoding, Filesystem and Network Functions, for, Splat and terraform console
The way to write Terraform expressions with confidence is to know the 119 built-in functions by family and to try every expression in terraform console first. We try string, collection, encoding, filesystem and network functions, for and splat, optional() defaults and AzureRM's provider functions, and in Bookstore we build a subnet per environment with cidrsubnet and the API settings from YAML.
Windows Containers: Containerising a .NET Framework 4.8 and IIS App, Server Core, Nano Server and Isolation Modes
A .NET Framework 4.8 and IIS application can go into a Windows container, but it only runs on a Windows host whose Windows version matches the image. For Bookstore's old back office we cover the Server Core based Dockerfile, configuration builders that override web.config with environment variables, ServiceMonitor and LogMonitor, process and Hyper-V isolation, the 2.6 GB image size, multi-OS tags and Windows node pools on AKS, and weigh the move to .NET 10.
Lesser-Known Dockerfile Instructions and Flags: Heredocs, SHELL, ADD from Git/URLs, COPY --exclude/--parents/--link and Build Checks
Used carefully, the newer Dockerfile flags make a build both shorter and safer: set -eu in heredocs, --checksum on ADD, --parents and --exclude on COPY, RUN --network=none for publish. With real output we show that the frontend's docker/dockerfile:1 tag was still on 1.27.1 on the day of writing and how behaviour changes with 1.29.0, then combine SHELL with pipefail, ONBUILD, build checks and hadolint in Bookstore.Api's new Dockerfile.
Extending Kubernetes: CRDs, Controllers and the Operator Pattern, ownerReferences and Finalizers
Extending Kubernetes means defining a new resource type with a CRD and letting a controller that watches it keep turning desired state into reality. With real output: a Campaign CRD for Bookstore, CEL validation, garbage collection via ownerReferences, a controller written in .NET with KubernetesClient, the safe fix for a namespace stuck in Terminating because of a finalizer, and the orphaned resource a forced finalizer removal leaves behind.
Working Efficiently with kubectl: Output Formats, jsonpath, custom-columns, Selectors, patch, events, k9s and Plugins
Using kubectl efficiently means getting exactly the answer you need with custom-columns, jsonpath, selectors and events when reading, and choosing the right patch type and safety catches when changing things. With real output we see a JSON patch overwrite the wrong variable by index and crash a pod, a server-side apply conflict, kubectl edit drift, a confirmed delete via kuberc, and krew, k9s and stern.
Where Do Pods Land? nodeSelector, Affinity/Anti-Affinity, Taints/Tolerations, Topology Spread and PriorityClass
kube-scheduler picks a node for a pod in a filter step and a score step; required rules can leave a pod Pending, preferences guarantee nothing. With real output we see required anti-affinity lock a rollout, zone spread get stuck on a cordon and recover with nodeTaintsPolicy: Honor, a tainted worker pool, preemption with PriorityClass, and how to read the "0/3 nodes are available" message.
SSH in Depth: ~/.ssh/config, ProxyJump Through a Bastion, Port Forwarding to a Remote Database and the Risk of Agent Forwarding
Hop to private servers through a bastion with ProxyJump, not ssh -A: with agent forwarding on, someone who is root on the bastion can log into the database server with your key, and ControlPersist keeps that window open for minutes. We cover the first-value-wins rule of ~/.ssh/config, Match and ControlMaster, tunnels with -L, -R and -D, narrowing the bastion with AllowTcpForwarding and PermitOpen, FIDO2 keys and SSH certificates, with real output.
Finding, Archiving and Moving Files: find, xargs, tar, gzip/zstd, zip, rsync, diff and Checksums
find -mtime +7 does not find files older than 7 days as people assume, but files at least 8 days old; read the list with -print before deleting, put -delete last and hand file names to xargs with -print0 | xargs -0. We cover building Bookstore's release package with tar --zstd, the difference between gzip, xz and zstd, the trailing / and --delete trap in rsync, and verifying the package with sha256sum, cmp and diff, with real output.
Editing Files on a Server: vim Essentials, nano, sudoedit and Changing Config Files Safely
Edit config files on a server with sudoedit instead of sudo vim, take a backup first, read the change with diff -u, test it with nginx -t or sshd -t and use reload instead of restart; a reload with a broken config keeps the site up, a restart takes it down. We cover vim's modes and essential commands, nano, why the editor falls back to nano under sudo, the CRLF and .swp pitfalls and tracking /etc in Git with etckeeper, with real output.
GitHub from the Terminal with the GitHub CLI: gh pr, gh issue, gh run, gh api and Extensions
The GitHub CLI (gh) is not a Git tool but a client for GitHub's API; with a token that has the right scopes you manage pull requests, issues, CI runs and releases from the terminal. We cover gh pr, gh issue, gh run, gh api, aliases and extensions, Bookstore's two-line flow from PR to auto-merge, and the GH_HOST and scope pitfalls.
Searching History: git grep, git log -S/-G/-L, blame -w -C, ignoreRevsFile and "When and Why Did This Line Change?"
Blame's first answer is not enough to find who changed a line, when and why: git log -S finds the commit where a text appeared, -G every change to those lines, -L the history of a single method. We trace Bookstore's institutional discount bug with git grep, the pickaxe, blame -w -C and .git-blame-ignore-revs on Git 2.56.