Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
SLO-Based Alerting: Burn Rate, Multiwindow Alerts and Ending Alert Fatigue
An SLO-based alert measures how fast the error budget is spent (burn rate) over a long and a short window together: the long window stays quiet on short spikes, the short one closes the alert soon after the outage. For Bookstore's checkout we turn the Workbook's 14.4/6/1 thresholds into Prometheus rules, test them with promtool, measure when they fire and stop in a live outage, and compare them with what Sloth generates.
Alerting: Symptom-Based Alerts, Alertmanager, Grafana Alerting, Runbooks and Alert Fatigue
Alerts go on symptoms the user feels, and each one has a runbook; causes are tickets, not pages. For Bookstore we write burn-rate, latency, retry, missing-telemetry and Watchdog alerts in Prometheus, route them with Alertmanager, watch grouping, silences and inhibition during a real failure, and measure why the error-rate alert goes quiet when the API is down.