Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
SLO-Based Alerting: Burn Rate, Multiwindow Alerts and Ending Alert Fatigue
An SLO-based alert measures how fast the error budget is spent (burn rate) over a long and a short window together: the long window stays quiet on short spikes, the short one closes the alert soon after the outage. For Bookstore's checkout we turn the Workbook's 14.4/6/1 thresholds into Prometheus rules, test them with promtool, measure when they fire and stop in a live outage, and compare them with what Sloth generates.
SLOs and Error Budgets: Setting Targets, the Math of Nines and an Error Budget Policy
An SLO is a target derived from today's measurements and user expectations and stricter than the SLA; the room it leaves is the error budget, and a policy written in advance says what happens when it runs out. For Bookstore's checkout we turn a 99.5 percent availability and an 800 ms latency SLO into a document, Prometheus rules and OpenSLO, convert nines into minutes, and measure how a new bucket and a short window misreport the budget.
Alerting: Symptom-Based Alerts, Alertmanager, Grafana Alerting, Runbooks and Alert Fatigue
Alerts go on symptoms the user feels, and each one has a runbook; causes are tickets, not pages. For Bookstore we write burn-rate, latency, retry, missing-telemetry and Watchdog alerts in Prometheus, route them with Alertmanager, watch grouping, silences and inhibition during a real failure, and measure why the error-rate alert goes quiet when the API is down.
Types of Performance Testing and Goals: Load, Stress, Spike, Soak, Breakpoint and From SLO to Acceptance Criteria
Each performance test type answers its own question: smoke shows the script works, average load the normal day, stress the peak, spike a sudden jump, soak what happens after hours and breakpoint the capacity limit. We derive acceptance criteria from an SLO as load, duration and limits, compute virtual users with Little's Law and show with real k6 runs on Bookstore how a closed model reduces load on its own and hides the problem when the system slows down.