What Is SRE? Its Relation to DevOps, Toil, the Error Budget Idea and SRE in a Small Team

Version note: This post is conceptual and has no code; the quotes and numbers were checked on 14 November 2026 against the online editions of Google's SRE book (2016) and the SRE Workbook (2018) on sre.google.

Guess First

One morning Selin, the Product Owner on the team behind Bookstore, the online bookstore used as the shared example across this site's series, says: "Checkout must never go down again. Our target is 100 percent." Emre pushes back: "100 is impossible, let's say 99.99." Ayşe says "99.9 is enough." Three questions: in a 30-day month, how many minutes of downtime does 99.9 percent allow, and how many does 99.99? If the customer's phone and mobile connection work without trouble 99 percent of the time, would the customer notice the difference between those two targets? And how does a 100 percent target affect the team's next release? The answers are halfway down.

Where Did SRE Come From?

SRE, Site Reliability Engineering, was born at Google. When Ben Treynor Sloss joined Google in 2003 he was put in charge of a seven-engineer "Production Team". He had always been a software developer, and he built the team the way a developer would want to work. The introduction of the SRE book defines it in one sentence: "SRE is what happens when you ask a software engineer to design an operations team" (Treynor Sloss, Site Reliability Engineering, 2016).

In practice this means: a classic operations team keeps the system running by hand, and as the system grows the team grows with it. An SRE team tries to solve the same work by writing code; it automates repetitive operational work, measures reliability and ties the question "how reliable should it be" to a number. Google described the approach in the 2016 book "Site Reliability Engineering" and in the hands-on "The Site Reliability Workbook" in 2018. Both can be read for free on sre.google and they are the main sources for this series.

A short history: first the software team wrote the code and "handed it over" to the operations team; releases were rare and frightening. In late 2008 the name "DevOps" appeared and said that development and operations should look at the same goal. SRE at Google had started earlier, but when the books came out they gave concrete answers to many of the questions DevOps was asking.

How SRE Relates to DevOps

The SRE book frames the relationship like this: DevOps is a set of principles (shared ownership, small and frequent changes, automation, measurement), and SRE is a specific implementation of those principles. The subtitle of the relevant Workbook chapter says it in words a developer understands: "class SRE implements interface DevOps". DevOps answers "what", SRE answers "how".

The concrete parts of that "how" are the posts of this series: the SLI, the service level indicator that measures how well the service serves its users; the SLO, the service level objective that sets a target for that measurement over a time window; the error budget, the room the target leaves; alerts built on the SLO; on-call rotations and runbooks; incident management; and the postmortem, the blameless review written after an incident. The DORA metrics in the CI/CD Fundamentals post "Measuring and Speeding Up a Pipeline: DORA Metrics, Caching and Parallelism" measured the speed of delivery; SRE adds a measurement of what the user experiences next to it.

Why Is 100 Percent the Wrong Target?

One of the most quoted ideas in the introduction of the SRE book is this: "100% is the wrong reliability target for basically everything" (Treynor Sloss, Site Reliability Engineering, 2016). It names pacemakers and anti-lock brakes as the exceptions. For a web service there are three reasons.

First, the user cannot see the difference. A customer reaches Bookstore from a phone, over a mobile network, through a home modem. Every link in that chain has its own outages. The "Embracing Risk" chapter of the SRE book puts it this way: someone on a phone that is 99 percent reliable cannot tell 99.99 percent from 99.999 percent. That answers the second question from Guess First: the outages on the customer's own side are far bigger than the gap between 99.9 and 99.99 percent.

Second, every "nine" costs more than the one before. Going from 99 to 99.9 percent usually takes a good deploy process and a few alerts. 99.99 needs a redundant database, automatic failover, several regions and 24/7 on-call. Each extra nine takes several times the effort of the previous one, and the user does not feel the return.

Third, and most important: a 100 percent target stops change. Every deploy is a risk; for a team with a 100 percent target the safest thing is never to deploy. On paper Selin's target asks for reliability; in practice it blocks new features. That is the answer to the third question.

The Error Budget: An Agreement Between Speed and Reliability

If the target is not 100 percent, the gap becomes a budget. With a 99.9 percent SLO the error budget is 0.1 percent: one thousandth of 43,200 minutes in 30 days, about 43 minutes. For 99.99 percent it drops to 4.3 minutes; rolling back a single bad deploy can eat the whole budget. For 99.5 percent it is 216 minutes a month, 3.6 hours. That was the first question in Guess First.

The real job of the error budget is to settle a disagreement. Developers want fast releases, operations wants the system not to break. The SRE book calls this tension a "structural conflict" and resolves it with the error budget: while there is budget left the team takes risks, ships new versions and runs experiments; when the budget runs out the priority shifts to reliability work. The decision is no longer an argument of "I think this deploy is risky" but a measured number. The budget is there to be spent: a budget that is never spent shows that the target is higher than it needs to be, or the team slower than it needs to be.

Three things make this work: an SLI that measures what the user experiences, an agreed SLO, and an error budget policy that says in advance what happens when the budget runs out. We cover the SLI in the second post of the series and the SLO and policy in the third.

Toil: Operational Work That Has to Be Measured

The second core idea of SRE is toil: work done to run production that is manual, repetitive and leaves no lasting value. The "Eliminating Toil" chapter of the SRE book lists the properties that make work toil: it is done by hand, it repeats, it could be automated, it fixes the problem of the moment but leaves no lasting improvement, and it grows linearly as the service grows. Not every piece of toil has all of these; the more it has, the more it is toil.

Toil does not mean work nobody likes. Meetings and HR paperwork are overhead; boring work that leaves lasting value (cleaning up the alert configuration from scratch, for example) is not toil. Toil examples from Bookstore:

• Logging into the server when the disk fills up and deleting old log files by hand.
• Importing the institutional customers' monthly order CSV file by hand every month and fixing the errors one by one.
• Restarting the API by hand when the payment provider slows down, because "it usually recovers".
• Applying the same seven steps by hand from a wiki page on every release.
• Renewing the certificate by hand from a calendar reminder before it expires.

Google's rule is that at most 50 percent of an SRE's time goes to toil and operational work; the rest is engineering, the work that reduces toil and improves the system. According to the book, Google's internal surveys put average toil at about 33 percent, but it ranges from 0 to 80 percent from person to person. The point of the cap: toil grows with the service; if it is not measured it eats the team's whole week and nobody finds time to improve the system. For a small team the first step is the same: write down the hours spent on operational work for one week.

SRE in a Small Team

Bookstore has no separate SRE team; Selin, Emre and Ayşe both build and run it. Applying everything in Google's book would drown this team. You have to be selective. What every team should take: SLOs for two or three critical user journeys, a small number of alerts tied to the SLO, a runbook for every alert, a weekly on-call rotation and a short postmortem after big incidents. What is a luxury: a separate SRE team, follow-the-sun on-call across several time zones, an automated chaos engineering platform, error budget policies signed by the board.

The first job before starting is an inventory: what runs in production, who looks after it, what is being watched today? This is the table the Bookstore team came up with:

Not a single row measures what the user experiences. The CPU alarm does not say whether checkout works, and the payment provider belonging to nobody will later be the longest part of an incident. The rest of the series fills these gaps one by one.

In Production

SRE is not a change of job title. Renaming the operations team to "SRE" and carrying on with the same work changes nothing; the change starts with measurement. First a measurable SLI, then its history, then a target. An SLO set without history is only a guess.

Make operational work visible. A team that does not write down who got paged how often, and how many hours went into manual intervention, does not notice toil growing. A simple table is enough: date, what happened, how many minutes it took, could it be automated?

Keep the responsibility for reliability in the team that writes the code. In small teams "let one person look after operations" is tempting, but if that person cannot change the code, all they can do is intervene by hand. The idea of SRE is that whoever runs the system can change it.

Common Pitfalls

Putting an SLO on everything. Twenty SLOs for twenty endpoints means twenty charts nobody looks at. Start with the two or three journeys users actually care about.

Using the error budget as a punishment. Asking "who broke it" when the budget runs out pushes the team to hide incidents. The budget is a decision tool; it talks about the system, not about people.

Solving toil with heroics. The person who logs in at midnight and fixes things by hand gets praised, but the same work comes back next week. The real job is to record a manual intervention the second time it happens and open a work item to automate it.

Mistaking infrastructure metrics for reliability. CPU, memory and disk alarms help when you look for the cause of a problem; but users do not see CPU, they see errors and waiting time.

When Is It Not Needed?

For a one-person hobby project or an internal prototype without users, a full SRE process is unnecessary. On-call rotations, an error budget policy and postmortem meetings make sense once there is a team of at least a few people and users who are waiting. In such a project two things are enough: a health check and an uptime check. But on the day the first paying user arrives, defining at least one SLI for checkout is worth it.

A Small Experiment

★☆☆ Calculate the error budget of a 99.95 percent SLO in minutes over a 28-day window. How many minutes is the same target over a 30-day window? Think about why the difference might matter.

★★☆ For one week, write down every piece of operational work in your team: what was done, how many minutes it took, was it manual, does it repeat? Classify it by the properties of toil and work out its share of the weekly working hours.

★★★ Build the inventory table above for a service you maintain. How many rows in the "watched today" column measure what the user experiences? Write down an owner for external dependencies such as the payment or e-mail provider.

For the error budget to be useful, you first have to measure what the user experiences correctly. In the next post we work out Bookstore's critical user journeys and derive SLIs for checkout from ASP.NET Core's built-in http.server.request.duration metric: which request is "good", which one is "valid", and what each measurement point cannot see.