Reliability basics: SLIs, SLOs and error budgets

How to say how reliable a service should be in numbers users would recognise, and how to turn the gap between perfect and good enough into a budget you can spend.

SLI (service level indicator)

Also called SLI, service level indicator, good events ratio

What: A measurement of how a service is behaving from the user's side, best written as a ratio: good events divided by valid events, such as successful requests over all requests, or requests faster than 300 ms over all requests.
Why: A ratio between 0 and 100 per cent can be compared with a target, summed over any window and understood by non-engineers. It is the raw material for SLOs, alerts and error budgets.

PROMQL

# availability SLI over 30 days (Prometheus)
sum(rate(http_requests_total{code!~"5.."}[30d]))
/
sum(rate(http_requests_total[30d]))

Watch out

Measuring CPU or pod restarts as an SLI: users do not feel those directly, so the number can be green while they suffer.

SLO (service level objective)

Also called SLO, service level objective, reliability target

What: A target for an SLI over a rolling window, for example 99.9 per cent of checkout requests succeed over 30 days. It is an internal promise the team plans its work around.
Why: It replaces arguments about whether the service is "reliable enough" with a number agreed with product owners, and it decides when reliability work beats feature work.
Example: Checkout: 99.9 per cent of requests return a non-5xx response, and 99 per cent complete within 500 ms, measured at the load balancer over 30 rolling days.

Watch out

Setting 100 per cent, or copying 99.99 from another team: every extra nine costs far more and users rarely notice it.

SLA (service level agreement)

Also called SLA, service level agreement, uptime commitment

What: A contract with customers that states a level of service and what happens, usually service credits, if it is missed. It is a business and legal document, not an engineering tool.
Why: Knowing the SLA tells you the floor. The internal SLO should be stricter, so you notice and react before the contract is breached.

Watch out

Using the SLA number as the SLO, which means the first time you notice a problem is when money is already owed.

Availability nines and allowed downtime

Also called nines, three nines, four nines, downtime table, uptime percentage

What: Each extra nine cuts allowed failure by ten times. The table shows how much full outage an availability target allows per 30 days and per year.
Why: Turning a percentage into minutes makes targets concrete: 99.99 per cent leaves about four minutes a month, less than one slow rollback.

Code

Target    per 30 days   per year
99%       7.2 hours     3.65 days
99.5%     3.6 hours     1.83 days
99.9%     43.2 min      8.77 hours
99.95%    21.6 min      4.38 hours
99.99%    4.32 min      52.6 min
99.999%   25.9 s        5.26 min

Watch out

Promising four nines on top of a single-region database and a vendor whose own SLA is three nines.

Error budget

Also called budget, allowed unreliability, 1 minus SLO

What: The share of events allowed to fail within the SLO window: 1 minus the SLO. At 99.9 per cent over 30 days and 10 million requests, the budget is 10,000 failed requests.
Why: It turns reliability into something you can spend on risky launches, experiments and planned maintenance, and it shows objectively when to slow down.
Example: Budget 10,000 errors a month. A bad deploy burns 6,000 on day 3, so 4,000 remain for 27 days, and the team pauses risky rollouts until the window recovers.

Watch out

Treating unspent budget as success: a budget that is never touched usually means the SLO is too loose or the team ships too cautiously.

Error budget policy

Also called budget policy, release freeze

What: A short written agreement, signed by engineering and product, that says what happens when the budget runs low or runs out, such as freezing non-urgent releases and prioritising reliability fixes.
Why: Without a policy the budget is just a chart. With one, the decision to stop shipping is made calmly in advance, not argued during an outage.

Watch out

Writing a policy nobody outside the SRE team agreed to, so it is ignored the first time a launch date is at stake.

Composite availability

Also called serial dependencies, dependency maths, availability multiplication

What: When a request needs several components in series, their availabilities multiply. Redundant copies in parallel do the opposite: the chance that all fail at once is the product of their failure rates.
Why: It shows quickly whether an SLO is even possible with your dependencies, and where redundancy buys the most.
Example: App 99.95, database 99.95 and a payment API 99.9 in series give about 99.8 per cent. Two independent 99 per cent replicas in parallel give 99.99 per cent.

Watch out

Assuming replicas fail independently when they share a zone, a config or a deploy pipeline.

Go deeper