Observability Best Practices

Chapter 1 of 6 · 4 practices · 3 min · 4-question quiz

Deciding what to measure

Start from what users experience. These practices turn that into a small set of indicators and objectives that every other signal supports.

Start: Define service level indicators as good events over valid events

4 practices in this chapter

  1. 1Define service level indicators as good events over valid eventsA service level indicator (SLI) is a measurement of something users care about. The Google SRE workbook recommends expressing every SLI as a ratio: good events divided by valid events, so it always…
  2. 2Set SLOs below 100% and agree what happens when the budget runs outA service level objective (SLO) is a target for an SLI over a window, such as 99.9% of requests successful over 28 days. The gap to 100% is the error budget: the amount of failure you accept.
  3. 3Cover the four golden signals for every serviceThe SRE book's chapter on monitoring distributed systems says that if you can only measure four things about a user-facing system, measure these: These four give every dashboard the same first row…
  4. 4Measure latency with histograms and percentiles, not averagesAverages hide the slow requests users complain about. The SRE book gives the example of a service with a 100 ms mean latency where 1% of requests take several seconds.

Check yourself on this chapter

4 questions, right here once you are signed in. Your progress is saved as you read.

Sign up for the quiz