Observability Best Practices

Chapter 2 of 6 · 4 practices · 3 min · 4-question quiz

Metrics that stay useful and affordable

How to name, type and label metrics so queries are obvious and the time series count does not explode.

Start: Name metrics with base units and standard suffixes

4 practices in this chapter

  1. 1Name metrics with base units and standard suffixesA metric called request_time leaves everyone guessing: milliseconds or seconds, a total or a histogram? The Prometheus naming guide removes the guesswork: Use base units: seconds, bytes, meters…
  2. 2Use counters for things that only go up, gauges for everything elseThe Prometheus instrumentation guide's rule is simple: if the value can go down, it is a gauge. Counters only increase, and reset to zero when the process restarts.
  3. 3Keep label cardinality low, and never label by user or requestEvery unique combination of label values is a separate time series, with its own storage and memory. One unbounded label can multiply your bill or take down the metrics backend.
  4. 4Instrument each kind of service for what it doesThe Prometheus instrumentation guide splits services into three kinds, each with its own key metrics: For offline pipelines, the guide suggests a heartbeat: a dummy item sent through the pipeline…

Check yourself on this chapter

4 questions, right here once you are signed in. Your progress is saved as you read.

Sign up for the quiz