Chapter 2 · Practice 4 of 4
Instrument each kind of service for what it does
Why it matters
The Prometheus instrumentation guide splits services into three kinds, each with its own key metrics:
| Kind | Key metrics |
|---|---|
| Online serving (APIs, web) | Requests, errors, latency, in-flight requests, measured on both client and server side |
| Offline processing (queues, pipelines) | Items in and out per stage, items in progress, time of last processing |
| Batch jobs | Last successful run time, duration of each stage, total run time |
For offline pipelines, the guide suggests a heartbeat: a dummy item sent through the pipeline regularly, so you can measure end-to-end delay even when real traffic is low.
Also export known series with a zero value at startup. A metric that only appears after the first error makes "no errors" look like "no data", and alerts that compare rates break.
Avoid
- Metrics that only appear after the first event