Chapter 2 of 9 · 4 checks · 2 min

Metric definitions

Two dashboards that disagree on the same number destroy trust in both. These checks make every metric defined once, in writing, and visible to readers.

Show

Write a definition for every metric: formula, filters, grain and time zoneMust
Why it matters and how to check
"Revenue" can mean gross or net, before or after refunds, booked or recognised, in local currency or converted. "Active user" depends on the event, the window and whether internal accounts are excluded. Each choice changes the number.
To verify, pick each metric on the dashboard and find its written definition: the formula, the filters applied (such as excluding test accounts), the grain (per day, per user), the time zone used for date boundaries, and the source table. If any part is only in someone's head or in an unreviewed query, write it down before launch.
Define each metric once in a shared layer, not separately in every chartMust
Why it matters and how to check
When each chart computes its own version of a metric, they drift apart. dbt's Semantic Layer lets you define metrics such as simple, ratio, cumulative, derived and conversion metrics in YAML in the dbt project, with a name, description, type, label and filter, and query them from BI tools. Power BI's optimization guidance makes the same point for measures: they aggregate columns in the semantic model to provide consistency across reports.
To verify, search the dashboard's queries for the metric's formula. It should appear once, in the semantic layer or model, and every chart should reference that definition rather than recompute it, for example cancellation_rate defined as a ratio metric with a numerator and a denominator.
Show definitions, data period and methodology on the dashboard itselfShould
Why it matters and how to check
The Analysis Function's testing guidance lists metadata a dashboard should carry: the last update, the next update date, the data period, and links to definitions, methodology and quality information. Readers who cannot see how a number is defined will either distrust it or misread it.
To verify, hover or click on each headline metric, or open the dashboard's info page, and check a non-specialist can find what it measures, what it excludes and where it comes from in one step.
Give each headline number a comparison: a target, a previous period or a benchmarkShould
Why it matters and how to check
A number on its own says little. 4,200 sign-ups is good or bad only relative to last week, the same week last year, or a target. Without a comparison, readers make their own, often the wrong one.
To verify, check every headline figure shows its comparison and that the comparison is labelled, for example "vs previous 7 days" or "target 5,000". Make sure the comparison period is complete and of the same length.