Chapter 2 of 8 · 4 checks · 2 min

Freshness

Stale data is the most common silent failure: every job succeeds while the numbers stop moving. These checks catch it.

Show

Every source has freshness checks with warn and error thresholdsMust
Why it matters and how to check
A source that stopped loading still returns rows, just old ones. dbt's freshness config compares the newest loaded_at_field value with warn_after and error_after thresholds, and table-level settings override the source default. If neither threshold is set, dbt does not check freshness at all. dbt recommends the dbt freshness command, which checks every source and model with freshness configured.
Set thresholds from how often each source should update, not from one default for everything. To verify, run dbt freshness and confirm every source used by a critical model appears in the output with a status. Then set a test source's threshold below its real lag and confirm the check errors.
Freshness is checked on output tables, not only on sourcesShould
Why it matters and how to check
A fresh source does not mean a fresh dashboard: a failed or skipped downstream model can leave the output days behind. Great Expectations' guidance says to track total lag through the pipeline and add checks after key processing steps. dbt supports freshness on models too; views need an explicit loaded_at_field.
To verify, list the tables your critical dashboards read and confirm each has its own freshness check. Pause one downstream model in staging and confirm its freshness check fails while the source check still passes.
Freshness uses the newest event or load timestamp, never an averageShould
Why it matters and how to check
Great Expectations warns that averaging timestamps can hide issues, and that checking only a sample of rows can mask staleness. Measure freshness with the maximum of a reliable timestamp column, for example with ExpectColumnMaxToBeBetween, or a Soda freshness check on a column. Prefer a load timestamp you control, such as _loaded_at, over an event time that late-arriving data can push backwards.
To verify, read the freshness query or config for each critical table and confirm it uses max of a load or event timestamp over the whole table or latest partition, not an average and not a sample.
Late data alerts fire at the deadline even when no job failedMust
Why it matters and how to check
A pipeline that hangs, or never starts because an upstream sensor is waiting, produces no error to alert on. If finance reads a table at 8 a.m., the alert should fire at the deadline when the data is not ready, whether or not anything failed.
To verify, for each critical table find the consumer deadline in its documentation and the alert that checks it. Delay a staging run past its deadline on purpose and confirm the owner receives the alert.