Recognises an aggregate hiding a total failure
A pass rate over the whole table that hides a part of it failing completely: one source system, one load, one region, one new product. 98 % overall can be 0 % for everything loaded since Tuesday.
Can AI do this?
Not tested yet
No AI benchmark covers this knobit yet. New results are announced in the Can AI Do the Job? newsletter.
What it looks like in practice
Expected result: The segment failing completely and the breakdown to report
How it is assessed: Flags the deficient case and passes the acceptable one. Deficient: Completeness of customer_segment_code measured over twelve million rows; a new source sends none for a week, the rate falls from 99.1 % to 98.9 %, and nothing alerts Acceptable: Measured per load batch or per source (population or a grouping in the set query), or a second check scoped to the latest load
Also known as: Aggregate that hides a total failure (data quality rule)
Related knobits
Builds on
All 31 knobits of Derive data quality rules.
Source: Review checklist 07-dq-rules, row 25; DAMA UK 2013; McGilvray 2021