Recognises an aggregate hiding a total failure
A pass rate over the whole table that hides a part of it failing completely: one source system, one load, one region, one new product. 98 % overall can be 0 % for everything loaded since Tuesday.
Can AI do this?
AI does this partly Some of the tested AI models master this; others do not show it. 2 of 4 models in the current benchmark master it.
| Model | Tier | Probability of mastery | Status |
|---|---|---|---|
| gemini-4-argon | Flagship | 91% | Mastered |
| gemini-3.5-flash-lite | Low cost | 2% | Not shown |
| gpt-6-luna | Low cost | 82% | Not shown |
| claude-haiku-5-5 | Low cost | 91% | Mastered |
Mastered means a probability of at least 90% that the model has this knobit, estimated from its graded answers. “Not shown” means the answers do not settle it either way. How the models are tested.
What it looks like in practice
Expected result: The segment failing completely and the breakdown to report
How it is assessed: Flags the deficient case and passes the acceptable one. Deficient: Completeness of customer_segment_code measured over twelve million rows; a new source sends none for a week, the rate falls from 99.1 % to 98.9 %, and nothing alerts Acceptable: Measured per load batch or per source (population or a grouping in the set query), or a second check scoped to the latest load
Also known as: Aggregate that hides a total failure (data quality rule)
Related knobits
Builds on
All 30 knobits of Derive data quality rules.
Source: Review checklist 03-dq-rules (data-quality), row 25; DAMA UK 2013; McGilvray 2021; formerly data-modelling K-725