Recognises one quality ratio over differing populations
One ratio where the populations in the config would give different answers.
Can AI do this?
AI does this partly Some of the tested AI models master this; others do not show it. 2 of 4 models in the current benchmark master it.
| Model | Tier | Probability of mastery | Status |
|---|---|---|---|
| gemini-4-argon | Flagship | 91% | Mastered |
| gemini-3.5-flash-lite | Low cost | 14% | Not shown |
| gpt-6-luna | Low cost | 9% | Not shown |
| claude-haiku-5-5 | Low cost | 90% | Mastered |
Mastered means a probability of at least 90% that the model has this knobit, estimated from its graded answers. “Not shown” means the answers do not settle it either way. How the models are tested.
What it looks like in practice
Expected result: The result split by population
How it is assessed: Flags the deficient case and passes the acceptable one. Deficient: QM-004 semantic accuracy of days_past_due 97.9%, not split Acceptable: Original book 99.6%, migrated 86.9%
Also known as: Overall result over populations that differ
Related knobits
Builds on
- Q-357 Explains segmented data quality reporting
- Q-453 Explains ISO/IEC 25024 data quality measures
- Q-1901 Explains a pooled rate as the weighted mean of its population rates
All 10 knobits of Profile and assess data quality.
Source: Review checklist 04-profiling-assessment, row 6 (DMBOK; BCBS 239 P3)