Recognises a scorecard average hiding a failing critical element
A headline green while a critical line misses its threshold.
Can AI do this?
AI does this well Most of the tested AI models master this. 3 of 4 models in the current benchmark master it.
| Model | Tier | Probability of mastery | Status |
|---|---|---|---|
| gemini-4-argon | Flagship | 94% | Mastered |
| gemini-3.5-flash-lite | Low cost | 73% | Not shown |
| gpt-6-luna | Low cost | 96% | Mastered |
| claude-haiku-5-5 | Low cost | 95% | Mastered |
Mastered means a probability of at least 90% that the model has this knobit, estimated from its graded answers. “Not shown” means the answers do not settle it either way. How the models are tested.
What it looks like in practice
Expected result: A headline derived from the critical lines
How it is assessed: Flags the deficient case and passes the acceptable one. Deficient: Headline 99.6% green, mean of seven lines; SC-002 days_past_due accuracy 97.9% vs 99% Acceptable: Headline = critical lines below threshold: 1 (days past due)
Also known as: Average hiding a failing critical element
Related knobits
Builds on
- Q-752 Explains deriving a scorecard headline from its critical lines
- Q-1802 Explains the concept of the arithmetic mean
All 10 knobits of Set data quality service levels and the scorecard.
Source: Review checklist 07-sla-scorecard, row 1 (BCBS 239 P7; DMBOK)