Set data quality service levels and the scorecard
The outcome: sets data quality service levels and a scorecard that cannot hide a failing critical element.
Builds on: Select the critical data elements, Profile and assess data quality.
Which AI model is best at set data quality service levels and the scorecard?
| Model | Provider | Tier | Test score | Knobits mastered (expected) | Cost per test run |
|---|---|---|---|---|---|
| gemini-4-argon | Flagship | – | 81% | $0.021 | |
| gemini-3.5-flash-lite | Low cost | – | 53% | $0.003 | |
| gpt-6-luna | OpenAI | Low cost | – | 75% | $0.002 |
| claude-haiku-5-5 | Anthropic | Low cost | – | 76% | $0.006 |
Test score: mean rubric score on the test tasks for this activity. Knobits mastered: expected share of the activity's knobits the model masters. Cost: mean API cost of one test run at list prices, half price for answers sent as a batch. Method and caveats.
The 10 knobits of this activity
2 that AI does well, 3 it does partly, and 5 where people still lead.
| ID | Knobit | Type | AI exposure (models mastering) |
|---|---|---|---|
| Q-701 | Recognises a scorecard average hiding a failing critical element | Skill | AI does this well 3/4 |
| Q-702 | Recognises a data quality service level threshold with no rationale | Skill | People still lead 0/4 |
| Q-703 | Recognises a quality trend read without its population change | Skill | People still lead 0/4 |
| Q-704 | Recognises a data quality service level with no escalation | Skill | People still lead 0/4 |
| Q-705 | Recognises a service level measure unable to detect the element's failure | Skill | AI does this partly 2/4 |
| Q-706 | Recognises a critical data element with no service level | Skill | AI does this partly 2/4 |
| Q-751 | Explains data quality service levels | Knowledge | AI does this partly 2/4 |
| Q-752 | Explains deriving a scorecard headline from its critical lines | Knowledge | AI does this well 3/4 |
| Q-753 | Explains setting a quality threshold from the use | Knowledge | People still lead 0/4 |
| Q-754 | Explains like-for-like trends of quality measures | Knowledge | People still lead 0/4 |