Explains rubric-based test item design
Task, analytic criteria with points and the knobit items they grade, caps, a reference answer that earns full marks, grading notes and automatic checks linked to criteria (Standards 2014, ch. 4 and 6).
Can AI do this?
AI does this well Most of the tested AI models master this. 3 of 4 models in the current benchmark master it.
| Model | Tier | Probability of mastery | Status |
|---|---|---|---|
| gemini-4-argon | Flagship | 100% | Mastered |
| gemini-3.5-flash-lite | Low cost | 100% | Not shown |
| gpt-6-luna | Low cost | 100% | Mastered |
| claude-haiku-5-5 | Low cost | 100% | Mastered |
Mastered means a probability of at least 90% that the model has this knobit, estimated from its graded answers. “Not shown” means the answers do not settle it either way. How the models are tested.
What it looks like in practice
Expected result: Task, analytic criteria with points and the knobit items they grade, caps, a reference answer that earns full marks, grading notes and automatic checks linked to criteria (Standards 2014, ch. 4 and 6).
How it is assessed: Answers a written question on the rubric-based test item correctly
Related knobits
Needed for
- T-201 Recognises a criterion grading two skills
- T-202 Recognises a criterion no grader can decide
- T-203 Recognises a reference answer failing its own rubric
- T-205 Recognises a criterion mapped to the wrong item
- T-206 Recognises a cap on a peripheral criterion
- T-209 Recognises an automatic check contradicting its criterion
- T-231 Writes a rubric criterion observing one knobit set
- T-233 Writes grading notes with acceptable alternatives and named failure modes
All 15 knobits of Write and review test items.
Source: Practice the stage 2 reference and review checklist rest on