Plan the modelling and set its standards
The outcome: plans a modelling assignment with a bounded scope, chosen standards and notations, located inputs and an independent review.
Which AI model is best at plan the modelling and set its standards?
| Model | Provider | Tier | Test score | Knobits mastered (expected) | Cost per test run |
|---|---|---|---|---|---|
| gemini-4-argon | Flagship | – | 92% | $0.021 | |
| gemini-3.5-flash-lite | Low cost | – | 82% | $0.003 | |
| gpt-6-luna | OpenAI | Low cost | – | 92% | $0.002 |
| claude-haiku-5-5 | Anthropic | Low cost | – | 90% | $0.006 |
Test score: mean rubric score on the test tasks for this activity. Knobits mastered: expected share of the activity's knobits the model masters. Cost: mean API cost of one test run at list prices, half price for answers sent as a batch. Method and caveats.
The 10 knobits of this activity
2 that AI does well, 1 it does partly, and 7 where people still lead.
| ID | Knobit | Type | AI exposure (models mastering) |
|---|---|---|---|
| K-001 | Recognises a modelling scope with no boundary | Skill | People still lead 1/4 |
| K-002 | Recognises a modelling standard named but not chosen | Skill | People still lead 1/4 |
| K-003 | Recognises a model layer planned with no notation | Skill | People still lead 0/4 |
| K-004 | Recognises model inputs the plan does not locate | Skill | AI does this partly 2/4 |
| K-005 | Recognises a model review planned by its modeller | Skill | People still lead 0/4 |
| K-006 | Recognises a model with no repository or baseline | Skill | AI does this well 3/4 |
| K-031 | Plans the deliverables of a modelling assignment | Skill | People still lead 1/4 |
| K-032 | Chooses a modelling scheme and notation for an audience and purpose | Skill | People still lead 0/4 |
| K-033 | Sets the modelling standards a project follows | Skill | People still lead 0/4 |
| K-034 | Selects the tools and repository for a modelling project | Skill | AI does this well 3/4 |