Skilirium

Capture data requirements

Activity 1 of 8 in Data modelling · Occupation: Data modeller

The outcome: captures atomic, sourced and testable data requirements.

Which AI model is best at capture data requirements?

Best result for capture data requirements: gemini-4-argon (87% test score).

ModelProviderTierTest scoreKnobits mastered (expected)Cost per test run
gemini-4-argonGoogleFlagship87% 88%$0.062
gemini-3.1-pro-previewGoogleFlagship54% 52%$0.20
gpt-6-astraOpenAIFlagship75% 78%$0.45
claude-fable-5-1AnthropicFlagship75% 78%$0.85
gemini-3.8-flashGoogleMid tier70% 67%$0.045
gpt-6.1-solOpenAIMid tier72% 73%$0.077
claude-sonnet-5-5AnthropicMid tier79% 75%$0.23
gemini-3.5-flash-liteGoogleLow cost37% 53%$0.008
gpt-6-lunaOpenAILow cost61% 59%$0.005
claude-haiku-5-5AnthropicLow cost74% 70%$0.018

Test score: mean rubric score on the test tasks for this activity. Knobits mastered: expected share of the activity's knobits the model masters. Cost: mean API cost of one test run at list prices. Method and caveats.

The 47 knobits of this activity

13 that AI does well, 10 it does partly, and 18 where people still lead.

IDKnobitTypeAI exposure (models mastering)
K-101Recognises a data requirement making several claimsSkillAI does this partly 3/10
K-102Recognises a data requirement prescribing a design solutionSkillNot tested yet
K-103Recognises a statement outside the scope of data requirementsSkillPeople still lead 1/10
K-104Recognises an ambiguous business term in a data requirementSkillAI does this partly 5/10
K-105Recognises untestable wording in a data requirementSkillAI does this partly 3/10
K-106Recognises an unverifiable data requirementSkillAI does this partly 3/10
K-107Recognises a reference with several possible referentsSkillAI does this partly 3/10
K-108Recognises a data requirement without a named subjectSkillAI does this partly 3/10
K-109Recognises a measure stated without unit or basisSkillAI does this well 8/10
K-110Recognises a data requirement silent on current versus past valuesSkillPeople still lead 1/10
K-111Recognises an absolute statement with its exception unaskedSkillAI does this partly 6/10
K-112Recognises a mandatory attribute without its moment of obligationSkillAI does this partly 6/10
K-113Recognises a business rule left inside another statementSkillAI does this well 9/10
K-114Recognises content filed under the wrong requirementSkillAI does this partly 3/10
K-115Recognises an unsourced data requirementSkillNot tested yet
K-116Recognises a missing or meaningless requirement prioritySkillPeople still lead 0/10
K-117Recognises a missing or wrong requirement typeSkillPeople still lead 0/10
K-118Recognises a status contradicting the requirement's contentSkillAI does this well 8/10
K-119Recognises a guess recorded as factSkillAI does this well 8/10
K-120Recognises a missing rationale for a non-obvious requirementSkillNot tested yet
K-121Recognises duplicate data requirementsSkillPeople still lead 0/10
K-122Recognises paraphrased data requirementsSkillPeople still lead 0/10
K-123Recognises conflicting data requirementsSkillPeople still lead 0/10
K-124Recognises inconsistent terminology across the registerSkillAI does this well 9/10
K-125Recognises a requirements register with unbounded scopeSkillNot tested yet
K-126Recognises a requirements register describing no eventsSkillNot tested yet
K-127Recognises an uncovered identity requirementSkillPeople still lead 1/10
K-128Recognises an uncovered volume or retention requirementSkillPeople still lead 1/10
K-129Recognises an uncovered sensitivity or access requirementSkillPeople still lead 1/10
K-130Recognises an uncovered lineage requirementSkillPeople still lead 1/10
K-131Recognises undecided history requirements across the registerSkillPeople still lead 1/10
K-132Recognises a requirements register serving no stated questionSkillPeople still lead 1/10
K-133Recognises a missing or wrong requirement traceSkillAI does this partly 4/10
K-134Recognises unstable requirement identifiersSkillAI does this well 7/10
K-151Explains singular and unambiguous requirement statementsKnowledgeAI does this well 7/10
K-152Distinguishes a business requirement from a design solutionKnowledgeNot tested yet
K-153Explains the data requirement types of the registerKnowledgePeople still lead 0/10
K-154Explains requirement attributes and traceabilityKnowledgeAI does this well 8/10
K-155Explains current value, history and point-in-time reproducibilityKnowledgeAI does this well 9/10
K-156Explains the identity of a business thingKnowledgePeople still lead 1/10
K-157Explains the non-functional data concernsKnowledgePeople still lead 1/10
K-158Explains the scope, events and questions a data register must stateKnowledgePeople still lead 1/10
K-159Explains referential and logical ambiguity in natural-language statementsKnowledgeAI does this well 10/10
K-160Explains persistent identifiersKnowledgeAI does this well 10/10
K-161Explains verifiable requirement statementsKnowledgeAI does this well 9/10
K-162Explains consistency within a requirements registerKnowledgePeople still lead 0/10
K-135Retires a superseded register entry instead of deleting itSkillAI does this well 10/10