Recognises a merge count used as a quality metric

R-503 · Skill knobit · Activity: Run the stewardship workflow and measure MDM · Competence: Reference and master data

Activity counted, false merges not.

Can AI do this?

Not settled yet The benchmark tested this, but the models' answers do not settle yet whether they master it. 0 of 4 models in the current benchmark master it.

Model	Tier	Probability of mastery	Status
gemini-4-argon	Flagship	84%	Not shown
gemini-3.5-flash-lite	Low cost	80%	Not shown
gpt-6-luna	Low cost	83%	Not shown
claude-haiku-5-5	Low cost	83%	Not shown

Mastered means a probability of at least 90% that the model has this knobit, estimated from its graded answers. “Not shown” means the answers do not settle it either way. How the models are tested.

What it looks like in practice

Expected result: The metric and a false-merge measure

How it is assessed: Flags the deficient case and passes the acceptable one. Deficient: MM-002 "merges this quarter: 12 400" Acceptable: False-merge rate on a labelled sample

Also known as: Merge count as a quality metric

Related knobits

Builds on

R-354 Explains measuring false merges and false splits

Usually followed by

R-532 Adds a false-merge measure taken before merging

All 13 knobits of Run the stewardship workflow and measure MDM.

Source: Review checklist 05-stewardship-metrics, row 3 (8000-8)