Recognises a regulation cited for data it does not cover
A control or requirement claims a regulation outside its data.
Can AI do this?
Not settled yet The benchmark tested this, but the models' answers do not settle yet whether they master it. 0 of 4 models in the current benchmark master it.
| Model | Tier | Probability of mastery | Status |
|---|---|---|---|
| gemini-4-argon | Flagship | 82% | Not shown |
| gemini-3.5-flash-lite | Low cost | 40% | Not shown |
| gpt-6-luna | Low cost | 82% | Not shown |
| claude-haiku-5-5 | Low cost | 84% | Not shown |
Mastered means a probability of at least 90% that the model has this knobit, estimated from its graded answers. “Not shown” means the answers do not settle it either way. How the models are tested.
What it looks like in practice
Expected result: The citation and the regulation that does cover the data
How it is assessed: Flags the deficient case and passes the acceptable one. Deficient: CT-005 IBAN encryption claimed as PCI DSS Req. 3.5.1 evidence Acceptable: IBAN under GDPR Art. 32 and banking secrecy; PCI only for card data
Also known as: Regulation cited for data it does not cover
Related knobits
Builds on
All 17 knobits of Register regulatory security requirements.
Source: Review checklist 01-regulatory-requirements, row 2 (PCI; DMBOK)