Recognises a vendor claim used as evidence
Evidence that is the vendor's own marketing.
Can AI do this?
People still lead Few or none of the tested AI models showed they master this. 1 of 4 models in the current benchmark master it.
| Model | Tier | Probability of mastery | Status |
|---|---|---|---|
| gemini-4-argon | Flagship | 86% | Not shown |
| gemini-3.5-flash-lite | Low cost | 7% | Not shown |
| gpt-6-luna | Low cost | 8% | Not mastered |
| claude-haiku-5-5 | Low cost | 96% | Mastered |
Mastered means a probability of at least 90% that the model has this knobit, estimated from its graded answers. “Not shown” means the answers do not settle it either way. How the models are tested.
What it looks like in practice
Expected result: The defect named on the score resting on vendor material, and its fix or the question for its owner
How it is assessed: Flags the deficient case and passes the acceptable one. Deficient: CA-003 EC-003 recovery: "Vendor brochure" Acceptable: Restore test in the PoC, or a reference customer
Also known as: Vendor claim as evidence
Related knobits
Builds on
All 14 knobits of Evaluate data storage technology.
Source: Review checklist 01-technology-evaluation, row 10 (DMBOK)