Recognises generated SQL edited by hand
A file in 08-ddl/ that regeneration does not reproduce byte for byte. The header says DO NOT HAND-EDIT; look for the edit anyway — an added constraint, a reordered statement, a tweaked type, a comment. From here the model and the database tell different stories and nothing mechanical can say which is right.
Can AI do this?
AI does this well Most of the tested AI models master this. 8 of 10 models master it.
| Model | Tier | Probability of mastery | Status |
|---|---|---|---|
| gemini-4-argon | Flagship | 100% | Mastered |
| gemini-3.1-pro-preview | Flagship | 100% | Mastered |
| gpt-6-astra | Flagship | 100% | Mastered |
| claude-fable-5-1 | Flagship | 99% | Mastered |
| gemini-3.8-flash | Mid tier | 100% | Mastered |
| gpt-6.1-sol | Mid tier | 100% | Mastered |
| claude-sonnet-5-5 | Mid tier | 100% | Mastered |
| gemini-3.5-flash-lite | Low cost | 31% | Not shown |
| gpt-6-luna | Low cost | 51% | Not shown |
| claude-haiku-5-5 | Low cost | 93% | Mastered |
Mastered means a probability of at least 90% that the model has this knobit, estimated from its graded answers. “Not shown” means the answers do not settle it either way. How the models are tested.
What it looks like in practice
Expected result: The hand edit and the model change replacing it
How it is assessed: Flags the deficient case and passes the acceptable one. Deficient: ck_fact_payment_settled_le_instructed added straight into 01_create_tables.sql after a deploy failure; the next regeneration drops it, nobody notices for two releases, and over-settled payments load again Acceptable: The check added to fact_payment.check_constraints in physical-model.yaml and the DDL regenerated; diff -r 08-ddl /tmp/ddl-check empty, and that diff run in CI
Also known as: Generated SQL edited by hand (generated DDL)
Related knobits
Builds on
Usually followed by
All 33 knobits of Generate and review DDL.
Source: Review checklist 08-ddl, row 1; BCBS 239; Humble & Farley 2010