Explains data cleansing, enhancement, parsing and standardisation
Cleansing corrects data to conform to standards and rules; enhancement adds attributes (time stamps, audit data, reference data); parsing splits a value into its parts by patterns; standardisation transforms values into one format and representation.
Can AI do this?
AI does this well Most of the tested AI models master this. 4 of 4 models in the current benchmark master it.
| Model | Tier | Probability of mastery | Status |
|---|---|---|---|
| gemini-4-argon | Flagship | 90% | Mastered |
| gemini-3.5-flash-lite | Low cost | 90% | Mastered |
| gpt-6-luna | Low cost | 90% | Mastered |
| claude-haiku-5-5 | Low cost | 90% | Mastered |
Mastered means a probability of at least 90% that the model has this knobit, estimated from its graded answers. “Not shown” means the answers do not settle it either way. How the models are tested.
What it looks like in practice
Expected result: Cleansing corrects data to conform to standards and rules; enhancement adds attributes (time stamps, audit data, reference data); parsing splits a value into its parts by patterns; standardisation transforms values into one format and representation.
How it is assessed: Answers a CDMP-style multiple-choice question on the data cleansing and enhancement
All knobits of Data quality.
Source: DAMA-DMBOK: Data Management Body of Knowledge, 2nd edition (revised), chapter 13: Data Quality, element QV-12