Data Platform & Analytics · Chapter 2 of 8
Modelling, and why it still decides everything
The least fashionable skill in the field and the one that determines whether the platform is usable in three years.
Storage is cheap and compute is elastic, which has produced a widespread belief that modelling no longer matters — land everything raw and figure it out later. What actually happens is a warehouse nobody can navigate, six tables that all claim to be the customer, and an analyst population that has quietly given up and gone back to spreadsheets.
The layers worth keeping separate
| Layer | What it holds | The rule |
|---|---|---|
| Raw | Source data, unmodified, append-only | Never transform here. It is your ability to reprocess |
| Staging | Lightly cleaned, typed, deduplicated | One staging model per source table, no business logic |
| Core | Conformed business entities — customer, order, account | One definition per concept. This is the whole point |
| Marts | Shaped for a consumer — finance, growth, a dashboard | Denormalise freely here; it is cheap and it is meant to be disposable |
The two questions that expose a bad model
- “How many places define active customer?” If the answer is more than one, every number in the business is negotiable and every meeting starts with a reconciliation.
- “Can I recompute last year's report and get last year's number?” If dimensions are overwritten in place, the answer is no, and every historical comparison is quietly wrong. This is what slowly-changing dimensions exist for, and it is routinely skipped because it is tedious.
Grain is the thing to get right first
State the grain of every table in one sentence — one row per order line per day. Almost every subtle data bug is a grain mistake: a join that fans out, a sum that double-counts, a metric that changes when a dimension gains a row.
If a table's grain cannot be stated in a sentence, it is not modelled, it is accumulated.