Data Platform & Analytics · Chapter 7 of 8
Cost, because half the postings name it
The warehouse bill is usually the second-largest line in a data organisation, and it is yours.
Cost appears in 50% of these postings. Consumption pricing means a single badly-written query can cost more than an engineer's week, and nobody notices until the invoice.
Where the money actually goes
- Full scans on unpartitioned tables. The most common and most fixable. Partition and cluster on what people actually filter by, which you can read from query history rather than guess.
- Recomputing what has not changed. Full refreshes where an incremental model would do. This is usually the single largest saving available.
- Dashboards on a schedule nobody needs. Hourly refreshes on a report read weekly. Check last-viewed timestamps — a meaningful share of scheduled refreshes serve dashboards nobody has opened in months.
- Idle warehouses with generous auto-suspend, and oversized ones chosen once and never revisited.
- Storage nobody owns — old snapshots, abandoned dev schemas, unpartitioned raw retained forever.
Attribution before optimisation
The blocker is almost never knowing what to optimise; it is that nobody can attribute spend to a team, so nobody owns it and every conversation is about the total.
Tag queries and warehouses by team, publish cost per team monthly, and let them see their own number. Visibility with ownership moves more spend than any central optimisation programme, because they know which pipelines are safe to turn off and you do not.