Data Platform & Analytics · Chapter 8 of 16
Serving the AI workload
AI appears in 92% of these postings. What it actually asks of a data platform is specific.
Almost every data-platform posting now mentions AI, and most of the time it means one of three concrete demands rather than anything exotic.
What AI workloads actually need from you
| Demand | What it means | The trap |
|---|---|---|
| Feature consistency | The same feature computed identically for training and serving | Two implementations that drift — the single most common cause of a model that works offline and not online |
| Point-in-time correctness | Training data must reflect what was known then | Leakage: joining a dimension as it is now, so the model learns the future |
| Documents for retrieval | Chunked, embedded, indexed, permission-filtered | The index becomes a copy of everything with none of the source access control |
| Throughput for training | Feeding accelerators fast enough that they are not idle | Object-store latency turning an expensive GPU fleet into a waiting room |
The two you will be asked about
Point-in-time correctness is the one that separates people who have shipped a model from people who have not. If you join today's customer dimension onto a two-year-old event to build training data, the model sees attributes that did not exist yet, scores brilliantly offline and fails in production. The fix is temporal joins against a dimension that retains history — which is why chapter 2's insistence on slowly-changing dimensions is not academic.
Permission-filtered retrieval is the one that becomes a security incident. Documents that were access-controlled in their source system get embedded into one index that everybody can query, and the permission model is silently gone. The control is to filter retrieval by the asking user's entitlements at query time, and to test it with two accounts of different privilege asking the same question.
What this means for the role
The data platform is now upstream of the AI platform, which raises the stakes on everything in this book: a wrong number in a dashboard is embarrassing, and the same wrong number in a feature store is a model making decisions on it at scale.
For the model side of this, read RAG — what it fixes and what it does not and Evaluation.
What changes when a data platform serves a model
Serving machine learning turns a data platform from something that answers questions into something on a request path, and three properties become requirements that were previously preferences.
| Property | Why it becomes a hard requirement | What it forces |
|---|---|---|
| Latency | A feature lookup sits inside a user request | A low-latency store, separate from the warehouse. The warehouse is not a serving system and using it as one is the most common early mistake |
| Point-in-time correctness | Training must see only what was known then | The subtlest and most damaging failure. Joining a current feature value onto a historical label leaks the future into training |
| The same computation in both paths | Training and serving must agree | One definition, executed identically — otherwise the model sees different inputs in production than it was trained on, and nothing errors |
Training-serving skew is the defect that produces a model that tested well and performs badly, with no error anywhere.
It happens when a feature is computed one way in the training pipeline — batch SQL over history — and another way at serving time, in application code, under a deadline. The two are almost the same. Almost is enough.
Common sources: a different null handling; a time window that is inclusive in one and exclusive in the other; a currency or unit conversion applied in one path only; a category encoded from a lookup that has since gained a value.
Nothing fails. The model returns predictions, the service returns 200, and the quality is quietly worse than the evaluation promised — which is why the team spends weeks tuning the model rather than checking the inputs.
The control is a single definition executed in both paths, which is the actual argument for a feature store — not caching, and not discovery, but definitional consistency. If you are not going to have one, log the served feature values and periodically compare them against a recomputation from the warehouse. That comparison finds skew and almost nobody runs it.
And on point-in-time correctness, the version worth remembering: if you build a training set by joining today's customer attributes onto last year's outcomes, the model learns from information that did not exist when the outcome occurred. It will look excellent in evaluation and fail in production, because in production the future genuinely is not available. Every feature join in a training pipeline needs an as-of timestamp, and a pipeline without one is producing an optimistic evaluation rather than a model.
Check yourself