Lakshya

Book

Data Platform & Analytics

What the job actually is now, modelling and grain, data contracts and who gets paged, quality that is not a dashboard, streaming, governance, cost, serving the AI workload, Databricks versus Snowflake, and statistics without the degree.

For the Data Platform archetypes. Built on 225 postings where compliance outranks SQL and real-time outnumbers batch seven to one.

CHAPTER 1 What the job actually is now Compliance outranks SQL. Half the postings name cost. Real-time outnumbers batch seven to one. CHAPTER 2 Modelling, and why it still decides everything The least fashionable skill in the field and the one that determines whether the platform is usable in three years. CHAPTER 3 Data contracts, and who gets paged The structural fix for the defining problem of the field: your pipeline breaks because of a change you did not make. CHAPTER 4 Data quality that is not a dashboard Most quality programmes produce alerts nobody actions. This is about the small number of tests that earn their place. CHAPTER 5 Streaming, and when you actually need it Named in 47% of postings against 6% for batch. Here is what that does and does not mean. CHAPTER 6 Governance, privacy and the 82% Named in more postings than SQL. Treat it as the job rather than as paperwork. CHAPTER 7 Cost, because half the postings name it The warehouse bill is usually the second-largest line in a data organisation, and it is yours. CHAPTER 8 Serving the AI workload AI appears in 92% of these postings. What it actually asks of a data platform is specific. CHAPTER 9 Databricks, Snowflake, and running both Named in 66% of enterprise postings in a reader-supplied sample, and 30% name both. The question is not which one. CHAPTER 10 Statistics and experimentation, without the degree Named in 44% of data postings in the reader sample. The part of the field that separates a data scientist from an analyst. CHAPTER 11 Streaming internals, and what the benchmarks are worth The coordinator moved server-side, the ZooKeeper era ended, storage went to object stores — and the one independently-retested vendor benchmark did not reproduce. CHAPTER 12 Stream processing: watermarks, state and the engines The concepts that transfer, the two major-version breaks that stranded people, and why one engine's micro-batch reputation is now out of date. CHAPTER 13 Orchestration, and the consolidation nobody predicted What actually changed in the dominant scheduler, why three-quarters of installs have not upgraded, and the acquisition that ended the alternatives debate. CHAPTER 14 Ingestion, transformation, and who owns your stack now Fivetran bought both sides of the dbt-versus-SQLMesh debate, one licence went more open while another went less, and the change-data-capture failure that fills a disk. CHAPTER 15 Data quality, observability and contracts What monitoring genuinely detects, the class of wrongness it structurally cannot, and why most contract tooling does not enforce contracts. CHAPTER 16 Table formats, catalogs and the credential fight The format war ended, a different war replaced it, and the thing no catalog will tell you is whether your table is actually healthy.