Databricks & Snowflake · Chapter 10 of 16
Choosing, migrating, and the interview
How to answer the question you will be asked, and what to actually learn.
You will be asked which you would choose. The strongest answer establishes that the comparison is largely stale, then answers on the dimension that still separates them.
What still genuinely differs
| If your primary user is… | Lean | Because |
|---|---|---|
| Engineers writing code and models | Databricks | Native Python/Scala, ML lifecycle, heavy unstructured transformation |
| A large analyst population on SQL | Snowflake | Concurrency, near-zero operations, predictable behaviour |
| Streaming as a first-class need | Databricks, usually | Structured Streaming and DLT are more mature for it |
| Sharing data with other organisations | Snowflake, usually | Secure sharing without copying is genuinely differentiated |
| A small team with no platform engineer | Snowflake | Far less to operate; Databricks rewards tuning you may not have time for |
Migration, honestly
Most consolidation programmes stall at roughly 70%, because the value is front-loaded and the long tail is unfunded — and then you carry both plus a half-finished migration, which is worse than either. If you are asked about migrating, the credible answer funds the finish before starting, publishes a decommission date, and does the long-tail work centrally rather than delegating it to teams for whom it is priority eleven.
What to actually learn, in order
- Pick one and go deep. Claiming both shallowly is transparent within two questions. Say which one you are deep in.
- Learn the cost model of that one — it is where enterprise conversations go and where most candidates are weakest.
- Learn one table format properly, Delta or Iceberg. Chapter 3 is the mechanism; then actually create a table, evolve its schema, delete a row and time-travel it.
- Read one query profile end to end on your chosen platform and be able to say where the time went.
- Then learn enough of the other to discuss the trade-off honestly. Two hours of reading is sufficient for that and it is the difference between sounding partisan and sounding experienced.
A free way to actually practise
Both offer free tiers — Databricks Community Edition and a Snowflake trial. Load a public dataset of a few hundred million rows, write a badly-performing query, look at the profile, fix it, and record the before and after.
That single exercise gives you a concrete performance story, which is what every technical round in this area is really asking for.
Choosing between them, on the axes that actually differ
Both platforms now do most of what the other does, so feature comparison has stopped discriminating. What still differs is the shape of the work and the shape of the bill, and those predict which one a given team will be happier on.
| Axis | One leans | The other leans |
|---|---|---|
| Default unit of work | Code — notebooks, jobs, a compute framework you configure | SQL — a warehouse that hides the cluster from you |
| Who is productive on day one | Engineers comfortable with distributed compute | Analysts who know SQL and want no infrastructure |
| Where tuning effort goes | Cluster shape, partitioning, file layout — more levers, more upside, more rope | Warehouse size and query shape — fewer levers, less to get wrong |
| Machine learning | Genuinely stronger — the training path is native rather than adjacent | Improving, and usually still the reason a team keeps a second platform |
| Cost failure mode | A cluster left running, or one sized for a peak that happens twice a week | Many short queries meeting a billing minimum, or a warehouse that never suspends |
The honest recommendation is about team composition rather than technology. A team of data engineers who write code will be more productive where code is the unit; a team of analysts and analytics engineers will be more productive where SQL is. Choosing against your team's centre of gravity means paying a training cost forever to win an argument about capability that no longer has a clear answer.
And the decision that outlives the platform choice: keep the table format and the catalog open.
The table format question has largely settled on one open format, and both platforms support it. That means your data can be readable by an engine you have not chosen yet, which is the only portability that survives a vendor relationship going badly.
Portability now lives at the format and specification layer, not the vendor layer. A proprietary storage format does not survive a change of mind; an open one does.
So the sequence for a platform decision is: pick the open format first, then the catalog, then the compute engine — because the last of those is the easiest to change and the first is the hardest.
Most evaluations run that order backwards, choosing the engine on a feature bake-off and inheriting whatever storage decision comes with it.
Check yourself
What still differentiates the two platforms?
The shape of the work — code versus SQL as the default unit — and the shape of the bill. Feature comparison has stopped discriminating.What should the choice actually be based on?
Your team's centre of gravity. Choosing against it means paying a training cost forever to win a capability argument that no longer has a clear answer.What decision outlives the platform choice?
Keeping the table format and catalog open. Portability lives at the format layer, not the vendor layer.What order should a platform evaluation run in?
Format, then catalog, then compute engine — hardest to change first. Most evaluations run it backwards.Answering the question you will actually be asked
The interview question is almost never “which is better”. It is some version of “we run one and are considering the other — what would you look at?”, and the answer that lands is a decision procedure rather than a preference.
- Start with what the team is. Engineers who write code are more productive where code is the unit of work; analysts and analytics engineers are more productive where SQL is. Choosing against your team's centre of gravity means paying a retraining cost forever to win a capability argument that no longer has a clear winner.
- Then the workload shape. Heavy machine learning training pulls one way. Concurrency-heavy interactive SQL pulls the other. Most estates have both, which is why so many organisations run both and should be honest about it rather than treating it as failure.
- Then the cost failure mode you are more exposed to — idle clusters, or many short queries meeting a billing minimum. These are different problems with different fixes, and one will describe your team.
- And name what you would not migrate. A migration that moves everything is a year. A migration that moves the twenty pipelines that matter and leaves the rest is a quarter, and the rest usually turns out to be unused.
What actually makes a migration expensive
Not the data. The data is the easy part — especially if it is already in an open table format, which is the whole argument for one.
What costs is everything around it: the SQL dialect differences that surface one query at a time; the orchestration and scheduling; the permissions model, which never maps cleanly; every BI report pointing at the old endpoint; and the undocumented business logic living inside pipelines whose author has left.
So the honest estimate is dominated by discovery, and the first task is an inventory: what exists, what is used, what feeds what. Teams that skip this migrate their dead weight and discover it a year later.
The sequencing that works: run both, move new work to the target, migrate the high-value pipelines deliberately, and set a date to switch off the source — then hold it. A dual-running period with no end date becomes permanent, and you pay for two platforms indefinitely while getting the benefit of neither.
And the decision that outlives the choice: keep the table format and catalog open. Portability lives at the format layer, not the vendor layer — which means picking the format first and the engine last, and most evaluations run that order backwards.
Check yourself