Databricks & Snowflake · Chapter 9 of 16
Governance and security on both
Named in 68% of postings. What each platform gives you and where the gaps are.
Governance is where these platforms have converged most, and where running both is hardest — because each has its own catalogue and they do not reconcile themselves.
| Capability | Databricks | Snowflake |
|---|---|---|
| Catalogue | Unity Catalog — catalog/schema/table, cross-workspace | Database/schema/table with a native information schema |
| Row and column control | Row filters and column masks in Unity Catalog | Row access policies and dynamic data masking |
| Lineage | Automatic, table and column level | Access history and object dependencies |
| Classification / tagging | Tags in Unity Catalog | Object tagging, with tag-based masking policies |
| Sharing | Delta Sharing — open protocol, works outside Databricks | Secure data sharing — no copy, within Snowflake |
| Audit | Audit logs of access and changes | Access history views |
The controls worth insisting on, whichever you run
- Classification at ingestion, applied at column level. Table-level tagging means everything inherits the strictest label, which produces either over-restriction and shadow copies, or a label nobody honours.
- Access derived from classification rather than granted per person. Otherwise nobody can answer who can see salary data.
- The two-user test. Two accounts with different entitlements querying the same object. If they see the same thing, the model did not survive whatever pipeline produced it.
- An erasure path you have rehearsed. Both platforms retain history — Time Travel and Fail-safe on Snowflake, old versions on Delta — so ‘we deleted the row’ is not the whole answer until retention has expired or been explicitly purged.
- Residency, which is an account and region decision made once and expensive to revisit.
The hard part when you run both
Two catalogues means two access models, and the failure is a table governed correctly in one and exposed in the other — usually because a copy was made to serve a different engine.
The fix is architectural rather than procedural: one storage layer in an open table format with one governance authority over it, and both engines reading through that. Delta Sharing and Iceberg interoperability make this workable now, and it is the answer an interviewer is listening for.
What a catalog is actually for
A catalog in a lakehouse is not a documentation tool, and treating it as one is why so many catalog projects deliver a searchable list nobody uses. Its load-bearing function is issuing short-lived, scoped credentials to query engines — which makes it, not the storage bucket policy, the place where access control is genuinely enforced.
That is why the catalog layer is contested by every vendor: whoever owns it owns authorisation for the entire estate, regardless of which engine runs the query. It is a far stronger position than owning a metadata listing.
- Enforce at the catalog, not in each engine. A rule applied in one query engine and not another is not a control — it is a convention, and someone will use the other engine.
- Row and column policies belong with the table definition, so they travel with the data rather than being reimplemented in every consumer. A masking rule that exists in the dashboard and not in the warehouse protects the dashboard.
- Lineage is the feature that pays for the platform, and not for the reason usually given. Its value is not documentation — it is answering “this number is wrong, what feeds it?” and “we are changing this column, what breaks?” Both are asked under time pressure, and neither can be answered by asking around.
- And the thing no catalog tells you: whether a table is healthy. Small-file accumulation, delete-file buildup, stale snapshots, metadata bloat — all invisible. The symptom that reaches you is never ‘this table needs compaction’; it is ‘queries got slower and nobody changed anything’.
One precision worth carrying into a vendor conversation: when a catalog is described as open source, ask whether the open build has the same capabilities as the managed service. For at least one major implementation the answer is no — and ‘it is open source’ and ‘the open version is not the product people buy’ are both true at once.
Check yourself