Gold Is a Location, Not a Guarantee

The medallion architecture divides a lakehouse into three layers, and anyone can read them without instruction: bronze is what arrived, silver is what’s been cleaned, gold is what you can use.

Then two teams query the same gold table and report different results. Or someone discovers midway through an outage that nobody knows who approves changes there.

A layer tells you where the data lives. It never tells you what it promises.

What the documentation promises, and what it doesn’t

Read the source before you argue with it. Databricks' documentation is explicit about the medallion pattern’s status: adopting it is “a recommended best practice but not a requirement”. It’s a naming convention, not a mechanism.

On guarantees, that same page states that “this architecture guarantees atomicity, consistency, isolation, and durability”. That deserves a pause.

ACID doesn’t come from the three-layer arrangement. It comes from the table format underneath, which is a separate, swappable piece. Three folder names do not buy you atomicity, isolation, or durability. The documentation attributes a property to the architecture that actually comes from something else — and never says what.

It’s the same displacement the rest of this post will describe, and it appears in Databricks’ own documentation. Even granting it entirely, ACID says nothing about whether the number is right, whether the grain matches your assumption, or whether anyone is accountable.

Validation does appear, but split across layers: bronze has minimal checks, and silver is where schema enforcement, deduplication, and quality tests land. Gold, according to that page, surfaces aggregated data for downstream consumers. Aggregating is not validating.

Trust gets inferred from the path

The problem isn’t the layers. It’s what the name does to the person who reads it.

flowchart LR
    Q{"Can I trust<br/>this GOLD table?"} --> A["It sits in GOLD"]
    A --> U["Which checks ran?<br/>Who owns it?<br/>What grain?"]
    U --> N["Unstated"]

Nobody argues this reasoning aloud. It just happens because a word like gold does rhetorical work that nobody asked it to do. Ananth Packkildurai, who wrote about data platforms in Data Engineering Weekly, says it plainly: “data architecture is not a medal competition.”

Because trust gets inferred rather than declared, there’s nothing that fails when it stops being true. A gold table that no longer holds what someone assumed remains in gold.

A guarantee is declared, not located

The alternative isn’t another layer. It’s writing what the table promises somewhere a machine can read it.

That’s a data contract. Bitol, the incubating standard at the Linux Foundation, defines it as a commitment between the data’s producer and its consumers, stating the quality rules it must meet, service levels, roles, and ownership. The difference from a layer is that here there’s a subject: someone stands behind something specific.

The same shift exists at the assertion level. Great Expectations formalizes expectations as verifiable claims about a dataset — literally “unit tests for your data”. An expectation either passes or fails, which is exactly what a folder cannot do.

A contract declares, and its enforcement depends on the tools that run it. The YAML file isn’t the guarantee.

But a written, verifiable promise is a different category from a promise inferred from a path.

That distinction is the entire argument. It echoes the same tension I laid out in the ontology of the semantic layer: what’s visible is not what bears the weight.

Gunnar Morling, who led Debezium at Red Hat, develops the same idea from the producer’s side.

The producer doesn’t expose its internal schema in the hope nobody depends on it. Instead, it decides what it promises and holds that promise steady even as it changes underneath.

Promote, don’t deposit

There’s a pattern that makes that promise part of the pipeline. The Apache Iceberg documentation describes write-audit-publish: writes land on an audit branch isolated from the main history, a validation flow examines that state, and only then is it promoted.

flowchart LR
    W["Write to audit branch"] --> V{"Run declared<br/>expectations"}
    V -- pass --> P["Publish to main"]
    V -- fail --> D["Discard branch"]
    P --> R["Consumer reads<br/>a checked promise"]

The verb is what changes. Data doesn’t get deposited into a trustworthy location; it earns the right to land there. The guarantee is the validation step, not the destination.

Failure modes worth naming

The pattern holds across all seven: someone trusts a promise that nobody is checking.

Where to start

Don’t migrate anything. Take the gold table that the most people query and write three sentences in a file next to it: what one row means, what it guarantees about freshness, and who answers when it stops being true.

Then convert one of those three into a check that runs with every load. Which one you choose matters less than the fact that it can fail on its own. Until something fails, you still have a well-named location, not a promise.