Posts on tosd - Theory of Small Decisions

Gold Is a Location, Not a Guarantee

Fri, 11 Sep 2026 00:00:00 +0000

The medallion architecture divides a lakehouse into three layers, and anyone can read them without instruction: bronze is what arrived, silver is what’s been cleaned, gold is what you can use.

Then two teams query the same gold table and report different results. Or someone discovers midway through an outage that nobody knows who approves changes there.

A layer tells you where the data lives. It never tells you what it promises.

The Chat Always Answers, and That's the Problem

Fri, 11 Sep 2026 00:00:00 +0000

A chat on top of your warehouse feels like you’ve finally crossed the finish line. Users ask in plain language, the model generates SQL, and the endless queue to your data team simply disappears.

Then someone asks how many active customers you had last quarter. They get a number back. They carry it into a meeting. Nobody can say what it counted as a customer, or as active, or whether it left out test accounts.

C4 for Data Architecture: The Wrong Container

Thu, 10 Sep 2026 00:00:00 +0000

Adopting C4 is sold as the end of diagram chaos. Four levels, one audience each, and everybody finally draws the same thing.

On a data platform it half works. The diagram gets drawn, the review approves it, and six months later nobody opens it to decide anything.

The level of detail was never the problem.

Every data platform diagram looks the same

Draw the container level for three platforms at three different companies and you get three versions of one picture: an ingestion layer, object storage, a processing engine, an analytical warehouse, an orchestrator and a BI tool.

An Ontology for the Semantic Layer

Wed, 02 Sep 2026 00:00:00 +0000

Semantic layers are usually sold as a place to put metric definitions. Write revenue once, expose it everywhere, stop arguing in meetings.

Then two teams report different revenue anyway, both technically correct, and the argument moves from the dashboard into the definition file.

The definitions were never the hard part.

Metrics are the visible layer, not the load-bearing one

A metric is an aggregation over a set of things. Change what counts as one of those things and the number changes, without the formula changing at all.

Batch or Real Time: A Tradeoff Usually Framed Wrong

Thu, 20 Aug 2026 00:00:00 +0000

The question arrives as a technology question. Should this integration be batch or streaming? And it gets answered like one, by comparing engines, latencies and throughput numbers.

That framing is the problem. Latency is not the requirement. It is a property you buy, and the thing worth arguing about is how much of it you actually need.

Start from the decision, not the pipeline

Data exists so somebody, or something, can act on it. So the useful question is not “how fresh can we make this?” but “how quickly does the action it triggers actually happen?”