Case Study 03 · Data Platform Engineering

How We'd Build a Governed Data Platform

A technical blueprint for the platform that ends the meeting where two people bring different numbers for the same metric.

Industry: Mid-Market / Multi-Source
Timeline: 10–12 weeks to production
Approach: Blueprint (How We'd Build It)
Stack: Snowflake · dbt · Dagster · Fivetran
How We'd Build a Governed Data Platform

A mid-market company reaches a point where every leadership meeting starts the same way: two people present the same metric with different numbers, and the first twenty minutes go on working out whose spreadsheet is right.

The data exists. It is in the CRM, the billing system, the product database, and four operational tools nobody centrally owns. Each team extracts what they need, applies their own assumptions, and builds reporting on top. None of it is wrong exactly. None of it agrees.

The Core Problem

Analysts spend most of their week rebuilding the same extracts and reconciling numbers instead of doing analysis. Leadership makes decisions on figures they do not fully trust, or defers decisions waiting for a number everyone accepts. The cost is not the analyst time — it is the decisions that get delayed or made badly.

$12.9M
Average annual cost of poor data quality
Gartner, cross-industry
60%
Of AI projects abandoned through 2026 on data readiness
Gartner forecast
37%
Of organisations confident in their data practices
The other 63% are guessing

The Solution We'd Build

A governed data platform: pipelines that land source data reliably, a dimensional model that encodes agreed definitions, and a semantic layer that makes those definitions the only way to query a metric.

The technical work is not the hard part. The hard part is the definition workshops — getting Finance, Sales, and Operations into a room to agree what active customer means, and recording the answer somewhere both auditable and enforceable.

Governed data platform architecture diagram
INGESTION
Fivetran / dlt
Managed connectors for SaaS sources, with dlt for anything custom. Incremental loading with schema-drift handling.
WAREHOUSE
Snowflake
Separate compute warehouses for pipelines and analytics, so a heavy query cannot delay the overnight load.
TRANSFORMATION
dbt
Dimensional models with tests on every table. Metric definitions in version control, reviewed like application code.
ORCHESTRATION
Dagster
Asset-based scheduling with lineage, retries, and freshness policies per dataset.
QUALITY
dbt tests + Great Expectations
Uniqueness, referential integrity, and business-rule assertions running on every load.
SEMANTIC LAYER
dbt Semantic Layer
One definition per metric, applied identically whether queried from BI, a notebook, or an API.

What We Would Not Do

  • We would not start with the warehouse. We would start with a data audit. Buying a platform before knowing what your data can support is how teams end up with an expensive warehouse full of numbers nobody trusts.
  • We would not migrate everything at once. We would move the highest-value domain first — usually revenue reporting — prove the pattern, then expand. Big-bang migrations fail slowly and expensively.
  • We would not build real-time unless a decision needs it. Streaming carries real operational cost. If nothing changes when data is fifteen minutes old rather than fifteen seconds, batch is the better engineering choice.
  • We would not skip the definition workshops. They are uncomfortable and they are the actual work. A perfect pipeline serving contested definitions solves nothing.

Delivery Timeline

WEEK 1–2
Data Audit
Source inventory, quality profiling, ownership mapping, and an honest assessment of what the data can currently support
WEEK 2–3
Definition Workshops
Facilitated sessions per domain to agree one definition per metric, with a named owner recorded for each
WEEK 3–5
Ingestion & Landing
Connectors configured, incremental loads with schema-drift handling, raw layer established
WEEK 5–8
Modelling & Tests
dbt dimensional models built incrementally, each shipped with tests and documentation
WEEK 8–10
Semantic Layer & BI
Metric definitions encoded, core dashboards rebuilt on the governed model
WEEK 10–12
Governance & Handover
Freshness monitoring, lineage, runbooks, and enablement so your team owns it

What Good Looks Like

OUTCOMES WE WOULD TARGET
Board reporting cycleDays → hoursGoverned model removes reconciliation
Conflicting metric definitionsEliminatedSemantic layer enforces one definition
Pipeline failure detectionUnder 15 minutesFreshness and volume alerting
Models covered by automated tests90%+dbt tests on every transformation
Ad-hoc report requestsSubstantial reductionSelf-service within a governed model
An Honest Caveat

These are targets based on the architecture and on what comparable platforms achieve — not results we are claiming from a specific past engagement. The audit in week one is what tells us which of them are realistic for your situation. Sometimes the finding is that your data cannot yet support the reporting you want, and the useful work for a quarter is fixing what sits underneath.

Services Involved

Can You Trust Your Numbers?

Tell us where your data lives, which metrics are contested, and what decisions depend on them. We will assess honestly what your data can support today and what it would take to get further — no obligation.