Data is no longer a byproduct of our systems — it is the product itself. Seven pillars for building sovereign, governed, and trustworthy data products that scale beyond the era of fragile pipelines and disconnected silos.
For two decades, data teams optimized for one thing: moving bytes from A to B. The resulting systems are familiar — centralized warehouses fed by hundreds of brittle pipelines, owned by a lone platform team, consumed by analysts who file tickets and wait. It worked, until it didn't. The failure mode is always the same: nobody knows where a number came from, nobody will commit to its freshness, and nobody is accountable when it's wrong.
Product thinking flips the model. A data product is built like a service: it has an owner, an SLA, a version, a contract, and a consumer experience. The team that understands the data best — the domain team — owns it end-to-end. The platform team builds the substrate (storage, compute, governance) that makes this feasible without every domain reinventing the wheel. This is the core bet of the Data Mesh paradigm, and it's the architectural frame this manifesto operationalizes.
What follows is a concrete commitment to seven pillars. None of them are optional. A data "product" that skips governance is just a pipeline with a better name.
Data is a first-class citizen, not residue. Every dataset is deliberately curated for a consumer, not harvested and hoped over.
Ingestion is not a dump; it is a strictly controlled port of entry. The wrong pattern here — nightly batch when the consumer needs sub-minute, or a stream when a file would do — poisons every downstream decision.
Storage is tiered and purpose-built. Each layer answers a different question: what actually happened, what does it mean, and how do we serve it fast?
Transformations are not dark magic maintained by one senior engineer's memory. They are transparent, version-controlled code, reviewed like any other.
Trust is earned through automated, relentless verification. We do not hope for data quality; we enforce it, and we fail loudly when it breaks.
A product is only as good as its usability. Different consumers want different surfaces — serve them where they are, not where the platform team finds it convenient.
Governance is not bureaucracy. Done right, it is the framework that lets a domain team ship a new data product next Thursday without a cross-org security review each time.
Viewed from the outside, a data product is a black box with declared input ports, output ports, and a published contract. Viewed from the inside, every pillar shows up as a concrete architectural layer. The boundary between inside and outside is the contract — and that contract is what consumers actually depend on.
┌──────────────────────────────────────────────┐
│ DATA PRODUCT (domain-owned) │
│ │
upstream│ ┌───────────┐ ┌────────────┐ ┌──────┐ │ downstream
───────►│──►│ INGEST │──►│ PROCESS & │──►│SERVE │──│──► consumers
sources │ │ (ports) │ │ ORCHESTRATE│ │(ports│ │ (BI, apps,
│ └───────────┘ └────────────┘ └──────┘ │ streams,
│ ▲ ▲ ▲ │ ML, humans)
│ │ │ │ │
│ ┌─────┴────────────────┴─────────────┴──┐ │
│ │ CONTRACTS · QUALITY · OBSERVABILITY │ │
│ └───────────────────────────────────────┘ │
│ ┌───────────────────────────────────────┐ │
│ │ GOVERNANCE · LINEAGE · ACCESS · PII │ │
│ └───────────────────────────────────────┘ │
│ ┌───────────────────────────────────────┐ │
│ │ METADATA · CATALOG · OWNER · SLA │ │
│ └───────────────────────────────────────┘ │
└──────────────────────────────────────────────┘
A pipeline ends at a table. A data product ends at a contract. The contract is the thing consumers depend on — schema, semantics, freshness, owner, SLA — and it is versioned independently of the pipeline implementation underneath.
The seven pillars are pattern-agnostic — you can build a data product inside a monolithic warehouse or across a federated mesh. What changes is where ownership lives and who carries the operational load. These three patterns cover most real-world deployments and none of them is universally correct.
| Pattern | Ownership | Best for | Fails when |
|---|---|---|---|
| Centralized Warehouse / Lakehouse |
Single platform team owns everything. | Small orgs, consistent semantics, tight compliance scope. | The central team becomes a bottleneck and domain knowledge erodes. |
| Data Mesh Federated |
Domains own their products; platform team owns the substrate. | Large orgs with strong domain boundaries and mature engineering culture. | Domains lack engineering capacity or governance is under-invested — you get chaos with a better name. |
| Data Fabric Integration layer |
Ownership stays where it is; a metadata/AI layer unifies access. | Heterogeneous legacy landscapes where re-platforming is infeasible. | The fabric becomes a leaky abstraction over inconsistent sources — quality issues surface at query time. |
Most large organizations end up with a hybrid: mesh-style domain ownership for new products, a central warehouse for cross-domain reporting, and fabric-style integration for the legacy systems nobody is going to rewrite. That's fine. The manifesto holds regardless — every data product, wherever it lives, commits to the seven pillars.
The abstractions get real when you write them down. Below is a minimal but complete product descriptor — the kind of file that lives next to the dbt models, gets reviewed in PRs, and is consumed by the catalog, the quality runner, and the access layer alike. Everything a consumer needs to decide whether to depend on this product is in this one file.
id: payments.settled_transactions version: 2.3.0 domain: payments owner: payments-platform@example.com description: "Settled payment transactions enriched with FX and merchant metadata." # ── input ports ──────────────────────────────────────── inputs: - name: raw_payment_events type: kafka topic: prod.payments.events.v4 - name: fx_rates type: table ref: finance.reference.fx_rates@^1 # ── output ports ─────────────────────────────────────── outputs: - name: settled_transactions type: iceberg_table location: warehouse.payments.settled_transactions - name: settled_events type: kafka topic: prod.payments.settled.v2 - name: settlement_api type: rest path: /v2/transactions/{id} # ── contract ─────────────────────────────────────────── schema: - { name: txn_id, type: string, nullable: false, pii: false } - { name: customer_id, type: string, nullable: false, pii: true } - { name: amount_usd, type: decimal(18,4), nullable: false } - { name: settled_at, type: timestamp, nullable: false } # ── quality ──────────────────────────────────────────── quality: tests: - unique(txn_id) - not_null(customer_id, amount_usd, settled_at) - range(amount_usd, min: 0, max: 10_000_000) freshness: <= 5m p99 volume: anomaly_detect(window: 7d, sigma: 3) # ── SLA ──────────────────────────────────────────────── sla: availability: 99.9 freshness_p99: 5m support: business-hours # ── governance ───────────────────────────────────────── access: classification: confidential default: deny grants: - { group: risk-analytics, dsa: DSA-2026-014 } - { group: finance-reporting, dsa: DSA-2026-017 }
If a new consumer can read this one file and decide — without asking anyone — whether the product meets their needs, you have a data product. If they have to ping the owning team on Slack to find out the freshness, you have a pipeline.
A data product that technically ticks every pillar can still fail in practice if it violates these six properties. Treat them as an acceptance test, not a wishlist.
Has a stable, unique identifier and endpoint. No consumer should ever hardcode a table path.
SLAs, quality scores, and lineage are visible without asking. Freshness is a published number, not a promise.
Schema, semantics, and examples live with the product. A new analyst is productive in a day.
Standard formats and contracts — Iceberg, Parquet, Avro — so products compose.
Access is controlled by policy, not convention. Every read is logged and auditable.
Version and deploy without breaking consumers. Contracts evolve via additive changes and deprecation windows, not flag days.
Most "data product" initiatives that stall do so for predictable reasons. None of them are technical. Watch for these and push back early.
Treating data as a product is not an architectural style choice. It is an operational commitment: to owners, to contracts, to SLAs, and to the consumers who depend on the output. The seven pillars are what that commitment looks like in concrete terms.
Pick a single high-value dataset in your organization. Give it an owner. Give it a contract file. Give it a published SLA. Wire up automated quality. Register it in the catalog. That one product, done rigorously, will do more for your data culture than a three-year platform strategy deck. Start there.