Data Mesh
Data Mesh, coined by Zhamak Dehghani in 2019, is a paradigm for designing and managing data architectures within organizations. It advocates a decentralized approach to data management, distributing ownership, access, and governance across different domains or business units.
The shift to decentralized data management
Centralized and monolithic data management, anchored in a data lake or warehouse, creates a massive bottleneck that stifles innovation and hinders data teams from meeting the rapidly evolving demands of the business. In response, the industry is progressively embracing decentralized data management, especially through the data mesh and its four core principles:
Domain-oriented decentralized data ownership and architecture
Data as a product
Self-serve data infrastructure as a platform
Federated computational governance
85%
of organizations recognize the relevance of Data Mesh.
(BARC 2023)
54%
of organizations are planning to implement or are implementing Data Mesh.
(BARC 2023)
70%
of organizations have or are in the process of implementing Data Products.
(Eckerson Group 2024)
Key factors driving the rise of data mesh
Economic Pressures
Economic PressuresExecutives face mounting pressure to justify investments in data infrastructure, yet measuring economic returns remains challenging despite significant resources allocated.
Competitiveness and AI Impact
Competitiveness and AI ImpactFear of losing competitiveness drives urgency in leveraging AI opportunities, facilitated by the emergence of accessible AI models. However, scaling AI applications necessitates high-quality, secure, and compliant data, adding pressure on centralized data management teams.
Flexibility in Implementation
Flexibility in ImplementationData mesh offers adaptable principles rather than rigid architectures or technologies, enabling diverse implementation approaches.
Endorsement and Enthusiasm
Endorsement and EnthusiasmGrounded in robust theoretical and empirical foundations, data mesh garners widespread support and enthusiasm among data teams and decision-makers, facilitating adoption and limiting resistance to change.
Accessibility and Cost-Effectiveness
Accessibility and Cost-EffectivenessImplementing data mesh principles requires reallocating existing resources rather than substantial investments, making it an accessible and cost-effective framework for transforming data management practices gradually and tailored to each organization’s context.
Zeenea supports the data mesh as its supervision system
Zeenea’s core innovation centers around an evolving knowledge graph, a structure that sets it apart from traditional catalog or metadata management tools. While we acknowledge there is no standalone technology to deliver a data mesh, we decided to evolve our Data Discovery Platform to offer a unique system – one that covers both the need for a cross-domain marketplace and the need for a private data catalog for each domain.
Private, federated domain data catalogs
In Zeenea Data Discovery Platform, each domain is responsible for a subset of the graph, a private Data Catalog, where it can potentially: organize data as desired, follow shared governance rules, feed the catalog, manage its users and permissions, identify the objects it wants to share with other domains – and control which information will be shared. This is our unique data catalog federation technology.
A marketplace to share the most valuable data across domains
Our federated catalogs seamlessly interconnect, extending into an organization-wide internal marketplace. This is what we call Enterprise Data Marketplace, another subset of the graph, containing not only data products but objects intended for sharing from domains – business definitions, dashboards, ML models, etc. The marketplace is an essential component to ensure the exploitation of the data mesh on a larger scale as it allows data consumers to have a simple solution to search for and access data products from the various domains.
Mirroring the mesh at the metadata management level
With this federated graph structure, Zeenea’s platform mirrors the mesh at the metadata level – a mirror that can be continually adapted to reflect the evolution of the data mesh’s topology and the architecture of the data platform as it is deployed throughout the organization. Structured as an evolving federation of interconnected graphs, the Zeenea platform thus provides the ideal foundation for building the data mesh supervision system and supporting its long-term deployment.
The Practical Guide to Data Mesh
Learn more about Data Mesh
