Over the past decade, data catalogs have emerged as important pillars in the landscape of data-driven initiatives. However, many vendors on the market fall short of expectations with lengthy timelines, complex and costly projects, bureaucratic data governance models, poor user adoption rates, and low-value creation. This discrepancy extends beyond metadata management projects, reflecting a broader failure at the data management level.

Given these shortcomings, a new concept is gaining popularity, the internal marketplace, or what we call the Enterprise Data Marketplace (EDM) at Zeenea.

In this series of articles, get an excerpt from our Practical Guide to Data Mesh where we explain the value of internal data marketplaces for data product production and consumption, how an EDM supports data mesh exploitation on a larger scale, and how they go hand-in-hand with a data catalog solution:

1. Facilitating data product consumption through metadata
2. Setting up an enterprise-level marketplace
3. Feeding the marketplace via domain-specific data catalogs

—–

Structuring data management around domains and data products is an organizational transformation that does not change the operational reality of most organizations: data is available in large quantities, from numerous sources, evolves rapidly, and its control is complex.

Data catalogs traditionally serve to inventory all available data and manage a set of metadata to ensure control and establish governance practices.

Data mesh does not eliminate this complexity: it allows certain data, managed as data products, to be distinguished and intended for sharing and use beyond the domain to which they belong. But each domain is also responsible for managing its internal data, those that will be used to develop robust and high-value data products – its proprietary data, in other words.

Metadata management in the context of an internal marketplace fed by domain-specific catalogs

In the data mesh, the need for a Data Catalog does not disappear, quite the contrary: each domain should have a catalog allowing it to efficiently manage its proprietary data, support domain governance, and accelerate the development of robust and high-value data products. Metadata management is thus done at two levels:

• At the domain level – in the form of a catalog allowing the documentation and organization of the domain’s data universe. Since the Data Catalog is a proprietary component, it is not necessary for all domains to use the same solution.

• At the mesh level – in the form of a marketplace in which the data products shared by all domains are registered; the marketplace is naturally common to all domains.

With a dedicated marketplace component, the general architecture for metadata management is as follows:

In this architecture, each domain has its own catalog – which may rely on a single solution or not – but should be instantiated for each domain to allow it to organize its data most effectively and avoid the pitfalls of a universal metadata organization.

The marketplace is a dedicated component, offering simplified ergonomics, and in which each domain deploys metadata (or even data) for its data products. This approach requires close integration of the different modules:

• Domain catalogs must be integrated with the marketplace to avoid duplicating efforts in producing certain metadata – especially lineage, but also data dictionaries (schema), or even business definitions that will be present in both systems.

• Domain catalogs potentially need to be integrated with each other – to share/synchronize certain information, primarily the business glossary but also some repositories.

Data catalog vs EDM capabilities

When we look at the respective capabilities of an Enterprise Data Marketplace and a Data Catalog, we realize that these capabilities are very similar:

In the end, on a strictly functional level, their capabilities are very similar. What distinguishes a modern Data Catalog from an EDM are:

• Their scope – The Data Catalog is intended to cover all data, whereas the marketplace is limited to the objects shared by domains (data products and other domain analytics products).

• Their user experience – The Data Catalog is often a fairly complex tool, designed to support governance processes globally – it focuses on data stewardship workflows. The marketplace, on the other hand, typically offers very simple ergonomics, heavily inspired by that of an e-commerce platform, and provides an experience centered on consumption – data shopping.

The Practical Guide to Data Mesh: Setting up and Supervising an enterprise-wide Data Mesh

Written by Guillaume Bodet, co-founder & CPTO at Zeenea, our guide was designed to arm you with practical strategies for implementing data mesh in your organization, helping you:

✅ Start your data mesh journey with a focused pilot project
✅ Discover efficient methods for scaling up your data mesh
✅ Acknowledge the pivotal role an internal marketplace plays in facilitating the effective consumption of data products
✅ Learn how Zeenea emerges as a robust supervision system, orchestrating an enterprise-wide data mesh

Over the past decade, data catalogs have emerged as important pillars in the landscape of data-driven initiatives. However, many vendors on the market fall short of expectations with lengthy timelines, complex and costly projects, bureaucratic data governance models, poor user adoption rates, and low-value creation. This discrepancy extends beyond metadata management projects, reflecting a broader failure at the data management level.

Given these shortcomings, a new concept is gaining popularity, the internal marketplace, or what we call the Enterprise Data Marketplace (EDM) at Zeenea.

In this series of articles, get an excerpt from our Practical Guide to Data Mesh where we explain the value of internal data marketplaces for data product production and consumption, how an EDM supports data mesh exploitation on a larger scale, and how they go hand-in-hand with a data catalog solution:

1. Facilitating data product consumption through metadata
2. Setting up an enterprise-level marketplace
3. Feeding the marketplace via domain-specific data catalogs

—–

As described in our previous article, an Enterprise Data Marketplace is a simple system in which consumers can search among the data product offerings for one or more eligible to perform a specific use case, become aware of the information related to these products, and then order them. The order materializes as access opening, physical data delivery, or even a request for data product evolution to cover the new use case.

Three main options for setting up an internal data marketplace

When establishing an internal data marketplace, organizations typically consider three primary approaches:

Develop it

This approach involves building a custom data marketplace tailored to the organization’s unique requirements. While offering the potential for a finely tuned user experience, this option often entails significant time and financial investment.

Integrate a solution from the market

Alternatively, organizations can opt for pre-existing solutions available in the market. Originally designed for data commercialization or external data exchange, these solutions can be repurposed for internal use. However, they may require customization to align with internal workflows and security standards.

Use existing systems

Some organizations choose to leverage their current infrastructure by repurposing tools such as data catalogs and corporate wikis. While this approach may offer familiarity and integration with existing workflows, it might lack the specialized features of dedicated data marketplace solutions.

The drawbacks of commercial marketplaces

Although often offering a satisfying user experience and native support for the data product concept, commercial marketplaces often have significant drawbacks: highly focused on transactional aspects (distribution, licensing, contracting, purchase or subscription, payment, etc.), they are often poorly integrated with internal data platforms and access control tools. They generally require data to be distributed by the marketplace, meaning they constitute a new infrastructure component onto which data must be transferred and shared (such a system is sometimes called a Data Sharing Platform).

Zeenea’s Enterprise Data Marketplace

In a pragmatic approach, it is not desirable to introduce a new infrastructure component to deploy a data mesh – it seems highly preferable to leverage existing capabilities as much as possible.

Therefore, at Zeenea, we’ve evolved our data discovery platform and data catalog to offer a unique solution, one that mirrors the data mesh at the metadata level to continually adapt to the organization’s evolving data platform architecture. This Enterprise Data Marketplace (EDM) integrates a cross-domain marketplace with private data catalogs tailored to each domain’s needs.

An approach that we detail in the next article of our series, made possible by what has long distinguished Zeenea and differentiates it from most other data catalogs or metadata management platform vendors: an evolving knowledge graph.

In our final article, discover how an internal data marketplace paired with domain-specific catalogs, provides a comprehensive data mesh supervision system.

The Practical Guide to Data Mesh: Setting up and Supervising an enterprise-wide Data Mesh

Written by Guillaume Bodet, co-founder & CPTO at Zeenea, our guide was designed to arm you with practical strategies for implementing data mesh in your organization, helping you:

✅ Start your data mesh journey with a focused pilot project
✅ Discover efficient methods for scaling up your data mesh
✅ Acknowledge the pivotal role an internal marketplace plays in facilitating the effective consumption of data products
✅ Learn how Zeenea emerges as a robust supervision system, orchestrating an enterprise-wide data mesh

Over the past decade, data catalogs have emerged as important pillars in the landscape of data-driven initiatives. However, many vendors on the market fall short of expectations with lengthy timelines, complex and costly projects, bureaucratic data governance models, poor user adoption rates, and low-value creation. This discrepancy extends beyond metadata management projects, reflecting a broader failure at the data management level.

Given these shortcomings, a new concept is gaining popularity, the internal marketplace, or what we call the Enterprise Data Marketplace (EDM) at Zeenea.

In this series of articles, get an excerpt from our Practical Guide to Data Mesh where we explain the value of internal data marketplaces for data product production and consumption, how an EDM supports data mesh exploitation on a larger scale, and how they go hand-in-hand with a data catalog solution:

1. Facilitating data product consumption through metadata
2. Setting up an enterprise-level marketplace
3. Feeding the marketplace via domain-specific data catalogs

—–

Before diving into the internal marketplace, let’s quickly go back to the notion of a data product, which we believe is the cornerstone of the data mesh and the first step in transforming data management.

Sharing and exploiting data products through metadata

As mentioned in our previous series on data mesh, a data product is a governed, reusable, scalable dataset offering data quality and compliance guarantees to various regulations and internal rules. Note that this definition is quite restrictive – it excludes other types of products such as machine learning algorithms, models, or dashboards.

While these artifacts should be managed as products, they are not data products – They are other types of products, which could be very generally termed “Analytics Products”, of which data products are one subset.

In practice, an operational data product consists of two things:

Metadata ensures consumers have all the information they need to use the product.

It typically covers the following aspects:

Schema – providing the technical structure of the data product, data classification, samples, and their origin (lineage).

Governance – identifying the product owner(s), its successive versions, its possible deprecation, etc.

Semantics – providing a clear definition of the exposed information, ideally linked to the organization’s business glossary and comprehensive documentation of the data product.

Contract – defining quality guarantees, consumption modalities (protocols and security), potential usage restrictions, redistribution rules, etc.

In the data mesh logic, these metadata are managed by the product team and are deployed according to the same lifecycle as data and pipelines. There remains a fundamental question: where can metadata be deployed?

Using a data marketplace to deploy metadata

Most organizations already have a metadata management system, usually in the form of a Data Catalog.

But data catalogs, in their current form, have major drawbacks:

They don’t always support the notion of a data product – it must be more or less emulated with other concepts.

They are complex to use – designed to catalog a large number of assets with sometimes very fine granularity, they often suffer from a lack of adoption beyond centralized data management teams.

They mostly impose a rigid and unique organization of data, decided and designed centrally – which fails to reflect the variety of different domains or the organization’s evolution as the data mesh expands.

Their search capabilities are often limited, particularly for exploratory aspects – it’s often necessary to know what you’re looking for to be able to find it.

The internal marketplace, or Enterprise Data Marketplace (EDM) is therefore a new concept gaining popularity in the data mesh circle. Like a general-purpose marketplace, the EDM aims to provide a shopping experience for data consumers. It is thus an essential component to ensure the exploitation of the data mesh on a larger scale – it allows data consumers to have a simple and effective system to search for and access data products from various domains.

In our next article, learn the different ways to set up an internal data marketplace, and how it is essential for data mesh exploitation.

The Practical Guide to Data Mesh: Setting up and Supervising an enterprise-wide Data Mesh

Written by Guillaume Bodet, co-founder & CPTO at Zeenea, our guide was designed to arm you with practical strategies for implementing data mesh in your organization, helping you:

✅ Start your data mesh journey with a focused pilot project
✅ Discover efficient methods for scaling up your data mesh
✅ Acknowledge the pivotal role an internal marketplace plays in facilitating the effective consumption of data products
✅ Learn how Zeenea emerges as a robust supervision system, orchestrating an enterprise-wide data mesh