Data fabric and data mesh are two concepts frequently discussed in the context of enterprise data and analytics. While they may seem similar on the surface, they are in fact very different in purpose for information flow. The ramifications of these concepts also have an increasingly significant impact on businesses today and in the future, as more aspects of business, such as product delivery, customer engagement, business development, and financial accounting, rely on data integration and management.

Over the past two decades, enterprises have managed data by oscillating through different approaches such as centralisation, decentralisation, database management, data warehousing, cloud data stores, and data lakes, among others. At present, we have cloud-based hyperscalers like Amazon Web Services, Microsoft Azure, and Google Cloud Platform as instantly recognisable choices.

Despite the abundance of options, the conundrum remains: Businesses want data to be in one place and easy to find. Gathering all the data into a single location continues to be a challenge. Data fabric and data mesh designs can help businesses solve these challenges in different ways.

For businesses today, finding a future-proof data framework that meets the changing needs of commerce is essential. Today, let’s take out the abstract understanding of data management and see how to approach data as new sources of revenue and value creation.

Decentralisation — the way out

Physically containing data in a single repository can be challenging in today’s environment where business units operate in silos. This means connecting to necessary data sources that might be stored in different formats, sizes, privacy restrictions, or other metadata traits.

Through logical data integration, business users harness virtualisation to connect and unify data and avoid issues relating to physically replicating data for ingestion. In logical data integration architectures, users do not access data directly, but through shared semantic models. These solutions provide virtualised representations of the data and leave the source data untouched. This is important as more stakeholders — executives and key decision makers — are involving themselves with the source data to draw more accurate macro-understandings of the business.

It is important to note that logical data fabric and data mesh are two very different architectural approaches. While data fabric is a data infrastructure stack, data mesh is a process-oriented approach that intends to solve data integration, management, and delivery in a distributed environment.

Data fabric for BI analysis

Let’s use an everyday analogy: Like different threads in the fabric of our clothing, a data fabric encompasses data from different locations, formats, and types woven together. In this configuration, the data is still understood to be physically integrated through traditional replication. A logical data fabric, on the other hand, replaces physical data integration with a logical data integration component. Data virtualisation makes this process possible, creating a logical data fabric.

This logical data fabric gives business users the option to layer business semantics on top without affecting the underlying data sources. Business leaders and data analysts can build customised virtual data stores without moving the underlying data sources. They can do so without fears of unintentionally modifying or corrupting the data sources.

For business leaders, their team of data scientists can use their preferred business intelligence (BI) tools and iteratively build their data models. This means fewer project management complications in collecting, replicating, and cleaning up the data for analysis. Logical data fabric makes data ready and accessible for use.

Toyota-Astra Motor (TAM) Indonesia applies data virtualisation as a core component of its enterprise-wide logical data fabric. Executives, data scientists, and business users have the flexibility to use their preferred BI tools. Additionally, the introduction of the logical data platform centralised security policies and improved overall data trust and confidence.

The logical data fabric removes complexities in data access and integration, enabling business users to make timelier decisions and cut down on product or service development cycles.

Data mesh for macro visibility

While a data fabric serves as an integration solution, a data mesh is an organising solution to structure data, individual access privileges, processes, and workflows within a singular enterprise.

In a data mesh, data ownership and management belong to assigned data domains that correspond to the department or function of the enterprise. Stakeholders within each data domain package their data as products to be delivered throughout the enterprise. Each line of business creates and maintains their own data products — such as consumer, asset, and finance data products.

When the enterprise’s key leadership lines and data domain owners need to create their own views or data products, granting access privileges may require complex and costly workarounds. A logical data mesh bridges the need for top-view visibility by business stakeholders with enterprise’s existing IT architecture design.

Through a logical data mesh, enterprises can connect the networks of relevant data available in an orderly and secure way to users, analysts, developers, and applications that need it. Data virtualisation enables enterprises to continue working with existing data assets, services, and project management workflows without disruptive trade-offs.

The logical data mesh grants BI teams access to privileges and business metadata in a separate, mutually exclusive layer. New semantic layers can be created for data domains without fear of modifying or corrupting the data sources. Data virtualisation is also a building block for creating new data domains that could be packaged for internal enterprise use or external, customer-facing products for sale.

Logical data fabric, data mesh or both?

Which approach works better? It depends on the operational workflow and size of the business. Logical data fabric is an intelligent, powerful way to integrate, manage, and deliver data which can be applicable to businesses of all sizes. A data mesh architecture, on the other hand, is an effective way to organise data within a large enterprise with complex organisational structures. So, it’s not a question of whether to choose data fabric or data mesh, but rather whether a business requires both implementations based on their needs.

Data virtualisation empowers organisations to use logical data integration and adopt the benefits of both approaches — data fabric and data mesh. Data virtualisation capabilities also provide a future-proofing data framework to meet the evolving needs of businesses. With a future-proof data framework, businesses can meet the changing needs and landscape of their industry, both today and in the future.