Data Fabric and Data Mesh are both hot topics in the world of data integration. There are lively debates on what the differences are, and which is better. I believe that they are founded on a common data platform technology, and will converge.
Data Fabric
Gartner defines data fabric as a design concept that serves as an integrated layer (fabric) of data and connecting processes that utilize continuous analytics over existing, discoverable and inferenced metadata assets to support the design, deployment and utilization of integrated and reusable data across all environments, including hybrid and multi-cloud platforms.
It has four principles:
- Collect and analyze all forms of metadata
- Convert passive metadata to active metadata
- Create and curate knowledge graphs
- Robust data integration backbone.
Data Mesh
Thoughtworks defines data mesh as an analytical data architecture and operating model where data is treated as a product and owned by teams that most intimately know and consume the data.
It has four principles too:
- Domain Ownership
- Data as a Product
- Self-serve data platforms
- Federated Computational Governance.
Common Technical Basis
Although their principles are different, data fabric and data mesh are very similar from a technical perspective. Both follow the model shown in a figure that I contributed to The Open Group White Paper on Technical Standards for Data Integration, of applications connecting to data sources through a mesh or fabric of data platforms.
Different Added Value
Data Fabric adds the dimension of automated analysis, with the data platforms building knowledge graphs within the common metadata.
Data mesh adds the human dimension, with guidance on how people should produce, use and manage the data in an enterprise.
Convergence
The power of analytics and AI is becoming increasingly evident. Enterprises now have to use these technologies to be competitive. At the same time, a successful enterprise cannot ignore the human dimension.
Enterprises will deploy data platforms, use automated analysis, and organise how their people use data. The result may be described as a data fabric or as a data mesh - but it will be both.
This is the future of data integration.