Companies Are Increasingly Relying On Data Mesh

The data architecture data mesh is becoming more and more popular with companies. Data Mesh is intended to distribute data and analysis work more evenly in the future through decentralization.

The data mesh is increasingly establishing itself in German companies as the dominant pattern for data platforms. Already 60 percent of companies with more than 1,000 employees will rely on the architecture paradigm in the future. 45 percent of companies with 500 to 1,000 employees are currently planning to introduce it. These are the results of the recently published study “Data Mesh – Just a buzzword or the next generation of corporate data platforms?”

In contrast to purely central approaches such as the data warehouse and the data lake, the data mesh is characterized by a higher degree of decentralization. The responsibility for the collection and maintenance of data and the data products based on it is shifting to the departments in which the data is generated – for example in marketing or accounting. At the same time, a series of guidelines and central components such as a data catalog and a data marketplace ensure that the data and data products provided can be used as easily as possible within the entire organization.

Higher data quality and availability

“The era of purely central data platforms is over,” “We see a clear trend towards decentralized forms of organization that spread data analytics capabilities in companies – and at the same time enforce company-wide standards through central components and guidelines.”

According to the survey results, large companies are at the forefront of modernizing data platforms: 23 percent are already using a data mesh, 37 percent are in the implementation phase and another 27 percent are planning to introduce it. In companies with 500 to 1,000 employees, only 7 percent have implemented a data mesh and 27 percent have started the first concrete implementation steps. 45 percent are currently planning to introduce a data mesh for this purpose.

Backlog demand for self-service

The modernization of the data platforms should help to draw more economic benefits from data than before and solve previous challenges. More than half of those surveyed currently see the greatest challenge in ensuring adequate data quality. 49 percent rate the handling of complex new technologies as challenging, while 43 percent perceive the availability of data and 41 percent the competent handling of it – the “data literacy” – as the greatest hurdles.

In addition, there are only a few companies that consistently provide all employees with easy-to-use data & analytics solutions. It is true that 90 percent of companies already have specialized teams for data analysis from which employees from business areas can request data & analytics solutions. However, only 8 percent of the companies enable their employees to carry out independent and ad hoc data analyses.

High expectations meet underestimated changes

The expectations of the positive effects of introducing a data mesh are high. 55 percent of those surveyed expect higher data quality and the implementation of a larger number of data-driven use cases. 51 percent hope for more reliable data, and 47 percent expect teams to work better together.

With regard to the changes that the IT managers surveyed expect, a technology-centric view dominates. 69 percent of them expect changes in terms of technology and architecture, 61 percent expect adjustments to processes and governance structures. Less than half, on the other hand, are prepared for changes in terms of organization, people and skills. Only 34 percent assume that there will be an impact on the work culture – although the adaptation of working methods in specialist departments outside of IT is a key success factor for the decentralized data mesh concept.

Also Read: Cloud Security: High Data Security In The Cloud!