24 Jul '22

Knitting it all together – How a Data Mesh offers a new decentralised approach to data management

Gold Medallist Tom Daley’s knitting skills were one of the unexpected takeaways from the Tokyo Olympics. Splashed across the pages of the tabloid press, his creations also went viral online. On a superficial level, we watched a man artfully knit as he sat poolside calmly waiting for his turn to dive.  At a deeper level, we were entranced by how he meshed reams of wool with needles to create beautiful yet functional designs. A Data Mesh could perhaps be similarly described.

Although a relatively new concept, at DTSQUARED our consultants are becoming ever more skilful at helping organisations monetise their data through the implementation of a Data Mesh. In technical terms, it is a decentralised solutions architecture approach to data management that uses existing business subject matter expertise to guide the resolution of data objects, which are then developed into multiple, context-based data product designs in a structured way.

Essentially, it is an approach that can help you to share, access, and manage the analytical data that resides in what are complex, large-scale, and fast-moving environments across your organisation – and sometimes beyond those boundaries too. It introduces a more logical architecture to managing and accessing all this data, and one that can help serve a good foundation for visualisation to make it more meaningful and accessible for technical and business users, not only those with technical knowledge.

What makes a Data Mesh different to other approaches to data management?

A Data Mesh is a very structured and logical approach to data management. As it is decentralised, this also makes it much easier in terms of control. Elements of privacy-by-design, standardisation, stewardship, and governance will all benefit from this decentralisation.

It also allows us to apply the principle of data as a product more widely to organisations and enables us to ensure the data is discoverable, addressable, and understood by the user as well as being trustworthy and accessible.  

Above all, it is interoperable which makes it much more manageable. When you have different types of data flowing through an organisation the data often needs to be changed, and the cost of this can be highly significant. If the data can be interoperable early in its lifecycle, then managing that data will become much easier and more cost-effective.

What about our traditional data lakes and data warehouses, are they now redundant?

The sheer volume of data that exists means that data lakes and data warehouses will not become redundant any time soon. We are experiencing an explosion in data creation from various data types including, but not limited to, personal data, customer data, insurance data, traffic data, location data, and internet traffic data. This exponential growth is predicted to last for the foreseeable future.

Data is of course not only a product but also a hugely valuable strategic asset that helps organisations thrive in an increasingly interconnected, global economy. The beauty of a Data Mesh is that it can help to manage this vast quantity of complex data in a secure, safe, and scalable way. When the analytics and the business intelligence kicks-in is when your organisation can properly monetise this data.

What is the easiest way to implement a Data Mesh approach?

Leveraging cloud-based technologies is probably the most efficient way to do this, because it is the most scalable. Using a cloud-based methodology is also reliable and easy to change. Plus, it is inherently more secure than any on-premise option. If transitioning from on-premises systems to cloud-based systems, a Data Mesh becomes one of the clearest options to follow.

Will a Data Mesh benefit all organisations?

A Data Mesh is best suited to larger organisations that typically have complex and disparate systems, not just because of the cost of implementation, but because of the complexity of gathering data that resides across the organisation. A smaller company may find that it is not feasible, but for larger companies where the sheer complexity of data can be overwhelming, this type of approach is strongly recommended.

How long will a Data Mesh take to implement?

This is very difficult to quantify. Once started, it becomes an ongoing process. However, many industry sources suggest that to implement a Data Mesh, and achieve a working model, a large organisation should consider a two-to-three-year timescale.

Where did the Data Mesh originate?

Some will suggest that the Data Mesh is of American origin, while others argue that elements of it have their roots in Europe. It really doesn’t matter; what does matter is that the guiding concepts and approach to data management have been unified. One of the keys to the success of the Data Mesh concept is a joined-up architectural approach that draws together technical and organisational design practices to help organisations scale up and package them in meaningful way; a way that makes sense to others in the data analytics space as well as business users and leaders.

At DTSQUARED we have been developing our own Data Mesh expertise for a long time. Data Strategy, Data Governance, Data Quality, Data Architecture, Data Stewardship, Data Lineage, and the handling of Master Data Management are all key principles of data management. On top of all these strands, and knitting them together, is the Data Mesh.   
Keen to learn more about implementing a Data Mesh in your business? Get in touch with us today to discuss your data challenges and the solutions we can offer to help you get the most out of your data. We would be delighted to talk to you.

Get in touch with our data experts

Get in touch for a free session with our data experts