As organisations enter 2025, a fundamental transformation in data strategy is underway, shaped by three key developments. First, the explosive growth of AI capabilities demands higher quality data. Second, early adopters are already demonstrating competitive advantages. Third, most companies are experiencing increasing regulatory pressures around data governance. 

Since the Deep Learning boom of 2012, data has always been treated as the "New Oil", leading organisations to hoard vast quantities in data lakes. But, today, we have arrived at a point where this traditional approach is reaching its technical and practical limits. John Naisbitt’s paradox “We are drowning in information but starved for knowledge” has never been more relevant. And so we find many organisations struggling to translate their data into relevant insights for decision-making. As if that wasn’t enough, they also face rising storage and management costs, along with increasing technical complexities such as deteriorating query performance. And then we haven’t even talked about the escalating trust and privacy concerns as well as a growing number of regulatory and compliance requirements.

It’s safe to say that we need a new approach to data which looks beyond mere accumulation. One that focuses on connection and value creation. In fact, pioneering companies like Paypal and Uber are already proving the benefits of modern data architectures which leverage AI effectively, maintain regulatory compliance, and secure competitive advantage.

So what’s their secret?

Like a Puzzle

Each of these leading companies understand that true value never emerges from isolated data points. They grasp that only connecting insights - across different business domains, departments, and even external sources - can reveal previously hidden patterns and relationships. Compare it to a puzzle: the story only emerges when all separate parts have been put together, right? In a similar way, data points need to be treated as pieces of a larger framework rather than as standalone facts so that they can paint a more comprehensive and nuanced business story.

This is where data mesh architectures come into play. Unlike traditional setups that rely on centralised data lakes or warehouses, data mesh takes a different path. Instead of pulling data from various business domains and putting it under the control of central data teams, it gives ownership of the data to domain experts—the people who truly understand the business context where the data originates.

This, of course, also means that these business owners have to be trained to see data not just as something to hoard, but as a product, with clear standards for quality, ownership, and value. One of the big advantages of this shift is that it makes data quality and insights a shared responsibility: each business area—like Marketing, Finance, or Operations—learns to take charge of their own data, manage it, and share it as a product with the rest of the organisation. 

In short, by adopting a data mesh approach, companies can move from simply collecting data to increasing their intelligence by connecting insights and enabling smarter, more confident decisions. 

Data Mesh: How to Get There

So what does the shift to a data mesh architecture entail? First, you need to transform your data architecture, moving from a bottleneck-prone, fragile and rigid central warehouse to a marketplace of high-quality data products that are owned and managed by business experts. It’s the perfect marriage of decentralisation and interconnection. This of course requires data governance standards as well as a shared data infrastructure that’s more scalable, better aligned with the business, and capable of generating insights faster with fewer bottlenecks.

Leading companies like Netflix, Zalando and ING have already successfully embraced the concept of data mesh architecture, moving from centralised data lakes to dynamic and collaborative networked domains. This approach has revolutionised their data strategies and gives their business experts the tools they need to lead data-driven innovation.

HOW NETFLIX TAMED THE DATA BEAST

Netflix's transformation to a data mesh architecture was driven by the sheer scale of their data challenges - their studios generate about 2 Petabytes of data per week worldwide, with storage costs increasing by 50% year over year. Their research showed that at least 40% of this data never gets used. By implementing a data mesh approach with distributed responsibility and treating data as a product, Netflix developed a scalable solution that:

• Reduced storage costs through better lifecycle management
• Enabled domain teams to manage their own data products
• Created clear data quality standards and validation processes
• Automated cleanup of unused data
• Improved visibility into data usage patterns

Traditionally, data expertise has always been centralised within dedicated data management teams. The data mesh approach, however, is all about the empowerment of business experts who learn to manage and leverage their own domain’s data. It may be a platitude, but sometimes it really is true that “If you want something done right, you’d better do it yourself”.

In practical terms, this means that marketing analysts might take charge of customer insights as data product owners. Operations managers could curate valuable process performance data products. The key challenge is to make sure that these business domain experts feel supported and confident in managing their data effectively. Providing them with the right tools, training, and authority is absolutely essential here. Examples are data literacy programs, self-service analytics tools and clear governance frameworks that guide rather than restrict. 

By democratising data responsibility and equipping teams properly, data shifts from being just a technical resource to becoming a shared business asset that everyone can contribute to and benefit from.

The Data Decision Confidence Framework

In a mature data mesh environment, generating insights becomes a lot more structured as well as more collaborative. At In The Pocket, we have developed our proprietary Data Decision Confidence Framework which provides a systematic approach to validating these insights. 

We always start the process with a range of business hypotheses that have been formulated by domain experts. Next, the business experts can start adding layers of validation and continue doing so until enough intelligence is gathered to make informed business decisions. 

Example:

• Hypothesis: "Customers who invest in solar panels show different energy and service consumption patterns." 
• First validation activity: Find evidence to support the hypothesis using smart meter data from Operations.
• Second validation activity: Adding an additional layer of insights through digital platform behaviour data obtained from Product Analytics.
• Third validation activity: The insights from the previous validation activities are further enriched with data from customer service interactions. 

This process is all about building confidence. The more data product layers come in from more domains, the better domain experts can validate or disprove their hypotheses. And when patterns are discovered within one business area, they can then also be further validated across different departments. For example, analysing how data evolves over time can uncover valuable insights into seasonal patterns and long-term trends. These strategic insights will only increase if you add external data—like market trends, industry benchmarks, and third-party sources. 

From Individual Data Mesh to Secure Data Ecosystems

It should perhaps not come as a surprise that this connected marketplace approach only becomes richer if it is scaled in broader data collaborations that reach outside of the walls of just one company. But with great scale come greater challenges. Organisations will then need to prevent data duplication across domains. At the same time, they must ensure that cross-domain analytics remain clear and easy to understand. Additionally, they must implement governance frameworks that maintain data quality without adding unnecessary technical hurdles.

As organisations get better at building well-documented, high-quality data products in collaborative environments, new opportunities will open up for secure data sharing between companies. In fact, industry-specific networks are already doing this, all the while keeping control of their data and protecting privacy through custom data clean rooms. Government open data projects are also benefiting, creating standardised frameworks for public-private data collaboration.

In these times of data explosion (thank you, AI) as well as increased regulatory and competitive pressure, the future of data value lies in connecting architectures across companies and industries, empowering business users and validating insights in multiple steps. To succeed, companies must choose flexible technologies to avoid vendor lock-in and adapt their culture to embrace this shift. 

The payoff? Deeper insights from broader data networks—whether in energy transitions, cross-industry innovation, or public service improvements. And as these networks mature, organisations using data mesh principles and strong validation practices will lead the way in unlocking the full value of collaborative data.

‍