May 15, 2025
Risk and Reward in Data Management
(VZ Art/Shutterstock)
If you can’t justify spending large sums of time and money to implement a top-notch data management system, you might not only be hurting your business’ potential to utilize AI for a competitive advantage, but you might also be increasing your regulatory risk.
It can be difficult sometimes to justify spending large amounts of time and money to manage your data. After all, unless you’re a data broker, data doesn’t directly make your customers happier or lead to rising sales–they’re products of hard work and running an efficient business, among many other factors.
However, with the current regulatory and technology environment, having a well-managed data estate has gone from a nice-to-have to a necessity.
Consider a recent report by Gresham, which asked 200 senior data professionals working at financial services firms in the US and the UK about their firms’ data management strategies. The survey found that 44% of decision-makers say they are either “warehousing data in too many places, or warehousing too much data,” which leads to “increased complexity and enhanced compliance risks,” the company says.
(Source: Gresham report)
“While organizations demonstrate eagerness to adopt cutting-edge AI solutions, many have not yet established foundational data management principles or implemented comprehensive data quality frameworks that they need to ensure compliance,” said Neil Vernon, Gresham CTO. “This strategic misalignment between tech ambition and core data infrastructure not only impedes innovation but also exposes organizations to operational inefficiencies and heightened regulatory risks.”
Gresham cites new regulatory frameworks like Markets in Financial Instruments Regulation (MiFIR), the Digital Operational Resilience Act (DORA), and T+0 (same-day settlement in stock markets) as potentially impacting companies that have failed to implement satisfactory data governance strategies, nor the specific data management policies needed to ensure high quality data.
Firms that can’t integrate their disparate data streams under one unified structure face heightened risks, including escalating costs and operational failures, along with regulatory breaches, Vernon said. “An integrated, strategic approach allows for accurate, high-quality data collection in one place, significantly reducing expenses and safeguarding compliance in a complex regulatory environment.”
Meanwhile, Fivetran is out with a new study that finds companies are risking their AI futures by not investing enough in good data management practices.
The data integration and ETL vendor surveyed about 400 data professionals in the US, UK Europe, and Asia, and concluded that poor data management practices are perhaps more prevalent than previously thought.
(Source: Fivetran report)
For example, while a majority (57%) of Fivetran’s survey-takers say their data centralization strategy is “very effective,” over 40% of them admit that more than half of their AI projects either fail or underperform. What’s more, the number one roadblock to AI success is data integration, which was cited by 33% of Fivetran survey takers.
The number of data sources that enterprises are managing is staggering. Fivetran says three-quarters of its survey-takers are sourcing data from 500 or more places. This is exactly the extract, transform, and load (ETL) process, as hated as it is, remains so critical to big data, advanced analytic, and AI operations today.
Fivetran’s survey also found that over 80% of data engineering resources are being spent just on maintaining existing ETL data pipelines, and that’s among the cohort that has already centralized the majority of their data. If the data engineering efficiency is that poor at more advanced companies, think about how many resources are essentially being wasted at companies trying to manage silos of data all over the place.
The data professionals surveyed by Fivetran cited regulatory compliance as the biggest challenge in AI data management, with a 59% share of survey-takers. Fivetran says that shows that while data centralization “is a critical first step, it’s not enough on its own.”
The two surveys reiterate the importance of good data management for running a business in the 21st century. Data generation isn’t slowing down, and there are plenty of opportunities to put that data to use, whether it’s looking at historic trends (traditional analytics), predicting what’s likely to happen next (predictive analytics and machine learning), or creating new experiences (generative AI).
However, without a foundation in solid data management, a company is unlikely to be able to take advantage of new technologies like GenAI. What’s more, the regulatory environment increasingly is mandating good data management practices as a necessary first step in being granted access to public markets.
When it comes to finding proper motivation, there are carrots and there are sticks. For data management, decision makers can find both carrots and sticks in the form of AI and regulatory compliance. In this light, it would seem there is only one clear path forward: investment in data management is non-negotiable anymore.
Related Items:
These Are the Top Challenges to GenAI Adoption According to AWS
Big Data Smackdown: Data Management Vs. Data Governance
Leading Solution Providers
Tabor Network
