News

Data Management Revamp Needed in Move to Cloud, Says Report

• By David Ramel
• 08/22/2022

A new survey-based report commissioned by financial firm Capital One explores data management in the cloud, concluding that new models are needed in the face of exponential data growth.

Organizational data migrations to the cloud are driving new data management solutions, says the "New Data Management Models Are Essential To Operate In The Cloud" report for which research firm Forrester Consulting polled 157 data management decision-makers in North America earlier this year. Although primarily known for banking and credit cards, Capital One recently entered the B2B software business by launching Capital One Software, whose first product is Slingshot, a data management solution for users of Snowflake Data Cloud.

"Data has become the currency businesses use to deliver value and build strong customer relationships," the report says. "Companies that leverage data at scale are best positioned to win, serve, and delight customers. As companies move more data to the cloud, they will require new data management models to handle their exponentially increasing quantity and variety of data. New models that offer better self-service, integration, security, and clarity on cost will help firms drive efficiency and overcome the myriad of data management challenges they currently face."

Data management provides many benefits, Capital One has explained, including:

• A more productive workforce
• Data consistency across the enterprise
• Quicker delivery of new products and services
• Better access to data for everyone
• Greater control over infrastructure and processing costs

As for what the term "data management" entails, the list is lengthy:

• Data governance establishes common standards and policies for data protection that align with governmental regulations.
• Data architecture describes the structure of an organization's data assets and how the data moves from one point in the system to another.
• Date integration involves combining data from multiple sources and presenting them in a unified view.
• Data stewardship is the oversight practices that ensure the quality and fitness of data assets for business users.
• Data quality measures the fitness of a data asset for its intended purpose through factors such as consistency, accuracy and timeliness.
• Data lineage gives visibility into the way data flows through an organization including its origin, transformations and destination points.
• Data remediation is the organizing, cleansing, moving, deleting and archiving of data to ensure data quality and the correction of any errors or mistakes.
• Sensitive data scan identifies sensitive and confidential data, where it's located and how secure it is.
• Extract, transform, load (ETL) is a set of processes for moving data from its original source to a data warehouse.
• Master data management ensures that a shared set of critical data exists for an organization that is accurate and up to date.
• Data security is the practice of protecting data from harmful, unwanted and corrupting forces and breaches.
• Metadata management is an agreement on how to identify, classify and describe data to improve how users access and use the data.
• Data lifecycle management ensures a data asset is managed correctly from the time of its creation to archiving.

Capital One highlighted three main takeaways from the study:

Data Estate Models Vary and Evolve as the Journey to the Cloud Progresses
"Nearly three-quarters of data management decision-makers have yet to manage most of their organizations' data on the cloud," the report said. "As they continue the journey to the cloud, they need vendors to support organizational models that require decentralized and federated data support.

"More than half of decision-makers (56 percent) note their organizations maintain a centralized data estate that stitches most or all organizational data. However, a small percentage (15 percent) are federated, which means there is a central set of data policies and platforms, but lines of business manage data independently through self-service. A federated model supports the inevitable expansion of a data estate as data volumes, sources, and tools multiply, eliminating potential bottlenecks in the enterprise. However, while federating and enabling self-service, the data estate must still remain well-managed."

The Market Demands an Integrated Experience and a Mix of Capabilities
"Eighty-two percent of data management decision-makers find controlling and forecasting data costs challenging, making ecosystem management tools critical," the report said. What's more: "Today more than eight in 10 data management decision-makers (82 percent) cite forecasting and controlling costs as a data ecosystem challenge. Usage-based pricing models that coincide with cloud computing introduce month-to-month cost variability. What was once meticulously planned and budgeted on-premises, is now unpredictable. As data volume and usage expands, opacity impedes both general accounting and opportunities to drive cost efficiency. Without the ability to predict and manage cloud costs, decision-makers can find themselves quickly over budget."

Talent Shortages and Misaligned Tools Plague Companies and Exacerbate Challenges
"Talent challenges are the biggest barriers to data management excellence," the report said. "Decision-makers do not have enough staff, in-house expertise, or collaboration between teams and tools to achieve their outcomes." What's more: "Surveyed decision-makers cite talent challenges as the biggest barrier to overcoming obstacles, followed by issues with tools. The 'Great Resignation' has made keeping and maintaining staff difficult. As skilled employees seek new opportunities, internal expertise leaves with them. Additionally, obstacles to onboarding and collaboration between teams and tools commonly cause delays and inefficiencies.

"Decision-makers need tools that are aligned to meet evolving business requirements and cater to a variety of roles. Onboarding and collaboration challenges are worsened when tools lack a simple user interface for different skill sets and areas of responsibility, and are not designed to work together. Partners who can fill staff, coordination, and knowledge gaps, as well as make technology recommendations, are highly appealing."

In conclusion, Capital One said data management decision-makers need to address key challenges to understand what their organizations have, what they need and how they share and exchange data in order to realize business value while continuing in their cloud journeys.

"As organizations progress on their cloud journey, they must adjust how they balance global and local data use with a federated approach to ensure governance requirements are met and data access, quality, and observability is at its best," the report said. "Decision-makers must figure out how operating groups can work together in a seamless way and address skill gaps. Organizations can't hire themselves out of the skills shortage facing so many businesses. Vendors and partners can fill the expertise and best-practices gaps left by employees with automation and no-code/low-code self-service."