DevOps Test Data Management Advances Along the Gartner® Hype Cycle™ for Agile and DevOps

Traditional test data management (TDM) has long been a critical factor in delivering high-quality, reliable systems and applications. The reason is simple: developing and testing software depends on teams having efficient access to production-like data. Yet over the past decade or so, a series of software development innovations and trends have paved the way for a vastly superior TDM process, known as DevOps test data management (DevOps TDM). 

DevOps TDM’s rise over the past few years has been a brisk one. Just four years ago, DevOps TDM wasn’t on a single Gartner® Hype Cycle™. In 2021, DevOps TDM appeared in five reports along the ‘Innovation Trigger’ — signaling awareness and enthusiasm for its technical and business benefits. Fast forward to 2023, and DevOps TDM has accelerated dramatically toward market maturity with an estimated time of 2 to 5 years.

This accelerated progression is exciting and Gartner lays out the story behind DevOps TDM’s existence and growth inside its report. I’d encourage everyone to read the full Gartner Hype Cycle for Agile and DevOps, 2023 (1) report. My three biggest takeaways are below.

1) Traditional TDM approaches are problematic and have the potential for greater risk

Too often, organizations still copy full production data sets to use in non-production environments without properly masking sensitive data, which increases the risk of data exposure and inappropriate access. Still others use artificially-generated data that doesn’t adequately represent all real-world edge cases found in production data, resulting in low quality software development.

Consider the real world case of Sky Italia, an Italian satellite TV platform. Prior to adopting DevOps TDM, Sky Italia employed a number of practices that limited its ability to produce software updates effectively. In a best case scenario, the firm would refresh its test data once per year. But at times the company would use stale data that was over four years old. In all cases, however, Sky Italia was using production data in non-production environments, exposing sensitive data to these unprotected environments. Their refresh processes were halting development and testing teams for 1-2 months, which caused them to take at least 6 months to deliver new business capabilities. A practice that’s troublesome for any business.

2) Modern Challenges Are Driving DevOps TDM’s Adoption

Test data management arose decades ago, when Waterfall development was the software development standard across industries and Agile was a pioneering concept. Traditional TDM doesn’t take modern challenges such as data privacy, increased development velocity, complex cloud models, or even DevOps itself into account. And that makes it even more inadequate and unsustainable for businesses competing today. DevOps TDM brings test data management into the 21st century by addressing these modern development challenges head on. 

DevOps’ increased rate of development: In a Waterfall model, software updates would be issued once every few months. Agile increased this output to about once per month— and DevOps increased it even further to once per week. When test data refreshes take 1-2 months to complete, they strain developers’ abilities to do their jobs well. But data virtualization shrinks refresh times, allowing testing to keep up with all other DevOps-enhanced processes.

Balance increased complexity with desire for agility: The multicloud has become a de facto standard for enterprises, with 89% of respondents surveyed in Flexera’s State of the Cloud Report possessing a multicloud strategy. However, multicloud environments are complex and require companies to navigate a variety of cloud operating models, and align their security and governance practices. This makes the management of data more complex, costly, and risky, and software development infinitely more challenging. DevOps TDM offers a cohesive, multicloud-ready approach that unifies test data processes across both apps and clouds.

Data privacy regulations: Today, over 70% of countries have data privacy regulations in place, according to the United Nations Conference on Trade and Development. Meanwhile, cyber attacks are more prevalent than ever— the Identity Theft Resource Center reported 1,802 data compromises in 2022, just 60 events below 2021’s all-time high of 1,862 events in one year. 

This has forced organizations to ensure that their test data is made compliant for regulators while also being protected from bad actors. This requires a degree of meticulousness that hadn’t previously been practiced in TDM. Keeping up with DevOps has led organizations to explore automating compliance— which has led masking to become an ideal solution.

3) DevOps TDM is Reaching Mainstream Adoption as Customers Achieve ROI

Jump back for a second to Sky Italia, which had been using legacy TDM practices for years prior to adopting DevOps TDM. In 2018, the 4,000-employee, $3.5 billion company faced a new challenge of needing to bring its non-production environments into GDPR compliance in a five month time span. This challenge led the company to adopt Delphix as a DevOps TDM solution— but the company discovered far more benefits than just achieving GDPR compliance.

Adopting Delphix allowed Sky Italia to reduce its infrastructure footprint by 90% and its operational costs by 30%. The company was also able to increase its release speed and quality, while also increasing the provisioning of its development and testing environments.

These feats line up with the benefits that other organizations have discovered by implementing DevOps TDM solutions. According to an IDC study of 10 large enterprises that had implemented DevOps TDM through the Delphix DevOps Data Platform, enterprises achieved an average five-year ROI of 560%.

It’s therefore no surprise that DevOps TDM has enjoyed the rapid rise that the Gartner Hype Cycle reports have documented over the past three years. DevOps TDM first landed on five separate Gartner Hype Cycle reports as a technology breakthrough, otherwise known as an Innovation Trigger, in 2021. At the time, DevOps TDM was expected to reach market maturation, known as the Plateau of Productivity, in 5-10 years.

In 2023, DevOps TDM appeared in the Hype Cycle for Agile and DevOps, Hype Cycle for Data Security (3), Hype Cycle for Operating Models (4), and the Hype Cycle for Software Engineering (5). 

By the following year, DevOps TDM had halved its time to market maturity to a mere 2-5 years. In addition, it had also moved forward to early adoption (known on the Gartner Hype Cycle as the Peak of Inflated Expectations) on four Hype Cycles. Fast-forward to 2023 and DevOps TDM landed on four Hype Cycles and progressed to the next phase of its technology lifecycle, the Trough of Disillusionment. Despite the gloomy name, this phase indicates that investment in the technology continues as successful solutions providers improve their products to customers’ satisfaction. Meanwhile, DevOps TDM continues to maintain its estimated time to market maturity of 2-5 years.

The Delphix Difference

As previously mentioned, continuing investment in a given technology as successful solutions providers improve their products to customers’ satisfaction is a hallmark of the Trough of Disillusionment, the phase in which DevOps TDM is currently situated. 

That’s precisely what we’ve been doing with our DevOps Data Platform at Delphix. We’re proud to have been named a Sample Vendor for DevOps TDM for the third year in a row in Gartner’s Hype Cycle reports. Throughout that time (and well before it), we’ve worked extensively with our customers to expand the functionality of our DevOps Data Platform and solve their needs more efficiently. 

Each year, we see more and more organizations investing in DevOps TDM solutions. Gartner currently estimates DevOps TDM market penetration to be about 20%-50% of its target audience (2). Our work at Delphix proves that this target audience spans across many industries, as we’ve helped organizations in healthcare, finance, manufacturing, consumer goods, telecommunications, education, and the public sector implement DevOps TDM solutions.

The future of DevOps TDM is a bright one, and we’re excited to watch the continued trajectory of DevOps TDM along the curve. The 2023 Gartner Hype Cycle for Agile & DevOps contains a lot of useful information for DevOps professionals. And if you find yourself ruminating on how your organization could benefit from faster, safer, and easier TDM, we’d love to chat. 

___

Sources:

(1) Hype Cycle for Agile and DevOps, Published 27 July 2023 • ID G00792717

(2) Figure 1: Hype Cycle for Agile and DevOps, Published 27 July 2023 • ID G00792717

(3) Hype Cycle for Data Security, Published 14 July 2023 • ID G00792194

(4) Hype Cycle for Operating Models, Published 25 July 2023 • ID G00791195

(5) Hype Cycle for Software Engineering, Published 1 August 2023 • ID G00791744

###

Gartner is a registered trademark of Gartner, Inc. and Hype Cycle is a registered trademark of Gartner, Inc. and/or its affiliates and is used herein with permission. All rights reserved.  

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.