Paxata is a provider of self-service data preparation software and an important Waterline Data partner. We work closely with them, integrating our solutions to make it easier for organizations to find, understand and prepare data more quickly in the burgeoning hybrid, multi-cloud tech landscape. All the while, we’re also ensuring governance and compliance with many internal and governmental regulations like GDPR and CCPA.
DataOps is an important topic these days because, as Erfan’s article mentions, it represents a new focus on improving collaboration and accelerating service delivery through iterative practices.
DataOps: The Evolution Before the Revolution
Winston Royce is widely credited with introducing the Agile method for software development in a paper published in 1970 that still reverberates. 30 years later, “Agile” was used to describe the iterative method of engineering by a group of software developers in their Agile Manifesto. What they came up with frames the Agile methodologies that now help software organizations deliver higher quality products, faster response to change, reduced risk and faster ROI.
This then gave rise to DevOps, an extension of the Agile method. A core component of DevOps, automation, is also a key enabler of DataOps, a process-oriented methodology that extends many of the tenets of DevOps to data. While DevOps brings application development and IT together, DataOps unites the data analytic and data teams with IT operations for improved and faster time to deployment of data analytics.
DataOps: Putting More Data to Work Faster
In other words, the point of DataOps is to deliver data’s value to the organization faster. This is key for a couple of reasons. One, data can’t deliver value if it’s just sitting there. Two, data’s value rises with how fast you can get to it.
As we wrote in an earlier blog post, putting your data—all of your data—to use faster means making decisions faster. It means moving faster. Company A may have better data than Company B, but if Company B puts its data to work faster than Company A, it’s going to win out.
Speed of Business = Speed of Data
In his now famous letter to Amazon stockholders a couple of years ago, Jeff Bezos wrote what he considers the key elements for the continued success of the business. Among them: high-velocity decision making. Without it, companies die. Slower organizations may make higher quality decisions, but they’re too late to the party for it to matter. He’s right: quick decision making can trump better decision making as long as you can quickly identify mistakes and correct them. It’s okay to fail as long as you fail (and correct) faster than the competition.
Bezos also says in order to continue in a state of constant motion, sometimes you have to move with 70% of the information you need instead of waiting for the 90%. While he is, of course, right to follow in the steps of Voltaire and not let the best be the enemy of the good, 70% is a surprisingly high bar for many companies. Most decision makers that I talk to estimate the amount of information that they have available to use to be around 10-20% of what they ideally need. With DataOps supported by automated data cataloging among other automated technologies, I believe organizations can have both–not just fast access to data, but fast access to all of their data.
Data Cataloging – DataOps’ Key Enabler
While much of Erfan’s article concentrates on where DevOps and DataOps meet and differ, she too touches on how automated data cataloging supports DataOps. “In DataOps,” she writes, “Success necessitates a unified catalog of data assets and data preparation flows, along with versioning and monitoring the environment.”
I wholeheartedly agree. We’ve always maintained that you can’t have data driven decision making without an AI-driven data catalog solution like Waterline’s in place that helps you 1) locate, identify and tag data faster, 2) govern and secure data faster, 3) prep data for self service faster, and 4) put data to work for your business faster.
Want to learn more about how DataOps and AI-driven data cataloging can unlock your data operation? Be sure to check out the informative InsideBIGDATA article and read about Waterline’s unique cataloging solutions here.