Agile Data Governance: Why Modern Data Challenges Require a New Approach to Governance

[jrineakter]

Long before starting data.world, I was enamored with creating data-driven cultures. We founded data.world to help the world adopt and improve data-driven decision-making and data literacy. I’ve seen firsthand how this substantially increases both corporate performance and democratization. And I’ve seen countless initiatives stall out or fail to launch, struggling with workflow between data producers and consumers, reuse and reproducibility, data literacy, security, and privacy and correctness concerns.

We need a new way of thinking about and running data analytics programs, starting with a modernized approach to data governance. This is the first in a series of blogs on what we call Agile Data Governance.

In Part One below we'll cover lessons learned from software history. Next we’ll publish Part Two, which profiles stakeholder types and explains the process at a high level. In Part Three we’ll explore how Agile Data Governance removes five key barriers to data-driven culture. And in Part Four, we’ll reveal the Principles of Agile Data Governance.

I sincerely hope you follow the entire series. But if you simply want the TL;DR, here it is:

Enterprises waste millions of dollars on failed data initiatives because they apply outdated thinking to new data problems. This results in overly-complex, rigid processes that benefit the few and make the rest of us less productive.

Agile Data Governance adapts the best practices of Agile and Open software development to data and analytics. It iteratively captures knowledge as data producers and consumers india whatsapp number data work together so that everyone can benefit.

We believe this methodology is the fastest route to true, repeatable return on data investment.

What we can learn from history
People have touted the advantages of being data-driven, the “big data revolution,” and ML/AI for years. Very few organizations make it to that promised land. Most end up stuck on a treadmill of building out new infrastructure or deploying the shiniest new self-serve BI or data science tool. But few big data initiatives earn adoption. This is because creating a data-driven culture is not a technology problem. It’s a people problem. Many before me have made this important point.

The data and analytics industry today reminds me a lot of the time before the dot-com crash. Back then every company thought they had to transform into a tech company. Today, it’s rare to meet a company that doesn’t, at some level, consider themselves a big data company.

Twenty-five years ago “tech-native” companies were eating the world, and every other company raced to get online. They hired teams of engineers, contractors, and consultants. Vendors rushed in to “support” them with all types of expensive technology they hyped as silver bullets. But most software projects went nowhere. According to a 1995 report from The Standish Group, 31% of projects were canceled before completion, while only 16% of projects were “completed on-time and on-budget.” These failures, wasting valuable time and money, were just as responsible for the dot-com crash as the hubris and mismanagement of the dot-coms themselves.