The next data governance challenge: Agility

the next data governance challenge is agility By now, odds are that you've heard the story of how Target used data to predict customer pregnancy with astonishing accuracy. Writing for The New York Times, Charles Duhigg caused quite the stir by reporting that a Target statistician:

was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a “pregnancy prediction” score. More important, he could also estimate her due date to within a small window, so Target could send coupons timed to very specific stages of her pregnancy.

And the company did just that. As the story goes, Target knew that a 16-year-old woman was pregnant before her own father did.

To be sure, few stories illustrate the power of data and analytics as vividly as Duhigg's account. By no accounts did Target do anything illegal. Its management simply wanted to market products of interest to its customers as effectively as possible. When viewed through the lens of a cut-throat retail environment, actions such as these aren't simply optional; they're usually required, especially for publicly traded companies.

Still, it's no overstatement to call the Target story a PR fiasco. Privacy advocates screamed bloody murder – and this was before the company copped to a massive data security breach. Put differently, Target served as a cautionary tale of how the power of data and analytics can collide with social media. Target may not have done anything illegal per se, but the story left a downright icky feeling among many.

Events such as these are becoming increasingly common. Case in point: Nextdoor, the free private social network for your neighborhood. It turns out that its users are engaging in racial profiling even though the company has never sanctioned it.

Lessons

The law of unintended consequences is alive and well. Make no mistake: Employees who hold formal data governance responsibilities in their organizations should pay close attention here. After all, these types of events fall into a gray area. They lie somewhere in between that which is clearly legal/acceptable and that which is illegal/unacceptable.

For instance, consider the first end of the spectrum. In my experience, most organizations take potential violations of Health Insurance Portability and Accountability Act (HIPAA) very seriously. In Europe, privacy regulations far exceed those in the United States. On the other end, no one can rightly criticize Amazon for recommending a Stephen King book because similar customers bought the latest from Dean Koontz.

But what about those thorny areas in between? Increasingly, organizations will have to make those types of judgment calls around data and analytics. Sure, a particular insight or data source may be valuable, but what happens when it may very well attract unanticipated and possibly catastrophic baggage? This is precisely the question that data governance bodies will need to address.

Simon says: Data governance professionals will need to be more agile than ever.

Expect more situations like Target and Nextdoor in the future. We live in an era of big data, social media, smartphones and APIs. As such, there's never been as great a tension between data governance and long-term planning.