I attended a recent data quality conference in the UK and had the following question posed to me by a delegate:
“We’ve just analysed a large section of our core business data and found major data quality issues for the first time. Our question is – where do we go from here?”
At first this question is akin to “how long is a piece of string,” but that is hardly constructive so let’s try and focus on some obvious starting points and feel free to jump in with your own personal experiences or advice.
The first question to answer is “so what?”
This might seem counter-intuitive given the delegate has clearly found some major data quality issues, but what struck me was the “first time” element of our conversation. This tells me that the business is probably coping with “what lies beneath” in their data.
I’ve never, ever been into an organisation and performed a data quality assessment that did not uncover some serious data quality issues. The problem is that over the years companies have developed “coping strategies” for working with bad data. More call centres. More truck rolls. More overtime. More bureaucracy.
Every organisation has data quality issues to some extent, but these are masked by wasteful processes that are in actual fact simply termed “business as usual.”
So the first challenge you face is to solve the “so what?” question.
To achieve this I find it useful to look outside the box. How do the data quality issues you’ve witnessed impact customer services? Revenue? Costs? Risk? Lead time? Bottlenecks? Churn?
If your business has coped with poor data quality for many years, then simply providing them with a stack of profiling results is going to fail the “so what?” challenge. Link the bad data to something your peers and leaders care deeply about.
Ideally, you need to drastically de-scope the data landscape and focus on one particular issue, the one that causes the most amount of grievance to your leaders.
The second question to answer is “what next?”
Now that you’ve got people’s attention, the obvious question you will be faced with is “great...so what do we do next?”
At this point you may start to consume endless books, articles, podcasts, guides and product tutorials in an effort to understand the “what next” dilemma.
This is incredibly difficult, particularly for organisations at the low end of the data quality maturity lifecycle. You can be mistaken for thinking that you have to implement governance councils and frameworks, technologies and resources.
Eventually these are admirable but you’re just not there yet and all of these will terrify your leaders.
The key to answering this question is to fully understand:
- How did our data arrive at its present condition?
- What are the implications for ignoring it?
- What are the costs and options for addressing it?
This all seems obvious, right? But how many times have you seen companies skipping 1) and even 2)?
The beauty of addressing the “how did our data get like this?” issue is that you can clearly demonstrate the likelihood of it happening again.
This is critical.
A lot of managers see data as historical, created by some past transaction that has limited value to the business. However, once you can demonstrate not only the scale of past defects but the likelihood of future defects (by understanding how defects are created), your case becomes much stronger.
Armed with this knowledge, you are far more able to understand the implications for ignoring poor data quality and the various options for resolving it.
With this knowledge you can then work out if the costs of prevention (and possible cleanup) are worthwhile.
This is important because many business cases I’ve seen for data quality focus on the past and ignore the future. Don’t make that mistake. Demonstrate that poor data quality is not some historical relic but a real, living problem that will impact the organisation well into the future (unless you act).
Over to you. If you were posed with the question “we’ve got data quality issues – now what?” what would your response be? There are no hard and fast rules so welcome your views.
