I want to create a new series on The Data Roundtable that focuses on providing practical tips for improving data quality.

Consistency is a key dimension often referred to in data quality practitioner circles. This refers to a rule where data must be consistent between two locations. For example, if a share price for Google is $1,069.87 in one part of your business, you would expect it to be the same in every other location.

In this article, however, I am not referring to that type of consistency.

I want to talk instead about the consistency experienced by a customer who buys your products or uses your services. When they experience poor consistency, they’re not bothered about data quality dimensions. They just want consistent quality of service.

When I was starting out with data quality 20 years ago, I noticed our service delivery quality was variable. We were inconsistently hitting our targets. Sometimes we’d surpass them, other times we were nowhere near.

Our clients felt those inconsistencies and became frustrated, so we looked to the root cause. At first it was difficult to spot because one thing I’ve discovered is there are no single points of failure. You might find a faulty piece of code, but what is the root cause? Was the developer trained in the appropriate standard? Was the testing team rigorous enough? Did the testing team and developer have the right awareness of data quality?

I prefer the term "cluster cause analysis" because it more accurately reflects reality. There are a cluster of variables, all with different weights, impacting the success or failure of a process.

Eventually I found that one of the primary causes of inconsistency in our service quality was due to lack of education and general awareness, particularly amongst our data entry workers. Some of the workers were experienced and took great care, whereas new recruits were simply trying to crank through the work quickly to impress their supervisors.

So here is something you can do to improve data quality today: provide greater awareness and education of how the data your knowledge workers are creating or maintaining will flow through the organisation. Listen to their problems and frustrations, then create simple training that fits their work schedule.

We then went one step further. We had standardised on having a username, sequence number and timestamp associated with each record. This helped us see who created or updated the data.

We then created simple reports that showed the quality of data across each member of the team each week. Data quality levels started to increase almost immediately. It really was that simple.

The key here is that you’re trying to eliminate some of the issues around an entire department, cluster cause or information chain. This is the key to improving consistency. Focusing on one individual or one data source or one bug will not give you the levels of consistency that your customers demand.

In Summary

Here is a recap of the techniques described in this article:

1. When you’re experiencing inconsistent levels of service, examine the cluster causes of poor data quality.

2. Provide awareness and education to anyone who is responsible for creating or maintaining the data (particularly knowledge workers such as data entry staff).

3. Measure the quality level across the team, but give each member visibility of their own performance.

Put these steps into action and let me know how things progress using the comments below.