Here on the Data Roundtable we've discussed many topics such as root-cause analysis, continual improvement and defect prevention. Every organization must focus on these disciplines to create long-term value from data quality improvement instead of some fleeting benefit.

Nowhere is this more important than the need for an appropriate education strategy, both in relation to data quality and the underlying systems and local policies. It is an area that often gets ignored in the quest for technological advances, so I wanted to recount a simple story that outlines the importance of factoring education into your root-cause discovery.

In a small retail research organisation, we performed a data quality assessment that uncovered clusters of data quality issues around a particular set of products. The data would appear to improve for a few weeks and then suddenly drop in quality for no obvious reason.

We were baffled until we started to enrich and extend the dataset by linking the data to the analyst who had created the records. Finally, we discovered that one person was responsible for the cluster of issues observed.

The data analyst in question was new to the role, having joined three months earlier. When interviewed, she explained how she had struggled with the assigned task. It was quite a complex activity, sourcing data from several external data feeds, merging them with manually entered data and coding a final cleanup process.

While the data had passed her basic checks, it had failed our further upstream data quality assessment. The team discussed ideas for preventing a repeat of the issue and it became clear that a principal failure was a lack of appropriate education:

• There was a data quality firewall policy that hadn't been explained or taught in a formal way; this would have ensured greater quality checks around the external data feeds
• The analyst did not adequately study an in-depth guide outlining the interface to the target system that her data was expected to conform to
• There was an internal policy to double-enter all manually entered information, the analyst had not been taught this process so simply entered the information herself, creating a number of errors

What this emphasised was the need to go deeper with root-cause analysis and explore the wider behavioural issues at play such as:

• Why didn't the supervisor provide the right level of training?
• Why didn't the analyst feel like they could ask for support?
• Why weren't local policies and procedures being followed?

Following the exercise, some simple improvements were made such as:

• Posting a full list of all the procedures and processes on a chart in the main staff areas
• Performing regular reviews with staff (particularly with new hires) to discuss any concerns
• Providing improved data quality training linked directly to relevant policies and tasks so staff could understand the bigger picture of how their work impacted the organisation

We could, of course, merely focused on cleaning up the data. However, by looking deeper we were able to establish not only technological but social improvements. As a bonus, these simple educational tweaks benefitted morale as well as operational performance. Once you start to impact people's lives for the good, that is when you really begin to see the growth of data quality in your organisation.

What educational strategies have you put in place to ensure data quality improvement in your organisation? Welcome your views below.