When you examine where most data quality defects arise from, you soon realise that your source applications are a prime culprit.
You can argue that the sales team always enter incomplete address details, or the surgeons can't remember the correct patient type codes but in my experience the majority of issues stem from underlying flaws in application design.
The challenge is that business models are constantly evolving. One year Amazon is selling physical books and DVD's, the next year it's e-books and streaming video. You can bet they've had to adapt their information systems radically to cope with the huge shift in their business model.
And the same is happening in your business, all the time. It may not be as profound a shift as Amazon, but the shifts are there all the same, and they're putting pressure on the quality of your underlying data.
You can identify these issues through data profiling and data quality assessments. You can spot the trends over time, and it's these trends that are vital in helping you re-engineer the next generation of solutions.
In one telecommunications company, we found that a substantial amount of equipment information was being entered in the engineering notes field. When we asked why this happened, the engineers explained that the system was never designed to deal with the different types of equipment and more complex configurations. Following an earlier merger, the system had been struggling to cope with the added complexity, so they had resorted to storing valuable technical data in a simple comments field.
This type of "local fix" is a firm indication that your system design has shifted from its original intent. You need to gather up these insights because they will help shape the next generation of applications and you, the data quality practitioner, must play a crucial role in that procurement process. You need to outline what functionality is required to ensure compliance with data quality guidelines. If you're not sure what those guidelines are, start looking at your past defect log and all the issues you've resolved, that is your starting point.
Don't wait for applications to be designed or procured without your involvement. Make a case for having close involvement with the procurement process because every sensible design decision taken at this point can save huge amounts of data quality improvement work over the lifetime of the system.
What do you think? Should data quality practitioners play a greater role in the design and procurement of new systems? Welcome your views.
