So your data migration project completes. The new target system has a green light following exhaustive functionality testing, and the users get to work with the new applications and screen designs.

Two weeks later everyone and his dog is moaning about the new system and the quality of the data. “It worked just fine in the old system,” people cry, and fingers start pointing to the migration team as the likely culprits. “They must have messed it up in the transformation!” management cry.

The problem with any data migration is that it’s typically carried out for one reason: to make an entirely new system operational. This new system invariably creates new pressures and demands on the legacy data, which can often expose weaknesses in the underlying quality of data.

In a lot of system implementations there is a lack of real depth in the data analysis. The focus is weighted to what amazing new functions the target system will perform, particularly if it’s a custom off The shelf (COTS) product.

If you’re leading a data migration, it’s important that you don’t put the blinkers on and solely focus on delivering a shunt of data from point A to point B. The data still has to deliver against the requirements of target users, too. It’s not enough to say that there were no failures during migration so you can pack up; your team is in a unique position to help predict what will happen when it lands in that target environment.

First, you need to push back on the target developers or COTS vendor. What data quality levels will they need to support the functions required by the users? Armed with this knowledge you can start to assess which of the legacy data will need to be improved. For example, I’ve witnessed legacy attributes with missing information that was valid in the legacy but will cause failure in the target. An attribute may have a flag of "N/A" to denote "not available" in the legacy system, but the target may demand an attribute be complete with a populated value in the target, contradicting the legacy system.

Second, your data profiling needs to be incredibly rigorous and leverage the right tools. You’re effectively trying to simulate what will happen when you migrate data to the target functions of the new system. For this you need to build rules, ideally in your data quality toolset because ETL tools, for example, often lack the richness found in data quality software.

Finally, you need to report on the data quality levels found to all stakeholders concerned. A lot of profiling reports of legacy data are meaningless to management, for example, but if you create a report highlighting where the new system will fail then you’ll get attention.

When things go wrong it’s easy to blame the data migration team for failing to move high quality data into the new environment. Preempt this situation by performing rigorous upfront data quality assessments and find those functionality blackspots before it’s too late.