At a recent data migration event, I met with several delegates who were amused at the thought of a zero-defect data migration. They both considered it technically unfeasible to deliver an entire migration without any defects occurring at run-time.

I asked how they dealt with the fall-out of problems on their last migrations. One said he used the age-old approach of assigning a dedicated mop-up team to fix any problems that crept through. The other delegate said he simply pushed the issues back to the business to resolve.

What strikes me as odd in these two situations is how anyone can perceive this as being a planned activity. It’s like saying, "We’re aiming to land a man on the moon successfully, but we’ll have an additional team up there at the ready because we’re bound to have endless issues that need fixing.”

I’m all for contingency planning during data migration, but you have to assume that your team is working toward - and is architecting for - a zero-defect migration.

It’s this second element that people generally misunderstand. Zero-defect migrations have to be designed. For example, if you carry out a data quality assessment in January but your migration is not until November, then you are guaranteed to find data defects that have been introduced in the ensuing months. This is why you have to architect a continuous data quality monitoring process to ensure that all of your data migration engines will fire and land the data on the target correctly.

All your transformation rules need to be mapped against known data quality rules so that those rules can be measured against the legacy data and the required transformation. This all has to be architected; it shouldn’t be guesswork at run-time.

Yes, you may get the odd alarm and there are occasionally hardware technology failures that are unforeseen, but in terms of data there should be no grey areas.

Zero-defect migrations should be the mantra, not a myth.