What constitutes a data migration success?
This sounds like an easy question to answer at first glance. A successful data migration is one where all the data was migrated. Easy, right?
Well of course we know that there is a heck of a lot more to it than this, but in actual fact many projects literally have no criteria for success other than “our data was moved from point A to point B.” They get into all kinds of trouble as a result.
I recounted a story some time ago on this blog of a healthcare organisation that paid off its contractors because they had migrated the data. Six months later the organisation subsequently found major data quality defects that posed a massive remediation challenge.
So we know that data quality is a criteria for success. But what exactly does that mean? Who defines the rules for data quality? How will they be tested for success? Who does the testing - supplier or internal staff? Do you have the skills to execute? What stages should you be testing for data quality?
Next, we have to think beyond the physical act of migration and consider the real end-goal: a fully operational target platform.
Our data has to meet the specifications of the target system interfaces, application logic and user requirements. So we need more rules and specifications to define the “right data.” Not always easy when we’re literally dealing with a moving target. You can seldom wait for application logic and data models to be firmed up; you have to crack on and start development.
Next, timing is everything. If we migrate all our data correctly but it takes 12 months longer than expected, is it really a success? Technically the data was migrated - but operationally it’s clearly a failure. So we need to factor in time to our success criteria.
So, can you see how complex this suddenly gets - but more importantly, how critical it is? The next time you’re putting together a data migration project plan, make sure you give more than a fleeting moment to the topic of data migration success criteria. Consider some of the factors above and seek expert opinion, because if you ignore your criteria for success then your project could quickly become another negative statistic.
3 Comments
Great reminder, Dylan, that migrating data is more than just "moving" them into a new technical environment.
Migrated data must have passed the same rules of integrity, completeness and correctness as if they were created directly through the new target system. If this cannot be guaranteed in the "batch" migration process, migrated data have to be flagged accordingly to warn "consuming" processes. Also, the new target system has to offer update functionality that allows to complete or correct migrated data (and eventually remove the flag).
Keeping the history of updates in the new system is as important as keeping the lineage of migrated data, particularly if they have been merged from different legacy sources.
Good tips Axel, thanks for sharing.
You're right, flagging migrated data is critical but even more so is, as you say, the need to apply future data quality rules against the legacy data to spot for gaps.
Thanks for contributing to the Roundtable, appreciated.
- Dylan
I am a Student in the Networking field. I have been giving the task in our final project to research a company that does data migration for medical offices in Las Vegas Nevada...Hard copies of Patient records, billing and ect...Can anyone point me in the right direction. Any and all help would be highly appreciated. I have found several companies, none of which are in Las Vegas.
Thank you,
Anthony
P.S.
Mr. Troike, My sons name is Axel and loves finding other people with the name Axel. He was excited to see your first name is Axel too.