How to re-balance a data migration project plan


How do you structure your data migration plan? How do you allocate resources and timescales so that you maximise your chance of success? In this article I want to explore some data migration project planning misconceptions and give you some pointers for an alternative approach.

One of the big improvements with data migration in recent years has been the splitting out of data migration as a distinct project. Historically, data migration was seen as a bit player in the target application implementation project, so it was just bundled into the planning for the major initiative.

Today, projects treat data migration as a distinct process that is best managed separately but feeds into the goals of the broader program.

This obviously means that you get a separate team of project management staff to help coordinate the flow and governance of the project. That’s the good news. The bad news is that they often try to approach data migration as just another software project.

Unfortunately, that is the path to failure.

The typical software development approach to data migration goes something like this:

Start > Analyse > Design > Build > Test > Execute > Validate > Close

More and more resources are added as each phase progresses. On some projects I've witnessed 30+ developers by the build phase, which is frankly insane from a project management perspective. You just can’t coordinate that level of development and expect a quality result.

What you end up with is a huge imbalance in your project plan with a small amount of resource and activity at the start and a huge amount of frenetic energy at the end.

Obviously, this is not ideal. Instead, I prefer something like the following approach:

Impact Assessment > Start > Discover > Analyse > Prototype > Release > Repeat > Close

I believe the days of the huge two-year ‘big-bang’ migration project are numbered. You simply can't hit a moving target two years off. Even 12-month projects are incredibly difficult to plan for at the outset.

I believe it is far better to front-load your project with highly specialized technology, seasoned migration experts and solid deep business engagement to create a more agile series of iterative migration milestones.

The impact assessment before the project start is critical because you simply don’t know what will lie ahead. You need activities such as data discovery, scoping and prototyping to be taking place before project initiation so you know what you’re dealing with and whether the project is even feasible.

When the project kicks off for real my preference is for a very small team of highly skilled practitioners, as opposed to a cast of thousands. I've found that migration projects don't scale by adding more developers; the converse is actually true, in that it makes life far more difficult to coordinate and quality issues creep in.

Another key requirement for this leaner approach is to use the right technology. "Hand cranking" migration platforms using custom scripts and whatever homegrown tools the IT team have to hand is a recipe for disaster. You need to use the right profiling, discovery, data quality, data transformation and data validation tools to have any chance of delivering an agile, results focused migration.

So the lesson is a simple one. Swap tail end panic for front-end discovery, prototyping and iterative release.

But what do you think? Welcome your views in the comments below.


About Author

Dylan Jones

Founder, Data Quality Pro and Data Migration Pro

Dylan Jones is the founder of Data Quality Pro and Data Migration Pro, popular online communities that provide a range of practical resources and support to their respective professions. Dylan has an extensive information management background and is a prolific publisher of expert articles and tutorials on all manner of data related initiatives.


  1. With that big diverse of data technologies being around with a lot of strict requirements there is no chance any off the shelf tool would do the job of migration.
    It always goes for the first requirement a good knowing and thinking on what you are doing and going to achieve. This will lead you to planning and milestones. Using tools in a way that fit the job to do. No not going for tools finding a job to do (hammer/screw).

  2. Thanks for your comments Jaap.

    Yes it's a good point, some source and target technologies e.g. Mainframe will require specialist approaches but the key is to apply the Impact Assessment at the start, that will help uncover what type of migration you're dealing with and what the optimal migration architecture needs to be.

    Thanks again - Dylan

  3. Pingback: Fresh Links Sundae – October 19, 2014 Edition from David Lowe of Disciplined IT and Actionable ITSM | Disciplined IT

Leave A Reply

Back to Top