20 encounters of the information management kind – #7 Data conversion strategies is where quality counts

0

I don’t believe you can commit to the success of a data conversion without addressing quality (or lack thereof). Do you agree? If so, then why are there so many conversion projects that just move data from one place to another? Let me give you an example. I had a client who did this HUGE database conversion from flat files to an Oracle database (yes, the flat files had seven years of history). Here is the chain of events:

  1. The business requirements were gathered, and they basically stated that the business users wanted the same reports they had now. Mistake #1: it is usually setting up a project to promise the same level of reports. Instead, offer an enhancement using words like "drill thru," "drill across," etc. This sets expectations that the new reports are BETTER, AND THEY SHOULD BE!
  2. ETL platform chosen
  3. Data model for new environment created with referential integrity
  4. Project plan created
  5. Resources acquired
  6. Kick-off planned

During the requirements gathering and the data modeling, no one really addressed the quality of the data. No profiling, and no quick SQL queries to check out the data. This cost time during implementation. Here's why:

  1. As soon as some data didn’t load, they dropped the referential integrity and loaded garbage
  2. The garbage had to be filtered out for the reports
  3. The data models had to reflect the lack of database enforced referential integrity

Planning for quality is definitely in my top 10!

Share

About Author

Joyce Norris-Montanari

President of DBTech Solutions, Inc

Joyce Norris-Montanari, CBIP-CDMP, is president of DBTech Solutions, Inc. Joyce advises clients on all aspects of architectural integration, business intelligence and data management. Joyce advises clients about technology, including tools like ETL, profiling, database, quality and metadata. Joyce speaks frequently at data warehouse conferences and is a contributor to several trade publications. She co-authored Data Warehousing and E-Business (Wiley & Sons) with William H. Inmon and others. Joyce has managed and implemented data integrations, data warehouses and operational data stores in industries like education, pharmaceutical, restaurants, telecommunications, government, health care, financial, oil and gas, insurance, research and development and retail. She can be reached at jmontanari@earthlink.net.

Leave A Reply

Back to Top