20 encounters of the information management kind – #6 Converting history! Does it make sense?

2

There are two cases, that I can think of, where you may have to consider whether to convert history in a data warehouse. They are:

1. Initial creation of the data warehouse. In the past, we have always entertained the feasibility of the conversion of history data even if the history data resides in the source system, a spreadsheet or another makeshift data warehouse. In some cases, the source system has gotten large and the historical data is not needed or used. So we consider bringing the historical data into the data warehouse that the business requirements dictate. You need to be careful here, as to not make the data warehouse your archival system. We only want the historical data that requires business usage!

2. When you are changing or revamping the data warehouse. In this instance, the business may have changed or you are incorporating enhanced data. The enhanced data may be something that is purchased (probably by marketing or sales), and we want to apply this data to historical data warehouse records.

So you have to ask yourself (and the business users) IS IT WORTH IT? If you google or bing "feasibility studies" it will give you good ideas on what you may want to include in your own feasibility study. Resources that are required to convert history are: people, hardware and software. Always consider giving the option of "START HISTORY FROM IMPLEMENTATION DATE"… this is the cheapest and easiest way to deal with history.

Share

About Author

Joyce Norris-Montanari

President of DBTech Solutions, Inc

Joyce Norris-Montanari, CBIP-CDMP, is president of DBTech Solutions, Inc. Joyce advises clients on all aspects of architectural integration, business intelligence and data management. Joyce advises clients about technology, including tools like ETL, profiling, database, quality and metadata. Joyce speaks frequently at data warehouse conferences and is a contributor to several trade publications. She co-authored Data Warehousing and E-Business (Wiley & Sons) with William H. Inmon and others. Joyce has managed and implemented data integrations, data warehouses and operational data stores in industries like education, pharmaceutical, restaurants, telecommunications, government, health care, financial, oil and gas, insurance, research and development and retail. She can be reached at jmontanari@earthlink.net.

2 Comments

  1. Does anybody have an opinion on what the time period should be before inactive customer data should be archived out of live production systems?

    After what time period should customer data be considered inactive?

    appreciate people's thoughts on this.

    • GREAT QUESTION! I think it is until the data is no longer relevant to drive business. For example, business changes pretty quickly in retail, so campaign information from 6 years ago is a bit worthless. Base it on the type of information, and how it is used in the enterprise.

Leave A Reply

Back to Top