There are two cases, that I can think of, where you may have to consider whether to convert history in a data warehouse. They are:
1. Initial creation of the data warehouse. In the past, we have always entertained the feasibility of the conversion of history data even if the history data resides in the source system, a spreadsheet or another makeshift data warehouse. In some cases, the source system has gotten large and the historical data is not needed or used. So we consider bringing the historical data into the data warehouse that the business requirements dictate. You need to be careful here, as to not make the data warehouse your archival system. We only want the historical data that requires business usage!
2. When you are changing or revamping the data warehouse. In this instance, the business may have changed or you are incorporating enhanced data. The enhanced data may be something that is purchased (probably by marketing or sales), and we want to apply this data to historical data warehouse records.
So you have to ask yourself (and the business users) IS IT WORTH IT? If you google or bing "feasibility studies" it will give you good ideas on what you may want to include in your own feasibility study. Resources that are required to convert history are: people, hardware and software. Always consider giving the option of "START HISTORY FROM IMPLEMENTATION DATE"… this is the cheapest and easiest way to deal with history.
2 Comments
Does anybody have an opinion on what the time period should be before inactive customer data should be archived out of live production systems?
After what time period should customer data be considered inactive?
appreciate people's thoughts on this.
GREAT QUESTION! I think it is until the data is no longer relevant to drive business. For example, business changes pretty quickly in retail, so campaign information from 6 years ago is a bit worthless. Base it on the type of information, and how it is used in the enterprise.