To many organizations, analytics means building a model that reveals new knowledge from data. Unfortunately, that’s too narrow a definition. Now that advanced analytical models are becoming high-value organizational assets – essential tools to attract and retain customers in telco, manage financial risk in insurance and banking, and predict demand and set prices in retail – these models and their underlying data must be managed and governed like other strategic assets. This is rarely done, however.  For example, we see that organizations are struggling to:

• Keep track of model versioning. Analysts don’t just develop one model to solve a business problem. They develop a set of competing models and use different techniques to address complex problems. They will have models at various stages of development and models tailored for different product lines and business units. As a result, your organization can quickly find itself managing thousands of models.
• Do structured and rapid deployment of new models and data. The model and data environment is anything but static. Models will be continually re-deployed as they are tested and as new results and data sources become available. As such, the model deployment process is a much more iterative process than the traditional IT process of building applications.
• Embed models into decision processes and decisions into model development. Models and data are not of any use if they only feed reports and dashboards. The results of analytics should guide business decisions, and the results of those business decisions should be fed back into models and model development. In a distributed and loosely managed modeling environment, this is hard to achieve. When different data sets and variables are used to create the models and there is little validation or back testing results become inconsistent.

As a consequence of the above, managers must make decisions based on the model results they receive, and everyone hopes for the best.  To solve these and similar challenges in a systematic fashion, you will need to establish proper governance of the analytics life cycle, i.e. the processes and technology support to ensure that your organisation's operational use of analytical models sustains and extends strategies and objectives.

In practice, you should consider establishing:

• A model repository. A central, secure repository that stores extensive documentation about the model, its scoring code and associated metadata. Using the repository, modelers can easily collaborate and reuse model code, with their activities tracked via user/group authentication, version control and audit controls.
• Automated workflow. Enable the model management process to become more automated and collaborative. As an example, users should be able to track each step of a modeling project, from problem statement through development, deployment and retirement.
• Governance. Accountability metrics and version control status reports on who is allowed to change what, when control is passed from one area to another, and more. A centralized model repository, lifecycle templates and version control can provide visibility into analytical processes and ensure that they can be audited to comply with internal governance and external regulations.
• Validation. Scoring logic should be validated before models are put into production, using a systematic template and process to record each test the scoring engine goes through, to ensure the logic embedded in the champion model is sound.
• Performance monitoring. As the champion model reaches test, stage and production lifecycle milestones, its status and performance metrics should be pushed to subject matter experts through standard reporting channels to gauge a model’s fitness for the business question at hand.
• Deployment. Deployment options and procedures aligned with IT and outsourcing partners – in batch or real time.