One of the lesser-known effects of COVID-19 has been on analytical risk models. Around the world, the pandemic has created a new reality. The situation became so unpredictable that financial institutions’ risk models lost their ability to make accurate predictions. Past performance, it turned out, was very definitely not a guide to the future.

These models include credit risk models, financial risk models, and nonfinancial models – and they underpin much of the work of these institutions. Banks have scrambled to develop new overlay models. This has considerably increased the number of models they are managing, and therefore their model risk.

Model risk management and model governance implications

The number of models now handled by financial institutions has reached frightening proportions. A recent McKinsey report noted:

“A strategic vision for model risk management [MRM]: there is a need to move MRM to a new level defined by a meaningful collaboration between the first and second lines of defense … The MRM function must establish governance of overlays covering business-as-usual models and regulatory models.”

I wrote about model governance and model risk management in an article back in 2019. I commented that “a model risk management framework aims to guarantee the right level of control for banks to use on all models supporting corporate processes.” Following the pandemic-driven proliferation of models, this now seems even more important.

Financial institutions must be able to manage their model ecosystems efficiently and cost-effectively, even with the increasing numbers and complexity of models. They need different tools throughout the model life cycle. And they must be able to satisfy various stakeholder and regulatory requirements.

Increased use of artificial intelligence and machine learning models also requires a more rigorous governance process. This must be able to quickly identify when a model begins to fail and provide defined operating controls on inputs (data) and output (model results). Governance of machine learning models has been already addressed in a previous article – you can learn more here.

Model validation and monitoring automation

However, there is another new challenge. We need to consider how to improve the internal model validation process and model monitoring activities. Over the last year, these two topics have been at the heart of my conversations with financial institutions across Europe, the Middle East and Africa. The McKinsey report commented:

“Validation backlogs and delays mount as existing validation capacity fails to cover expanding demand. Inventory is increasing as new models are developed outside traditional areas of financial risk. The rapid development of AI is increasing model complexity and adding to the backlog.”

As a result, model validation quality is suffering. Financial institutions are therefore considering how to industrialise and automate model validation processes. It is not yet possible to replace the stakeholders involved in the internal validation process with an automated mechanism. However, it should be possible to improve and automate some of the tasks and routines required to validate a model, or for quarterly or annual review.

In particular, it is possible to automate repetitive analyses (e.g., back-testing, model benchmarking and data quality assessments). This will leave room and time to analyse the results from back-testing, or to build other models using new machine learning techniques. However, it may not be as simple as that.

What customers want

Customers tell me that they need more than simply to modify or customize existing processes. They also want procedures they can:

• Link and store with the model for which they were used.
• Track with version control.
• Document.
• Link and store with related procedures and models.
• Keep safely with the data used to run the validation analysis procedures to provide the replicability required by regulators.

These issues apply to many, if not most, financial institutions, regardless of the model.

Model monitoring and maintenance are also important and require feedback into model development. The pandemic has added to the need to monitor model performance to understand if they need to be retrained, retired or replaced. Model monitoring can be repetitive, and financial institutions are working to understand how to schedule and automate these tasks to deliver continuously updated model performance.

Model validation and monitoring automation: Myth or reality?

In the past, there has been considerable skepticism from financial institutions about whether the effort required for model validation and monitoring could be reduced. This was mainly because of the lack of proper control of internal processes, the proliferation of data preparation procedures and different versions of model procedures.

Today, however, things have changed. Many financial institutions now have model risk and model governance frameworks in place. The aim is to manage the design, development, testing and validation of models, both internally and externally. This includes determining whether a model fits the organisation’s definition, categorizing models, keeping model information in a model inventory and conducting periodic assessments of models. It is possible to extend these frameworks with automated model validation and monitoring.

This is not a silver bullet. However, it provides a way to standardize and industrialize some model validation and monitoring activities that have typically been done manually. It can therefore free up time for more complex tasks, saving both time and effort. Financial institutions are taking careful note.