Why model risk management is good for business

Insurers rely heavily on models to support many business decisions ranging from pricing to capital management, and including reserving, reinsurance, and asset valuation. How these models are managed to ensure that business decisions are supported appropriately is an important business challenge, which is only increasing as models grow in terms of scope and complexity.

Some insurance models, e.g., capital models, are subject to intense external scrutiny through regulatory model approval and oversight. The Prudential Regulation Authority (PRA) will, following the review of Solvency UK, be turning its attention to other models used in insurance.

For those insurers that have not formally considered model risk, this is likely to become a game changer impacting risk functions and model owners. The crucial ingredient to meet this challenge is encapsulated in the often-quoted statement from the statistician George Box that “all models are wrong, but some are useful” and that it is important to be “… alert to what is importantly wrong” by understanding where the model is right-enough to be useful: “It is inappropriate to be concerned about mice when there are tigers aboard.”

For insurers, this means there is a need to understand specific model risks within their business and prioritise appropriate remedial actions.

Model risk management as a regulatory concern

Last year the PRA published its requirements for model risk management for banks (SS1/23), setting out its expectations for model development, validation, and governance, as well as management’s understanding of complex models, see Figure 1 for a summary of the principles.

Figure 1: The PRA’s model risk management principles

Previously, the PRA primarily focused on model risk management in banks, with some attention given to models used by insurers for capital requirements and risk management. However, in their latest business plan, the PRA has shifted its focus for 2024 to address specific model drift within individual general insurance firms, with an emphasis on enhancing the effectiveness of internal model governance and validation. By doing so, the PRA aims to encourage firms to better recognise and resolve issues within their own models. It is therefore clear that insurers, not just banks, need to consider model risk from a regulatory perspective and now is the time for insurers to start developing this properly to get ahead of the inevitable PRA policies that we expect will arrive in the not so distant future..

The case for insurers’ early engagement with model risk

We think there are two main reasons for insurers to think early about model risk.

Model risk is material for insurers, and is growing

Solvency II prompted a significant investment in controls and governance for capital models and bolstered the use of such models through wider business understanding. As a result, capital models and the data that feed them typically go through a high degree of technical accuracy testing. However, such scrutiny is often not replicated in models that are crucial to the running of the business, including pricing, reserving, and internal rating models, even where such models feed the well documented capital model. In this context, model risk management is therefore both a technical and business challenge.

At the same time, the use of models and their complexity is increasing. 

• The growing use of machine learning in insurance creates its own set of issues with model reproducibility, understanding and challenge, and data ethics; particularly relevant in light of consumer duty rules, where such models affect consumer outcomes. 
• Climate metrics are moving from being a directional tool to be increasingly used in decision-making. The more this happens, the more important it is that boards understand the limitations of the climate models that create these metrics.

The main model risk exposure tends to arise from the model scope and how risks evolve. Current heightened inflation is a case in point where model risk arises from the limited inclusion of inflation in models built during the decades of low inflation. In these types of scenarios, one would expect effective model risk management to lead to a timely response and adjustment of models to reflect a changed inflationary environment.

There are several documented examples of insurers getting into difficulties because of crystallisations in model risk: 

• the Australian Securities and Exchange Commission has recently commenced litigation against two subsidiaries of IAG, alleging that IAG’s pricing algorithm has led to misleading customers. The court explicitly called out that robust systems and controls are needed, especially where complex pricing systems and algorithms are used 
• JP Morgan’s billion-dollar losses in 2012 were partly the result of an error prone risk modelling system, including volatility calculations that required data to be cut and pasted between spreadsheets, later compounded by traders concealing losses using the same flaws 
• a meta-analysis of insurer failures conducted by Canada’s Property and Casualty Insurance Compensation Corporation, found that insurers entering new lines of business regularly struggle to adequately price for risks owing to incorrect model assumptions, and that this is one of the leading causes of new insurer failures.

Developing proportionate, business-relevant solutions for model risk takes time

The biggest danger for firms seeking to enhance their approach to model risk is a failure to develop a proportionate approach to model risk, often by seeking to replicate the existing Solvency II Internal Model control environment across the business. An approach that fits the range of models a company uses requires tailoring, and a ranking of the models in question is key to ensuring that the controls are pitched at an appropriate level that encourages model development and innovation rather than stifling it.

Our experience is that when companies are under pressure to meet regulatory deadlines, it can lead to an ineffective ‘tick box’ type approach. While these approaches can ensure that regulatory expectations are met, they do not typically contribute to the effectiveness and efficiency of risk management, which in this case requires a proportionate and business-relevant solution.

Where to start with model risk management?

For firms to understand and manage their full universe of risks effectively, they need to be able to manage the risks created by using models to support decision-making. Doing so is, simply, good risk management practice.

The PRA’s model risk management Principles (see Figure 1) represents a useful structure within which to think about model risk. We would encourage insurers to consider as a first step the roles and responsibilities of business owners and risk function in respect of the firm’s overarching approach to model risk, to agree where responsibility sits for defining and challenging the approach vs implementing mitigation.

Developing an inventory of models that includes an appropriate risk ranking is a crucial next step to understand an organisation’s model environment and begin to determine where the firm should focus its attention on conducting full risk assessments of individual models.

A risk assessment of models might seem like an insurmountable challenge unless it is approached in a pragmatic and risk-ranked manner. Our experience of supporting insurers with their approach to model risk has identified internal and external drivers that represent both opportunities and challenges towards the management of models, set out in Figure 2.

Figure 2: internal and external opportunities and challenges of models

Models are an integral part of how insurance companies are run. Although their development and use require specialist knowledge, that does not mean they should be excluded from good management disciplines. Managing model risk makes good business sense.

Insurers have to date not felt the same regulatory pressure that banks have, however the PRA is clear in its business plan that it will now start to apply model risk principles to insurers. Establishing practices and processes early on can reduce the cost of applying them whilst ensuring better business outcomes, and ultimately help firms to get ahead of regulations.

Finally, and most importantly, better models yield better results! While it may be true that all models are wrong, it is inescapable that we use them to support or even take over business decision-making every single day, and ensuring decision makers know about the tigers, even if they miss a few mice, is essential to success.

How can Crowe help?

Crowe’s risk consulting team is experienced in risk management, actuarial modelling, and model approval. Our team has supported insurers with ensuring models are appropriately developed for new business lines, improving the efficiency and effectiveness of models, and supporting firms through the internal model approval process.

For more information contact Isaac Alfon and Lloyd Richards.