26 July 2010

Published in: Risk governance, Regulation - supervision

The use test: a simple suggestion

Many are concerned about how the Solvency II's "use test" for internal models will be conducted. David Ingram has an elegant solution

As most of us know, the use test is required for an internal model to be allowed for determining capital requirements under Solvency II. That test is meant to determine whether the internal model is "widely used and plays an important role in" the business decisions and particularly the risk management of the insurer.

In the October 2009 paper, Tests and Standards for Internal Model Approval, (former consultation paper 56), the Committee of European Insurance and Occupational Pensions Supervisors (CEIOPS) lays out a foundation principle and nine general principles for the use test of internal models (see box).

To ensure these principles are followed, regulators will need to accomplish the very difficult process of discerning the difference between reports that "were in the room" when the decisions were made and "reports that drove" the decisions. And in the end, that is the only really important question. All the rest of the review process is window dressing around that key test.

But there is an alternative -- a simple and fairly foolproof test.

Focus on outcomes

The test would focus on two things - the outcomes of the risk management systems and management's understanding of those outcomes. This is significantly different from a focus on the processes or the content, or even the purported use of the internal models.

By focusing upon outcomes rather than process, the firms which are creating a system that looks like a risk management system but which is not actually driving any important decisions from that system can be eliminated from consideration for regulatory reliance upon their internal models.

By focusing upon outcomes, firms that are not competently managing their risks can also be eliminated from consideration. And finally, by focusing upon outcomes, the reviewers can avoid the lengthy scrutiny of the entire risk management system.

The focus upon management's understanding of the outcomes would take care of the reliance issue. And the reliance issue is at the center of the issue here. As CEIOPS puts it, "if an undertaking does not trust its model sufficiently to use it, why should the supervisory authority?"

A conversation with the regulator

This would all be accomplished by requiring top management (not risk management or modelling staff) to hold a conversation with the regulator about their risk profile each year in which management would explain:

• why the insurer is taking each of the major risks
• what the firm expects to get out of that risk exposure and
• how is it making sure that the potential losses that the firm experiences are not worse than represented by its risk model. This discussion should include recognition of gross risk before offsets, as well as net retained risk.

This proposed format for the use test assumes that everyone agrees that it is important for top management to understand the risks of their firm. And it is important that this conversation be held without the direct participation of the CRO or the modelling expert.

It goes without saying that the people responsible for the model, the people whose very jobs depend upon the model, will be able to tell the story. But if the reports were just "in the room", then top management will have a difficult time remembering what they mean. And the whole point of the modelling exercise is to get top management to look at their risk positions. There has always been a presumption that information about the risk profile will lead to better decision-making.

The second time management goes through this exercise with the regulator, the discussion would include an explanation of the reasons for the changes in the risk profile: did the profile change because the world shifted or did it change because of a deliberate decision on the part of management to take more or less of a risk or retain more or less of a risk?

Finally, a third part of the discussion would be to compare the actual experience of the past year with what was predicted by the model, and to determine the degree to which that experience caused the firm to recalibrate its view of each risk.

The first time that a firm's top management demonstrates that they can competently hold this conversation would not be the real use test.

The real use test

The real use test would be the second or third time. To pass the test, management would merely need to have a complete story that makes sense from year to year.

Now, some might doubt whether the regulator would necessarily have the ability to discern whether the story does make sense or not. But I would contend that the regulator who could not make that call would have a difficult time discerning whether or not the reports were actually being relied upon by management and the board or simply "in the room."

Those who fail the test would be those making large changes to their model calibration and their story from year to year, stretching to make it look like the model information was a part of management decisions.

The regulator would have the added advantage of hearing many of these stories. One firm might be able to pull off telling a story that is incredible, but the regulator will be hearing from a dozen firms which all lived through the same markets, risks and calamities in a year, so the credible stories will easily stand out from the incredible ones.

Some firms which might have passed the test before the crisis but which should have failed were those which in successive years told the same story of good intentions without taking action to reduce outsized risks.

No preparation time for this test will be needed by firms which are genuinely using their models. Their story for this test will be the story of their firm's financial management.

And when pillar 3 is being defined, the information from this conversation should be considered for inclusion in the public disclosures.

Satisfying the 10 criteria

This approach does, in fact, satisfy most of the 10 criteria laid out by CEIOPS.

Foundation principle: the undertaking's use of the internal model shall be sufficiently material to result in pressure to improve the quality of the internal model.

If the top management of the firm is held to a standard of being required to explain their results as described above, their personal involvement will ensure that the internal model will be of sufficient quality to provide them personally with the information to perform that task. If, as events unfold and the risk environment changes, the internal model starts to drift away from a tight match with the risks of the firm or the real risk environment, then they will personally and immediately realize that as they prepare to give their explanations, because these explanations will become strained and convoluted.

This personal involvement of top management in the internal model approval process might well be the best way to prove to the supervisors they have put their trust in their internal model.

Dave Ingram, CERA, FRM, PRM, is senior vice president, Willis Re, in New York

Back to top