Upload
dodi-mulyadi
View
92
Download
0
Embed Size (px)
Citation preview
© 2010 Deloitte
Modelling in Solvency II:from regulation to practice
November 2010
© 2010 Deloitte
Solvency II Processes: What are we talking about?
As depicted hereunder, the Solvency II value chain is built upon
complex processes and information flows:
Life:
Contracts and
claims
Non life / Health:
Contracts and
claims
Asset Data
Accounting Data
Data sourcesData
CollectionCalculation
Conso. and
aggreg.Reporting
SFCR &
RTS Group
SFCR &
RTS Solo
Ris
k C
lass a
gg
reg
ato
r and
co
nso
lid
atio
n
Market
Risk
Life Insurance
Risk
Non-life and
Health Risk
Counterparty
Default Risk
Operational
Risk
Economic
Scenario
Generation
Cash Flow
projection Life
Cash Flow
projection
Non-life /
Health
Cash Flow
projection
Assets
Extr
act Tra
nsfo
rm a
nd
Lo
ad
Co
here
nce c
ontr
ol, v
alid
atio
n and
sig
n o
ff
Valid
atio
n a
nd
sig
n o
ff
Monitor and Control
Results:
SII Balance
sheet, Capital
Requirements
Operational
Losses
Counterparty
information
Life
Liabilities
Model Point
Assets
Model Point
Data
Preparation
Experience
analysis and
assumptions
setting
External Market
Data
Co
here
nce c
ontr
ol, v
alid
atio
n and
sig
n o
ff
MVL & Tech
Provisions Life
MVL & Tech
Provisions Non
life and Health
Asset
Pricing
Internal
Reports
2 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
5 options can be envisaged which will influence the level of complexity and the
accuracy of the implemented SCR calculations.
What are the Solvency II options for the calculation of
the SCR?
Simplified calculation in the Standard
formula
Standard
Formula
Standard
Formula using own
data to
calibrate the parameters
in the risk modules
Partial
Internal Model
Internal
Model
Complexity,
Accuracy
Complexity,
Accuracy
1
2
3
4
5
And of course, don’t forget Technical Provisions …
3 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
77%
76%
74%
71%
71%
71%
71%
64%
64%
62%
60%
56%
54%
54%
53%
49%
48%
47%
45%
44%
44%
43%
41%
37%
37%
36%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Liquidity
Regulatory/compliance
Market
Credit
Legal
Budgeting/financial
Tax
Mortality
Liability management
Privacy
Morbidity
Business continuity/IT security
Country/sovereign risk
Fraud
Property and casualty
Reputation
Strategic
Operational
Lapse
Data integrity
Vendor/service provider
Human resource
Model
Systemic
Catastrophe
Geopolitical
How effective do you think your organization is in managing
each of the following types of risks?Percent responding extremely or very effective
7th Edition Global Risk Management Survey Review, Draft Report
4 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Avatars of model risk
There are many aspects to model risk. For instance:
• The model design, and the choices made in representing reality, do not correspond
to the objectives of the model. I.e. some essential features of the economic reality or
of the contracts modelled, are not captured.‒ Pricing model for mortgages that ignores prepayment risk; valuing a life insurance company while ignoring
the cost of guarantees, correlation effects in defaults,…
• Somewhere along the lines of the mathematical developments, a mistake was
made. Describing the setting correctly is no guarantee for a correct solution …‒ Using results that only hold for a normal distribution in credit or operational risk modelling
• A model is used for a purpose it was not designed for. For instance, a model may
only work for a specific range of market parameters, but is applied outside that range‒ A model developed for valuing a complex financial instrument for accounting purposes, where there is a
materiality concept and where timeliness is key, may not be the best guide in setting up the hedge, since
it is probably a simplified model.
• Absence of accurate and adequate input data leads to non-sensical output‒ Correlations, length of the liquid market curve, unlisted equity, historical loss data, mortality trends,…
• The model’s implementation in a software environment, does not correspond to the
specifications‒ Numerical instability e.g. Excel has its limits
‒ Software bugs, version control, use of libraries from vendors,…
5 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Agenda
Some ideas on model governance2
Adding value through model validation3
Solvency II Internal model requirements1
Model documentation4
6 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
How will internal models be approved?
Internal model tests (principles-based approach)
To gain sign-off an internal model needs to pass six tests which need to be repeated when
the model is changed to enable on-going appropriateness of the model. Where the
application for that approval relates to a partial internal model, the requirements shall be
adapted to take account of the limited scope of the application of the model.
The tests explicitly refers to economic capital under Pillar I.Use test
(Article 120)
Demonstrate that the internal model is widely used and plays an important role in system governance , in particular in the risk-management system en decision-making processes, the economic and Solvency Capital assessment and allocation processes, including ORSA
Demonstrate that internal model complies with adequate actuarial and statistical techniques and data quality requirements. Statistical qualityVerify it assesses all material risks the company is exposed to and the mitigation actions for these risks taking into account standardspolicyholders and management actions using realistic assumptions.
Calibration Demonstrate calibration details of the internal model and verify the reconciliation to regulatory standard i.e. the level of
standards protection within 1 year being at 99.5% confidence interval.
Profit and loss Demonstrate that the causes and sources of profit and losses for each major business unit are reviewed at least annually attribution and verify how categorisation of risks chosen in the model explains the causes of profits and losses.
Validation Demonstrate that there is a regular cycle of model validation that includes monitoring performance, appropriateness of standards specification and testing results against experience.
Document the design and operational details of the internal model to provide a detailed outline of theory, assumptions and Documentation
mathematically and empirical basis underlying the model and indicate any circumstances where the model does not work standardseffectively.
(Article 121)
(Article 122)
(Article 123)
(Article 124)
(Article 125)
7 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Internal Model
Insurers have the choice of:
• Standard model
• Partial internal model
• Full internal model
Internal
Model
Risk Strategy
• Risk appetite
• Understanding risk impacts
• Accountability
Information
• Data
• Assumptions
• Technology
Management
• Decision Structure
• Remuneration Policy
• Responsibilities
Decision Making
• Pricing
• Capital allocation
• Reinsurance
• Planning
• Interaction and
integration
‒ Reserving
‒ Pricing
‒ Planning
‒ Management information
• Parameters
‒ Frequency
‒ Interaction and
consistency
‒ Ownership
‒ Sign-off
8 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Internal model
Requirements for Approval
Approvalcriteria
Use test
ModelGovernance
Statistical quality
standards
Calibrationstandards
Profit and loss statement
Validation standards
Documentation standards
Is the internal model
adequately documented?
Is the internal model widely accepted within
the organisation and does it capture the
fundamental risks?
Are the quality of the data,
assumptions and methods
used sufficient?
Does the internal model
produce the same level
of prudence as the
standard formula?Can the realised gains and losses be explained
using the internal model?
Does the internal model
produce the proper
results?
Is there an ongoing feedback
loop between the administrative
and management board (AMB)
and the risk management
function?
Solvency II Directive, Articles 120 - 126
CEIOPS Advice on Level 2 Implementing measures (former CP 56)9 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Use Test
General Principles
Theinternal model
…
… is well understood by
senior management
… is appropriate in the business
model
… covers risk sufficiently to be
useful in risk management and decision making
… is widely integrated into
the risk management
system
... is integrated into the risk
management system on a
consistent basis for all uses
… is used to support and
verify decision making
… is used to calculate SCR
… is used to improve the risk
management system
… is designed to facilitate the analysis of business decisions
Is the internal model widely accepted within
the organisation and does it capture the
fundamental risks?
Solvency II Directive, Articles 120
CEIOPS Advice on Level 2 Implementing
measures (former CP 56)
10 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Use Test
Some examples on the use of the internal model
Capital
Risk
appetite
Strategy
• Business strategy
should be formulated
and implemented in
daily operations.
• The risk strategy has a
direct impact on capital
requirements
• The risk appetite should be determined and be related to the strategy and solvency position. The
appetite depends on available capital and the willingness to take more risks in exchange for a
higher expected return.
• The risk appetite depends on the risk-taking capacity in terms of both capital and organisation
• Volatility of capital is
required to withstand
unexpected losses. The
volatility depends on the
adopted risk management
strategy.
• A high risk profile requires
more capital, with the
possibility of a higher return
on capital. An insurer should
understand their risk
appetite and the resulting
capital requirements.
An important aspect of the use test is the interaction between strategy, capital
and risk appetite
Solvency II Directive, Articles 120
CEIOPS Advice on Level 2 Implementing measures (former CP 56)
11 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
In which areas do you plan to use your internal model as
part of the decision-making process for Solvency II?
87%
87%
80%
67%
67%
60%
60%
53%
47%
40%
33%
33%
20%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Risk-based performance reporting
Capital management and planning
Management information, e.g., providing information on how the risk position compares …
Decisions on asset mix strategy and the possible effects of investment decisions
Strategy and planning, e.g., as an input to planning and strategy by providing an …
Analysis, design or purchase of reinsurance
Pricing of business
Assessment of the risks, value and impact to the business of potential mergers, …
Product development
Prioritization of risk management activity
Purchase of hedging assets or changes to existing hedges
Excess surplus investigations (with profit funds)
Executive compensation
7th Edition Global Risk Management Survey Review, Draft Report
12 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Use Test
Some examples on the use of the internal model
Governance systemRisk management
systemDecision Making
Capital Assessment
and Allocation
• Connection between
the internal model
and technical
provisions
• Connection between
the internal model
output and internal
and external
reporting
• Connection between
the internal model
and the technical
implementation of
management actions
• Measurement of
material risks
• ALM
• External risk
reporting
• Internal risk reporting
• Draft reinsurance
program
• Development of risk
strategies
• Efficient use of
Capital
• Investment Decisions
• Identifying targets for
return on capital and
rewards
• Product development
and pricing
• Business planning
and strategy
• Company policy for
taking on risk
• Capital management
• ORSA
• Calculation of SCR
• Economic capital
allocation for each
entity, business line,
risk or business unit
• Solvency capital
allocation by entity,
business line, risk or
business unit
Solvency II Directive, Articles 120
CEIOPS Advice on Level 2 Implementing measures (former CP 56)
13 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
The Statistical Quality Standards philosophy: an internal model is not just a
black box or an expert with good predictive power for the probability
distribution forecast.
Statistical Quality StandardsAre the quality of the
data, assumptions and
methods used sufficient?
Potential discussion areas:
• Probability distribution forecast (does a probability distribution forecast always has to
consist of a full distribution, i.e. whether every quantile must be known?)
• The use of expert judgement related to data.
• Materiality (circumstances under which risks which are in the scope of the internal
model must actually be modelled in the internal model?)
• Aggregation in internal models
The methods used to calculate the probability distribution forecast shall be based
on:
• Adequate actuarial and statistical techniques
• Consistency of calculation methods used for the probability distribution forecast and
technical provisions
• Current and credible information
• Justification of underlying assumptions
Solvency II Directive, Articles 121
CEIOPS Advice on Level 2 Implementing measures (former CP 56)
14 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Statistical Quality StandardsAre the quality of the
data, assumptions and
methods used sufficient?
• Risk ranking & model coverage‒ Relative importance of risks, comparables, relative classification
‒ Link with use test: coverage, resolution, consistency,...
• Recognition of diversification effects‒ What drives dependence? Is there diversification gain?
‒ Account for extreme scenarios and tail dependence
‒ Part of business decisions?
• Recognition of risk mitigation‒ Economic view, but legal effective/enforceable?
‒ How incorporated in model?
• Financial Guarantees and options‒ Identify and document
‒ Influencing factors
‒ Coherence with TP
• Future management actions
15 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
The Calibration Standards aim to assess whether the SCR derived from the
internal model has the appropriate level of prudence.
Calibration StandardsDoes the internal model
produce the same level of
prudence as the standard
formula?
Article 122.1
Insurance and reinsurance undertakings may use a different time period or risk
measure than that set out in Article 101(3) for internal modelling purposes as long as the
outputs of the internal model can be used by those undertakings to calculate the
Solvency Capital Requirement in a manner that provides policy holders and
beneficiaries with a level of protection equivalent to that set out in Article 101.
Article 101(3)
The Solvency Capital Requirement shall be calibrated so as to ensure that all
quantifiable risks to which an insurance or reinsurance undertaking is exposed are taken
into account. It shall cover existing business, as well as the new business expected to be
written over the following 12 months. With respect to existing business, it shall cover only
unexpected losses.
It shall correspond to the Value-at-Risk of the basic own funds of an insurance or
reinsurance undertaking subject to a confidence level of 99,5 % over a one-year period.
Solvency II Directive, Articles 122
CEIOPS Advice on Level 2 Implementing measures (former CP 56)
16 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Calibration Standards
Different time period and risk measure:
• If the time period used is different from the one set out in Article 101:
‒ Demonstrate that the time effects of the risks are taken into account
‒ Demonstrate that all significant risks over a one-year-period are properly managed
‒ Give special attention to the choice of the data used
‒ Justify the choice of time horizon in view of the average duration of the liabilities of the
undertaking, of the business model and of the uncertainties associated with too far time horizons
• Consider the practical implications if the undertaking uses different time periods and/or risk
measures within the same internal model, especially when aggregating the capital for the different
risks.
• When taking this approach, the undertaking shall show that the level of protection provided by
the SCR is equivalent to that set out in Article 101(3), and specifically that the approach taken to
aggregate the risks is appropriate.
Equivalent protection of policy holders:
• If the SCR cannot be derived directly from the probability distribution:
‒ Explain how risks are rescaled and justify that the bias introduced when doing so is immaterial.
‒ Explain the shortcuts used to reconcile the outputs of its internal model with the distribution of
the Basic Own Funds, if any.
‒ If considering a longer time horizon than that set out in Article 101(3), show due consideration of
the solvency position at the earlier time horizons.
‒ If considering a different time horizon than that set out in Article 101(3), justify the particular
assumptions made in order to properly take into account the dependencies between
consecutive time steps.
Does the internal model
produce the same level of
prudence as the standard
formula?
Solvency II Directive, Articles 122
CEIOPS Advice on Level 2 Implementing measures (former CP 56)
17 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
The Profit and Loss attribution aims to demonstrate how the categorisation
of risk chosen in the model explains the causes and sources of profits and
losses.
Profit and loss attributionCan the realised gains and
losses be explained using the
internal model?
Profit and loss attribution and the Use test
• The P&L attribution will give the undertaking information relating to the risk profile of the
undertaking, and therefore this information is also used in the ORSA.
• The P&L attribution may also provide an unbiased view on the risks of the portfolio to better
understand the portfolio exposures and assess whether the risk management framework is
appropriate.
• The results of the P&L attribution can also be used for other internal purposes such as
budgeting, forecasting, reinsurance-program testing.
Form of profit to be taken
• The variable may differ from basic own funds, because a different internal definition may be used
for economic capital resources.
• Be aware what causes differences in the profits and losses used in the Profit and loss attribution
and the profits and losses reported in the accounting systems.
Categorisation of risks
• The economic capital requirements resulting from the internal model shall lead directly to a
categorisation of all material risks. The qualitative assessment of non-material risks or non-
quantifiable risks completes the categorisation of risks based upon the internal model results.
Solvency II Directive, Articles 123
CEIOPS Advice on Level 2 Implementing measures (former CP 56)
18 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Agenda
Some ideas on model governance2
Adding value through model validation3
Solvency II Internal model requirements1
Model documentation4
19 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Internal Model Governance under SOLVENCY II regulationThe administrative or management body (high level internal model governance):
• Approving the application for approval to use the internal model to calculate the SCR, and the application for approval
for major changes or extensions to the model.
• Deciding roles and responsibilities for the internal model governance.
• Deciding on the strategic direction of the model and hence any changes to the model.
• Agreeing major changes in advance of the change being made.
• Aligning the model design and operations with the undertaking’s risk profile and operations.
• Ensuring there are sufficient resources to develop, monitor and maintain the model.
• Monitoring on-going compliance with the requirements for internal model approval, and informing the
supervisory authorities if the model ceases to comply.
• Ensuring there are adequate independent review procedures in place around the internal model design, operation and
validation.
• Ensuring that outputs are aligned with use – i.e. that the management information produced by the model assists in
decisions made at Board level.
• If the internal model ceases to comply with the requirements for approval, the administrative or management body
must ensure that a plan to restore compliance is developed or assess the non-compliance as immaterial.
The risk management function (detailed internal model governance):
• Design and implementation of the internal model.
• Testing and validation of the internal model.
• Documentation of the internal model and any changes to it.
• Analysing the performance of the internal model, and reporting on the performance to the high-level
governance, including compliance with the internal model approval requirements.
• Suggesting areas for improvement and reporting on the status of efforts to improve previously identified weaknesses
to the high level governance.
• Liaise closely with users of the outputs of the internal model.
• Develop a communication loop with the actuarial function to pass the detailed actuarial perspective to the risk
management function and in return receive the insights on the internal model.
Ongoin
g feedback lo
op
Undertakings may use external validation
review, however the ultimate responsibility
for signing off the appropriate validation
processes shall fall on the board, and may
not be delegated to any third party.
The administrative,
management or
supervisory body may
set up an internal
control committee
20 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Conceptual considerations
• Model Governance should support the
achievement of at least two objectives:
‒ The orderly development and usage of models
‒ The containment of model risk in the organisation
• Some criteria that can be used to prioritise the
need for model governance in the universe of
models discussed before:
‒ Regulatory expectations (explicit and implied)
‒ Stakeholder expectations
• Counterparts like reinsurers, dealing rooms,...
• Credit rating agencies
• Clients (e.g. Investment models)
‒ Risks for the group
• Financial risks; e.g. due to mispricing,
mishedging
• Reputational risks
• Materiality concept to be incorporated!
‒ The assessment of model risk, as hinted at in the
previous section
‒ Cost of installing a rigorous process vs. the
expected benefitsFDIC, Supervisory Insights, Winter 2005, Vol. 2, Issue 2.
21 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
A bit more pragmatic thinking
• Any fair value or risk model is composed of three
ingredients:
1. A mathematical recipe, the model
2. Position data/contract data
3. Model data (e.g. market data such as interest rates; lapse
rates; correlations, behavioural parameters,...)
Similar set-up seems possible for most models.
• Of these three ingredients, only the mathematical part
can be legitimately be claimed to require dedicated
expert skills.
• Different departments have different skills/roles relevant
for each of the above ingredients:
‒ Risk department often is a knowledge centre for
mathematical modelling within the organisation (so, it can
act as internal consultant, even if it is not the
“owner”/”responsible” for the model)
‒ Under Solvency II, the risk department gets the model
validation role explicitly
‒ Setting of all sorts of parameters is a management decision,
and should remain their ultimate responsibility
‒ The gold copy of any contractual data will mostly be the
financial accounts (since audited, and rigorous internal
control procedures mostly apply to them)
Fair Value Or
Risk Measure
Contract data Market & behavioural data
A mathematical recipe
Reasonably easily
verifiable
Least accessible to broad user base
Ease of control depends on
market or risk type
A tentative role for Finance Department:
• In Risk Models
‒ Ensuring accurate and complete
capture of exposure data
‒ Back-testing any performance
predictions coming from the risk
models
• In Fair Value models
‒ Validate consistency of parameters
across the group
‒ Understand sensitivity w.r.t. to
parameters
22 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Conceptual considerations
• It would be best practice to ensure that there is
always a 4-eyes principle applied. The rigour with
which this is applied should be function of the
regulatory expectations, the risk the model presents
to the company,...
• In terms of controlling model risk, there are at least
two control layers:
‒ MODEL VALIDATION: model validation is a formal
process, that challenges the assumptions, mathematical
algorithm, the implementation in a tool of a model.
‒ MODEL REVIEW: a set of periodic activities, not
necessarily requiring in-depth mathematical skills, but
that are confronting the “predictions” of the model with
reality. E.g. realised losses or earnings over the first
quarter, prices realised in actual transactions, excess
losses beyond the VaR amount,... Such controls can
provide indications that the model is not reliable.
• The question of scope. Which models will you include
in such a formal process?
Often performed by professionals close to
the business, like performance
controllers,….
Requires in-depth technical modelling
skills, so mostly concentrated in an expert
centre, often within the risk department
23 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
An example: ALM model
Potential allocation of responsibilities related to ALM modelling and ALM
management
What Who Performs Who Controls (second line of defense)
Development of the model Risk departmentIndependent validation by a model validation team or
external third party
Running the model (including
setting of model data and
uploading contract data)
Balance sheet management/ALM
department
• Model data subject to challenging validation of the
Risk department (formal sign off)
• Position data subject to formal reconciliation
process (documented checklist) at each run,
possibly in collaboration with finance department
Periodically reviewed by internal audit
Analysis of the results and
suggestions for management
action
Balance sheet management/ALM
department
• Formally reviewed by Risk department before
being presented to ALCO/MT Finance
Analysis of new balance sheet
management
proposals/instruments
Balance sheet management/ALM
departmentValidation and challenge by Risk department
Back testing of earnings
forecasts by reconciliation with
realised
Finance department (financial
controlling)NA
24 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Agenda
Some ideas on model governance2
Adding value through model validation3
Solvency II Internal model requirements1
Model documentation4
25 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
In designing the internal model, the Level 1 Text allows a large amount of freedom. Two examples of
this are given below:a. Article 121(4) sets out that “No particular method for the calculation of the probability distribution forecast shall be
prescribed.”
b. Article 122(1) sets out that “Insurance and reinsurance undertakings may use a different time period or risk
measure than that set out in Article 101(3) for internal modelling purposes as long as the outputs of the internal
model can be used by those undertakings to calculate the Solvency Capital Requirement in a manner that provides
policy holders and beneficiaries with a level of protection equivalent to that set out in Article 101.”
In addition to being used for the undertaking’s own risk management purposes, the internal model is
also used to calculate the regulatory capital. Materially misstating the regulatory capital, especially
holding regulatory capital that is not high enough, will result in a decrease in the level of the policy holder
protection provided by the undertaking.
In addition, as the internal model should also be widely used in and play an important role in the system
of governance, any material misestimation within the internal model will affect not only policy holder
protection, but also the whole risk management and decision making processes of the undertaking.
Therefore, supervisory authorities will require undertakings to take appropriate steps to validate
that the internal model is appropriate for the calculation of regulatory capital to ensure that the
level of regulatory capital is not materially misstated so as to decrease the level of the policy
holder protection, for use within the undertaking’s risk management and decision making processes.
It is the undertaking which has the primary role in this validation process, not the supervisory
authority. The validation process ≠ the approval process, in respect of which the supervisory authority
needs to take a decision. As part of the approval process, the supervisor will need to evaluate the
validation processes which the undertaking has in place.
Introduction to the validation cycle
26 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Model validation function
Model validation is a key activity to help make sure that models function as
intended, both when they are implemented and over time. Ongoing monitoring
and validation of risk management models are important in order to assess a
model’s sensitivity to structural changes and to changes in parameters and
assumptions. Fifty-nine percent of institutions reported having a model
validation function, an increase from 53 percent in 2008 . Larger
institutions were more likely to have a model validation function, with 79
percent institutions with more than $100 billion in assets have such a
function, up from 66% in 2008.
Model validation is most often placed in an independent risk management
function. Among institutions with a model validation function, 65 percent reported
that this model validation resides in independent risk management, while 19
percent placed it in Internal Audit and 8 percent in Actuarial. Larger institutions
were even more likely to have risk management handle model validation. Among
institutions with more than $100 billion in assets, 77 percent said that model
validation was placed in independent risk management.
7th Edition Global Risk Management Survey Review, Draft Report
27 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
The validation policy which sets out the way in which undertakings will validate
their own internal model and why that way is appropriate.
How to demonstrate compliance: The validation policy shall set out at least the following items:
• Purpose and Scope of Validation: the areas of the internal model that need to be validated shall
include at least:‒ Data
‒ Methods
‒ Assumptions
‒ Expert judgement
‒ Documentation
‒ Systems/IT
‒ Model governance
‒ Use test
• Tools used: may be a mathematically well defined test, a qualitative judgement or any other
process designed to gain comfort that the internal model is appropriate and reliable. ‒ Some validation tools will need to be used by all undertakings using an internal model to calculate the SCR:
• Testing results against experience
• Testing the robustness of the internal model (shall include at least sensitivity testing)
• Stress and scenario testing
• Profit and loss attribution
‒ Examples of what further validation tools undertakings may want to use:• Benchmarking
• Analysis of change
• Hypothetical portfolio
• Qualitative reviews
The validation policy
Modelling in Solvency II: from regulation to practice28
© 2010 Deloitte
• Frequency of validation process
• Governance of validation results‒ The risk management function shall be tasked with the validation of the internal model.
‒ Nevertheless, certain parts of the validation process may be carried out by other parts of the undertaking, as
long as there are clear lines of reporting and the risk management function retains overall responsibility for the
validation process.
‒ Set out how the senior management is involved in the validation processes
‒ Consideration needs to be given to what is validated at group level and what is validated at the related
undertaking level
• Limitations and future developments: expect that the model and its associated validation
processes will constantly develop as the risk profile of the undertaking develops and as new
validation tools become available
• Documentation of the validation policy
• Independent review: set out how independent review, external or internal, is being used within the
validation process
The validation policy (cont.)
Modelling in Solvency II: from regulation to practice29
© 2010 Deloitte
Internal Model Change Policy
Model Change Policy: The development of a model change policy is a requirement on
all undertakings applying for approval to use the internal model to calculate the SCR.
This is part of the application and must be approved by the supervisory authorities.
Model changes
• Major changes to the internal model are subject to prior supervisory approval
• The amended internal model should not be used to calculate the SCR until approval is granted
• Undertakings may further assign model changes to the following subcategories:
‒ The calculation kernel of the internal model
‒ The risk management
‒ The internal model governance
‒ The existing and regulatory approved internal model change policy of the undertaking
‒ Other aspects of the internal model
The policy for model changes shall contain an indication of the undertaking internal governance for
changes to the model (e.g. internal approval of changes, escalation path, internal communication,
documentation and validation of changes, etc.).
30 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Model risk and an extra capital buffer under ORSA?
CEIOPS Level 2: Technical Provisions – Article 86 h Simplified methods and techniques to
calculate technical provisions
3.109 Therefore the undertaking shall assess the model error that results from the use of a given
valuation method, having regard to the nature, scale and complexity of the underlying risks The
valuation method should be regarded as proportionate if the model error is expected to be non-
material.
3.110 For this purpose the undertaking should define a concept on materiality which should lay down
the criteria on basis of which a decision on the materiality of a potential misstatement of technical
provisions is made. This materiality concept and should be reflected in the undertaking’s own risk and
solvency assessment (ORSA).
3.112 An assessment of the model error may be carried out, by:
• Sensitivity analysis in the framework of the applied model: this means to vary the parameters
and/or the data thereby observing the range where a best estimate might be located.
• Comparison with the results of other methods: applying different methods gives insight in
potential model errors. These methods would not necessarily need to be more complex.
• Descriptive statistics: in some cases the applied model allows the derivation of descriptive
statistics on the estimation error contained in the estimation. Such information may assist in
quantitatively describing the sources of uncertainty.
• Back-testing: comparing the results of the estimation against experience may help to identify
systemic deviations which are due to deficiencies in the modelling.
31 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Making model validation concrete and tangible
32 Modelling in Solvency II: from regulation to practice
• The following areas related to an internal model need to be
considered and addressed in the context of a model validation:
• The level II CEIOPS also suggests the following validation tools:
Use TestExpert judgment
AssumptionsModel
Governance
Methods Systems/IT
Data Documentation
MUST CONSIDER MAY CONSIDER
Testing against experience Benchmarking
Robustness Analysis of change
Stress & Scenario Testing Hypothetical portfolios
P&L attribution Qualitative review
© 2010 Deloitte
Model Review Questions to address
Reading documentation of the
ESG, and assessment of some
content questions
• Is the objective of what needs to be modelled clearly stated?
• Does the documentation justify the choice of approach?
• Are the strengths and weaknesses of the chosen approach explained?
• Is part of the documentation readable by an interested, non-technical user,
and will it allow that person to assess strengths and weaknesses of the
approach? I.e. roughly speaking, there would be three levels of
documentation, one for non-technical users, one providing the full
mathematical detail, and one describing the implementation in a tool and the
process to be used to run the model.
• Is the calibration process well-described (market data to be used,
formulae,...) both theoretically and operationally?
• How is the choice for other asset classes (real estate, shares) modelled
based on the interest rate model, justified? How are correlations accounted
for/incorporated?
Review the mathematical
model
• Check all formulae, either by reference to the literature (include in review file)
or by recomputation (document in file). This includes the stochastic
simulation engine, the closed form formulae e.g. used for calibration, the
numerical discretisation schemes,...
Acceptability of the model for
the stated aim
• Benchmarking with approach/model chosen in the literature, by peers or in
vendor packages. Collect and store benchmark information.
• How was the choice of model argued in the past? Who approved its use?
Has it been subject to review/challenge by (senior) users?
• Where is the ESG used in practice in the organisation (as compared to
stated objectives)? Is this acceptable given its characteristics? (USE test)
ESG Validation Workplan along the above lines (1/4)
33 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Input Testing Questions to address
Input Data • Review of the procedures used to extract input parameters (e.g. implied or
historical volatilities, zero coupon curve, risk premia) from market data.
• Review of interpretation and acceptance controls of such data when
received from vendors.
Stability of the calibration • Hull-White type models are calibrated using a minimization on a set of
chosen derivatives for which the price is known (swaptions, caplets).
Impact of choosing a different set of instruments? Impact of choosing a
different seed point for the optimisation?
• How is correlation between stochastic processes linked to observable
parameter of correlation between asset classes?
ESG Validation Workplan along the above lines (2/4)
34 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Computational Process Questions to address
Review of the code • Does the code correspond to what was documented in the model?
• Have pre-coded subroutines been tested/assessed or is there general
documentation available? (e.g. random number generation for a given
distribution, numerical minimizers)
Assessment of the numerical
procedures
• Impact of the discretisation, no creation of fictitious value ("1=1" test,
martingale test)?
• Is the starting yield curve found back as E[Product(exp(-r.dt))]?
• Are the original parameters such as volatilities and correlations that one
calibrated again, found back in the runs?
• Stability for extreme values of parameters (inverted curve, high volatility,
no mean reversion/strong mean reversion, …)?
• Sensitivity relative to the parametrisation/interpolation/extrapolation for the
yield curve to which the mode is calibrated?
• Comparison of the simulated value of an instrument with the closed form
formula (in the chose model) if available?
• Convergence enhancing tricks, such as control variates or anti-thetic
variables.
• Numerical stability of the simulated variables in particular for longer time
horizons or extreme environments (high or low rates, ...)?
• Estimation error in function of the number of simulation runs (e.g. for
known instruments)?
ESG Validation Workplan along the above lines (3/4)
35 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Outcome Analysis Questions to address
Back Test • Review the implemented back testing procedures. If incomplete, design
and possibly perform some back testing procedures: shape of the
distributions for rates that are generated e.g. the 10y swap rate in 20years;
standard deviation of these distributions,…
Sensitivity analysis • Controlled assessment of the impact on the output of changing input
parameters. The output here should primarily be interpreted as being the
value or risk measures that one wants to compute with the ESG. This step
may therefore require considering the ESG and the applications that rely
on it, jointly. The purpose is to indentify and separate the important from
the less important input parameters, in order to focus the calibration and
expert judgement controls.
Use of scenarios • How are the generated scenarios stored? Are the labels of the data stored
aligned with their mathematical meaning and definition?
ESG Validation Workplan along the above lines (4/4)
36 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Agenda
Some ideas on model governance2
Adding value through model validation3
Solvency II Internal model requirements1
Model documentation4
37 Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
The Documentation standards aim to instill a
culture of good documentation praxis leading to
all model developments being documented.
38
Documentation
standards
The documentation shall demonstrate compliance with the internal model tests.• Detailed outline of the theory, assumptions, and mathematical and empirical basis underlying the internal model.
• Describe the drawbacks and weaknesses of the model, including the circumstances under which the model does
not work effectively, such as:‒ Any specific features of the internal model or circumstances or limitations that present potential concerns or would significantly
increase the uncertainty of the results of the internal model beyond what would reasonably be expected. For instance:
• Limitations in risk modelling and the cover of risk captured
• The nature, degree and sources of uncertainty surrounding the results of the internal model and sensitivity to key assumptions
• Shortcoming and/or deficiencies in input data
‒ Insufficiencies in IT-systems, governance and related controls surrounding the internal model.
• Tailored for key bodies and key personnel
• Include evidence such as training that all levels and functions of management, for example the board, senior
management, and the internal audit, of the undertaking understand the relevant aspects of the internal model
• Include a list of all documents that the undertaking considers relevant to the internal model, and where and how
these documents can be accessed.
• Identify those responsible for pulling together and/or updating documents.
The documentation does not have to be one single document, provided there is a list or a mapping
process that brings it all together.
The documentation of an internal model shall be thorough, sufficiently detailed and sufficiently complete
to satisfy the criterion that an independent knowledgeable third party could form a sound judgment as
to the reliability of the internal model and the compliance with Articles 120 to 126 and could understand
the reasoning and the underlying design and operational details of the internal model.
Modelling in Solvency II: from regulation to practice
© 2010 Deloitte
Documentation: Qualitative Questions from the QIS 5
39 Modelling in Solvency II: from regulation to practice
1. Model description and overview
General Model Governance:
2. Policies, controls and procedures for the management of the internal model
3. Model change policy
4. Evidence of Use Test
5. Training Manual for Management and staff training in use and understanding of
Internal Model
Model Changes:
6. Record of Major and Minor changes of the model
Technological Specifications:
7. Description of the Information Technology platform(s) used in the internal model
8. Description of Contingency plans relating to the technology platform(s) used
9. User guide
10. Source code
© 2010 Deloitte
Documentation: Qualitative Questions from the QIS 5
40 Modelling in Solvency II: from regulation to practice
Data:
11. Data policy
12. Documentation evidencing or justifying the accuracy, completeness and
appropriateness of the data
13. Data directory
Statistical Quality Standards:
14. Detailed description of Internal Model Methodology
15. Description of underlying assumptions
Expert Judgement:
16. Description of where Expert Judgement is applied in the model
17. Validation of Expert Judgement as applied in the model
Calibration:
18. If your model output uses a risk measure other than 1-year VaR at 99.5%, do you
have documentation evidencing or verifying that the chosen risk measure is at least as
strong as 1 year VaR at 99.5%?
© 2010 Deloitte
Documentation: Qualitative Questions from the QIS 5
41 Modelling in Solvency II: from regulation to practice
Profit and Loss Attribution:
19. Results of Profit and Loss Attribution
Validation:
20. Description and report/results of Validation Tests
Scope:
21. Partial Internal Model Scope
22. Qualitative and Quantitative indicators for the coverage of risk
Other:
23. Description of risk mitigation techniques accounted for in the internal model
24. Description of Future Management Actions accounted for in the internal model
25. Description of known internal model shortcomings / weaknesses, including
circumstances under which the internal model does not work effectively
© 2010 Deloitte
Our contact details
Email: [email protected]
Phone: +32 2 800 20 45
Yves Dehogne
Audit Partner
Partner, Bedrijfsrevisoren
Email: [email protected]
Phone: + 32 2 800 24 73
Arno De Groote
Director, Enterprise Risk Services
Director, Actuarial & Financial Risk Advisory
Email: [email protected]
Phone: + 32 2 800 21 46
Dirk Vlaminckx
Audit Director
Director, Bedrijfsrevisoren
Email: [email protected]
Phone: + 32 2 800 24 94
Christophe Vandeweghe
Manager, Enterprise Risk
Services
Manager, Actuarial & Financial Risk Advisory
42 Modelling in Solvency II: from regulation to practice
About Deloitte
Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private
company limited by guarantee, and its network of member firms, each of which is a
legally separate and independent entity. Please see www.deloitte.com/about
detailed description of the legal structure of Deloitte Touche Tohmatsu Limited and
its member firms.
Deloitte provides audit, tax, consulting, and financial advisory services to public
and private clients spanning multiple industries. With a globally connected network
of member firms in more than 140 countries, Deloitte brings world-class
capabilities and deep local expertise to help clients succeed wherever they
operate. Deloitte's approximately 170,000 professionals are committed to
becoming the standard of excellence.
Deloitte in Belgium
A leading audit and consulting practice in Belgium, Deloitte offers value added
services in audit, accounting, tax, consulting and financial advisory.
In Belgium, Deloitte has more than 2,240 employees in over 12 offices across the
country, serving national and international companies, from small and middle-sized
enterprises, to public sector and non-profit organisations. The turnover reached 317
million euros in the financial year 2009.
The Belgian practice is part of the international firm Deloitte Touche Tohmatsu
Limited. For the fiscal year ended 31 May 2009, DDTL realised a turnover of
US$26.1 billion worldwide, with approximately 170,000 members of staff in more
than 140 countries.
This document is confidential and prepared solely for your information.
Therefore you should not, without our prior written consent, refer to or use our
name or this document for any other purpose, disclose them or refer to them
in any prospectus or other document, or make them available or communicate
them to any other party. No other party is entitled to rely on our document for
any purpose whatsoever and thus we accept no liability to any other party who
is shown or gains access to this document.