OpRisk Scenario Analysis

8/2/2019 OpRisk Scenario Analysis

1/30

Using Scenario Analysis to estimate

Operational Risk Capital

James ElderDirectorCredit Suisse First Boston([email protected])

OpRisk Europe, March 2004

The author makes no representation as to the accuracy or completeness of the information provided. The views expressed hereare those of the author, and do not necessarily represent those of Credit Suisse First Boston or Credit Suisse Group.


2/30

2Operational Risk

Overview

Background

The fundamental challenges in measuring OpRisk?

The 4 elements of the AMA Practical implementation issues The irrelevance of small losses on the capital charge

Correlation assumptions

Implementing a scenario-based capital estimation approach

Conclusion


3/30

3Operational Risk

Background


4/30 4Operational Risk

A1. Types of OpRisk

Goal of OpRisk management is to reduce the frequency & severity of large, rare events

Not Relevant(Otherwise would already be out of business!)

MINOR events(Secondary challenge)

Generally not firm threatening Experience makes it easier to understand

problems, to measure issues & to takerelevant action

Can often be incorporated into pricing - costof doing business (eg. credit card fraudlosses)

Generally generates efficiency savings ratherthan reduce material risks

HighFrequency

MAJOR events

(Primary challenge) Can put banks (eg. Barings) out of business orseverely harm reputation

Difficult to understand and prioritize in advance Similar to issues faced in several other industries:

aviation, healthcare, railways, chemical

processing

Doesnt matter much

L

owFrequenc

y

Large LossesSmall Losses


5/305

Operational Risk

A2. Swiss cheese model Major OpRisk events

Swiss cheese analogy holes exist inall systems

Risk of accidents can be mitigated bydeveloping effective defenses-in-depth

Successive layers of protection each designed

to protect against the possible breakdown ofthe one in front

Defensive control layers try to minimizeoccurrence of large organizationalaccidents

Ideal ControlEnvironment

Real ControlEnvironment

Risks

Potentiallosses

Risks

Defences

Major OpRisk events moreunlikely as they require

alignment of holes insuccessive control layers

e.g. bad person; flawed systems;poor management; weak

controls, on a bad day . . .

Some holesfrom active

failures

Some holes

due to latentconditionsLosses

Source: Swiss cheese model (Adapted from Reason, 1997)


6/306

Operational Risk

The fundamental challenges in measuring

OpRisk


7/30

7Operational Risk

B1. Position equivalence - Lack of a quantifiable size

Both market and credit risk exposures are typically explicit & normally accepted as aresult of a discrete trading decision

Risk taking decision depends on the ability to measure the risk of a transaction relative to its expected profitability

Credit risk: Exposures can be measured as money lent, mark-to-market exposure of potential exposure. The risk can beestimated using credit ratings, market-based models and other tools

Market risk: Positions can be decomposed into risk sensitivities and exposures. The risk can be quantified withscenarios, value-at-risk models, etc.

Generally, market prices are observable/quoted & frequent

In both market and credit risk there is a direct linkage to the driver of risk, the size ofthe position, and the level of risk exposure

These risk models allow the user to predict the potential impact on the firm for different risk positions in various marketmovements

OpRisk is normally an implicit event it is accepted as part of being in business, ratherthan part of any particular transaction

No market prices observed/quoted; Relevant data infrequent

No inherent size for the OpRisk inherent in any transaction, system, or process Eg. how much rogue trader risk does a bank have? How much fraud risk? How much could a bank lose from

implementing a new IT system incorrectly? Has the risk grown since yesterday?

The equivalent position for OpRisk is difficult to identify


8/30

8Operational Risk

B2. Completeness What is the portfolio?

there are known knowns there are things that we know that we

know. There are unknowns that is to say, there are things that we

now know we dont know. But there are also unknown unknowns

there are things we do not know we dont know.Donald Rumsfeld quote

For both market and credit risk, modelling starts with a known portfolio of risks Usually a key test of a banks risk management systems and processes to ensure that there is complete risk capture

In OpRisk modelling, the portfolio of risks is not available with any reasonable degreeof certainty

Even if a bank knows its processes and could ascertain the size of the risk in those processes, it is difficult to identifyunknown risks or non-process types risks

Many Major OpRisk events are from unknown or non-process types risks they aresimply outside the banks normal set of understood risks Many of the biggest operational risk losses arise from fundamentally new issues, which even the most far-sighted and

active risk management might not be able to foresee

Many OpRisk approaches effectively imply the portfolio from historic loss events

Imagine taking this approach to credit risk modelling i.e.deduce the loan portfolio from historic defaults instead ofobtaining it from books and records


9/30

9Operational Risk

B3. Context dependency & relevance of loss data

Context dependency describes whether the size or likelihood of an incident varies indifferent situations It is important in modelling because it determines how relevantyour data are to the current problem

Eg. an analysis of transportation accidents over the last century would clearly contain data that had lost relevance due todifferent modes of transport, changing infrastructure, better communications, etc.

Consider the following questions:

(1) Are your businesses, people or processing systems similar to 10 years ago? (eg. manybanks have merged and/or materially changed their systems and processes)

(2) Are the threats to those systems similar to 10 years ago? (eg. did firms worry about

internet virus attacks in 1993?)Answering No illustrates the high context dependency of OpRisk

Context dependency is driven by how quickly the underlying system or processchanges

Market risks: Appear to have a moderate level of context dependency as stock market prices tend to exhibit statisticalproperties that appear to be somewhat stable across time

Credit risk: Credit ratings an loss statistics have been measured for many decades and show some reliable properties

The level of context dependency has a fundamental impact on the ability to model andvalidate a system the higher the context dependency the less the past will be a good

predictor for the future High context dependency degrades the relevance of information over time


10/30

10Operational Risk

The 4 elements of the AMA - Practical

implementation challenges


11/30

11Operational Risk

C1. The 4 elements of the AMA

A banks AMA OpRisk model must include the following 4 elements:

(1) Internal loss data (2) External loss data

(3) Scenario analysis (4) Business environment & internal control factors

There are a number of practical implementation issues with each of these 4 elements:

Completeness; Accuracy; Auditability; Relevance

LOW/MEDIUM(4)

MEDIUM

LOW (3)

HIGH

Accuracy

HIGHLOWMEDIUM/HIGHScenario analysis

HIGHLOW/HIGH (4)LOWBusiness environment &internal control factors

MEDIUMLOW/MEDIUM (3)LOW (3)External loss data

LOW (2)HIGHLOW/MEDIUM (1)Internal loss data

RelevanceAuditabilityCompleteness

Notes

(1) More difficult to ensure completeness for high-frequency, small-loss events Minor events; easier for Major events(2) Low rating as most firms unlikely to have suffered numerous Major events to provide sufficient data sample(3) Low/medium rating due to reporting bias and collection bias(4) Medium accuracy and auditability for factors that are countable but Low otherwise

Conclusion

Some elements are auditable but not relevant

& others are relevant but not auditable


12/30

12Operational Risk

C2. Relevance of internal/external loss data

Internal loss data

Internal loss data is not a good predictor of future events

After major event, management actions lead to improved controls & reduced chance of re-occurrence

Given lack of historic data and scarcity of meaningful data points, data can only be used as inputinto defining potential types of scenarios

Data generally more relevant at department level for efficiency/process improvements

External loss data

Main purpose is to provide guidance on types of scenarios and parameters & for lessons learned toidentify potential actions to reduce likelihood of event occurring within own organization

Numerous reporting/data capture issues: Reporting bias: Relies on companies disclosing significant OpRisk loss events. Some events have to be disclosed

others may not. Relies on events to be reported correctly in publicly available documents (figures often inflated)

Capture bias: Relies on firms capturing accurately OpRisk loss events and amounts from publicly available documents

External data needs to be cleansed to ensure it is relevant

After reviewing for relevance (ie. similarity to your firm; whether your firm has similar business; whether circumstancessurrounding the loss could be repeated in your firm) meaningful data points can be reduced significantly

External data can be used to assist in determining an estimate of the event severity and probability

Estimate of the OpRisk event probability can be estimated as number of events divided by institution years (typically 1 in2, 3, 5, 10, 20, 50 years); Number of institution years estimated from number of peer institutions and the number of yearsover which the external data was likely to be reasonably reliable


13/30

13Operational Risk

C3. Scenario analysis

Given the limitations of internal and external loss data, scenario analysis is the mostappropriate approach to determining an OpRisk capital charge

Can better take account of context dependency and the evolution of the organization, the business environment andinternal control factors

To address the issue of completeness of the portfolio of OpRisk exposure one needs

to list out a set out exposures (and their associated probabilities of occurring) Primary focus is on the major events, eg. rogue trader, building unavailability, etc.

Important to note that many of thebiggest OpRisk losses arise fromfundamentally new issues & hencedifficult to foresee

There will be some element of the Event

space not covered by a known risk unknown risks but with top-down approachcan include Unexpected Event scenario

Unexpected

Event

BCPRogueTradingMis-

selling

Fraud

Risk

A

Risk B

Risk N

OpRisk Event Space

Unknown Risk


14/30

14Operational Risk

C4. Business environment & internal control factors

There are a number of potential dimensions of business environment and internalcontrol factors, Eg:

Complexity (business/product, technology, business processes, organization, legal entity)

Rate of change of markets/products/volume (developing vs matured)

Management (centralised vs remote; own managed vs outsourced)

Processing maturity (automatic straight-through-processing vs manual)

Personnel (level of turnover; level of resourcing; competency of resourcing)

Although it is possible to justify each business environment and internal control factoras a driver of risk, it is generally only possible from a directional basis rather than

absolute basis Generally more effective to develop action plans and monitor risk reduction of each risk factor to an acceptable risk level

Some elements are auditable at the specific factor level but is it difficult to translate thefactor into an economic amount Even harder to aggregate across factors

Eg. what is the economic value of one outstanding confirmation acceptance vs one depot break? Translating economic amounts is necessarily judgmental and qualitative

Easier for factors that are countable, eg. process type risks, rather than non-process type risks

Can be taken account of, to some extent, in determining scenarios and associatedseverity and probability parameters


15/30

15Operational Risk

The irrelevance of small losses to capital


16/30

16Operational Risk

D1. Experience/observations of loss data collation

It is important to understand why internal loss data is being collected. How is the datagoing to be used? Must not confuse data with information

5,0

00

10,0

00

25,0

00

50,0

00

100,0

00

250,0

00

500,0

00

750,0

00

1,0

00,0

00

1,5

00,0

00

2,0

00,0

00

2,5

00,0

00

5,0

00,0

00

More

USD loss band

Frequency

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

5,0

00

10,0

00

25,0

00

50,0

00

100,0

00

250,0

00

500,0

00

750,0

00

1,0

00,0

00

1,5

00,0

00

2,0

00,0

00

2,5

00,0

00

5,0

00,0

00

More

USD loss band

Loss

Loss frequency

Value of losses

Lotsof data

No

information

More

relevantinformation

Limiteddata

Minor events Majority of collectedevents contributelittle to total loss

Major events Relatively fewevents contributemajority of totalloss

Conclusion: All relevant information is obtained from Major OpRisk loss events


17/30


18/30

18Operational Risk

D3. Small OpRisk losses vs Large OpRisk losses

Shouldnt we collect small OpRisk losses to learn more about potential large OpRisklosses and for better risk management purposes?

No Small losses and large losses are different in character and how they arise

Large OpRisk losses

Events can threaten the capital or solvency of the firm Typically result from a combination of control failures across a number of departments

Examples: Rogue trader; Fraud; Class-actions

Benefit from central collation and analysis to identify cross-department functionalissues and lessons learned

Key difference: Do not get small rogue traders, class-actions, etc.

Small OpRisk losses

Generally result from basic human errors (eg. lack of attention, forgetfulness, poorcommunication, etc.)

Incidents typically correspond to single breaches of the control layers; Shows that the successivedefense layers provide adequate control to capture upstream control breaches

Examples: Settlement errors; Credit card fraud

Data collation and actions should be taken at appropriate level of organization

Small losses best handled by dept experts little benefit from central collation/analysis Key difference: Do not get large losses of this type of error


19/30

19Operational Risk

D4. Small OpRisk losses vs Large OpRisk losses (cont.)

Risks

Big LossSmall Loss

Fraud; rogue trader; business interruptionSettlement errorsTypical examples

Central aggregation and reporting requiredDepartment escalation and reportingprocesses sufficientAppropriate lossreporting

Lessons learned often read across multiple

departmentsOften require new controls or significant re-design of existing controls

Escalation required across departments

Lessons learned only relevant to

processes within the particular deptconcernedOften actions taken only requirereinforcement of minor changes tocontrols already in place

Escalation only relevant to deptmanagement

Appropriate

Actions

Typically many control layers breachedControl failures cross a number ofdepartments

Small number of control layers breachedGenerally control failure is specific to aparticular department

How caused?Large LossesSmall Losses


20/30

20Operational Risk

Correlation assumptions


21/30

21Operational Risk

E1. Correlation assumptions

Correlation should be considered at 2 levels: (1) within a scenario, (2) across scenarios Correlation within scenarios: 100% correlation between individual control failures

eg. Major rogue trader event is the combination of: lack of supervision & failure to obtain confirmations& failure to independent test prices & failure to perform independent P&L analysis &

Correlation across scenarios: 0% correlation between major event scenarios

No evidence to suggest that OpRisk events are correlated, eg. what is the likelihood of documentationfailure impacting building unavailability

Lack of supervision

Poor challenge, issue escalation Failure to obtain transaction confirmations

Failure to obtain independent FX prices Failure to analyse P&L

Major event eg. Rogue Trading

Major event is the combination of individual controlfailures that alone would not give rise to the incident

(ie. 100% correlation between individual control failing)


22/30

22Operational Risk

E2. OpRisk aggregation: Across scenarios

Scatter graph of severity of loss event vs date of arrival shows no pattern Indicates no relationship between one event and another

There is no strong relationship between the number of loss events and the aggregatevalue of loss events

No obvious relationship between number of losses and aggregate value of losses is evident suggeststhat the level of OpRisk is not related to the number of events suffered

[insert scatter graphof loss event vs time]

1-Jan-01 1-Jan-02 1-Jan-03 1-Jan-04Date of loss

Los

samount(Logscale)

Loss event

2001

Q1

2001

Q2

2001

Q3

2001

Q4

2002

Q1

2002

Q2

2002

Q3

2002

Q4

2003

Q1

2003

Q2

2003

Q3

2003

Q4

Numberoflossevents

Valueoflossevents

Number of loss events Sum of loss events


23/30


24/30

24Operational Risk

Implementing a scenario based estimationapproach


25/30

25Operational Risk

F1. Putting it all together & determining a capital charge

4 elements of the AMA

Businessenvironment &internal control

factors

Scenariodefinitions &parameters

Internal loss data

External loss data

50

Very

Low

2351020

High

Medium

High

Medium

Low

Medium

LowProbability

1 in xyears 50

Very

Low

2351020

High

Medium

High

Medium

Low

Medium

LowProbability

1 in xyears

Scenario exposures & probabilities

S

cenarioexposureamount

Loss Distribution

0.00%

0.50%

1.00%

1.50%

2.00%

2.50%

3.00%

3.50%

4.00%

Annual loss

Probability

Capitalcharge

Aggregation ofscenarios using

OPRISK+


26/30

26Operational Risk

F2. Components of a scenario

Each scenario should use: Internal loss data, external loss data, business environmentand internal control factors to determine the scenario severity and probabilityparameters

Base

Risk

Internal loss data What losses has the firm experienced for this givenscenario?

What were the size of losses; frequency of major events? What management actions have been taken to prevent

future occurrence or reduce potential size of loss?

External loss data What major events of this particular scenario have occurred

to other firms similar to the firm? What is the potential range of losses? How frequently have

the events occurred? What is the potential loss and likelihood of occurrence for

the firm

Business environment & internal control factors

What are the business environment and internal controlfactors that could affect size and likelihood of loss?

Complexity of product/business; pace of change of marketor regulation; volumetrics; key risk indicators

UnexpectedEvent

BCPRogueTrading

Mis-selling

Fraud

Risk ARisk B

Risk N

Scenario, eg. rogue traderScenario loss: $?mProbability: x

BE&ICFs


27/30

27Operational Risk

F3. Calculating the OpRisk capital charge

Loss Distribution

0.00%

0.50%

1.00%

1.50%

2.00%

2.50%

3.00%

3.50%

4.00%

Annual loss

Probability

An actuarial approach is taken. The OpRisk economic capital is calculated usingOPRISK+ - an event risk model

Using a credit portfolio analogy the inputs into the OPRISK+ model are: OpRisk scenario exposures (cf.credit exposures) and OpRisk event probability (cf. default probability)

A correlation assumption is made that major OpRisk scenario events are independent

Top-down scenario approach considering major events allows an independence assumption to be made

Bottom-up scenario approach looking at the combination of minor OpRisk events needs to consider howthese minor events could correlate to generate a major event

Capitalcharge

50

Very

Low

2351020

High

Medium

High

Medium

Low

Medium

LowProbability

1 in xyears 50

Very

Low

2351020

High

Medium

High

Medium

Low

Medium

LowProbability

1 in xyears

Scenario exposures & probabilities

Scenarioexposureamount


28/30

28Operational Risk

F4. Building scenarios An approach

Define OpRisk Scenario definitions Understanding of risks facing the bank

Basel event categories

Internal/External loss data to indicate where banks can lose money

Draft up scenario templates using standardised format Description of scenario risk

Description of primary controls mitigating risk

Summary of internal loss experience related to scenario risk

Summary of relevant external loss experience related to scenario risk

Description of any relevant Business Environment and Internal Control Factors affecting scenario riskor control environment

Assumptions used to determine parameter assumptions

Summary of scenario parameters (frequency and severity)

Review each scenario template with Business Unit experts Discuss scenario risk, controls and scenario parameters with relevant line experts utilizing their expert

judgment (eg. discuss the Fraud scenario with experts from Legal, Corporate Security and Operations).

Review capital assessments with Senior Management

Provides an additional sense check over capital numbers

Provides comparison check between business units


29/30

29Operational Risk

F5. Benchmarking OpRisk capital results

Validation of OpRisk models is a major challenge pure statistical validation ofOpRisk models may not be possible for many years, probably never

However, there are benchmark tests that can be performed for scenario approaches:

Against Internal OpRisk Loss Data

Graphs show loss distribution from actualinternal OpRisk loss data over 3 yr period

No assumptions are made regarding the underlyingdistribution of events

Against External OpRisk Loss Data

Graph shows loss events for 10 key peers over10yrs (ie. 100 institution yrs of relevant data)

99%-ile capital figure is equivalent to 1 in 100year event

ERC

$1,434m

Loss Distribution from Actual Op Risk Losses

0.0%

0.1%

0.2%

0.3%

0.4%

0.5%

0.6%

Loss $m

ScenarioCapital(99.9%)

Internal

LossDataCapital(99.9%)

"Backtest" of peer loss data (present value) vs 99% OpRisk Capital

Based on 10 key peers and 10 years - ie 100 Bank Years

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

SumOfInflated Loss AmountCapital (99%)


30/30

30Operational Risk

G1. Conclusion

Characteristics of Scenario Approach

A scenario based capital estimation approach is: pragmatic; implementable; costeffective

Sensible capital numbers can be derived in a systematic and transparent manner The approach is forward-looking, utilizing all types of data available

Expert judgment is used to blend available data with understanding of the controlenvironment to produce forward-looking assessments of risk

Have a go yourself

Re-perform the analysis in this presentation on your data

What does your loss data tell you? Frequency plot; Value of losses vs number of losses plot; Cumulative loss ranking; Scatter plot vs time;

Interarrival times

Documents

OpRisk Scenario Analysis