KRIs – a failed experiment or a misunderstood tool?

Scottish Chapter Annual Conference1 November 2013

Disclaimer• Your speaker is a Fellow of the Institute of Directors, a

Director of Council and Vice-Chairperson of the IOR, as well as the Chief Executive of RiskBusiness International Limited.

• The views expressed in this presentation are the sole responsibility of the presenter and do not and can not be construed as representing the view of either the Institute of Operational Risk or RiskBusiness International Limited.

What is a KRI?

• Basically, whatever you want it to be!– It’s a metric, a piece of information– It only has value to those who can use it– There is no difference between KRIs, KPIs and KCIs –

depends on source and use of the information

Why is Op Risk “Hard” to do?

• It’s fundamentally different to other types of risk:– No direct correlation to volumes, market volatility, economic

cycles or other easily quantifiable factors– Its embedded into each of the other types of risk – how and

when do we separate them?– It has a direct link to the “human factor”– The business intuitively accept it as part of “business as usual”

and have difficulty in understanding the “regulatory” rationale behind elevating it to a distinct risk class

– The issues of “face”, deniability, ignorance, the “Titanic” effect and historical audit processes

– Multiple independent programs

Why are KRIs left to last?

• How do we correlate them to……– Risk?– Exposure?– Loss Data?– Capital Requirements?– Senior Management Requirements and General

Business Management Needs?• Where do we start?• Can they ever be predictive? If not, why

bother?

Predictive metrics

• By themselves, individual indicators will never be truly predictive

• Avoid thinking that trends are predictive• KRIs can be used to “predict” events before they manifest

themselves – if:– There is a common structure against which risks and losses are

classified and to which indicators are linked– Indicators are monitored on a real-time basis by business units

carrying the exposure– Management reacts to stimuli as and when warning patterns

emerge– We have had sufficient history to test and prove the correlations

The role of KRIs in time

Internal Event Data

External Loss Data

KRIs RCSA

Risk Profiling

Scenario Analysis

HistoryMedium- toLong-termFuture

Short-termFutureNow

OpRisk Framework: Time Perspective

Who owns KRIs?• Who decides which KRIs to monitor?• Who decides how the KRI values are calculated?• Who decides who receives KRI reports and when they

receive them?• Who decides when to escalate a KRI submission?• Who decides to discontinue using a specific KRI?

Providers

Roles in the KRI Process

Consumers Observers

Producers

Risk Management

Operational Risk as a “Bow Tie”

EventsCauses Impacts

Card - Retail FX Swap Lockbox

Products

Glob Serv Inv Mgmt Sec Lend

Org’l Hierarchy

Market and Sell Products and

Services

Trading and/or Investment

ManagementTrade Settlement

Processes

Detective Controls

Internal causes

External causes Oversight Controls

EVENT

Preventive Controls

Defined Business Process

Market and Sell Products and

Services

Trading and/or Investment

ManagementTrade Settlement

Processes

Glob Serv Inv Mgmt Sec Lend

Org’l Hierarchy

Card - Retail FX Swap Lockbox

Products

Expected Result

Defined Business Process

Risk Category

KRIs

The Role of KRIs

However....• The Bow-Tie Model suggests that causal factors drive

everything else, therefore the normal business model needs to in corporate cause as a major component

Causal chainsManager

S’visor A S’visor B

Clerk A Clerk AClerk A Clerk B Clerk B Clerk B

Trade Record

Review and check details

Correct? Send to Confirm

Return to F/O

Yes

No

Annual holidays

Market volatility

Complex trade

Lack of training

Staff € problems

Staff personal problems

M’mentoffsite

Short staffed

System failure

Evolution of risk classification

Market and Sell Products and

Services

Trading and/or Investment

ManagementTrade Settlement

Processes

Glob Serv Inv Mgmt Sec Lend

Org’l Hierarchy

Card - Retail FX Swap Lockbox

Products

Business Process

Risk

Process Risks

Internal causes

External causes

Process Risk Causes

Preventive

Controls

Risk Categor

y

Detective

Controls

Preventive

Controls

Detective

Controls

Data Management• Quality management is one of the most critical aspects of the KRI

programme:– Use standard specifications covering what, when and how

data will be measured, along with who and why– Investigate discrepancies and challenges– Utilise other LoD to assist in QM

• Collect data at the earliest opportunity, disseminate it as quickly as you can

• Consumers often already have the data, add value by augmenting the data with context– #/$ of suspense account items– Broken down by time-buckets– Analysed by party/system/entity

Top-Down v Bottom-Up• The concept is to obtain reports on the same metrics from

across the firm• Can we identify any metrics which can actually be

compared across the entire firm?• Are these metrics in any way correlated to risk exposure?

Derived indicators• “Manufactured” metrics whose value is derived from a series

of underlying metrics• Examples of underlying metrics:

– Staff availability– Average error count– Average cost per error– Transactional volumes– Economic cycles– Cycle volatility

• The resultant metric seeks to measure the impact of staff taking leave at specific points in time, assessing the increased potential cost

Thresholds• Thresholds are essentially pre-defined limits which, when

the value of a KRI reaches that level, generates warnings or alerts

• Different types:– Touch– Repetitive touch– Percentile breach– Trend threshold

• Thresholds should never be absolute; they need to address cycles and correlations.

Threshold issues• Never start with a “big bang” – management will be

confused if they suddenly start receiving numerous “alerts” and “warnings”

• Start with high-level thresholds and fine tune them over time – what is the correct level for any metric?

• Use layered structures so that increasing levels of seniority receive warnings when appropriate – and the “worker-bees” do not get a surprise

• Revise thresholds from time to time

In summary……• There is no such thing as a global set of “top 10” indicators

which everyone should monitor• Excluding composite or index-based metrics, indicators are

not in of themselves predictive• A good indicator programme will involve a large number of

players• Your risk profile is constantly changing, causal drivers

continuously morph into different impact chains – the indicator set being monitored should thus not be set in stone

• Don’t expect instant gratification – the benefits of the indicator programme will take time to manifest themselves

• Read the IOR SPG on KRIs issued November 2010!

Questions and Comments

• Contact Details:• Mike Finlay, Chief Executive, RiskBusiness

– Telephone : +44 7721 969 224– E-mail : mike.finlay@riskbusiness.com or mfinlay@ior-institute.org – URL : www.riskbusiness.com and www.ior-institute.org