Macroprudential stress tests – performance evaluation

IMF/Bank of Russia Workshop on RECENT DEVELOPMENTS IN MACROPRUDENTIAL STRESS TESTING Moscow, 4-5 September 2018

Christoffer Kok Deputy Head / Stress Test Modelling Division DG Macroprudential Policy and Financial Stability

DISCLAIMER: The views expressed are my own and do not necessarily reflect those of the ECB or the Eurosystem.

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

ECB-RESTRICTED DRAFT

2

Source: Darracq Pariès, M., Kok C. and Rodriguez Palenzuela, D. (2011), “Macroeconomic propagation across different regulatory regimes: evidence from an estimated DSGE model for the euro area”, International Journal of Central Banking, Vol. 7, No. 4, pp. 49-113.

Contribution of “financial shocks” to real GDP growth (annual growth in percentage points)

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2003

Q1

2003

Q3

2004

Q1

2004

Q3

2005

Q1

2005

Q3

2006

Q1

2006

Q3

2007

Q1

2007

Q3

2008

Q1

2008

Q3

2009

Q1

2009

Q3

2010

Q1

2010

Q3

2011

Q1

2011

Q3

2012

Q1

2012

Q3

2013

Q1

2013

Q3

Financial accelerator factors

Bank balance sheet factors

Overall financial factors

The crisis illustrated that financial “frictions” tend to amplify the business cycle

– More positive growth in the upturns

– And more negative growth in the downturn

Macroprudential stress

testing is a useful tool to capture real-financial linkages

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

Stress testing for macroprudential (and micro-prudential) purposes have gained importance since the crisis

Authorities have made substantial efforts to develop and implement relevant frameworks

But macroprudential stress testing remains an evolving area

How to get it right?

This presentation focuses on some of the key challenges

ECB-RESTRICTED DRAFT

3

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

ECB-RESTRICTED DRAFT Relevant recent background material

4

An ECB e-book, staff tools for “macroprudential ST”

http://www.ecb.europa.eu/pub/pdf/other/stampe201702.en.pdf See also: http://www.ecb.europa.eu/press/key/date/2016/html/sp160426.en.html http://www.ecb.europa.eu/press/key/date/2017/html/ecb.sp170922_3.en.html Constâncio et al., (2018), “Macroprudential Policy at the ECB”, ECB Occasional Paper (forthcoming).

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

A. ECB Stress Testing Framework: Overview

ECB staff toolkit for Systemic Risk analyses (and EBA/SSM/NCA STs)

5

The ECB Top-Down stress test “workhorse” – the basis for STAMP€

Contagionmodels

Macro feed back models

Insurance + shadow banks

Fire sales

Micro house-holds and NFC data

Scenario Balance sheet FeedbackSatellite models

Macromodels

Credit riskmodels

Profitmodels

Market riskmodels

Loan lossmodels

Balance sheet and P&L tool => Solvency

Dynamic adjustmentmodel

Funding shock

RWA

Financial shocks

Adapted from Henry and Kok (eds.), ECB Occasional Paper 152, October 2013 https://www.ecb.europa.eu/pub/pdf/scpops/ecbocp152.pdf .

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

Contribute to supervisory stress test exercises (e.g. EBA EU-wide Sress Test / IMF-EU country programmes)

Top-down challenger models for quality assurance of

banks’ bottom-up submissions

Covering areas such as Credit risk Market risk Net interest income Net fees and commission income Operational risk Other P&L

ECB-RESTRICTED DRAFT

6

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

ECB-RESTRICTED DRAFT

7

Source: ECB Financial Stability Review, May 2018.

Distribution of banks CET1 ratio under adverse scenario (in per cent)

• ECB produces and publishes on a regular basis…

• …a quantitative assessment of the resilience of euro area financial institutions…

• …to a materialisation of the main systemic risks identified in its Financial Stability Review

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

ECB-RESTRICTED DRAFT

8

Source: Henry, J. and Kok C. (eds., 2013), “A macro stress testing framework for assessing systemic risks in the banking sector”, ECB Occasional Paper No. 152.

Second-round macro feedback effects on bank solvency (end-horizon CET1 ratio; in per cent)

• Going beyond (bottom-up) supervisory stress test

• …by incorporating amplification effects due to macroeconomic feedback

• …as banks respond to initial adverse shock

-5

0

5

10

15

20

-5 0 5 10 15 20

CET1

ratio

incl.

chan

ging l

oan v

olum

es an

d 2n

d rou

nd ef

fect

s

CET1 ratio incl. loan volumes consistent with scenario

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

ECB-RESTRICTED DRAFT

9

Source: Darracq Pariès, M., Halaj, G. and Kok C. (2016), “Bank capital structure and the credit channel of asset purchases”, ECB Working Paper No. 1916.

Macro propagation due to dynamic bank reactions with / without macroprudential capital buffer (bps lending rate impact; annual GDP growth in percentage points; in comparison to baseline)

• By exploiting bank heterogeneity…

• …macroprudential stress testing tools

• …can be employed to assess macro implications of banks’ dynamic responses

• With / without additional macroprudential buffers

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

1. Dynamic bank behaviour

2. Micro-to-macro

3. Feedback effects

4. Performance evaluation

ECB-RESTRICTED DRAFT

10

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

How do banks respond to shocks?

Deleverage through asset reductions vs. raise equity

Banking book (loan supply) vs. trading book (fire sale losses)

Adjustments via prices or quantities

Subject to regulatory and other constraints (solvency and liquidity)

ECB-RESTRICTED DRAFT

11

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

ECB-RESTRICTED DRAFT

12

Source: Darracq Pariès, M., Halaj, G. and Kok C. (2016), “Bank capital structure and the credit channel of asset purchases”, ECB Working Paper No. 1916.

Banks’ responses to shocks are determined by balance sheet structure and risk-return characteristics

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

A variety of potential modelling approaches Simple loan regressions (Gross-Vendetti, 2017: Stamp€ ch. 9) Empirical models of bank balance sheet adjustments (Maurin-

Toivanen, 2012; Kok-Schepens, 2013) VAR-based approaches with individual bank balance sheets

(GVAR: Gross-Kok-Zochowski, 2016; FAVAR: Budnik-Bochmann, 2017)

Structural portfolio optimisation models (Halaj, 2015; Darracq Pariès-Halaj-Kok, 2016; Behn-Daminato-Salleo, 2018)

Modelling link between solvency positions and funding costs (Arnould-Pancaro-Zochowski, 2018; Gross-Hansen-Kok, 2018)

Agent-based models (Halaj-Kok, 2015; Calimani-Halaj-Zochowski, 2017; Halaj, 2018)

ECB-RESTRICTED DRAFT

13

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

Key strengths: Allow for high level of granularity and accounting for bank heterogeneity Allow for impact assessments of macroprudential policies along both time

dimension and cross-section dimension

But not trivial task going from micro level impact assessments

to the macro level How to make bank heterogeneous responses consistent with system-

wide effects?

More than just the sum of the parts Ideally, heterogeneous bank behaviour should be modelled

within a comprehensive and consistent macro model framework

Realistically, more piecemeal approaches are likely to remain ‘state-of-the-art’ in the near future

ECB-RESTRICTED DRAFT

14

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

ECB-RESTRICTED DRAFT

15

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

Macro feedback effects

How to disentangle macro feedback effects (e.g. loan supply reductions) from the scenario?

How to model the “residual” amplifying macro effects? Not only bank responses but also other sectors’ responses matter =>

need for modelling heterogeneity at sector level (bank, HH, NFC, etc.)

Financial contagion due to Interconnectedness

Need to capture both interlinkages with other banks and with other (non-bank) financial intermediaries

Often modelled as an add-on (‘satelite’) to the rest of the macroprudential stress test apparatus

How to ensure consistency with other dynamic bank reactions? How feed back contagion effects to the broader stress test results? Static cascades vs. dynamic contagion effects

ECB-RESTRICTED DRAFT

16

Rubric Use as rubric line or delete on slide master

www.ecb.europa.eu

Some limitations: Macroprudential stress testing is fundamentally about

running counterfactual / “what if” scenarios Reliance on the consistent linking of suites of models Time series of granular data often short

More efforts needed to evaluate how well stress testing

frameworks capture reality / predictive power Back testing

Case studies

Sensitivity analysis of key assumptions

Non-linear effects (e.g. when going from baseline to adverse)

Reverse stress testing

ECB-RESTRICTED DRAFT

17