Model Risk of Correlation Products PRESENTED TO:Risk Magazine's Training Course: Advanced Correlation Modelling & Analysis BY:Martin Goldberg, Director

Model Risk of Correlation Products

PRESENTED TO: Risk Magazine's Training Course: Advanced Correlation Modelling & Analysis

BY: Martin Goldberg, [email protected] of Model ValidationRisk ArchitectureCitigroupNew York, New York

DATE: May 11, 2006

PLACE: New York City

Page 2 of 44Martin Goldberg, Citigroup April, 2006

Outline How does your model perform

Testing copula density and marginals separately

Estimating correlations and volatility - implied vs. historical

Stress Events, Extreme Value Theory, conditional dependencies, and VaR

Summary






Summary


Performance metrics

Most models can calibrate to the price of some standard instrument at

inception. What distinguishes good from bad is the performance over time or

over changing markets

Backtesting

Ex Ante Profit Attribution Analysis

Paper Trading

Hypothetical Paper Trades

Stress Tests


Sources of Error

There are many possible sources of model risk. Some that I will discuss today

are:

Calibration error - the model does not fit the market, either because

It is not flexible enough

It is too flexible and you are calibrating to noise

Input error - the data you are fitting your model with are wrong or come from

a different market

Regime shift - the model used to work well, but the market has changed and

the model can’t handle the changes.

Wild West error - there are not enough data to distinguish a good model from

a bad one


Not Flexible Enough - Simplistic models

No correlation skew as an analogy to B_S models without vol skew

CAPM one-factor models with no correlation between alphas

Multiplying VaR by and neglecting GARCH-type effects

Confusing correlation with contagion, or assuming constant correlation

t


Too Flexible - Careful Fit to Noise

In a variance-covariance Value-at-Risk calculation, or in a portfolio optimization, a key

component is a correlation matrix.

A matrix calculated from a year of daily returns has rank at most 252, no matter how

many timeseries you put in.

Random Matrix Theory [3], [4] describes the principal components you would get if

the timeseries were pure noise. How does it compare to the eigenvalues of, say, the

S&P 500 correlation matrix?

About 94% of the spectrum is random, leaving the meaningful signal a rank of about

30.

Moral: Whenever possible, use a control group of synthetic data of known

properties to separate out real effects from artefacts of the modeling process.


Calibrating to wrong data

First example

Hedge fund managers and private equity funds are required to post monthly

estimates of their portfolio value.

The estimates are noisy due to illiquidity

The managers, either unconsciously or deliberately, “smooth” the earnings

by shading their estimates.

Correlations due to similar smoothing, or from market moves?

Next example

Historical correlations for trading in liquid markets should probably use daily

timeseries

For buy-and-hold, probably less frequent, like monthly or quarterly.

Related issues in estimation of default correlation from equity data.

Spectral analysis, for example [5].


Regime Shifts

Abrupt jumps should be handled differently than gradual non-stationarity.

Key questions for jumps:

Is this really a jump and not just a fat tailed diffusion?

Is the jump a one-time regime shift invalidating all prior history, or is it a

feature of a jumpy market?

Is there a news story explaining the jump?

Was there a knock-on effect (possibly contagion?) in other markets?


Illiquid and New Markets and Cowboys

Here you need to use heuristics and trader intuition, since there isn’t anything

else.

A wide enough bid-ask spread covers all sins.

This is not exactly model risk since there is no model as such.

Anything worth doing is worth doing badly at first.






Summary


Quick Introduction to Copula Theory

Copula theory is a generalization of the concept of correlation.

A copula is expressed as a quantile of the distribution in N dimensions.

A copula in one dimension is a tautology - x% of the data are at or below the x

% quantile.

The one-dimensional copula density, also called a marginal, is a uniform

distribution from 0 to 1, with the copula

density of each point being its quantile in the data series.

The copula is related to the copula density by

),(

),Pr(),(),(

)0,0( VUc

vVuUvuCvu


Example

In two dimensions, the copula density of changes in USD Libor and JPY Libor

looks like

Note that it is not actually continuous, since some days are unchanged.

3m Libor Weekly Changes Copula Density Rank Correlation 10.6%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

JPY Libor

US

D L

ibor


Copulas again

The Copula for this data is at each point (x,y) the fraction of the data where X<x and Y<y.

For a good introduction to copulas see [9].

10

%

20

%

30

%

40

%

50

%

60

%

70

%

80

%

90

%

10

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

JPY Libor

USD Libor

Cumulative Copula of weekly Changes in 3M Libor

80%-100%

60%-80%

40%-60%

20%-40%

0%-20%


Marginal Probability Distributions

The first step in constructing a copula is to transform all the marginals to uniform

densities.

Not everything is Gaussian.

Alternative distributions, which can be fitted to data and give better results

usually than a Gaussian or lognormal, are Student-t, CEV, Madan’s VG, and my

favorite for exploratory data analysis, the Tukey gXh distribution [15],[16]

Where g controls skew and h controls smile. This nests Gaussian, lognormal,

Student-t, and even Cauchy distributions, and is more tractable than CEV or VG.

Later I will show what can go wrong if the marginals are mis-specified.

Art form to trade-off between accuracy/flexibility and tractability/speed.

2

2

1)(

hZgZ

gXh eg

eBAZX


Fitting the Tukey gXh distribution to single-B bond spreads

-5 -4 -3 -2 -1 0 1 2 3 4 50

500

1000

1500

2000

2500

3000

3500

Absolute change in yield (%)

Normal

Rescaled

gXh

Comparison of normal, rescaled normal and (g X h) distribution fits to 10 day changes in idiosyncratic spread for single-B bonds using EJV data. Rescaled Cumulative Normal fits at 99th percentile.


Families of Copulas

There are many “families” of copulas described in the literature [7].

Note that most of the literature stops at 2 dimensions; more than that is

much harder to work out the math, and there are constraints.

Some popular families of copulas are:

#

Parameters

in 2-D

Name

0 Independence

1 Gaussian, Plackett, Frank, Gumbel, Morgenstern

2 Student-t, but most others have no good names -

see [7] page 149 and following

As many as

needed

Bernstein (see [8]) which is the copula equivalent

of a Taylor series


Properties of Copulas

Any multivariate density can be expressed as a copula connecting marginal densities.

The copula and the marginals are completely separate - any marginal pdf’s can be

connected by any valid copula.

For distributions with continuous marginals the copula is unique.

The average of two copulas may not be a copula, but the average of two copula

densities is a copula density.

A Gaussian copula connecting two Gaussian marginals is a Pearson (ordinary)

correlation.

Pearson Correlation is not a good measure if the copula is not Gaussian or any marginal

is not Gaussian.

The rank correlation is a non-parametric copula equivalent, for any distributions, of

Pearson correlations for multivariate normals. Spearman rho is easier than Kendall Tau.

Spearman rank correlation in 2-D is the Pearson correlation between ranks of the

entries of the 2 data series.

Easy to compute in Excel using the RANK() function.


Tail Dependence

The Upper Tail Dependence is defined for any copula density c(U1,U2) as

By flipping the < to > and replacing the u with 1-u, we can get a Lower Tail Dependence

More generally, in N dimensions, the hypercube has 2N corners, so we can define 2N Tail

Dependences

Tail Dependence is my preferred definition of contagion.

A Gaussian Copula has zero tail dependence - a very extreme move in one dimension is

never simultaneous with a similarly big move in any other dimension.

)(u

)1/(),Pr(1

),()( 21 uuUuU

uuuC

u


Fiendish Copula Density

Upper and lower tail dependence of 1;

middle “local dependence” -1

The rank correlation is constructed to be

exactly zero.

It is more pathological than what you will

ever actually find, but it is a good stress

test.

You can find funnel-shaped and galaxy-

shaped copula densities in real data, but

in a less exaggerated form than below.

Extreme Funnel Extreme Galaxy

Fiendish Copula Density

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Gaussian Copula Density


Fiendish Copula

100

200

300

400

500

600

700

800

900

1000

X

Y

Fiendish Cumulative Copula It is not immediately obvious why this is

so fiendish.

Although all the theory is done using the

cumulative distributions, the copula

densities are more informative and make

prettier pictures.

Caveat: Copula theory is not nearly as

well developed for more than 2

dimensions. The standard cheat scheme

used in Credit Derivatives is to assume

every underlying looks like all the others,

which means all 2-dimensional slices

look alike.


Distinguishing Correlation from Contagion

Non-Linear CorrelationContagion can be defined as a significant difference in the association between large moves (tail events) relative to the association between smaller moves (ordinary days). This is Tail Dependence. As an example study, here is a test of contagious increases in Pearson correlation between Brent oil and kerosene, using 215 pairs of weekly historical spot data. Since you always should use a control, I have also used 215 pairs of random numbers with the same correlation of 63%.

Random fake normals correlated 62.82%

-4

-3

-2

-1

0

1

2

3

4

-4 -3 -2 -1 0 1 2 3 4

Brent vs Kerosene 62.82% CorrelationAll weekly data

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

-0.35 -0.3 -0.25 -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15

Brent % Moves

Ker

o %

Move

s


Tonsured densityTonsured Density

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

-0.35 -0.3 -0.25 -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15

Null hypothesis is an elliptical distribution, so eliminate center of distribution where

Xba

b

i

a

i 2

2

2

2


Nonlinearity? Tonsured correlation

Tonsured Correlation

0.6

0.7

0.8

0.9

1

Tonsured Fake data

Tonsured real data


Tail Dependence for Brent and Kerosene

Upper Tail Dependence - Brent and Kero

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

U

Lam

bda

Lower Tail dependence - Brent and Kero

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

00.10.20.30.40.50.6

u

Lam

bda

�Upper tail dependence, but not lower, is found in the US Treasury curve[14].

�Barry Schachter’s gloriamundi.org website has many other such tail dependence papers.


Another Parametric Correlation Curve

Bjerve and Doksum[1] have a correlation curve, which does not generalize

well to n>2 data series.

Regress Y against X, then the local variance

The local regression slope

And the global standard deviation of the X variable

And use these to get a local correlation

Note this is not symmetric in X and Y

)|()(2 xXYVarx )(x

)()()(

)(222 xx

xx


Causes of Non-constant Correlation

Use of a standard Pearson correlation assumes multivariate normal distributions.

The changes in correlation could be due to contagion, or just to skewed or fat-tailed

underlyings.

Kurtosis alone still has elliptical distribution and does not have much effect on

correlation.

Copula theory - not well developed for n>2. For example, a Student-t copula has the

same degrees of freedom for every pair of variables.

If two Gaussian marginals are associated by a Gaussian copula, the rank-correlation is

constant, and equal to the Pearson correlation.

Copulas can be made flexible enough to represent arbitrarily screwy associations.

Mathematical theories use the copula, which is the cumulative function; easier to

visualize copula density.


Problems with Pearson Correlation

High leverage data points


Picking the right copula

Fitting the data exactly with a copula density of delta functions at each data point

is valid, but not very useful. Trade-off between smooth function and accuracy [2].

Hurd et al[10] use an expansion in Bernstein copulas for the copula between

EUR/GBP and USD/GBP implied by the volly smile surfaces of traded options. FX

is special because triangular arbitrage gives an implied correlation. On some days,

the copula is bimodal and they need 11 terms to get the extra hump qualitatively

right. Is this noise or a feature?

Malvergne and Sornette[11] find that most of the time, a Gaussian copula works

for most currency pairs and most pairs of stocks, but not for pairs of commodities.

However, if your portfolio is actively traded or has optionality, this may not be

the case. Boyson et al[12] find contagion between hedge fund styles, so a fund-of-

funds is not well-modeled by a Gaussian copula.

Some other caveats are found in [13], where they show how hard it is to find the

best parametric copula to fit to real data.


Is stock market contagion an urban legend?

On the previous slide I mentioned[11] that a Gaussian copula works well for equities.

Does this contradict the received wisdom that correlations between stocks goes to one

in a crash?

One example of the studies of correlation in a crash is [17]. Note that they always use

ordinary Pearson correlations.

In contrast, [18] notes that a constant association with a fat-tailed distribution leads to

the appearance of changing correlation.

This is an example of invoking higher-order effects to explain illusory phenomena

caused by not fully capturing the lower order effects. In quantum chemistry, this is

called lack of basis set saturation.

A similar example is the simpler versions of jump-diffusion models, where they pretend

the diffusion part is lognormal or Gaussian, pretend that the drift term is linear, and then

invoke jumps to explain the rest. Jumps in financial time series are real, but they are

always accompanied by headlines in the Wall Street Journal or Financial Times. If your

model has more jumps than headlines, fix the diffusion part.






Summary


Nonstationary time series

Historical estimate is always a convolution with a filter.Square wave - equally weighted data more recent than -RiskMetrics exponential decay

Whatever other filter you like, or better, one suggested by the data

Greg Sullivan[6] describes an optimal estimator for the length of the square

wave filter, which could be easily modified to find, for example, the optimal

exponent for the RiskMetrics method.

For a stationary time series, the error in the correlation estimate is

In the absence of any obvious regime shift, the non-stationarity can be

modeled as a linear drift

t )1( 2

1

2||

)(

2

b

btat


Nonstationary time series 2

Then the optimum observation period, dictated by the data itself, is

This represents the trade-off between longer periods reducing statistical

noise, and shorter periods being closer to stationary. The paper[6] has detailed

instructions for finding this optimum point.

Of course there are data series where there is a genuine regime shift, like the

start of a new equity listing due to IPO, and there the cutoff is obvious.

32

22* )1(2

bt


Implied and Historical Correlations

I n t h e s t a n d a r d r i s k - n e u t r a l p r i c i n g o f d e r i v a t i v e s w i t h m u l t i p l e u n d e r l y i n g s s 1 a n d s 2 , a c o r r e l a t i o n e n t e r s t h e e q u a t i o n s a s :

MdwdwE

ddwtsdttds

ddwtsdttds

)(

,...),(,...)(

,...),(,...)(

21

2222222

1111111

W h e r e t h e d r i f t s a r e t h e r i s k - n e u t r a l d r i f t s , a n d t h e d i f f u s i o n s w a r e c o n t i n u o u s m a r t i n g a l e s w i t h n o d r i f t , d i s t r i b u t e d a s G a u s s i a n s u n d e r s o m e f o r w a r d m e a s u r e . T h e j u m p i n t e n s i t i e s w i t h p r o b a b i l i t i e s a r e n o t p r e s e n t i n p u r e d i f f u s i o n m o d e l s . T h e r e c a n b e c o r r e l a t i o n s b e t w e e n w 1 a n d w 2 , b e t w e e n

1 a n d 2 , a n d / o r b e t w e e n t h e a n d t h e w ,

o r e v e n m o r e i f t h e a r e s t o c h a s t i c : a s f e w a s o n e o r a s m a n y a s s i x o r m o r e p o s s i b l e m o d e l c o r r e l a t i o n s . W h a t w e o b s e r v e i n t h e r e a l w o r l d a r e h i s t o r i c a l t i m e s e r i e s S 1 a n d S 2 , w i t h h i s t o r i c a l c o r r e l a t i o n H m e a s u r e d a s t h e u s u a l P e a r s o n c o r r e l a t i o n e i t h e r b e t w e e n a r i t h m e t i c c h a n g e s [ S 1 ( t ) - S 1 ( t - 1 ) ] a n d [ S 2 ( t ) - S 2 ( t - 1 ) ] o r b e t w e e n l o g r a t i o s l o g ( S 1 ( t ) / S 1 ( t - 1 ) ) a n d l o g ( S 2 ( t ) / S 2 ( t - 1 ) ) . T h e q u e s t i o n i s w h a t r e l a t i o n s h i p i s t h e r e b e t w e e n t h e m o d e l M a n d t h e h i s t o r i c a l H .


Changing the skewness – fake data with correlation .5

6.2 5.0 3.7 2.3 1.1 0.1 -0.9 -1.9 -3.0 -4.2-7.5

-3.6

-0.7

2.1

5.9

0

0.1

0.2

0.3

0.4

0.5

0.6

Skew1

Skew2

Linear Correlation


Changing the Skew with Added Jumps – fake data with correlation .5

13.8 12.3 9.9 6.1 2.5 0.4 -0.9 -2.1 -3.5 -5.1-7.7

-3.4

0.3

11.2

0

0.1

0.2

0.3

0.4

0.5

0.6

Skew1

Skew2

Linear Correlation


Rank Correlations

In the absence of jumps, any amount of skewness or kurtosis leaves the rank

correlation between time series unchanged.

This is one of the best reasons to use copula theory.

Rank correlations are very little extra effort compared to Pearson

correlations, and do not assume Gaussian marginals.

Enhancing your model of the univariate marginals does not require redoing

the copula / rank correlation matrix.

Non-Gaussian copulas can capture tail dependence and not confuse it with the

tail shapes.






Summary


Extreme Value Theory

EVT says that, for quantiles outside the observation data set, the marginal pdf

can take only 3 different shapes:

Gaussian decay

Abrupt cutoff (Gumbel)

Power-law decay (Frechet)

Most financial time series have Frechet tails. Estimate the exponent, and fit it

with the bulk of the distribution, and you are done. You can use this to predict

the 99%ile (VaR), the 99.9%ile (Basel 2), or the 99.97%ile (economic capital for

a AA firm.) The result will differ from scaling up using the assumption of a

multivariate normal density.

What copula to use? A quick series of tests include tail dependences, stability

over the sample period, and the X test.


X test for copula skew

By construction, half the copula density is on the left half (0-.5), and half is on the

lower half (0-.5).

You can test for quadrant dependence by seeing how far off the density is from ¼ in

each quadrant, but that may not be very informative – this is just a crude measure of

rank correlation.

Far more informative a test is (in a 2D copula density) drawing an X from upper left

to lower right and upper right to lower left. If the density is elliptical, all 4 triangles

will be mirrored images of each other.

Test comparing each of the 4 to the average of all 4.

Extend to quartets of diamond-shaped sub-regions?

This X-test is not in the literature, as far as I know, but it is referred to in [10].

The extension of the X test to N dimensions involves 2N diagonals.


Summary

Not everything is multivariate (log-)Gaussian.

Rank correlations are a better and more flexible measure than correlations.

Copula theory is useful.

Tradeoffs between accuracy and speed, complexity vs ease of use.

Liquidity of market important in selecting model.

Get the simpler features right before invoking higher-order effects.

Keep up with the literature – there are lots of good ideas out there.

Disclaimer: This talk represents my personal views and is not

intended to be in agreement with anything Citigroup says or does.


References

1. Bjerve, S. and Doksum, K.A. (1993). Correlation curves: measures of association

as functions of covariates. Ann. of Statist., 21, 890-902.

2. http://xxx.lanl.gov/abs/physics?papernum=0406023 Maximum Entropy

Multivariate Density Estimation: An exact goodness-of-fit approach

3. http://arxiv.org/abs/cond-mat?papernum=0111503 Noisy Covariance Matrices

4. http://arxiv.org/abs/cond-mat?papernum=0205119 Noisy Covariance Matrices II

5. http://www.ofce.sciences-po.fr/pdf/dtravail/wp2003-07.pdf and

http://ideas.repec.org/p/fce/doctra/0307.html, Iacobucci, Spectral Analysis for

Economic Time Series

6. Greg Sullivan, Risk 8(8), August 1995, page 36, Correlation Counts

7. H. Joe, “Multivariate Models and Dependence Concepts” Chapman&Hall, 1997

8. http://greywww.kub.nl:2080/greyfiles/center/2003/doc/122.pdf Multivariate

Option Pricing Using Dynamic Copula Models by R.W.J. van den Goorbergh, C.

Genest, B.J.M. Werker


References

9. http://gro.creditlyonnais.fr/content/wp/copula-survey.pdf Copulas for Finance - A

Reading Guide and Some Applications by Bouyé, Durrleman, Nikeghbali, Riboulet,

Roncalli

10.http://www2.warwick.ac.uk/fac/soc/wbs/research/wfri/rsrchcentres/ferc/

wrkingpaprseries/wp05-20.pdf Using Copulas to Construct Bivariate Foreign

Exchange Distributions with an Application to the Sterling Exchange Rate Index by

Matthew Hurd, Mark Salmon, Christoph Schleicher

11.http://arxiv.org/abs/cond-mat?papernum=0111310 Testing the Gaussian Copula

Hypothesis for Financial Assets Dependences Authors: Y. Malevergne , D. Sornette

12.http://www.nber.org/papers/w12090 Is There Hedge fund contagion? Authors: N.

Boyson, C. Stahel, R. Stulz

13.http://www.crest.fr/pageperso/fermanian/pitfalls_copula.pdf Some Statistical pitfalls

in Copula Modeling for Financial Applications Authors J Fermanian, O Scaillet

14.http://gloriamundi.org/picsresources/mjasnw.pdf Nonlinear Term Structure

Dependence by M Junker, A Szimayer, N Wagner


References

15.http://fic.wharton.upenn.edu/fic/papers/02/0225.pdf

16.http://fic.wharton.upenn.edu/fic/papers/02/0226.pdf

17.http://arxiv.org/abs/cond-mat?papernum=0302546 Dynamics of market

correlations: Taxonomy and portfolio analysis Authors: J.-P. Onnela, A.

Chakraborti, K. Kaski, J. Kertesz, A. Kanto

18.http://ideas.repec.org/p/cam/camdae/0319.html. "Changing Correlation and

Portfolio Diversification Failure in the Presence of Large Market Losses by

Sancetta, Satchell

Documents

Model Risk of Correlation Products PRESENTED TO:Risk Magazine's Training Course: Advanced Correlation Modelling & Analysis BY:Martin Goldberg, Director