Embed Size (px)
Citation preview
Gaussian Process Conditional Copulaswith Applications to Financial Time Series
Jose Miguel Hernandez–Lobato1,
October 28, 2013
joint work withJames Robert Lloyd1 and Daniel Hernandez-Lobato2
1Cambridge University.2Universidad Autonoma de Madrid.
1 / 28
Copulas and Multivariate Probabilistic Models
Copulas link univariate marginal distributions into jointmultivariate probabilistic models.
−3 −2 −1 0 1 2 3
0.0
0.1
0.2
0.3
0.4
x
0 2 4 6 8 10
0.0
0.1
0.2
0.3
y
Marginal Densities
Copula Joint Density
Copulas specify dependencies between the random variables.
2 / 28
Separating Marginals and Dependencies Using Copulas
Sklar’s Theorem: any joint distribution can be written in termsof its copula and its univariate marginals [Sklar, 1959].
p(x1, x2) = c(u1, u2)︸ ︷︷ ︸copula
p1(x1) p2(x2)︸ ︷︷ ︸marginals
,
where ui := Pi(xi) and Pi denotes the marginal cdf for xi.
CopulasMarginals
MultivariateModel
CopulaMarginalMarginal
Copula
Marginal
Marginal
We can use different copulas and different univariate marginalsas building blocks for new multivariate models.
3 / 28
Additional Advantages of Copulas
AR(1)-GARCH(1,1) Time Series Model
Copula Model
AR(1)-GARCH(1,1) Time Series Model
Joint MultivariateModel for the Data
CopulaMarginal Marginal
Copula
Marginal
Marginal
−20
−10
010
20Returns AXP
Time
Ret
urn
Apr 05 Mar 06 Feb 07 Jan 08 Nov 08 Oct 09 Sep 10 Aug 11 Jun 12 May 13
−5
05
1015
Returns BA
Time
Ret
urn
Apr 05 Mar 06 Feb 07 Jan 08 Nov 08 Oct 09 Sep 10 Aug 11 Jun 12 May 13
0.0
0.4
0.8
Mapped Returns AXP
Time
Map
ped
Ret
urn
Apr 05 Mar 06 Feb 07 Jan 08 Nov 08 Oct 09 Sep 10 Aug 11 Jun 12 May 13
0.0
0.4
0.8
Mapped Returns BA
TimeM
app
ed R
etur
n
Apr 05 Mar 06 Feb 07 Jan 08 Nov 08 Oct 09 Sep 10 Aug 11 Jun 12 May 13
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Approximate Sample from the Copula
Mapped Returns AXP
Map
ped
Ret
urns
BA
Copulas easily extend univariatemodels to the multivariate regime.
Copulas simplify the estimation ofmultivariate models [Joe, 2005].
4 / 28
Parametric Copula Models
Any parametric density p(x1, x2) has corresponding copula
c(u1, u2) =p(x1, x2)
p1(x1)p2(x2),
where xi = P−1i (ui). Other parametric copulas are not directly
derived from a parametric density, e.g., Archimedean copulas.
Gaussian Copula
0.2 0.4 0.6 0.8
0.2
0.4
0.6
0.8
Student's t Copula
0.2 0.4 0.6 0.8
0.2
0.4
0.6
0.8
Symmetrized Joe Clayton Copula
0.2 0.4 0.6 0.8
0.2
0.4
0.6
0.8
5 / 28
Conditional CopulasThe distribution of x1 and x2 may depend on a covariate vector z.
CopulaMarginal MarginalCopulaMarginal Marginal
CopulaMarginal Marginal
CopulaMarginal Marginal
The copula framework can be extended to conditional distributions[Patton, 2006]. In this case,
p(x1, x2|z) = c(u1, u2|z)︸ ︷︷ ︸conditional
copula
p1(x1|z) p2(x2|z)︸ ︷︷ ︸conditionalmarginals
,
where ui := Pi(xi|z) and Pi denotes the marginal cdf for xi given z.6 / 28
Semiparametric Conditional CopulasMain Assumption: c(u1, u2|z) is given by a parametric copula modelcpar[u1, u2|θ1(z), . . . , θk(z)] specified by k scalar parameters θ1, . . . , θkthat are functions of z. We select θi(z) = σi[fi(z)], where fi is a realfunction and σi is a link function that maps R to a set Θi of validconfigurations for θi.
Example: conditional Student’s t copula. cstudent[u1, u2|ρ(z), ν(z)],ρ(z) = σρ[fρ(z)], ν(z) = σν [fν(z)], σρ(x) = 2Φ(x)− 1, σν(x) = exp(x).
−4 −2 0 2 4
050
100
150
−4 −2 0 2 4
−1.0
0.0
0.5
1.0
4−4
4−4
1
2
3
Our objective is to estimate f1, . . . , fk from data.7 / 28
A Bayesian Approach for the Estimation of f1, . . . , fk
Given a dataset formed by Dz = {zi}ni=1, Du1,u2= {u1i, u2i}ni=1,
where (u1i, u2i) ∼ cpar[u1, u2|θ1(zi), . . . , θk(zi)], we place priordistributions on f1, . . . , fk and compute a posterior distribution.
The distribution over functions is going to be a Gaussian Process.
Animations generated using the method desribed in [Henning, 2013]
8 / 28
From Multivariate Gaussians to Gaussian ProcessesLet us consider a 20 dimensional vector f = (f1, . . . , f20)T. We plotthe value of each component fi against its index i where f is randomwith distribution f ∼ N (m,V).
0.0
0.2
0.4
0.6
0.8
1.0
5 10 15 20
20
15
10
5
Covariance Matrix V
i
i
Intuitively, we can understand a GP as a Gaussian distribution on a
vector f whose dimension is infinite and is indexed by numbers in R.
9 / 28
Definition and Basics of Gaussian ProcessesDefinition:A Gaussian process p(f) = P(f |m, k) is a distribution over functionsf : Rd → R such that every finite dimensional vector of functionvalues f = [f(z1), . . . , f(zn)]T at locations Z = [z1, . . . , zn]T follows amultivariate Gaussian distribution p(f) = N (f |mZ,VZ,Z).
The mean vector and covariance matrix mZ and VZ,Z are the resultof evaluating the mean function m : Rd → R and the covariancefunction k : Rd × Rd → R at the locations Z.
Choices for µ and k:m(z) = µ,k(z, z′) = β exp{−(z− z′)Tdiag(λ)(z− z′)}+ γδ(z− z′).
Prediction:Given f , the conditional for the value of f at at new locationsZ? = [z?n+1, . . . , z
?n+h]T is p(f?|f) = N (f?|mpred,Vpred) with
mpred = mZ? + VZ?,ZV−1Z,Z(f −mZ) ,
Vpred = VZ?,Z? −VZ?,ZV−1Z,ZVZ,Z? .
10 / 28
Sparse Gaussian Processes
Working with GPs has cost O(n3).
A low rank covariance approximation reduces the cost to O(m2n).We use the FITC approximation [Snelson, 2006]. The covariancematrix VZ,Z is approximated using
VZ,Z ≈ VZ,ZV−1Z,Z
VZ,Z + Λ ,
where Λ is diagonal, Λ = diag(VZ,Z)− diag(VZ,ZV−1Z,Z
VZ,Z) and Z is
a set of m additional locations or pseudo-inputs.
11 / 28
Posterior Distribution and Predictive DistributionWe are given Dz = {zi}ni=1 and Du1,u2 = {u1i, u2i}ni=1. We denote byfj = [fj(z1), . . . , fj(zn)]T the vector with the evaluation of fj at Dz.Then,
p(f1, . . . , fk|Du1,u2,Dz) =
[n∏i=1
cpar [u1i, u2i|σ1[f1(zi)], . . . , σk[fk(zi)]]
] k∏j=1
p(fj)
[p(Du1,u2 ,Dz)]−1
.
We make predictions at z? using f? = [f1(z?), . . . , fk(z?)] and
p(u?1, u?2|z?,Dz,Du1,u2
) =
∫cpar [u?1, u
?2|σ1[f1(z?)], . . . , σk[fk(z?)]]
p(f?|f1, . . . , fk, z?,Dz)
p(f1, . . . , fk|Du1,u2,Dz) df1 · · · dfk df? .
We approximate these distributions using Expectation Propagation
[Minka, 2001].12 / 28
Expectation PropagationEP approximates the unnormalized joint p(f1, . . . , fk,Du1,u2
|Dz) with
an unnormalized Gaussian Q(f1, . . . , fk) = k∏kj=1N (fj |mj ,Vj).
EP adjusts gi by minimizing KL[giQ\i||giQ\i], where Q\i is the ratio
of Q and gi. EP with sparse GPs [Naish-Guzman et al. 2008].
13 / 28
An Additional Approximation
Minimizing KL[giQ\i||giQ\i] requires to compute the first and secondmoments of gi(f1i, . . . , fki)Q\i(f1i, . . . , fki) with respect to each fji.
No analytical solution!k-dimensional numerical integration infeasible!
Additional approximation:∫f1igi(f1i, . . . , fki)Q\i(f1i, . . . , fki) df1i · · · dfki ≈
≈ C ×∫f1igi(f1i, f2i, . . . , fki)Q\i(f1i, f2i, . . . , fki) df1i
where f1i, . . . , fki are the means of f1i, . . . , fki with respect to Q\i,and C is a constant that approximates the width of the integrandaround its maximum in all dimensions except f1i.
Now we only need a 1-dimensional numerical integration!
14 / 28
An Illustrative ExampleThe approximation is very accurate since gi is locally very flat.To illustrate this, we plot for a conditional Student’s t copula thenormalized versions of
q?1(f1i) =
∫gi(f1i, f2i)Q\i(f1i, f2i) df2i , q1(f1i) = gi(f1i, f2i)Q\i(f1i, f2i) ,
and similarly for q?2(f2i) and q2(f2i).
1.0 1.1 1.2 1.3 1.4 1.5
01
23
45
x x xx
x
x
x
x
x
x x
x
x
x
x
x
xx x xo o o
o
o
o
o
o
oo
o
o
o
o
o
oo o o
ox
−5.0 −4.5 −4.0 −3.5 −3.0
0.0
0.5
1.0
1.5
x x x xx
x
x
x
x
xx
x
x
x
x
xx x x xo o o o
o
o
o
o
o
o o
o
o
o
o
oo o o
ox
q?1(f1i) and q1(f2i) are almost identical. Finding the moments of
q?1(f1i) requires 2-dimensional numerical integration. Finding the
moments of q1(f2i) requires only a 1-dimensional integral.15 / 28
ExperimentsWe evaluate the proposed Gaussian Process Conditional Copulas(GPCC) in experiments with synthetic and financial time series.
One-step-ahead rolling-window prediction task where we assume thecopula function changes as a function of time.
We consider three parametric copula families for our GPCC:Gaussian: GPCC-G.Student’s t: GPCC-T.Symmetrized Joe Clayton: GPCC-SJC.
Copula Parameters Transformation Synthetic parameter functionGaussian correlation, τ 0.99(2Φ[f(t)]− 1) τ(t) = 0.3 + 0.2 cos(tπ/125)
Student’s tcorrelation, τ 0.99(2Φ[f(t)]− 1) τ(t) = 0.3 + 0.2 cos(tπ/125)
degrees of freedom, ν 1 + 106Φ[g(t)] ν(t) = 1 + 2(1 + cos(tπ/250))
SJCupper dependence, τU 0.01 + 0.98Φ[g(t)] τU (t) = 0.1 + 0.3(1 + cos(tπ/125))lower dependence, τL 0.01 + 0.98Φ[g(t)] τL(t) = 0.1 + 0.3(1 + cos(tπ/125 + π/2))
Table: Copula parameters, modeling formulae and parameterfunctions used to generate synthetic data. Φ is the standard Gaussiancumulative density function f and g are GPs.
16 / 28
Benchmark Methods
We compare with Gaussian, Student’s t and symmetrized Joe Claytoncopulas with static parameters: CONST-G, CONST-T, CONST-SJC.
Time-varying Correlation Model (TVC). Student’s t copula with νconstant and ρt = (1− α− β)ρ+ αεt−1 + βρt−1, where εt−1 is theempirical correlation of the last 10 data points [Jondeau et al. 2006].
Dynamic Symmetrized Joe Clayton Copula (DSJCC). Theparameters τU and τL satisfy
τUt = 0.01 + 0.98Λ[ωU + αUεt−1 + βUτ
Ut−1
],
τLt = 0.01 + 0.98Λ[ωL + αLεt−1 + βLτ
Lt−1
],
where Λ[·] is the logistic function, εt−1 = 110
∑10j=1 |ut−j − vt−j |,
(ut, vt) is a copula sample at time t [Patton 2006].
Two-state Hidden Markov Model (HMM). At any time the copula is
Student’s t with different parameters for each state. Two regimes of
low/high dependence [Jondeau et al. 2006].
17 / 28
Visualization of Synthetic Time SeriesEach time series has length 5001. Rolling-window of size 1000.
Training data sampled from a dynamic Student’s t copula.
Red: Evolution of τ . Blue: Evolution of ν.
18 / 28
Results on Synthetic Time Series
0 200 400 600 800 1000
515
Student's t Time Series,
Mean GPCC−TGround truth
0 200 400 600 800 1000
0.2
0.4
0.6
0.8
Student's t Time Series
Mean GPCC−TGround truth
19 / 28
Experiments with Foreign Exchange Time Series
Daily returns of nine currency pairs from 02-01-1990 to 15-01-2013(6011 observations). Each pair includes the Swiss franc (CHF) andanother currency.
CHF is a safe haven currency. We expect correlations between CHFand other currencies to have large variability across time.
Data preprocessed using an asymmetric AR(1)-GARCH(1,1) processwith non-parametric innovations [Hernandez-Lobat et al. 2008].
Code Currency NameCHF Swiss FrancAUD Australian DollarCAD Canadian DollarJPY Japanese YenNOK Norwegian KroneSEK Swedish KronaEUR EuroNZD New Zeland DollarGBP British Pound
20 / 28
Results on Foreign Exchange Time Series0
2040
6080
100
120
140
EUR−CHF GPCC-T,
Oct 06 Mar 07 Aug 07 Jan 08 Jun 08 Nov 08 Apr 09 Oct 09 Mar 10 Aug 10
Mean GPCC−T
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
EUR−CHF GPCC-T,
Oct 06 Mar 07 Aug 07 Jan 08 Jun 08 Nov 08 Apr 09 Oct 09 Mar 10 Aug 10
Mean GPCC−T
0.2
0.4
0.6
0.8
1.0
1.2
EUR−CHF GPCC-SJC,
Oct 06 Mar 07 Aug 07 Jan 08 Jun 08 Nov 08 Apr 09 Oct 09 Mar 10 Aug 10
Mean GPCC-SJCMean GPCC-SJC
21 / 28
Visualization of Foreign Exchange Time Series
Predictions of GPCC-T on the training data for CHF-EUR.
Red: Evolution of τ . Blue: Evolution of ν.
22 / 28
Experiments on Equity Exchange Time SeriesResults on 8 pairs of financial time series with equity returns:
0.35
0.40
0.45
0.50
0.55
0.60
0.65
0.70
RBS−BARC Time Series, Tau
Apr 09 Sep 09 Aug 10 Jan 11 Jul 11 Dec 11 Jun 12 Nov 12 Apr 13
Mean GPCC−T
010
2030
4050
RBS−BARC Time Series, Nu
Apr 09 Sep 09 Aug 10 Jan 11 Jul 11 Dec 11 Jun 12 Nov 12 Apr 13
Mean GPCC−T
23 / 28
Summary and Conclusions
I We have proposed a framework for conditionaldependencies based on parametric copula models withmultiple parameters.
I Our approach is based on sparse GPs and expectationpropagation for fast approximate inference.
I We have evaluated this framework on synthetic andfinancial time-series.
I We outperform static copula models and other dynamiccopula models.
I Our approach can be easily extended to higher dimensionsusing vine factorizations [Lopez-Paz et al. 2013].
24 / 28
Vine Factorizations
1 2
3 4
e 1=
1,3|∅
e 2=
2,3|∅
e3 = 3, 4|∅
1, 3|∅ 2, 3|∅
3, 4|∅
e4 = 1, 2|3
e 5=
1,4|3
1, 2|3 1, 4|3
e6 = 2, 4|1, 3
G1/T1G2/T2
G3/T3
c1234 = c13|∅︸︷︷︸e1
c23|∅︸︷︷︸e2
c34|∅︸︷︷︸e3︸ ︷︷ ︸
T1
c12|3︸︷︷︸e4
c14|3︸︷︷︸e5︸ ︷︷ ︸
T2
c24|13︸ ︷︷ ︸e6︸ ︷︷ ︸T3
25 / 28
Useful References I
I J. M. Hernndez-Lobato, J. R. Lloyd and D. Hernndez-Lobato.Gaussian Process Conditional Copulas with Applications to FinancialTime Series. In Advances in Neural Information Processing Systems26 (NIPS), 2013.
I A. Sklar. Fonctions de repartition a n dimensions et leurs marges.Publ. Inst. Statis. Univ. Paris, 8(1):229–231, 1959.
I H. Joe. Asymptotic efciency of the two-stage estimation method forcopula-based models. J. Multivar. Anal., 94(2):401–419, 2005.
I A. J. Patton. Modelling asymmetric exchange rate dependence.International economic review, 47(2):527–556, 2006.
I P. Hennig. Animating Samples from Gaussian Distributions. MaxPlanck Institute for Intelligent Systems, Technical Report 8, 2013.
I E. Snelson and Z. Ghahramani. Sparse Gaussian processes usingpseudo-inputs. In NIPS, 1257–1264, 2006.
26 / 28
Useful References II
I A. Naish-Guzman and S. B. Holden. The generalized FITCapproximation. In NIPS, 1057–1064, 2008.
I E. Jondeau and M. Rockinger. The copula-garch model of conditionaldependencies: An international stock market application. Journal ofinternational money and nance, 25(5):827–853, 2006.
I J. M. Hernandez-Lobato, D. Hernandez-Lobato, and A. Suarez.GARCH processes with non-parametric innovations for market riskestimation. In ICANN, 718–727, 2007.
I D. Lopez-Paz, J. M. Hernandez-Lobato and Z. Ghahramani. GaussianProcess Vine Copulas for Multivariate Dependence, In ICML, 2013.
27 / 28
Thank you for your attention!
28 / 28
