Optimal Transport between Copulas for Clustering Time Series

IntroductionDependence measures & Copulas

Optimal TransportThe Target Dependence Coefficient

Clustering Credit Default SwapsLimits & Future Developments

Optimal Transport between Copulasfor Clustering Time Series

IEEE ICASSP 2016

Gautier Marti, Frank Nielsen, Philippe Donnat

March 22, 2016

Gautier Marti Optimal Transport between Copulas for Clustering Time Series




1 Introduction

2 Dependence measures & Copulas

3 Optimal Transport

4 The Target Dependence Coefficient

5 Clustering Credit Default Swaps

6 Limits & Future Developments





Clustering of Time Series

We need a distance Dij between time series xi and xj

If we look for ‘correlation’, Dij is a decreasing function of ρij ,a measure of ‘correlation’

Several choices are available for ρij . . .





1 Introduction


3 Optimal Transport








Common dependence measures

(a) ρ = 0.66 (b) ρ = 0.23 (c) ρS = 0.65 (d) ρS = 0.64

500 data points (xi , yi ) from N((

00

),

(1 0.6

0.6 1

)).

(a) Pearson correlation ρ between X and Y .

(b) Pearson correlation ρ between X and Y with one outlier introduced in the dataset.

(c) Spearman correlation ρS between X and Y which is Pearson correlation on the rank-transformed data.

(d) Spearman correlation ρS between X and Y with one outlier introduced in the dataset.





Copulas

Sklar’s Theorem:

F (xi , xj) = Cij(Fi (xi ),Fj(xj))

Cij , the copula, encodes the dependence structureFrechet-Hoeffding bounds:

max{ui + uj − 1, 0} ≤ Cij(ui , uj) ≤ min{ui , uj}

Figure: (left) lower-bound copula, (mid) independence, (right)upper-bound copula





Dependence measures and their relations to copulas

Bivariate dependence measures:

deviation from Frechet-Hoeffding bounds

Spearman’s ρS ,Gini’s γ,Kendall distribution distance [2],

deviation from independence uiujSpearman,Copula MMD [6],Schweizer-Wolff’s σ,Hoeffding’s Φ2





Motivation: Target specific dependence, forget others

Motivation: We want to detect y = f (x2) and y = f (x), but noty = g(x), where f , g are respectively strictly increasing, decreasing.Problem: A dependence measure which is powerful enough todetect y = f (x2) will generally also detect y = g(x).

Dependence to detect (ρij := 1)

Dependence to ignore (ρij := 0)





1 Introduction


3 Optimal Transport








Optimal Transport

Wasserstein metrics:

W pp (µ, ν) := inf

γ∈Γ(µ,ν)

∫M×M

d(x , y)pdγ(x , y)

In practice, the distance W1 is estimated on discrete data bysolving the following linear program with the Hungarian algorithm:

EMD(s1, s2) := minf

∑1≤k,l≤n

‖pk − ql‖fkl

subject to fkl ≥ 0, 1 ≤ k, l ≤ n,n∑

l=1

fkl ≤ wpk , 1 ≤ k ≤ n,

n∑k=1

fkl ≤ wql , 1 ≤ l ≤ n,

n∑k=1

n∑l=1

fkl = 1.





EMD: How does it work? Showcase in 1D

Earth Mover Distance is the minimum cost, i.e. the amount of dirtmoved times the distance by which it is moved, of turning piles ofearth into others.

EMD = |x1 − x2| EMD = 16 |x1 − x3|+ 1

6 |x2 − x3|





1 Introduction


3 Optimal Transport








EMD between Copulas - The Methodology

Why the Earth Mover Distance?

Figure: Copulas C1,C2,C3 encoding a correlation of 0.5, 0.99, 0.9999respectively; Which pair of copulas is the nearest? For Fisher-Rao,Kullback-Leibler, Hellinger and related divergences:D(C1,C2) ≤ D(C2,C3); EMD(C2,C3) ≤ EMD(C1,C2)





EMD between Copulas - The Methodology

Probability integral transform of a variable xi :

FT (xki ) =1

T

T∑t=1

I (x ti ≤ xki ),

i.e. computing the ranks of the realizations, and normalizingthem into [0,1]





Application: A target-oriented dependence coefficient

Now, we can define our bespoke dependence coefficient:

Build the forget-dependence copulas {CFl }l

Build the target-dependence copulas {CTk }k

Compute the empirical copula Cij from xi , xj

TDC(Cij) =minl EMD(CF

l ,Cij)

minl EMD(CFl ,Cij) + mink EMD(Cij ,CT

k )





Target Dependence Coefficient: Two examples

Motivating example

Figure: Dependence is measured asthe relative distance from thenearest forget-dependence(independence) to the nearesttarget-dependence (comonotonic)

Classical dependence

Figure: Dependence is measured asthe relative distance fromindependence to the nearesttarget-dependence: comonotonicityor counter-monotonicity





Benchmark: Power of Estimators

0.0

0.4

0.8

xvals

pow

er.cor[typ,]

xvals

pow

er.cor[typ,]

0.0

0.4

0.8

xvals

pow

er.cor[typ,]

xvals

pow

er.cor[typ,]

cordCorMICACERDCTDC

0.0

0.4

0.8

xvals

pow

er.cor[typ,]

xvalspow

er.cor[typ,]

0 20 40 60 80 100

0.0

0.4

0.8

xvals

pow

er.cor[typ,]

0 20 40 60 80 100

xvals

pow

er.cor[typ,]

Noise Level

Pow

er

Figure: Dependence estimators power as a function of the noise forseveral deterministic patterns + noise. Their power is the percentage oftimes that they are able to distinguish between dependent andindependent samples. Experiments similar to [3]





1 Introduction


3 Optimal Transport








Clustering Financial Time Series

East Japan Railway Com-

pany vs. Tokyo Electric

Power Company:

ρ = 0.49,

ρS = 0.17,

τ = 0.12

TDC = 0.19





Impact of different coefficients

Which is best? One can look at:

stability criteria [5],

convergence rates [4],





1 Introduction


3 Optimal Transport








Computational Limits

The methodology presented can be applied in higher dimensions,but it has some scalability issues:

non-parametric density estimation is hard (problem oftenreferred as the curse of dimensionality),

costly to compute due to the exponential number of bins.

Partial solutions:

Approximation schemes can drastically reduce thecomputation time [1]

Parametric modelling (optimal transport between Gaussianmeasures [7]) can alleviate these issues but loses genericity.





Marco Cuturi.Sinkhorn distances: Lightspeed computation of optimaltransport.In Advances in Neural Information Processing Systems, pages2292–2300, 2013.

Fabrizio Durante and Roberta Pappada.Cluster analysis of time series via kendall distribution.In Strengthening Links Between Data Analysis and SoftComputing, pages 209–216. Springer, 2015.

David Lopez-Paz, Philipp Hennig, and Bernhard Scholkopf.The randomized dependence coefficient.In Advances in Neural Information Processing Systems, pages1–9, 2013.





Gautier Marti, Sebastien Andler, Frank Nielsen, and PhilippeDonnat.Clustering financial time series: How long is enough?2016.

Gautier Marti, Philippe Very, Philippe Donnat, and FrankNielsen.A proposal of a methodological framework with experimentalguidelines to investigate clustering stability on financial timeseries.In 14th IEEE International Conference on Machine Learningand Applications, ICMLA 2015, Miami, FL, USA, December9-11, 2015, pages 32–37, 2015.

Barnabas Poczos, Zoubin Ghahramani, and Jeff G. Schneider.Copula-based kernel dependency measures.





In Proceedings of the 29th International Conference onMachine Learning, ICML 2012, Edinburgh, Scotland, UK, June26 - July 1, 2012, 2012.

Asuka Takatsu et al.Wasserstein geometry of gaussian measures.Osaka Journal of Mathematics, 48(4):1005–1026, 2011.


Data & Analytics

Optimal Transport between Copulas for Clustering Time Series