Complexity, tails & trends ...Nick Watkins (nww@pks.mpg.de)

Max Planck Institute for the Physics of Complex Systems, Dresden, Germany (2013-14)

MCT, Open University

Centre for the Analysis of Time Series, LSE

Centre for Fusion Space & Astrophysics, University of Warwick

Addendum:

These slides were given as a contributed talk at the Trends 2014 meeting at Clare College, Cambridge, UK on 28th July 2014. I have also included the spare slides which were prepared in case of technical questions etc.

Nick Watkins29/7/2014

BIG QUESTION: How big, how long lasting, and how serially dependent are real world risks and hazards ?

BIG QUESTION: How big, how long lasting, and how serially dependent are real world risks and hazards ?

BIG QUESTION: How big, how long lasting, and how serially dependent are real world risks and hazards ?

SUBTOPIC: How do 2 types of complexity affect trend detection ?

This presentation complements those by Sandra and Christian and discusses aspects of two problems we face in real world time series:

• Long range dependence, if present in a system, would complicate the

trend detection, because it implies the presence of low

frequency "slow" fluctuations.

• Another effect, the presence of heavier than Gaussian tails in probability distributions, is, meanwhile, a source of "wild" fluctuations.

• Important to distinguish these two concepts, a process begin in the 60s by Benoit B Mandelbrot.

I will mainly be surveying ideas today, the detail is here:

[1] Watkins, “Bunched black (and grouped grey) swans”, Frontiers review article, GRL, 2013, http://onlinelibrary.wiley.com/doi/10.1002/grl.50103/full[2] Franzke et al, Phil. Trans. A, 2012, doi:10.1098/rsta.2011.0349[3] Graves et al, “A brief history of long memory”, submitted 2014 arXiv:1406.6018v1 [stat.OT][4] Graves et al, “Efficient Bayesian inference for long memory processes”, submitted 2014, arXiv:1403.2940v1 [stat.ME]

and will be developed in:

[5] Franzke et al, “Hurst without Joseph”, in prep.[6] Watkins and Franzke, in prep.

2Variance

Correlation length

2 2( )x

( ) ( ) ( )

e.g. ( ) ~ exp( )

x t x t

How do we quantify “big”, “correlated”?

2Variance

Correlation length

2 2( )x

( ) ( ) ( )

e.g. ( ) ~ exp( )

x t x t

iid Gaussian

white ~ ( )

(0, )N

2

Correlation length

Damped Brownian motion

"Physics"mv v

1

First order autoregressive AR(1)

(1 ) "stats"N N N

x x x

,x v

Variance But what if fluctuations “wild” in amplitude. Consider fat tailed pdf , wildest case of which is power law with small tail exponent ?

Light tail

Power law tail

(1 )~

stable

p

range

df (

: 0 2

) xp x

Fat tails not just curiosity: a well known topic in financial risk

[S&P 500] Mantegna &

Stanley, Nature, 1996

Fat tails not just curiosity: a well known topic in financial risk … lead to erratically fluctuating moments

[S&P 500] Mantegna &

Stanley, Nature, 1996

1963

We also observe fat(tish) tails in solar wind, ionosphere and magnetosphere

AE: Chapman, Hnat, Rowlands

& Watkins, NPG, 2005

Polar UVI: Uritsky et al, reviewed in

Freeman & Watkins, Science, 2002

SW Poynting flux: Freeman,

Watkins & Riley, PRE, 2000

See also Chapman, et al, GRL, 1998;

Watkins et al, GRL, 1999;

Lui et al, GRL, 2000 etc

Correlation length

But what if fluctuations “slow” in time, consider long rangedependence

Light tail

Power law tail

] ~

( ) ( ) ( )

( )

( ) F[

x t x t

S f f

(1 )~

stable

p

range

df (

: 0 2

) xp x

We just saw case

Variance

Nile river minima

600 700 800 900 1000 1100 1200 13009

10

11

12

13

14

15Annual minimum level of Nile: 622-1284

Annual m

inim

um

:

Time in years

Nile minima, 622-1284

Long range dependence contentious topic since Hurst reported anomalous growth of range in Nile river minima …

600 700 800 900 1000 1100 1200 13009

10

11

12

13

14

15Annual minimum level of Nile: 622-1284

Annual m

inim

um

:

Time in years

Hurst, Nature, 1957

Nile minima, 622-1284

Long range dependence contentious topic since Hurst reported anomalous growth of range in Nile river minima … and Mandelbrot proposed LRD as one possible explanation …d parameter … J=d+1/2

600 700 800 900 1000 1100 1200 13009

10

11

12

13

14

15Annual minimum level of Nile: 622-1284

Annual m

inim

um

:

Time in years

Hurst, Nature, 1957

Walk built from noise with d=-1/2

As above for d=1/2

Nile minima, 622-1284

Joseph Effect:

29 July 2014 19

Pharoah’s dream of 7 years of plenty and 7 years of drought. Now shuffle

... there came seven years of great plenty throughout the land of Egypt. And there shall arise after them seven years of famine ...

Genesis: 41, 29-30.

Joseph Effect:

29 July 2014 20

Pharoah’s dream of 7 years of plenty and 7 years of drought. Now shuffle

Point is that marginal distribution, of sample at least, unaffected by

shuffling, but that the two series represent very different worlds for insurers,

or Pharoahs. Former unlikely to happen in random trend free process without LRD.

... there came seven years of great plenty throughout the land of Egypt. And there shall arise after them seven years of famine ...

Genesis: 41, 29-30.

We see some classic long range dependence indicators in our space weather data …

AE power spectrum

Tsurutani et al, GRL, 1991

We see some classic long range dependence indicators in our space weather data …

AE power spectrum .

Tsurutani et al, GRL, 1991

ACF. Watkins, NPG, 2002

after Takalo and Timonen

( )

( ) F[ ] ~S f f

So how have I been modelling competing effects of LRD and heavy tails so far ? LFSM

• Use linear fractional stable motion (LFSM) model

• Self-similarity exponent H depends both on memory parameter d and tail exponent alpha in this class of models.

1 11( ) ( ) ( ) ( )

H H

H HR

X t C t s s dL s

1/ d H

Memory kernel: d measures LRD

α-stable jump: heavy tails

LFSM model unpicks the different scaling of solar wind Poynting flux driver & response in AE/U/L

Watkins et al, Space Science

Reviews, 2005

Data

Model

Peak pdf vs. diff. time Std dev vs tau

Need better model with LRD & heavy tails & ability to add HF damping: α-stable AutoRegressive Fractionally Integrated Moving Average (ARFIMA(p,d,q)):

29 July 2014 25

Ionospheric index

power spectrum .

Tsurutani et al, GRL,

1991

( )(1 ) ( )d

t tB B X B

1

( ) 1p

j

j

j

z z

1 t tBX X

Granger (& Joyeux), 1980

Need better model with LRD & heavy tails & ability to add HF damping: α-stable AutoRegressive Fractionally Integrated Moving Average (ARFIMA(p,d,q)):

29 July 2014 26

0.15(1 ) t tB X 1.5 Ionospheric index

power spectrum .

Tsurutani et al, GRL,

1991

( )(1 ) ( )d

t tB B X B

1

( ) 1p

j

j

j

z z

1 t tBX X

Granger (& Joyeux), 1980

Need better inference: Cambridge-BAS PhD Tim Graves developed Bayesian method: tested on α-stable ARFIMA(0,d,0) where heavy tails & LRD co-exist

Graves, Gramacy, Franzke & Watkins, submitted 2014; and in prep.

1.5 0.15 d

BUT …

• Both Mandelbrot (1965, 1967), and his critics (notably VitKlemes, WRR, 1974) noted that there were nonstationarymodels that also produced 1/f spectra-close relatives of what is now called Alternating Fractional Renewal Process.

• Mandelbrot was at pains to emphasise that needed to use our eyes as well as mathematical rigour (e.g. his Selecta series of annotated collected papers).

• Work in progress with Christian on how best to distinguish “true” LRD from models which share the power spectral and/or growth of range features but actually show much shorter typical lengths for dependent runs …

29 July 2014 28

Why does LRD matter in already fat-tailed hazards ? [Riley, Space Weather, 2012 ]: • Knowing relative frequency of a coronal mass ejection of a given

magnitude …

Why does LRD matter in already fat-tailed hazards ? [Riley, Space Weather, 2012 ]: … Knowing relative frequency of a coronal mass ejection of a given magnitude doesn’t fully specify the hazard:

Bunching, whether short range, or full blown LRD, affects overall hazard.

“Grouped grey swan” problem …[Watkins, GRL Frontiers, 2013]

Link to interacting hazards problem, c.f. UCL-Kings-Southampton Workshop & EGU session, 2013

Conclusions:

Bold paradigms, including some imported from outside space

physics

Stimulus to statistical inference-confrontation

of early claims with rigour

Critical thinking about

assumptions and methods

Risk and hazard questions: how big, how correlated ? Basic statistical concepts (finite) variance and (finite) correlation length. How we might relax these limits, and why we might want to [Watkins, GRL Frontiers, 2013]. Showed you some examples of work in this area [see also Watkins et al, GRL, 1999; Freeman, Watkins et al, PRE, 2000; Watkins et al, PRE, 2009 and outcomes of BAS Natural Complexity project & Warwick collaboration].

Statistical methodology: Importance of confronting these exciting new results and ideas with statistical inference, and more flexible models such as ARFIMA: e.g. Bayesian inference of long range dependence [Graves, Gramacy, Franzke & Watkins, first 2 papers now submitted].

Closing the loop: Better inference alone is not enough, though-work is in progress on how to distinguish between competing models. [Franzke et al; Watkins and Franzke]

SPARESSubtitle

Ideal reservoir • Average influx over

years, need to ensure annual released

volume equals mean influx:

Accumulated deviation

of the influx from the

mean:

29 July 2014 33

1

1( )

t

t

1

) { ( )( },t

u

uX t

Range:

Standard deviation:

Form against interval Plot loglog.

White Gaussian noise prediction

(Rescaled) range

29 July 2014 34

R

S

0 0max ) min ( ), ,(t tx t x t

1S

1/2/ ~R S

AR(1): 1st order AutoRegressive

29 July 2014 35

1 1t t tX X

0 100 200 300 400 500 600 700 800 900 1000-8

-6

-4

-2

0

2

4

6

8Example series of AR(1)

1 0.9

AR(1): 1st order AutoRegressive

29 July 2014 36

1(1( ) ) t tB B X

1 1t t tX X

1ttBX X 0 100 200 300 400 500 600 700 800 900 1000

-8

-6

-4

-2

0

2

4

6

8Example series of AR(1)

1 0.9

1

( ) 1p

j

j

j

z z

AutoRegressive Fractionally Integrated Moving Average [ARFIMA(p,d,q)]

29 July 2014 37

( )(1 ) ( )d

t tB B X B

Autoregressive term of order p

Moving average of order q

1

( ) 1q

j

j

j

z z

Fractional integration of order d

Granger (& Joyeux), 1980

(1 )d

t tB X Pure LRD

ARFIMA(0,d,0 ):

Exact Bayesian inference on ARFIMA for d in Gaussian special case

• Have data x, assumed to be a realization from a distribution with likelihood function L, parameters psi are the objects of interest.

• ARFIMA has parameters μ (location), σ (scale), d (order of fractional difference), φ (autoregression), θ. All but d, and its sign are essentially nuisance parameters here.

• First assume Gaussian innovations.

• Assume flat priors for μ, log σ and d …

• Even with this, likelihood for d very complex

• No analytic posterior p --- use MCMC sampling 29 July 2014 38

( | ) ( ) ( | )x p L x

Key features

• Don’t want to assume form of p, q – use reversible jump MCMC [Green, Biometrika 1995]

• Reparameterisation of model to enforce stationarityconstraints on φ and θ.

• Efficient calculation of Gaussian likelihood (long memory correlation structure prevents use of standard quick methods)

• Necessary use of Metropolis-Hastings requires careful selection of proposal distribution

• Parameter correlation (φ,d) requires blocking

29 July 2014 39

Approximate inference in more general case

• Drop Gaussianity assumption.

• Go to more general distribution (t,α-stable,…)

• Seek joint inference on d, α

• Approximate long memory process as very high order AR

• Construct likelihood sequentially

• Use auxiliary variables to integrate out unknown history

29 July 2014 40

Further developments

29 July 2014 41

~ 1/d n• In addition to joint inference problem.

have studied how posterior variance

of d depends on sample size n,

Motivated by Kiyani, Chapman & Watkins PRE,

2009 study of how errors in structure functions

depend on sample size. Watkins et al, in prep.

LFSM model allowed initial burst scaling study

H2

1

) '( 't

ts x t dt

Prediction: ( ) ~ s 1/ (2 H) p s

Watkins et al,

PRE, 2009