Bayesian Analysis of Non-Gaussian Long range Dependent Processes

Tim Graves (Statistics Laboratory, Cambridge) Christian Franzke (BAS, Cambridge)

Bobby Gramacy (Booth School of Business, Chicago) & Nick Watkins (BAS, LSE & Warwick) [nww@bas.ac.uk]

NG22A-04 11am Tuesday 4th December 2012 Scaling and Correlations and their use in forecasting Natural Hazards I Room 300 Moscone South

Summary: 1. Standard climate noise models have short range dependence

(SRD). Recent evidence of long range dependence in surface temperatures. Example is Antarctic study [Franzke, J. Climate, 2010]. LRD hampers trend identification and quantification of significance.

2. LRD idea originated at same time as H-selfsimilarity, so not always realised that a model doesn’t need to be H-ss to show LRD, e.g. ARFIMA, [Watkins, GRL Frontiers, 2013].

3. Graves PhD has developed MCMC method to perform Bayesian inference on ARFIMA(p,d,q). Treated Gaussian ARFIMA first, tested on model (& real) data. Study dependence of posterior variance of inferred d on length of time series.

4. However, many real datasets not Gaussian. ARFIMA can allow alpha stable innovations. Graves has modified method to allow joint inference.

5 December 2012 2

Short & long range dependence

5 December 2012 3

( ) exp( )ρ τ λτ−

( ) ( )ρ τ δ τ

Delta correlated white noise

2 1( ) dk ckρ −

0 1/ 2d< <

Exponentially correlated red AR(1), SRD

0( )

k

kkρ

=∞

=

= ∞∑LRD: power law correlated, “1/f” noise

LRD & Antarctic temperature trends

4

[Franzke, J. Climate, 2010] found evidence of LRD in station temperature series. 3 have significant EMD residual trend (dashes) against SRD null model. One station [Faraday] still significant against LRD.

LRD and selfsimilarity, common history ...

Mandelbrot (& Wallis), mid 1960s: 2 departures from AR(1), “Biblical geoscience” illustrations, selfsimilarity exponent H

• heavy tails in amplitude, cotton prices. “Noah” effect, 40 days and 40 nights of rain. • long range dependence in Nile level. 7 lean & 7 fat years: “Joseph effect”. 5 December 2012 5

LRD and selfsimilarity, common history ...

Mandelbrot (& Wallis), mid 1960s: 2 departures from AR(1), “Biblical geoscience” illustrations, selfsimilarity exponent H

• heavy tails in amplitude, cotton prices. “Noah” effect, 40 days and 40 nights of rain. • long range dependence in Nile level. 7 lean & 7 fat years: “Joseph effect”. 5 December 2012 6

LRD and selfsimilarity, common history ...

Mandelbrot (& Wallis), mid 1960s: 2 departures from AR(1), “Biblical geoscience” illustrations, selfsimilarity exponent H

• heavy tails in amplitude, cotton prices. “Noah” effect, 40 days and 40 nights of rain. • long range dependence in Nile level. 7 lean & 7 fat years: “Joseph effect”. 5 December 2012 7

… not necessarily common origin • Both fBm & Levy flights are H-selfsimilar, • Frac. Brownian motion: H (here = J) = d + ½ LRD (persistence): 0 < d < ½; “Hurst” exponent

increases from Brownian value of ½ as memory parameter d increases

• Levy flights: H = 1/α α is exponent of pdf heavy tail. • Both are limits of H = 1/ α +d • Many methods (e.g. R/S …) inspired by self-

similarity, & measure d, not α, via geometry. 8

Can model LRD (d) without assuming complete H-selfsimilarity Don’t actually need completely H-selfsimilar

models to exhibit LRD (just asymptotic) In 1980s Granger and Joyeux modified SRD Auto

Regressive Moving Average [ARMA(p,q)] models to allow LRD via Fractional Integration of order d [ARFIMA(p,d,q)].

Physically interesting: High frequency p term(s) that turns nonstationary, H-ss random walk into weakly stationary AR(p) i.e. dissipation

5 December 2012 9

AR(1): 1st order AutoRegressive

5 December 2012 10

1 1t t tX Xφ ε−= +

0 100 200 300 400 500 600 700 800 900 1000-8

-6

-4

-2

0

2

4

6

8Example series of AR(1)

1 0.9φ =

AR(1): 1st order AutoRegressive

5 December 2012 11

1(1( ) ) t tB B Xφ εΦ − ==

1 1t t tX Xφ ε−= +

1ttBX X −=0 100 200 300 400 500 600 700 800 900 1000

-8

-6

-4

-2

0

2

4

6

8Example series of AR(1)

1 0.9φ =

1( ) 1

pj

jj

z zφ=

Φ = −∑

AutoRegressive Fractionally Integrated Moving Average

[ARFIMA(p,d,q)]

5 December 2012 12

( )(1 ) ( )dt tB B X B εΦ − = Θ

Autoregressive term of order p

Moving average of order q

1( ) 1

qj

jj

z zθ=

Θ = +∑

Fractional integration of order d

Granger (& Joyeux), 1980

(1 )dt tB X ε− =Pure LRD

ARFIMA(0,d,0 ):

Exact Bayesian inference on ARFIMA for d

• ARFIMA has parameters μ, σ, d, φ, θ. All but d essentially nuisance parameters here.

• First assume Gaussian innovations. • Assume flat priors for μ, log σ and d … • Even with this, likelihood for d very complex • No analytic posterior --- use MCMC sampling

5 December 2012 13

( | ) ( ) ( | )x p L xψ ψπ ψ ψ ψ∝

Key features

• Don’t want to assume form of p, q – use R J MCMC [Green, Biometrika 1995]

• Reparameterisation of model to enforce stationarity constraints on φ and θ.

• Efficient calculation of Gaussian likelihood (long memory correlation structure prevents use of standard quick methods)

• Necessary use of Metropolis-Hastings requires careful selection of proposal distribution

• Parameter correlation (φ,d) requires blocking 5 December 2012 14

“Calibration”

5 December 2012 15

~ 1/d nσ

• Have studied how posterior variance of d depends on sample size n, c.f. Kiyani et al, PRE, 2009. study of structure functions etc • Looked at standard test series like Nile river, find d of about .4 and ARFIMA(0,d,0) most probable model. Confims e.g Beran, 1994. • Looked at CET. Dependence more complicated and a model incorporating seasonality performs better.

Approximate inference in more general case

• Drop Gaussianity assumption. • Go to more general distribution (α-stable). • Seek joint inference on d, α • Approximate long memory process as very

high order AR • Construct likelihood sequentially • Use auxiliary variables to integrate out

unknown history 5 December 2012 16

Pure symmetric α-stable ARFIMA

5 December 2012 17

0.15(1 ) t tB X ε− = 1.5α =

Posterior estimates of d, α

5 December 2012 18

1.5α =0.15d =

Scatter of d and α

5 December 2012 19 1 1t t tX Xφ ε−= +

Good estimation of all parameters. Posteriors of d and α are independent.

Conclusions: 1. Standard climate noise model AR(1). Discretises Ornstein-Uhlenbeck

physics. Short range dependence (SRD). However recent evidence of long range dependence (d nonzero) in surface temperatures. Example is multistation Antarctic study [Franzke, J. Climate, 2010]. LRD hampers trend identification and quantification of significance.

2. LRD idea originated at same time as H-selfsimilarity. Not always realised in physics that model doesn’t need to be H-ss to show LRD, e.g. ARFIMA, [Watkins, GRL Frontiers, 2013]. Corollary is that SRD can blur classic LRD methods, range of methods desirable [Franzke et al, Phil. Trans. Roy. Soc A, 2012].

3. Graves PhD: Develop MCMC method to perform Bayesian inference on ARFIMA(p,d,q). Gaussian first, tested on model (& real) data.

4. However, many real datasets not Gaussian. ARFIMA can allow alpha stable innovations. Modify method to allow joint inference of d,alpha.

5. Study dependence of posterior variance of inferred d on length of time series.

5 December 2012 20

Models in Physics & Time Series Analysis

“Models in physics are like Austrian train timetables. Trains in Austria are always late, but without a timetable we wouldn’t know how late they are”.

--- attributed to Pauli, in Kleppner & Kolenkow, “An Introduction to Mechanics” “Remember that all models are wrong; the practical

question is how wrong do they have to be [in order] to not be useful”.

--- Box & Draper, “Empirical Model Building”

21