Bayesian Analysis of Non-Gaussian Long range Dependent Processes
Tim Graves (Statistics Laboratory, Cambridge) Christian Franzke (BAS, Cambridge)
Bobby Gramacy (Booth School of Business, Chicago) & Nick Watkins (BAS, LSE & Warwick) [[email protected]]
NG22A-04 11am Tuesday 4th December 2012 Scaling and Correlations and their use in forecasting Natural Hazards I Room 300 Moscone South
Summary: 1. Standard climate noise models have short range dependence
(SRD). Recent evidence of long range dependence in surface temperatures. Example is Antarctic study [Franzke, J. Climate, 2010]. LRD hampers trend identification and quantification of significance.
2. LRD idea originated at same time as H-selfsimilarity, so not always realised that a model doesn’t need to be H-ss to show LRD, e.g. ARFIMA, [Watkins, GRL Frontiers, 2013].
3. Graves PhD has developed MCMC method to perform Bayesian inference on ARFIMA(p,d,q). Treated Gaussian ARFIMA first, tested on model (& real) data. Study dependence of posterior variance of inferred d on length of time series.
4. However, many real datasets not Gaussian. ARFIMA can allow alpha stable innovations. Graves has modified method to allow joint inference.
5 December 2012 2
Short & long range dependence
5 December 2012 3
( ) exp( )ρ τ λτ−
( ) ( )ρ τ δ τ
Delta correlated white noise
2 1( ) dk ckρ −
0 1/ 2d< <
Exponentially correlated red AR(1), SRD
0( )
k
kkρ
=∞
=
= ∞∑LRD: power law correlated, “1/f” noise
LRD & Antarctic temperature trends
4
[Franzke, J. Climate, 2010] found evidence of LRD in station temperature series. 3 have significant EMD residual trend (dashes) against SRD null model. One station [Faraday] still significant against LRD.
LRD and selfsimilarity, common history ...
Mandelbrot (& Wallis), mid 1960s: 2 departures from AR(1), “Biblical geoscience” illustrations, selfsimilarity exponent H
• heavy tails in amplitude, cotton prices. “Noah” effect, 40 days and 40 nights of rain. • long range dependence in Nile level. 7 lean & 7 fat years: “Joseph effect”. 5 December 2012 5
LRD and selfsimilarity, common history ...
Mandelbrot (& Wallis), mid 1960s: 2 departures from AR(1), “Biblical geoscience” illustrations, selfsimilarity exponent H
• heavy tails in amplitude, cotton prices. “Noah” effect, 40 days and 40 nights of rain. • long range dependence in Nile level. 7 lean & 7 fat years: “Joseph effect”. 5 December 2012 6
LRD and selfsimilarity, common history ...
Mandelbrot (& Wallis), mid 1960s: 2 departures from AR(1), “Biblical geoscience” illustrations, selfsimilarity exponent H
• heavy tails in amplitude, cotton prices. “Noah” effect, 40 days and 40 nights of rain. • long range dependence in Nile level. 7 lean & 7 fat years: “Joseph effect”. 5 December 2012 7
… not necessarily common origin • Both fBm & Levy flights are H-selfsimilar, • Frac. Brownian motion: H (here = J) = d + ½ LRD (persistence): 0 < d < ½; “Hurst” exponent
increases from Brownian value of ½ as memory parameter d increases
• Levy flights: H = 1/α α is exponent of pdf heavy tail. • Both are limits of H = 1/ α +d • Many methods (e.g. R/S …) inspired by self-
similarity, & measure d, not α, via geometry. 8
Can model LRD (d) without assuming complete H-selfsimilarity Don’t actually need completely H-selfsimilar
models to exhibit LRD (just asymptotic) In 1980s Granger and Joyeux modified SRD Auto
Regressive Moving Average [ARMA(p,q)] models to allow LRD via Fractional Integration of order d [ARFIMA(p,d,q)].
Physically interesting: High frequency p term(s) that turns nonstationary, H-ss random walk into weakly stationary AR(p) i.e. dissipation
5 December 2012 9
AR(1): 1st order AutoRegressive
5 December 2012 10
1 1t t tX Xφ ε−= +
0 100 200 300 400 500 600 700 800 900 1000-8
-6
-4
-2
0
2
4
6
8Example series of AR(1)
1 0.9φ =
AR(1): 1st order AutoRegressive
5 December 2012 11
1(1( ) ) t tB B Xφ εΦ − ==
1 1t t tX Xφ ε−= +
1ttBX X −=0 100 200 300 400 500 600 700 800 900 1000
-8
-6
-4
-2
0
2
4
6
8Example series of AR(1)
1 0.9φ =
1( ) 1
pj
jj
z zφ=
Φ = −∑
AutoRegressive Fractionally Integrated Moving Average
[ARFIMA(p,d,q)]
5 December 2012 12
( )(1 ) ( )dt tB B X B εΦ − = Θ
Autoregressive term of order p
Moving average of order q
1( ) 1
qj
jj
z zθ=
Θ = +∑
Fractional integration of order d
Granger (& Joyeux), 1980
(1 )dt tB X ε− =Pure LRD
ARFIMA(0,d,0 ):
Exact Bayesian inference on ARFIMA for d
• ARFIMA has parameters μ, σ, d, φ, θ. All but d essentially nuisance parameters here.
• First assume Gaussian innovations. • Assume flat priors for μ, log σ and d … • Even with this, likelihood for d very complex • No analytic posterior --- use MCMC sampling
5 December 2012 13
( | ) ( ) ( | )x p L xψ ψπ ψ ψ ψ∝
Key features
• Don’t want to assume form of p, q – use R J MCMC [Green, Biometrika 1995]
• Reparameterisation of model to enforce stationarity constraints on φ and θ.
• Efficient calculation of Gaussian likelihood (long memory correlation structure prevents use of standard quick methods)
• Necessary use of Metropolis-Hastings requires careful selection of proposal distribution
• Parameter correlation (φ,d) requires blocking 5 December 2012 14
“Calibration”
5 December 2012 15
~ 1/d nσ
• Have studied how posterior variance of d depends on sample size n, c.f. Kiyani et al, PRE, 2009. study of structure functions etc • Looked at standard test series like Nile river, find d of about .4 and ARFIMA(0,d,0) most probable model. Confims e.g Beran, 1994. • Looked at CET. Dependence more complicated and a model incorporating seasonality performs better.
Approximate inference in more general case
• Drop Gaussianity assumption. • Go to more general distribution (α-stable). • Seek joint inference on d, α • Approximate long memory process as very
high order AR • Construct likelihood sequentially • Use auxiliary variables to integrate out
unknown history 5 December 2012 16
Pure symmetric α-stable ARFIMA
5 December 2012 17
0.15(1 ) t tB X ε− = 1.5α =
Posterior estimates of d, α
5 December 2012 18
1.5α =0.15d =
Scatter of d and α
5 December 2012 19 1 1t t tX Xφ ε−= +
Good estimation of all parameters. Posteriors of d and α are independent.
Conclusions: 1. Standard climate noise model AR(1). Discretises Ornstein-Uhlenbeck
physics. Short range dependence (SRD). However recent evidence of long range dependence (d nonzero) in surface temperatures. Example is multistation Antarctic study [Franzke, J. Climate, 2010]. LRD hampers trend identification and quantification of significance.
2. LRD idea originated at same time as H-selfsimilarity. Not always realised in physics that model doesn’t need to be H-ss to show LRD, e.g. ARFIMA, [Watkins, GRL Frontiers, 2013]. Corollary is that SRD can blur classic LRD methods, range of methods desirable [Franzke et al, Phil. Trans. Roy. Soc A, 2012].
3. Graves PhD: Develop MCMC method to perform Bayesian inference on ARFIMA(p,d,q). Gaussian first, tested on model (& real) data.
4. However, many real datasets not Gaussian. ARFIMA can allow alpha stable innovations. Modify method to allow joint inference of d,alpha.
5. Study dependence of posterior variance of inferred d on length of time series.
5 December 2012 20
Models in Physics & Time Series Analysis
“Models in physics are like Austrian train timetables. Trains in Austria are always late, but without a timetable we wouldn’t know how late they are”.
--- attributed to Pauli, in Kleppner & Kolenkow, “An Introduction to Mechanics” “Remember that all models are wrong; the practical
question is how wrong do they have to be [in order] to not be useful”.
--- Box & Draper, “Empirical Model Building”
21