Based on the book by Fan/Yao Nonlinear Time Serieshomepage.univie.ac.at/robert.kunst/nonlin1.pdfCharacteristics of Time Series Threshold models ARCH and GARCH models Bilinear models

Characteristics of Time Series Threshold models ARCH and GARCH models Bilinear models

Nonlinear time seriesBased on the book by Fan/Yao: Nonlinear Time Series

Robert M. [email protected]

University of Viennaand

Institute for Advanced Studies Vienna

October 27, 2009

Nonlinear time series University of Vienna and Institute for Advanced Studies Vienna


Outline

Characteristics of Time Series

Threshold models

ARCH and GARCH models

Bilinear models



What is a nonlinear time series?

Formal definition: a nonlinear process is any stochastic processthat is not linear. To this aim, a linear process must be defined.Realizations of time-series processes are called time series but theword is also often applied to the generating processes.

Intuitive definition: nonlinear time series are generated bynonlinear dynamic equations. They display features that cannot bemodelled by linear processes: time-changing variance, asymmetriccycles, higher-moment structures, thresholds and breaks.



Definition of a linear process

DefinitionA stochastic process (Xt , t ∈ Z) is said to be a linear process if forevery t ∈ Z

Xt =∞

∑

j=0

ajεt−j ,

where a0 = 1, (εt , t ∈ Z) is iid with Eεt = 0, Eε2t <∞, and∑∞

j=0 |aj | <∞.



For comparison: the Wold theorem

Theorem (Wold’s Theorem)

Any covariance-stationary process (Xt) has a unique representationas the sum of a purely deterministic component and an infinitesum of white-noise terms, in symbols

Xt = δt +∞

∑

j=0

ajεt−j ,

with a0 = 1,∑∞

j=0 a2j <∞, and the terms εt defined as the linear

innovations Xt − E∗(Xt |Ht−1), where E

∗ denotes the linearexpectation or projection on the space Ht−1 that is generated bythe observations Xs , s ≤ t − 1.



Linear processes and the Wold representation

◮ There are many covariance-stationary processes that are notlinear: either the innovations are not independent (thoughwhite noise) or the absolute coefficients do not converge.

◮ If it is just the absolute coefficients, the processes are longmemory. Those are not really nonlinear, and we will nothandle them here.

◮ Many nonlinear processes have a Wold representation thatlooks linear. It is correct but it just describes theautocovariance structure. It is an incomplete representation.

◮ If Eε2t <∞ is violated, (Xt) becomes an infinite-variancelinear process, conditions on coefficient series must beadjusted. Outside the scope here.



A simple example: an AR-ARCH model

The dynamic generating law

Xt = 0.9Xt−1 + ut , (1)

ut = h0.5t εt , (2)

ht = 1 + 0.9u2t−1, εt ∼ NID(0, 1), (3)

defines a stable AR-ARCH process. It is an AR(1) model withWold innovations ut , which are white noise but not independent.



A time series plot of 1000 observations

0 200 400 600 800 1000

−30

−20

−10

010

Impression: not too nonlinear, but outlier patches point tofat-tailed distributions: variable has no finite kurtosis.



Correlograms

0 5 10 15 20 25 30

0.0

0.2

0.4

0.6

0.8

1.0

Lag

0 5 10 15 20 25 300.

00.

20.

40.

60.

81.

0

Lag

Impression: AR-ARCH on the left inconspicuous, fairly identical toa standard AR(1) correlogram on the right.



Plots versus lags

−20 −10 0 10 20

−20

−10

010

20

y(−1)

y

−6 −4 −2 0 2 4 6−

6−

4−

20

24

6

x(−1)

x

Impression: a bit more dispersed than the standard AR(1) plot onthe right: leptokurtosis. Basically linear (as should be).



Plots of squares versus lagged squares

0 100 200 300 400

010

020

030

040

0

y²(−1)

y²

0 10 20 30 400

1020

3040

x²(−1)

x²

Impression: even this device, suggested by Andrew A. Weissallows no reliable discrimination to the standard AR case on theright.



I. Characteristics of Time Series

The meaning of this section:

This section corresponds to Section 2 of the book by Fan & Yaoand is meant to review the basic concepts of (mostly linear)time-series analysis.



Stationarity

DefinitionA time series (Xt , t ∈ Z) is (weakly, covariance) stationary if (a)EX 2

t <∞, (b) EXt = µ ∀t, and (c) cov(Xt ,Xt+k) is independentof t ∀k .

Remark. For k = 0, this definition yields time-constant finitevariance.

DefinitionA time series (Xt , t ∈ Z) is strictly stationary if (X1, . . . ,Xn) and(X1+k , . . . ,Xn+k) have the same distribution for any n ≥ 1 andn, k ∈ Z.

Remark. For nonlinear time series, often strict stationarity is themore ‘natural’ concept.



ARMA processes

The ARMA(p, q) model with p, q ∈ N

Xt = b1Xt−1 + . . .+ bpXt−p + εt + a1εt−1 + . . .+ aqεt−q,

for (εt) white noise, is technically rewritten as

b(B)Xt = a(B)εt ,

with polynomials a(z) and b(z) and the backshift operator Bdefined by BkXt = Xt−k .

This is the most popular linear time-series model. Often, theexpression ARMA process is reserved for stable polynomials.



Stationarity of ARMA processes

TheoremThe process defined by the ARMA(p, q) model is stationary ifb(z) 6= 0 for all z ∈ C with |z | ≤ 1, assuming that a(z) and b(z)have no common factors.

Remark. Note that t ∈ Z. If t ∈ N, the ARMA process can be‘started’ from arbitrary conditions, and the usual distinction of‘stable’ and ‘stationary’ applies. Clearly, here a pure MA process(p = 0) is always stationary.



Causal time series

DefinitionA time series (Xt) is causal if for all t

Xt =∞

∑

j=0

djεt−j ,∞

∑

j=0

|dj | <∞,

with white noise (εt).

Remark. A causal process is always stationary. Xt = 2Xt−1 + εtviolates the ARMA stability conditions and nonetheless has astationary non-causal solution. These are at odds with intuitionand will be excluded.



Stationary Gaussian processes

A process (Xt) is called Gaussian if all its finite-dimensionalmarginal distributions are normal. For Gaussian processes, theWold representation holds with iid innovations.

◮ The purely nondeterministic part of a Gaussian process islinear.

◮ A Gaussian MA(q) process is q–dependent, i.e. Xt andXt+q+k are independent for all k ≥ 1.

◮ For a Gaussian AR process, Xt is independent of Xt−k , k > pgiven Xt−1, . . . ,Xt−p.



Autocovariance and autocorrelation

DefinitionLet (Xt) be stationary. The autocovariance function (ACVF) of(Xt) is

γ(k) = cov(Xt+k ,Xt), k ∈ Z.

The autocorrelation function (ACF) of (Xt) is

ρ(k) = γ(k)/γ(0) = corr(Xt+k ,Xt), k ∈ Z.

It follows that γ(−k) = γ(k) and ρ(−k) = ρ(k) (even functions).



Characterization of the ACVF

TheoremA function γ(.) : Z → R is the ACVF of a stationary time series ifand only if it is even and nonnegative definite in the sense that

n∑

i=1

n∑

j=1

aiajγ(i − j) ≥ 0

for all integer n ≥ 1 and arbitrary real a1, . . . , an.

Remark. One direction is easy to show, as it uses the properties ofa covariance matrix of Xt , . . . ,Xt−n+1. The reverse direction ishard to prove and needs a Theorem by Kolmogorov.



ACF of stationary ARMA processes

Any ARMA process has an MA(∞) representation

Xt =

∞∑

j=0

ajεt−j ,

with∑∞

j=0 |aj | <∞. It follows that

γ(k) = σ2∞

∑

j=0

∞∑

j=0

ajaj+k , ρ(k) =

∑∞j=0 ajaj+k∑∞

j=0 a2j

.

Clearly, for pure MA(q) processes, these expressions become 0 fork > q.



Properties of the ACF for ARMA processes

Proposition

1. For causal ARMA processes, ρ(k) → 0 like ck for |c | < 1 ask → ∞ (exponential);

2. for MA(q) processes, ρ(k) = 0 for k > q.

This proposition does not warrant a clear distinction between ARand general ARMA processes.



Estimating the ACF

The sample ACF (the correlogram) is defined byρ̂(k) = γ̂(k)/γ̂(0), where

γ̂(k) =1

T

T−k∑

t=1

(Xt − X̄T )(Xt+k − X̄T ),

for small k , for example k ≤ T/4 or k ≤ 2√

T , withX̄T = T−1

∑Tt=1 Xt .



Statistical properties of the mean estimate

TheoremLet (Xt) is a linear stationary process defined byXt = µ+

∑∞j=0 ajεt−j , with (εt) iid(0, σ2) and

∑∞j=0 |aj | <∞. If

∑∞j=0 aj 6= 0,

√T (X̄T − µ) ⇒ N(0, ν2

1), where

ν21 =

∞∑

j=−∞

γ(j) = σ2

∞∑

j=0

aj

2

.

Remark. The variance of the mean estimate depends on thespectrum at 0 and may be called the long-run variance.



The long-run variance

A stationary time-series process Xt =∑

ajεt−j has the variance

varXt = γ(0) = σ2ε

∞∑

j=0

a2j ,

which usually differs from the long-run variance

σ2ε(

∞∑

j=0

aj)2.

The long-run variance may also be seen as the spectrum at 0,∑∞

j=−∞ γ(j), or as the limit variance

limn→∞

n−1var

n∑

t=0

Xt .

For white noise (Xt), variance and long-run variance coincide.



Statistical properties of the variance estimate



∑∞j=0 |aj | <∞. If

Eε4t <∞,√

T{γ̂(0) − γ(0)} ⇒ N(0, ν22), where

ν22 = 2σ2(1 + 2

∞∑

j=1

ρ(j)2).

Remark. This is reminiscent of the portmanteau Q.



Statistical properties of the correlogram



∑∞j=0 |aj | <∞. If

Eε4t <∞,√

T{ ˆ̺(k) − ̺(k)} ⇒ N(0,W), where

wij =∞

∑

t=−∞

{ρ(t + i)ρ(t + j) + ρ(t − i)ρ(t + j) + 2ρ(i)ρ(j)ρ(t)2

−2ρ(i)ρ(t)ρ(t + j) − 2ρ(j)ρ(t)ρ(t + i)},

where ̺(k) = (ρ(1), . . . , ρ(k))′.

Remark. This is Bartlett’s formula, impressive but not immediatelyuseful. In simple cases, it allows determining confidence bands forthe correlogram. White noise yields

√T ρ̂k ⇒ N(0, 1) for k 6= 0.



Partial autocorrelation function

Definition(Xt) is a stationary process with EXt = 0. The partialautocorrelation function (PACF) π : N → [−1, 1] is defined byπ(1) = ρ(1) and

π(k) = corr(R1|2,...,k ,Rk+1|2,...,k),

for k ≥ 2, where Rj |2,...,k denotes residuals from regressing Xj onX2, . . . ,Xk by least squares.

Remark. For non-Gaussian processes, the thus defined PACF doesnot necessarily correspond to partial correlations, if partialcorrelations are defined via conditional expectations.



Properties of the PACF

Proposition

1. For any stationary process, π(k) is a function of the ACVFvalues γ(1), . . . , γ(k);

2. For an AR(p) process, π(k) = 0 for k > p, i.e. the PACF ‘cutsoff at p’.

Remark. Mathematically, the PACF does not provide any newinformation on top of the ACVF. The sample PACF facilitatesvisual pattern recognition and may indicate AR models and theirlag order.



Mixing

White noise is usually insufficient for ergodic theorems—such aslaws of large numbers (LLN) or central limit theorems (CLT). Forlinear processes with iid innovations, some moment conditions willsuffice. For nonlinear processes, more is needed.

Generally, mixing conditions guarantee that Xt and Xt+h are moreor less independent for large h. The metaphor is mixing drinks: adrop of a liquid will not remain close to its origin. If we mix twoseparate glasses, the outcome will be stationary but not mixing.



Five mixing coefficients

α(n) = supA∈F0

−∞,B∈F∞

n

|P(A)P(B) − P(AB)|,

β(n) = E

{

supB∈F∞

n

|P(B) − P(B|X0,X−1,X−2, . . .)|}

,

ρ(n) = supX∈L2(F0

−∞),Y∈L2(F∞

n )

|corr(X ,Y )|,

φ(n) = supA∈F0

−∞,B∈F∞

n ,P(A)>0

|P(B) − P(B|A)|,

ψ(n) = supA∈F0

−∞,B∈F∞

n ,P(A)P(B)>0

|1 − P(B|A)/P(B)|.

Here, F lk is the σ–algebra generated by Xt , k ≤ t ≤ l , and L2 are square

integrable functions. Good mixing coefficients converge to 0 as n → ∞.



Mixing processes

DefinitionA process (Xt) is α–mixing (‘strong mixing’) if α(n) → 0 asn → ∞, and similarly for β–mixing etc.

A basic property is

α(k) ≤ 1

4ρ(k) ≤ 1

2

√

φ(k),

and generally φ–mixing implies ρ–mixing (and β–mixing), andρ–mixing (or β–mixing) implies α–mixing. ψ–mixing impliesφ–mixing and thus all others. Even for simple examples, the mixingcoefficients cannot be evaluated directly.



Some properties of mixing processes

1. If (Xt) is mixing (any definition) and m(.) is a measurablefunction, then (m(Xt)) is again mixing. The property is‘hereditary’.

2. If (Xt) is a linear ARMA process and εt has a density, it isβ–mixing with β(n) → 0 exponentially.

3. A strictly stationary process on Z is mixing iff its restriction toN is mixing (any definition).

4. Finite-dependent (such as strict MA) or independentprocesses are mixing.

5. Deterministic processes are not mixing.



A LLN for α–mixing processes

Proposition

Assume (Xt) is strictly stationary and α–mixing, and E|Xt | <∞.Then, as n → ∞, Sn/n → EXt a.s., where Sn =

∑nt=1 Xt .

A comparable LLN with slightly stronger conditions guaranteesconvergence of n−1

varSt to the long-run variance.



A CLT for α–mixing processes

TheoremAssume (Xt) is strictly stationary with EXt = 0, the long-runvariance σ2 is positive, and one of the following conditions hold

1. E|Xt |δ <∞ and∑∞

j=0 α(j)1−2/δ <∞ for some δ > 2;

2. P(|Xt | < c) = 1 for some c > 0 and∑∞

j=1 α(j) <∞.

Then,Sn/

√n ⇒ N(0, σ2),

with Sn =∑n

t=1 Xt .

Remark. The theorem cares for processes with bounded support(case 2) as well as for some quite fat-tailed ones (case 1 forδ = 2 + ǫ), which however need fast decrease of their mixingcoefficients.








Documents

Based on the book by Fan/Yao Nonlinear Time Serieshomepage.univie.ac.at/robert.kunst/nonlin1.pdfCharacteristics of Time Series Threshold models ARCH and GARCH models Bilinear models