ACF/PACF Estimation and AR/MA models · 2008-01-14 · §2.5 (cont): ACF & PACF Estimation§2.6 MA(1) and AR(1) Representations PACF Estimation The sample partial autocorrelation

ACF/PACF Estimation and AR/MA models

§2.5 (cont): ACF & PACF Estimation §2.6 MA(∞) and AR(∞) Representations

Outline

1 §2.5 (cont): ACF & PACF Estimation

2 §2.6 MA(∞) and AR(∞) Representations

Arthur Berg ACF/PACF Estimation and AR/MA models 2/ 13


Autocovariance Estimation

The sample autocovariance function.

γ̂(h) =1n

n−h∑t=1

(Zt+h − Z̄)(Zt − Z̄)

And compare γ̂(h) with γ(h).

γ(h) = E [(Zt+h − µ) (Zt − µ)]

The following estimator is slightly less biased, but it’s variance is often larger.

ˆ̂γ(h) =1

n− h

n−h∑t=1

(Zt+h − Z̄)(Zt − Z̄)



Autocovariance Estimation

The sample autocovariance function.

γ̂(h) =1n

n−h∑t=1

(Zt+h − Z̄)(Zt − Z̄)

And compare γ̂(h) with γ(h).

γ(h) = E [(Zt+h − µ) (Zt − µ)]

The following estimator is slightly less biased, but it’s variance is often larger.

ˆ̂γ(h) =1

n− h

n−h∑t=1

(Zt+h − Z̄)(Zt − Z̄)



Asymptotics of γ̂

When the process {Zt} is Gaussian, Bartlett (1946) derived the followingasymptotics of γ̂k:

cov(γ̂k, γ̂k+j) ≈1n

∞∑i=1

(γiγi+j + γi+k+jγi−k)

var(γ̂k) ≈1n

∞∑i=−∞

(γ2i + γi+kγi−k)

To guarantee γ̂ → 0, the following condition is imposed:

∞∑−∞|γi| <∞



Asymptotics of γ̂

When the process {Zt} is Gaussian, Bartlett (1946) derived the followingasymptotics of γ̂k:

cov(γ̂k, γ̂k+j) ≈1n

∞∑i=1

(γiγi+j + γi+k+jγi−k)

var(γ̂k) ≈1n

∞∑i=−∞

(γ2i + γi+kγi−k)

To guarantee γ̂ → 0, the following condition is imposed:

∞∑−∞|γi| <∞



Autocorrelation Estimation and Asymptotics

The sample autocorrelation function is defined by

ρ̂k =γ̂k

γ̂0=∑n−k

t=1 (Zt − Z̄)(Zt+k − Z̄)∑nt=1(Zt − Z̄)2

with asymptotics given by

cov(ρ̂k, ρ̂k+j) ≈1n

∞∑i=−∞

(ρiρi+j + ρi+k+jρi−k

− 2ρkρiρi−k−j − 2ρk+jρiρi−k + 2ρkρk+jρ2i )

var(ρ̂k) ≈1n

∞∑i=1

(ρ2i + ρi+kρi−k − 4ρkρiρi−k + 2ρ2

kρ2i )

?≈ 1

n

(1 + 2ρ2

1 + 2ρ22 + · · ·+ 2ρ2

m)

The second variance approximation holds if ρk = 0 for k > m.Arthur Berg ACF/PACF Estimation and AR/MA models 5/ 13


PACF Estimation

The sample partial autocorrelation function is computed via theDurbin-Levinson recursive algorithm (1960).Start with φ̂11 = ρ̂1then recursively compute

φ̂k+1,k+1 =ρ̂k+1 −

∑kj=1 φ̂kjρ̂k+1−j

1−∑k

j=1 φ̂kjρ̂j

andφ̂k+1,j = φ̂kj − φ̂k+1,k+1φ̂k,k+1−j j = 1, . . . , k

Quenouille (1949) showed that for a white noise process the followingapproximation holds

var(φ̂kk) ≈1n

Hence ±2/√

n can be used as critical limits on φkk to test for white noise.



PACF Estimation


φ̂k+1,k+1 =ρ̂k+1 −


1−∑k

j=1 φ̂kjρ̂j



var(φ̂kk) ≈1n

Hence ±2/√




PACF Estimation


φ̂k+1,k+1 =ρ̂k+1 −


1−∑k

j=1 φ̂kjρ̂j



var(φ̂kk) ≈1n

Hence ±2/√




PACF Estimation


φ̂k+1,k+1 =ρ̂k+1 −


1−∑k

j=1 φ̂kjρ̂j



var(φ̂kk) ≈1n

Hence ±2/√




PACF Estimation


φ̂k+1,k+1 =ρ̂k+1 −


1−∑k

j=1 φ̂kjρ̂j



var(φ̂kk) ≈1n

Hence ±2/√




PACF Estimation


φ̂k+1,k+1 =ρ̂k+1 −


1−∑k

j=1 φ̂kjρ̂j



var(φ̂kk) ≈1n

Hence ±2/√




Outline

1 §2.5 (cont): ACF & PACF Estimation

2 §2.6 MA(∞) and AR(∞) Representations



Herman Ole Andreas Wold (1908-1992)

Theorem (Wold Decomposition)A stationary time series Zt that is purelynondeterministic can always beexpressed in the form

Zt = µ+ at + ψ1at−1 + ψ2at−2 + · · ·

= µ+∞∑

j=0

ψjat−j

where ψ0 = 1, {at} is a zero meanwhite noise process, and

∑∞j=0 ψ

2j <∞.

Moreover, this decomposition is unique.



Backshift and Difference OperatorsDefinition (Backshift Operator)The backshift operator, B, is defined by

BZt = Zt−1

and repeated backshift operations are represented by

BkZt = BB · · ·B︸︷︷︸k times

Zt = Zt−k

Definition (Difference Operator)The difference operator, ∆, is defined by

∆Zt = (1− B)Zt = Zt − Zt−1

and repeated d times is called differences of order d which represented by

∆d =

d times︷︸︸︷(1− B)(1− B) · · · (1− B) Zt



Backshift and Difference OperatorsDefinition (Backshift Operator)The backshift operator, B, is defined by

BZt = Zt−1

and repeated backshift operations are represented by

BkZt = BB · · ·B︸︷︷︸k times

Zt = Zt−k

Definition (Difference Operator)The difference operator, ∆, is defined by

∆Zt = (1− B)Zt = Zt − Zt−1

and repeated d times is called differences of order d which represented by

∆d =

d times︷︸︸︷(1− B)(1− B) · · · (1− B) Zt



Moving Average Model — MA(q)

Definition (Moving average model — MA(q))The moving average model of order q is defined to be

Zt = µ+ at + θ1at−1 + θ2at−2 + · · ·+ θqat−q

where θ1, θ2, . . . θq are parameters in R.

The above model can be compactly written as

Zt = µ+ θ(B)at

where θ(B) is the moving average operator.

Definition (Moving Average Operator)The moving average operator is

θ(B) = 1 + θ1B + θ2B2 + · · ·+ θqBq








Zt = µ+ θ(B)at



θ(B) = 1 + θ1B + θ2B2 + · · ·+ θqBq








Zt = µ+ θ(B)at



θ(B) = 1 + θ1B + θ2B2 + · · ·+ θqBq



MA(1)

Consider the mean zero MA(1) process

Zt = at + θat−1

Then the autocovariance function is computed as

γ(h) = cov(Zt+h,Zt)

= E [(at+h + θat+h−1)(at + θat−1)]

= E(at+hat)︸︷︷︸h=0

+θ E(at+h−1at)︸︷︷︸h=1

+θ E(at+hat−1)︸︷︷︸h=−1

+θ2 E(at+h−1at−1)︸︷︷︸h=0

=

(1 + θ2)σ2, h = 0θσ2, |h| = 10, |h| > 1



MA(1)


Zt = at + θat−1


γ(h) = cov(Zt+h,Zt)

= E [(at+h + θat+h−1)(at + θat−1)]

= E(at+hat)︸︷︷︸h=0

+θ E(at+h−1at)︸︷︷︸h=1

+θ E(at+hat−1)︸︷︷︸h=−1

+θ2 E(at+h−1at−1)︸︷︷︸h=0

=

(1 + θ2)σ2, h = 0θσ2, |h| = 10, |h| > 1



MA(1)


Zt = at + θat−1


γ(h) = cov(Zt+h,Zt)= E [(at+h + θat+h−1)(at + θat−1)]

= E(at+hat)︸︷︷︸h=0

+θ E(at+h−1at)︸︷︷︸h=1

+θ E(at+hat−1)︸︷︷︸h=−1

+θ2 E(at+h−1at−1)︸︷︷︸h=0

=

(1 + θ2)σ2, h = 0θσ2, |h| = 10, |h| > 1



MA(1)


Zt = at + θat−1



= E(at+hat)︸︷︷︸h=0

+θ E(at+h−1at)︸︷︷︸h=1

+θ E(at+hat−1)︸︷︷︸h=−1

+θ2 E(at+h−1at−1)︸︷︷︸h=0

=

(1 + θ2)σ2, h = 0θσ2, |h| = 10, |h| > 1



MA(1)


Zt = at + θat−1



= E(at+hat)︸︷︷︸h=0

+θ E(at+h−1at)︸︷︷︸h=1

+θ E(at+hat−1)︸︷︷︸h=−1

+θ2 E(at+h−1at−1)︸︷︷︸h=0

=

(1 + θ2)σ2, h = 0θσ2, |h| = 10, |h| > 1



MA(1) — ACF Computation

Therefore

ρ(h) =

1, h = 0θ

1+θ2 , |h| = 1

0, |h| > 1

Note that when θ = 1θ′ , we have

ρ(h) =

1, h = 0θ′−1

1+θ′−2 = θ′

θ′2+1 , |h| = 1

0, |h| > 1



MA(1) — ACF Computation

Therefore

ρ(h) =

1, h = 0θ

1+θ2 , |h| = 1

0, |h| > 1

Note that when θ = 1θ′ , we have

ρ(h) =

1, h = 0θ′−1

1+θ′−2 = θ′

θ′2+1 , |h| = 1

0, |h| > 1



Invertibility

Recall the MA(q) process:


We wish MA(q) processes to be uniquely determined by their ACFs.Invertibility guarantees this.

Definition (Invertibility)An MA model process is called invertible if Zt has the AR(∞) representation

Zt =∞∑

j=0

πjZt−j + at

In the MA(1) case, this amounts to |θ| < 1.We later see an easy way of determining whether a given MA model is invertiblebased on the roots of the MA operator.



Invertibility





Zt =∞∑

j=0

πjZt−j + at




Invertibility





Zt =∞∑

j=0

πjZt−j + at




Invertibility





Zt =∞∑

j=0

πjZt−j + at




Invertibility





Zt =∞∑

j=0

πjZt−j + at



Documents

ACF/PACF Estimation and AR/MA models · 2008-01-14 · §2.5 (cont): ACF & PACF Estimation§2.6 MA(1) and AR(1) Representations PACF Estimation The sample partial autocorrelation