1Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Marcel DettlingInstitute for Data Analysis and Process Design
Zurich University of Applied Sciences
http://stat.ethz.ch/~dettling
ETH Zürich, May 9, 2011
2Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Prewhitening the Gas Furnace DataWhat to do:
- AR(p)-models are fitted to the differenced series
- The residual time series are Ut and Vt, white noise
- Check the sample cross correlation (see next slide)
- Model the output as a linear combination of pastinput values: transfer function model.
3Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Prewhitening the Gas Furnace Data
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
AC
F
PrW.Out
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
PrW.Out & PrW.Inp
-20 -15 -10 -5 0
-0.5
0.0
0.5
1.0
Lag
AC
F
PrW.Inp & PrW.Out
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
PrW.Inp
4Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Transfer Function ModelsProperties:
- Transfer function models are an option to describe the dependency between two time series.
- The first (input) series influences the second (output) one, but there is no feedback from output to input.
- The influence from input to output only goes „forward“.
The model is:
2, 2 1, 10
( )t j t j tj
X X E
5Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Transfer Function ModelsThe model is:
- E[Et]=0.
- Et and X1,s are uncorrelated for all t and s.
- Et and Es are usually correlated.
- For simplicity of notation, we here assume that the series have been mean centered.
2, 2 1, 10
( )t j t j tj
X X E
6Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Cross CovarianceWhen plugging-in, we obtain for the cross covariance:
- If only finitely many coefficients are different from zero, we could generate a linear equation system, plug-inand to obtain the estimates .
This is not a statistically efficient estimation method.
21 2, 1, 1, 1, 110 0
( ) ( , ) , ( )t k t j t k j t jj j
k Cov X X Cov X X k j
1̂21̂ ˆ j
7Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Special Case: X1,t UncorrelatedIf X1,t was an uncorrelated series, we would obtain thecoefficients of the transfer function model quite easily:
However, this is usually not the case. We can then:
- transform all series in a clever way- the transfer function model has identical coefficients- the new, transformed input series is uncorrelated
see blackboard for the transformation…
21
11
( )(0)kk
8Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Gas Furnace Transformed
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
AC
F
pr.Out
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
pr.Out & pr.Inp
-20 -15 -10 -5 0
-0.5
0.0
0.5
1.0
Lag
AC
F
pr.Inp & pr.Out
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
pr.Inp
9Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Gas Furnace: Final Remarks• In the previous slide, we see the empirical cross correlations
of the two series and .
• The coefficients from the transfer function model will beproportional to the empirical cross correlations. We can al-ready now conjecture that the output is delayed by 3-7 times, i.e. 27-63 seconds.
• The formula for the transfer function model coefficients is:
21ˆˆ ˆ ( )ˆ
Zk
D
k
21ˆ ( )k tD tZ
ˆk
10Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
State Space ModelsBasic idea: There is a stochastic process/time series
which we cannot directly observe, but only under the addition of some white noise.
Thus: We observe the time series
where , i.i.d.
Example: = # of fish in a lake= # estimated number of fish from a sample
tZ
t t tY Z U 2~ (0, )tU N
tZtY
11Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Notation and Terminology for an AR(1)We assume that the true underlying process is an AR(1), i.e.
,
where
are i.i.d. innovations, „process noise“.
In practice, we only observe , as realizations of the process
, with , i.i.d.
and additionally, the are independent of , for all s,t, thus they are independent „observation white noise“.
1 1t t tZ a Z E
t t tY Z U 2~ (0, )tU N
ty
2~ (0, )t EE N
tU sEsZ
12Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
More TerminologyWe call
the „state equation“, and
the „observation equation“.
On top of that, we remember once again that the „process noise“ is an innovation that affects all future values and thus also , whereas only influences the currentobservation , but no future ones.
1 1t t tZ a Z E
t t tY Z U
tYtU
tE t kZ
t kY
13Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Example with α=0.7
Time
yt1
0 20 40 60 80 100
-2-1
01
2
Zustands-Prozess X_tBeobachtungs-Prozess Y_t
AR(1)-Example
14Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
ACF/PACF of Zt
Time
serie
s
0 20 40 60 80 100
-1.0
-0.5
0.0
0.5
1.0
-0.2
0.4
1.0
Lag k
Aut
o-K
orr.
0 5 10 15 20
-0.2
0.2
0.6
Lag k
part.
Aut
okor
r
1 5 10 15 20
15Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
ACF/PACF of Yt
Time
serie
s
0 20 40 60 80 100
-2-1
01
2-0
.20.
41.
0
Lag k
Aut
o-K
orr.
0 5 10 15 20
-0.2
0.1
Lag k
part.
Aut
okor
r
1 5 10 15 20
16Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Example with α=1
Time
yt1
0 20 40 60 80 100
-20
24
6
Zustands-Prozess X_tBeobachtungs-Prozess Y_t
AR(1)-Example
17Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
ACF/PACF of Zt
Time
serie
s
0 20 40 60 80 100
-10
12
34
5-0
.20.
41.
0
Lag k
Aut
o-K
orr.
0 5 10 15 20
-0.2
0.4
1.0
Lag k
part.
Aut
okor
r
1 5 10 15 20
18Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
What is the goal?The goal of State Space Modeling/Kalman Filtering is:
To uncover the „de-noised“ process Zt from theobserved process Yt.
• The algorithm of Kalman Filtering works with non-stationary time series, too.
• The algorithm is based on a maximum-likelihood-principle where one assume normal distortions.
• There are extensions to multi-dimensional state spacemodels. This is partly discussed in the exercises.
19Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Summary of Kalman FilteringSummary:
1) The Kalman Filter is a recursive algorithm
2) It relies on an update idea, i.e. we update theforecast with the difference .
3) The weight of the update is determined by therelation between the process variance and theobservation white noise .
4) This relies on the knowledge of . In practicewe have procedures for simultaneous estimation.
1,ˆ
t tZ 1 1,ˆ( )t t ty Y
2E
22 2, , ,Eg h
20
Applied Time Series AnalysisFS 2011 – Week 11Additional Remarks1) For the recursive approach of Kalman filtering, initial
values are necessary. Their choice is not crucial, theirinfluence cancels out rapidly.
2) The procedures yield forecast and filter intervals: and
3) State space models are a very rich class. Every ARIMA(p,d,q) can be written in state space form, andthe Kalman filter can be used for estimating thecoefficients.
4) We can also use Kalman filtering for smoothing, i.e.providing with T>t.
1, 1,ˆ 1.96t t t tZ R 1, 1 1, 1
ˆ 1.96t t t tZ R
1| TtZ Y
21Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Example with α=0.7
Time
yt1
0 20 40 60 80 100
-2-1
01
2
Zustands-Prozess X_tBeobachtungs-Prozess Y_tFilter-Werte mit 95%-Filter-Intervall
AR(1)-Example with alpha=0.7
22Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Example with α=1.0
Time
yt2
0 20 40 60 80 100
-3-2
-10
12
3
Zustands-Prozess X_tBeobachtungs-Prozess Y_tFilter-Werte mit 95%-Filter-Intervall
AR(1)-Example with alpha=1.0
23Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Smoothing with α=1.0
Time
yt2
0 20 40 60 80 100
-3-2
-10
12
3
Zustands-Prozess X_tBeobachtungs-Prozess Y_tFilter-Werte mit 95%-Filter-Intervall
Smoothing instead of filtering