View
4
Download
0
Category
Preview:
Citation preview
2IS55 TSA with R 1
1
Time Series Analysis using
In a Nutshell
d J J M Rij k
Stochastic Processes:
X XX14
x 104
RealizationData Collection
dr. J.J.M. RijpkemaEindhoven University of Technology,
dept. Mathematics & Computer ScienceP.O.Box 513, 5600 MB Eindhoven, NL
© 2012 j.j.m.rijpkema@tue.nl
jr / department of mathematics and computer science 2IS55 TSA with R
Individual stochasts, that might be dependent
1 2
, , , ,t tt X XX
StochasticProcess
1940 1950 1960 1970 1980 1990 2000 20105
6
7
8
9
10
11
12
13
year
labo
rfor
ce -
dat
a
Model Fit• Model Identification
• Parameter Estimation
• Model Validation
Model Use
2
Overview• Introduction & Preliminaries• Exploratory Data Analysis• Exploratory Data Analysis
• Time Sequence Plot
• (Partial) Autocorrelation Function
• Exponential Smoothing Methods• Simple Exponential Smoothing
• Holt and Holt-Winters Models
• ARMA Models
jr / department of mathematics and computer science 2IS55 TSA with R
• The Bare Essentials
• Model Fitting
• More Time Series Modeling
2IS55 TSA with R 2
3
Introduction & Preliminaries
Stochastic Processes: RealizationData CollectionStochastic Processes:
Individual stochasts, that might be dependent
1 2
, , , ,t tt X XX
StochasticProcess
1940 1950 1960 1970 1980 1990 2000 20105
6
7
8
9
10
11
12
13
14x 10
4
year
labo
rfor
ce -
dat
a
Realization
Model Fit• Model Identification
• Parameter Estimation
• Model Validation
Model Use
jr / department of mathematics and computer science 2IS55 TSA with R
2IS55Restriction
Exploratory Data Analysis?
Relevant Models?
4
• References:
jr / department of mathematics and computer science 2IS55 TSA with R
http://a-little-book-of-r-for-time-series.readthedocs.org/
Coghlan, Avril, A Little Book of R for Time Series,Cowpertwait, Paul S.P. et al., Introductory Time Series with R, chapters 1& 2
2IS55 TSA with R 3
5
• Software:
http://www r project org/
jr / department of mathematics and computer science 2IS55 TSA with R
http://www.r-project.org/
Metaphor:
• Download and Install :
CRAN mirror: http://cran-mirror.cs.uu.nl/
jr / department of mathematics and computer science 2IS55 TSA with R
2IS55 TSA with R 4
• Start -Console:
jr / department of mathematics and computer science 2IS55 TSA with R
• Alternative: R-studio
jr / department of mathematics and computer science 2IS55 TSA with R
http://rstudio.org/
2IS55 TSA with R 5
• Install package ‘forecast’ (& ‘tseries’):
Netherlands (Utrecht)
jr / department of mathematics and computer science 2IS55 TSA with R
• Load package ‘forecast’ & ‘tseries’:
jr / department of mathematics and computer science 2IS55 TSA with R
> library(forecast)
2IS55 TSA with R 6
Exploratory Data Analysis
11
• Example: Age of Death of 42 Successive Kings of England
> load("D:/..../kings.RData")
> class(kings)[1] "numeric"
> kings [1] 60 43 67 50 56 42 50 65 68 43 65 34 47 34 49 41 13 35 53 56 16 43 [23] 69 59 48 59 86 55 68 51 33 49 67 77 81 67 71 81 68 70 77 56
• General Properties:
jr / department of mathematics and computer science 2IS55 TSA with R
> kings.ts <- ts(kings)> kings.ts
Time Series: Start = 1 End = 42 Frequency = 1 [1] 60 43 67 50 56 42 50 65 68 43 65 34 47 34 49 41 13 35 53 56 16 43 [23] 69 59 48 59 86 55 68 51 33 49 67 77 81 67 71 81 68 70 77 56
• Conversion to time series object:
12
• Time Sequence Plot:
> plot(kings.ts)
kin
gs.
ts
40
60
80
Exploratory Data Analysis
• Main Properties:– Trend?
– Cyclic Variation?
jr / department of mathematics and computer science 2IS55 TSA with R
Time
0 10 20 30 40
20
Cyclic Variation?
– Irregular Variation?
• Sudden Changes?
• Outliers?
• Missing Values?
2IS55 TSA with R 7
13
Are successive observations related ?
S tt l t & S l C l ti C ffi i t f {( )}
• Autocorrelation Function:
• Scatterplot & Sample Correlation Coefficient of {(xt,xt-1)}
11 2
t t
t
x x x xr
x x
Approximation !
> lag.plot(kings.ts,do.lines=F)
06
08
0
kin
gs.
ts
jr / department of mathematics and computer science 2IS55 TSA with R
r=0.4
0 20 40 60 80 100
20
40
lag 1
k
Autocorrelation Coefficient at time-lag 1
20 20 60 100 20 20 60 100
14
• Generalisation:
2
kt t
t
k
x x x xr
x x
> lag.plot(kings.ts,lags=12,do.lines=F)
lag 1
king
s.ts
2060
-20 20 60 100
lag 2
king
s.ts
lag 3
king
s.ts
-20 20 60 100
lag 4
king
s.ts
lag 5
king
s.ts
lag 6
king
s.ts
2060
jr / department of mathematics and computer science 2IS55 TSA with R
lag 7
king
s.ts
2060
lag 8
king
s.ts
lag 9
king
s.ts
lag 10
king
s.ts
lag 11
king
s.ts
-20 20 60 100lag 12
king
s.ts
2060
2IS55 TSA with R 8
15
• Autocorrelation Function:
> acf(kings ts)
> acf(kings.ts)$acf
[,1] [1 ] 1 000000000 > acf(kings.ts)
0.2
0.6
1.0
AC
F
[1,] 1.000000000 [2,] 0.400584149 [3,] 0.238081338 [4,] 0.259547258 [5,] 0.347780525 [6,] 0.160923687 [7,] 0.031201729 [8,] 0.115411950 [9,] 0.078385052 [10,] -0.036221926 [11,] -0.001138564
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 10 15-0
.2Lag
[12,] 0.110712654 [13,] -0.010138296 [14,] -0.116699180 [15,] 0.034735266 [16,] 0.025070163 [17,] -0.119576411
16
• Partial Autocorrelation:
Direct Pathway
xt-k
xt-k-1
xt
xt-k-2 xt-2 xt-1
Indirect Pathway
• Autocorrelation Function:
• Measure for the overall correlation between {xt-k} and {xt}
jr / department of mathematics and computer science 2IS55 TSA with R
• Both through the direct and the indirect pathway !
• Partial Autocorrelation Function:
• Measure for the direct correlation between {xt-k} and {xt}
• Indirect pathway corrected for !
2IS55 TSA with R 9
17
• (Partial) Autocorrelation Function:
> pacf(kings.ts)3
5 10 15
-0.3
-0.1
0.1
0.
L
Pa
rtia
l AC
F
> acf(kings.ts)
jr / department of mathematics and computer science 2IS55 TSA with R
Lag
kings.ts
60
18
• EDA-Summary: > tsdisplay(kings.ts)
0 10 20 30 40
206
0.4
0.4
jr / department of mathematics and computer science 2IS55 TSA with R
2 4 6 8 12
-0.4
0.0
Lag
AC
F
2 4 6 8 12
-0.4
0.0
Lag
PA
CF
2IS55 TSA with R 10
19
• Example: Births per Month in New York City> load("D:/..../births.RData")> births.ts <- ts(births,start=c(1946,1),frequency=12) > plot(births.ts)
bir
ths.
ts
22
24
26
28
30
Exploratory Data Analysis
• Main Properties:– Trend?
– Seasonal Variation?
jr / department of mathematics and computer science 2IS55 TSA with R
Time
1946 1950 1954 1958
20
2 Seasonal Variation?
– Cyclic Variation?
– Irregular Variation?
• Sudden Changes?
• Outliers?
• Missing Values?
births.ts
26
20
tsdisplay(births.ts)• EDA-Summary:
1946 1948 1950 1952 1954 1956 1958 1960
202
0.6
0.6
F
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 15 25 35
-0.4
0.2
Lag
AC
F
0 5 15 25 35
-0.4
0.2
Lag
PA
CF
2IS55 TSA with R 11
21
• Example: Monthly Sales for Australian Souvenir Shop
> load("D:/..../souvenir.RData")> souvenir.ts <- ts(souvenir,start=c(1987,1),frequency=12) > plot(souvenir.ts)
sou
ven
ir.t
s
4e
+0
48
e+
04
Exploratory Data Analysis
• Main Properties:– Trend?
– Seasonal Variation?
jr / department of mathematics and computer science 2IS55 TSA with R
Time
1987 1989 1991 19930e
+0
0
Seasonal Variation?
– Cyclic Variation?
– Irregular Variation?
• Sudden Changes?
• Outliers?
• Missing Values?
souvenir.ts
1e+
05
22
tsdisplay(souvenir.ts)• EDA-Summary:
1987 1988 1989 1990 1991 1992 1993 19940e+
000.
6
0.6
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 10 20
-0.2
0.2
Lag
AC
F
0 5 10 20
-0.2
0.2
Lag
PA
CF
2IS55 TSA with R 12
23
• Time Sequence Plot after Log-Trafo:
> souvenir.log <- log(souvenir.ts) > plot(souvenir.log)
sou
ven
ir.lo
g
91
01
1
Exploratory Data Analysis
• Main Properties:– Trend?
– Seasonal Variation?
jr / department of mathematics and computer science 2IS55 TSA with R
1987 1989 1991 1993
8 Seasonal Variation?
– Cyclic Variation?
– Irregular Variation?
• Sudden Changes?
• Outliers?
• Missing Values?
souvenir.log
10
24
> tsdisplay(souvenir.log.ts)• EDA-Summary:
1987 1988 1989 1990 1991 1992 1993 1994
80.
4
0.4
F
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 10 20
-0.6
0.0
Lag
AC
F
0 5 10 20
-0.6
0.0
Lag
PA
CF
2IS55 TSA with R 13
25
• Example: Annual Rainfall in London> load("D:/..../rain.RData")> rain.ts <- ts(rain,start=1813)> plot(rain.ts)
rain
.ts
25
30
35
Exploratory Data Analysis
• Main Properties:– Trend?
jr / department of mathematics and computer science 2IS55 TSA with R
Time
1820 1840 1860 1880 1900
20
– Trend?
– Cyclic Variation?
– Irregular Variation?
• Sudden Changes?
• Outliers?
• Missing Values?
rain.ts
0
26
> tsdisplay(rain.ts)• EDA-Summary:
1820 1840 1860 1880 1900
2030
0.2
0.2
jr / department of mathematics and computer science 2IS55 TSA with R
5 10 15 20
-0.3
0.0
0
Lag
AC
F
5 10 15 20
-0.3
0.0
0
Lag
PA
CF
2IS55 TSA with R 14
27
• Example: Annual Diameter of Women’s Skirts> load("D:/..../skirts.RData")> skirts.ts <- ts(skirts,start=1866)> plot(skirts.ts)
skir
ts.t
s
08
00
10
00
Exploratory Data Analysis
• Main Properties:– Trend?
jr / department of mathematics and computer science 2IS55 TSA with R
Time
1870 1880 1890 1900 1910
60
0 – Cyclic Variation?
– Irregular Variation?
• Sudden Changes?
• Outliers?
• Missing Values?
skirts.ts
0
28
> tsdisplay(skirts.ts)• EDA-Summary:
1870 1880 1890 1900 1910
600
900.
8
0.8
jr / department of mathematics and computer science 2IS55 TSA with R
2 4 6 8 12
-0.4
0.2
Lag
AC
F
2 4 6 8 12
-0.4
0.2
Lag
PA
CF
2IS55 TSA with R 15
29
• Example: Volcanic Dust Veil Index> load("D:/..../volcano.RData")> volcano.ts <- ts(volcano, start=1500) > plot(volcano.ts)
volc
an
o.t
s
20
04
00
60
0
Exploratory Data Analysis
• Main Properties:– Trend?
jr / department of mathematics and computer science 2IS55 TSA with R
Time
1500 1600 1700 1800 1900
0
– Cyclic Variation?
– Irregular Variation?
• Sudden Changes?
• Outliers?
• Missing Values?
volcano.ts
700
30
> tsdisplay(volcano.ts)• EDA-Summary:
1500 1600 1700 1800 1900
030
04 4
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 10 15 20 25
0.0
0.4
Lag
AC
F
0 5 10 15 20 25
0.0
0.4
Lag
PA
CF
2IS55 TSA with R 16
31
• Trend– Long term changes in mean
• Seasonal Variation– Periodic variations over time
• Types of Variation:
– Can be estimated & modeled
– Can be corrected for
– Can be estimated & modeled
– Can be corrected for
• Stationary Time Series– No systematic change in mean and variance
– No strictly periodic variations
jr / department of mathematics and computer science 2IS55 TSA with R
No strictly periodic variations
• Other Irregular Fluctuations– Random or with structure ?
• Other Cyclic Variation– Business cycle (~7 year)
32
Simple Exponential Smoothing
Example: Annual Rainfall in London
• Series with NO Trend and No Seasonality
• Basic Idea: one-step ahead prediction
Stationary Series• No trend (??)• No Seasonality
rain
.ts
1820 1840 1860 1880 1900
20
25
30
35
jr / department of mathematics and computer science 2IS55 TSA with R
0 1 1 2 2ˆ 1T T T Tx c x c x c x
1i
ic Geometric Weights:
• Remark: mainly used for short-term forecasts !
2IS55 TSA with R 17
33
ˆ ˆ1 1 1x x x
• Interpretation:
1 2ˆ 1 1 1T T T Tx x x x
Weighted average of 11 1 1T T Tx x x
1 1ˆ ˆ1 1T T Tx x x
ˆ
Simple Exponential Smoothing• Overview:
g gpast and present
α =0 pastα =1 present
jr / department of mathematics and computer science 2IS55 TSA with R
tx 1 1
ˆ 1t tx L 1tx 1t̂L t̂L
1ˆ ˆ1t t tL x L
• Remark:
• Simple Exponential Smoothing allows for updates of level estimates
34
• Example: fixed parameter = 0.2
• Adequate initial value for the level estimate needed
> rain.ses1 <- HoltWinters(rain.ts,alpha=0.2,beta=FALSE,gamma=FALSE)
1 1L̂ x
A il bl I f ti
> rain.ses1
H lt Wi t ti l thi ith t
• Smoothing Parameters:
> names(rain.ses1) [1] "fitted" "x" "alpha" "beta“ "gamma“ [6] "coefficients" "seasonal" "SSE" "call"
• Available Information:
jr / department of mathematics and computer science 2IS55 TSA with R
Holt-Winters exponential smoothing without trend and without seasonal component.
Smoothing parameters: alpha: 0.2 beta : FALSE gamma: FALSE
Coefficients: [,1]a 25.30941• Final Component Estimate:
2IS55 TSA with R 18
35
• ‘Running’ Component Estimates:
> rain.ses1$fitted
xhat level 1814 23.56000 23.56000
1 1ˆˆ 1t tx L
1815 24.06200 24.06200 1816 23.62160 23.62160 1817 25.14528 25.14528 1818 24.84622 24.84622 1819 24.65298 24.65298 1820 25.00438 25.00438 1821 24.53751 24.53751 1822 25.96801 25.96801 1823 25.54640 25.54640
.........
> plot(rain.ses1$fitted)
22
52
8
xha
t
jr / department of mathematics and computer science 2IS55 TSA with R
22
22
25
28
1820 1840 1860 1880 1900
leve
l α = 0.2
5
36
> plot(rain.ses1)
• In-Sample Prediction vs. Realization:
Ob
serv
ed
/ F
itte
d
20
25
30
35
> rain.ses1$SSE
[1] 1972.197
Running SSE
0
2
1ˆ 1
T
t tt s
x x
jr / department of mathematics and computer science 2IS55 TSA with R
1820 1840 1860 1880 1900
• Remark:
• In-sample comparison uses the same data for fitting and validation.
• Independent hold-out sample for validation would be preferred!
2IS55 TSA with R 19
37
• Goodness-of-Fit Measures:
• Bias:
1
ˆ1 nt t
t t
x xMPE
n x
• Mean Percentage Error
1
1 ˆn
t tt
ME x xn
• Mean ErrorThe closer to 0,
the
• Variability:
1
1 ˆn
t tt
MAE x xn
• Mean Absolute Error
• Root Mean Squared Error
1
ˆ100 nt t
t t
x xMAPE
n x
• Mean Absolute Percentage Error
ˆ
• Mean Absolute Scaled Error
jr / department of mathematics and computer science 2IS55 TSA with R
• In-Sample Accuracy:
> accuracy(rain.ses1)
ME RMSE MAE MPE MAPE MASE0.08835385 4.46331492 3.41468131 -2.41358228 13.79285103 0.70334023
2
1
1 ˆn
t tt
RMSE x xn
1
ˆ1 nt t
t
x xMASE
n q
> rain.ses1.fore
38
• Holt-Winters Forecasts:
> rain.ses1.fore <- forecast.HoltWinters(rain.ses1,h=10)
Point Forecast Lo 80 Hi 80 Lo 95 Hi 95 1913 25.30941 19.56146 31.05736 16.51867 34.10014 1914 25.30941 19.44762 31.17119 16.34458 34.27423 1915 25.30941 19.33596 31.28285 16.17381 34.44500 1916 25.30941 19.22635 31.39247 16.00617 34.61264 1917 25.30941 19.11867 31.50014 15.84150 34.77732 1918 25.30941 19.01284 31.60597 15.67964 34.93917 1919 25.30941 18.90876 31.71005 15.52046 35.09835 1920 25.30941 18.80634 31.81247 15.36383 35.25498 1921 25.30941 18.70551 31.91330 15.20962 35.40919 1922 25 30941 18 60620 32 01261 15 05774 35 56107
Forecasts from HoltWinters
35
> plot.forecast(rain.ses1.fore)
jr / department of mathematics and computer science 2IS55 TSA with R
1922 25.30941 18.60620 32.01261 15.05774 35.56107
1820 1840 1860 1880 1900 1920
15
20
25
30
2IS55 TSA with R 20
-55
15
39
• Diagnostics: In-Sample Forecast Errors
Constantover time?
1820 1840 1860 1880 1900
0.3
0.0
0.2
AC
F
0.3
0.0
0.2
PA
CFNon-zero
Autocorrelations?
> tsdisplay(rain.ses1.fore$residuals)
jr / department of mathematics and computer science 2IS55 TSA with R
5 10 15 20
-0
Lag
5 10 15 20
-0
Lag
> Box.test(rain.ses1.fore$residuals,lag=20,type="Ljung-Box")
Box-Ljung test data: rain.ses1.fore$residuals X-squared = 22.5621, df = 20, p-value = 0.3108
2
2 2#
1
2k
iBL N p
i
rX N N
N i
40
• Example: optimal estimated
• Principle:
• Optimal value for the parameter to be determined from ‘running’ 1-step ahead prediction:
> rain.ses2 <- HoltWinters(laborforce.ts,beta=FALSE,gamma=FALSE)
> rain.ses2
S thi t l h 0 02412151
• Smoothing Parameters:
2
10
ˆmin 1T
t tt
x x (or related, eg. AIC or BIC)
jr / department of mathematics and computer science 2IS55 TSA with R
Smoothing parameters: alpha: 0.02412151 beta : FALSE gamma: FALSE
Coefficients: [,1][,1] a 24.67819
• Final Component Estimate:
1ˆ ˆ1 1 1T T Tx x x More emphasis on the past!
2IS55 TSA with R 21
41
• ‘Running’ Component Estimates:
> rain.ses2$fitted
xhat level 1814 23.56000 23.56000
1 1ˆˆ 1t tx L
1815 23.62054 23.62054 1816 23.57808 23.57808 1817 23.76290 23.76290 1818 23.76017 23.76017 1819 23.76306 23.76306 1820 23.82691 23.82691 1821 23.79900 23.79900 1822 23.98935 23.98935 1823 23.98623 23.98623 .........
.52
4.5
xha
t
> plot(rain.ses2$fitted)
jr / department of mathematics and computer science 2IS55 TSA with R
23
.2
3.5
24
.5
1820 1840 1860 1880 1900
leve
l
optimal α
42
• In-Sample Prediction vs. Realization:
5
> plot(rain.ses2)
> rain.ses2$SSE
[1] 1828.855
Running SSE
0
2
1ˆ 1
T
t tt s
x x
Ob
serv
ed
/ F
itte
d
20
25
30
35
jr / department of mathematics and computer science 2IS55 TSA with R
• In-Sample Accuracy:
> accuracy(rain.ses2)
ME RMSE MAE MPE MAPE MASE0.4682496 4.2980556 3.3501092 -0.8688727 13.4015953 0.6900400
1820 1840 1860 1880 1900
2IS55 TSA with R 22
> rain.ses2.fore
43
• Holt-Winters Forecasts:
> rain.ses2.fore <- forecast.HoltWinters(rain.ses2,h=10)
Point Forecast Lo 80 Hi 80 Lo 95 Hi 95 1913 24.67819 19.17493 30.18145 16.26169 33.09470 1914 24.67819 19.17333 30.18305 16.25924 33.09715 1915 24.67819 19.17173 30.18465 16.25679 33.09960 1916 24.67819 19.17013 30.18625 16.25434 33.10204 1917 24.67819 19.16853 30.18785 16.25190 33.10449 1918 24.67819 19.16694 30.18945 16.24945 33.10694 1919 24.67819 19.16534 30.19105 16.24701 33.10938 1920 24.67819 19.16374 30.19265 16.24456 33.11182 1921 24.67819 19.16214 30.19425 16.24212 33.11427 1922 24 67819 19 16054 30 19584 16 23968 33 11671
Forecasts from HoltWinters
35
> plot.forecast(rain.ses2.fore)
jr / department of mathematics and computer science 2IS55 TSA with R
1922 24.67819 19.16054 30.19584 16.23968 33.11671
1820 1840 1860 1880 1900 1920
20
25
30
-55
44
• Diagnostics: In-Sample Forecast Errors
Constantover time?
1820 1840 1860 1880 1900
0.3
0.0
0.2
AC
F
0.3
0.0
0.2
PA
CFNon-zero
Autocorrelations?
> tsdisplay(rain.ses2.fore$residuals)
jr / department of mathematics and computer science 2IS55 TSA with R
5 10 15 20
-0
Lag
5 10 15 20
-0
Lag
> Box.test(rain.ses2.fore$residuals,lag=20,type="Ljung-Box")
Box-Ljung test data: rain.ses2.fore$residuals X-squared = 17.4008, df = 20, p-value = 0.6268
2
2 2#
1
2k
iBL N p
i
rX N N
N i
2IS55 TSA with R 23
45
• Series with Trend but No Seasonality
Holt’s Exponential Smoothing
Example: Annual Diameter of Women’s Skirts
• Principle:
Time
skir
ts.t
s
1870 1880 1890 1900 1910
60
08
00
10
00
Example: Annual Diameter of Women’s Skirts
• No Seasonality(annual data!)
jr / department of mathematics and computer science 2IS55 TSA with R
1 1 1ˆ ˆˆ 1T
t t tx L T
Holt’s Exponential Smoothing
Ttx
t̂T
1 1ˆ ˆ ˆ ˆ1t t t tT L L T
t̂L 1 1ˆ ˆ ˆ1Tt t t tL x L T 1
Ttx 1t̂L 1t̂T
p
46
• Remarks:
• Holt’s ES allows for updates of level and trend estimations
• Two parameter version of Exponential Smoothing
• Special: Brown’s Exponential SmoothingSpecial: Brown s Exponential Smoothing
• Both parameters are equal: = β
• Similar to ARIMA(0,2,1) model (to be discussed)
• Practical:
• Adequate initial values for the level and trend estimates needed
ˆ ˆ
jr / department of mathematics and computer science 2IS55 TSA with R
1 1L̂ x 1 2 1T̂ x x
• Optimal value for the parameters and to be determined from ‘running’ 1-step ahead prediction:
2
1,0
ˆmin 1T
t tt
x x (or related, eg. AIC or BIC)
2IS55 TSA with R 24
47
• Example: optimal and estimated
> skirts.hes <- HoltWinters(x=skirts.ts,gamma=FALSE)
• Smoothing Parameters:
> skirts.hes
Smoothing parameters: alpha: 0.8383481 beta : 1 gamma: FALSE
Main emphasis on recent values!
1 1ˆ ˆ ˆ1Tt t t tL x L T
1 1ˆ ˆ ˆ ˆ1t t t tT L L T
jr / department of mathematics and computer science 2IS55 TSA with R
Coefficients: [,1] a 529.308585b 5.690464
• Final Component Estimate:
48
• ‘Running’ Component Estimates:
> skirts.hes$fitted
xhat level trend 1868 626.0000 617.0000 9.0000000
1 1 1ˆ ˆˆ 1T
t t tx L T
1869 633.3233 625.1617 8.1616519 1870 645.9730 635.5673 10.4056551 1871 674.8676 655.2175 19.6501514 1872 721.5669 688.3922 33.17471011873 765.5280 726.9601 38.5679053 ... ...
> plot(skirts.hes$fitted)
500
800
xha
t90
0
el
jr / department of mathematics and computer science 2IS55 TSA with R
600le
ve-6
00
40
1870 1880 1890 1900 1910
tre
nd
optimal α and
2IS55 TSA with R 25
49
• In-Sample Prediction vs. Realization:
> plot(skirts.hes)
> skirts.hes$SSE
[1] 16954.18
Running SSE
0
2
1ˆ 1
T
t tt s
x x
Ob
serv
ed
/ F
itte
d
07
00
90
0
jr / department of mathematics and computer science 2IS55 TSA with R
• In-Sample Accuracy:
> accuracy(skirts.hes)
ME RMSE MAE MPE MAPE MASE255.25079 454.78440 269.86059 33.42056 35.34162 11.21304
1870 1880 1890 1900 1910
50
> skirts.hes.fore
50
• Holt-Winters Forecasts:
> skirts.hes.fore <- forecast.HoltWinters(skirts.hes,h=19)
Point Forecast Lo 80 Hi 80 Lo 95 Hi 95 1912 534.9990 509.55210 560.4460 496.08130 573.9168 1913 540.6895 491.01052 590.3685 464.71204 616.6670 1914 546.3800 465.36129 627.3987 422.47258 670.2874 1915 .... .....
02
00
0
> plot.forecast(skirts.hes.fore)
jr / department of mathematics and computer science 2IS55 TSA with R
1870 1890 1910 1930
-10
00
01
00
0
??
2IS55 TSA with R 26
51
• Diagnostics: In-Sample Forecast Errors
-20
20Constantover time?
1870 1880 1890 1900 1910
-0.4
0.0
0.4
AC
F
-0.4
0.0
0.4
PA
CFNon-zero
Autocorrelations?
> tsdisplay(skirts.hes.fore$residuals)
jr / department of mathematics and computer science 2IS55 TSA with R
2 4 6 8 12
-
Lag
2 4 6 8 12
-
Lag
> Box.test(skirts.hes.fore$residuals,lag=20,type="Ljung-Box")
Box-Ljung test data: skirts.fore$residualsX-squared = 19.7312, df = 20, p-value = 0.4749
2
2 2#
1
2k
iBL N p
i
rX N N
N i
52
• Example: Births per Months in New York City
Holt-Winters Exponential Smoothing(Additive Seasonality)
0
Time
bir
ths.
ts
1946 1950 1954 1958
20
22
24
26
28
30
• Trend ?
• Additive Seasonality ?
P i i l
jr / department of mathematics and computer science 2IS55 TSA with R
1 1 1ˆ ˆ ˆˆ 1TS
t t t t sx L T I
Additive Holt-Winter’s ES• Principle:
1TStx 1t̂L 1t̂T 1t̂I
TStx
t̂T
1 1ˆ ˆ ˆ ˆ1t t t tT L L T
t̂I
ˆ ˆ ˆ1TSt t t t sI x L I
t̂L 1 1ˆ ˆ ˆ ˆ1TSt t t s t tL x I L T
2IS55 TSA with R 27
53
• Remarks:
• Holt-Winter’s ES allows for updates of level, trend and seasonality estimations
• Three parameter version of Exponential Smoothing
• Similar to SARIMA (0,1,1)x(0,1,1)s model (to be discussed)
• Special: Simple Seasonal ES No trend (Tt=0), only seasonality !
1 1ˆ ˆˆ 1S
t t t sx L I
Simple Additive Seasonal ES
1Stx 1t̂L 1t̂I
ˆ ˆ ˆ1TSt t t t sI x L I
1ˆ ˆ ˆ1St t t s tL x I L
Stx t̂It̂L
jr / department of mathematics and computer science 2IS55 TSA with R
• Practical:
• Adequate initial values for the level, trend and seasonality estimates needed
• Optimal values for the parameters , and to be determined from:
2
1, ,0
ˆmin 1T
t tt
x x (or related, eg. AIC or BIC)
54
• Example: optimal and estimated
births.hw <- HoltWinters(births.ts,seasonal="additive")
• Smoothing Parameters: • Final Component Estimates:
births.hw
Smoothing parameters: alpha: 0.4823655beta : 0.02988495gamma: 0.563186
• Smoothing Parameters:
Coefficients: [,1]
a 28.04366357 b 0.04199921
s1 -0.78546221 s2 -2.19944507 s3 0.87813012 s4 -0.65164728 s5 0 63427267
• Final Component Estimates:
ˆ ˆ ˆ1TL x L T
jr / department of mathematics and computer science 2IS55 TSA with R
Emphasis for trend on the past!
s5 0.63427267 s6 0.21182821 s7 2.23177191 s8 2.17167733 s9 1.52077678 s10 1.16900861 s11 -0.97500043 s12 -0.18636055
1 11Tt t t tL x L T
1 1ˆ ˆ ˆ ˆ1t t t tT L L T
ˆ ˆ ˆ1TSt t t t sI x L I
2IS55 TSA with R 28
> births.hw$fitted
xhat level trend season Jan 1947 23.13579 23.81055 -0.1567618007 -0.51798958
55
• ‘Running’ Component Estimates: 1 1 1ˆ ˆ ˆˆ 1TS
t t t t sx L T I
Feb 1947 21.83089 22.83531 -0.1812218860 -0.82319792 Mar 1947 23.90724 22.29623 -0.1919165635 1.80292708 Apr 1947 21.58463 22.00869 -0.1947742244 -0.22928125 May 1947 21.51602 21.85461 -0.1935580066 -0.14503125 Jun 1947 20.43661 21.77488 -0.1901562399 -1.14811458 Jul 1947 22.44490 21.74120 -0.1854799895 0.88917708 Aug 1947 22.51935 22.05453 -0.1705728887 0.63538542 Sep 1947 22.50969 22.51328 -0.1517657072 0.14817708 Oct 1947 22.96787 22.64867 -0.1431840736 0.46238542 Nov 1947 21.63717 22.57404 -0.1411352421 -0.79573958 Dec 1947 22.07360 22.49168 -0.1393790022 -0.27869792
2226
xha
t24
27
leve
l
> plot(births.hw$fitted)
jr / department of mathematics and computer science 2IS55 TSA with R
Jan 1948 21.19997 22.35201 -0.1393876391 -1.01264664 Feb 1948 .......
optimal α, and
21
l-0
.20
0.00
tre
nd
-20
2
1948 1950 1952 1954 1956 1958 1960
sea
son
56
• In-Sample Prediction vs. Realization:
30
> plot(births.hw)
> births.hw$SSE
[1] 90.94058
Running SSE
0
2
1ˆ 1
T
t tt s
x x
Ob
serv
ed
/ F
itte
d
20
22
24
26
28
jr / department of mathematics and computer science 2IS55 TSA with R
• In-Sample Accuracy:
> accuracy(births.hw)
ME RMSE MAE MPE MAPE MASE12.60682 17.77135 12.93276 50.15378 51.52523 10.81909
1948 1952 1956 1960
2IS55 TSA with R 29
> births.hw.fore
57
• Holt-Winters Forecasts:
> births.hw.fore <- forecast.HoltWinters(births.hw,h=48)
Point Forecast Lo 80 Hi 80 Lo 95 Hi 95 Jan 1960 27.30020 26.32517 28.27523 25.80902 28.79139 Feb 1960 25.92822 24.83950 27.01694 24.26316 27.59327 Mar 1960 29.04779 27.85040 30.24518 27.21654 30.87905 Apr 1960 27.56001 26.25756 28.86247 25.56808 29.55195 May 1960 28.88793 27.48307 30.29280 26.73938 31.03649 Jun 1960 28.50749 27.00220 30.01278 26.20535 30.80963 Jul 1960 30.56943 28.96521 32.17365 28.11598 33.02288 Aug 1960 30.55133 28.84929 32.25338 27.94828 33.15439 Sep 1960 29.94243 28.14338 31.74148 27.19102 32.69384 O t 1960 29 63266 27 73720 31 52813 26 73380 32 53153
> plot.forecast(births.hw.fore)
35
40
jr / department of mathematics and computer science 2IS55 TSA with R
Oct 1960 29.63266 27.73720 31.52813 26.73380 32.53153 Nov 1960 27.53065 25.53918 29.52213 24.48496 30.57635 Dec 1960 28.36129 26.27407 30.44852 25.16916 31.55342 Jan 1961 27.80419 25.52190 30.08648 24.31373 31.29466 Feb 1961 26.4322 .......
1950 1955 1960
20
25
30
3
-20
2
58
• Diagnostics: In-Sample Forecast Errors
Constantover time?
1948 1950 1952 1954 1956 1958 1960
-0.2
0.0
0.2
AC
F
-0.2
0.0
0.2
PA
CFNon-zero
Autocorrelations?
> tsdisplay(births.hw.fore$residuals)
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 15 25 35
-
Lag
0 5 15 25 35
-
Lag
> Box.test(births.hw.fore$residuals,lag=20,type="Ljung-Box")
data: births.hw.fore$residualsX-squared = 70.9555, df = 35, p-value = 0.0003082
2
2 2#
1
2k
iBL N p
i
rX N N
N i
2IS55 TSA with R 30
59
Stochastic Processes:
x 104
Realization:
ARMA Models: the Bare Essentials
Individual stochasts, that might be dependent
1 2
, , , ,t tt X XX
StochasticProcess
1940 1950 1960 1970 1980 1990 2000 20105
6
7
8
9
10
11
12
13
14x 10
year
labo
rfor
ce -
dat
a
Model Fit• Model Identification
• Parameter estimation
• Model Validation
jr / department of mathematics and computer science 2IS55 TSA with R
• Purely Random Processes
• Moving Average Processes: MA(q)
• Autoregressive Processes: AR(p)
• ARMA(p,q) & ARIMA(p,d,q) Processes
60
Basic Building Block
• Purely Random Process: (white noise)
1 2
, , , ,t tt Z ZZ
and mutually independent 2~ 0,t ZZ N
• Simulation:> ts.sim <- arima.sim(list(order=c(0,0,0)),n=100)> plot(ts.sim)
im 01
23
Exploratory Data Analysis
• Main Properties:– Trend?
jr / department of mathematics and computer science 2IS55 TSA with R
Time
ts.s
i
0 20 40 60 80 100
-3-2
-10 – Trend?
– Seasonal Variation?
– Cyclic Variation?
– Irregular Variation?
2IS55 TSA with R 31
ts.sim
02
61
• EDA-Summary: > tsdisplay(ts.sim)
0 20 40 60 80 100
-30
0.2
0.2
F
jr / department of mathematics and computer science 2IS55 TSA with R
5 10 15 20
-0.3
0.0
Lag
AC
F
5 10 15 20-0
.30.
0Lag
PA
CF
62
• Moving Average Process: MA(q)
1 1t t t q t qX Z Z Z
• Interpretation: Present Value = Moving Average of Past Disturbances (=shock)
• Process is mainly influenced by ‘random’ events from the past: Economics ??
• MA-models are often used to model time series which show short term dependencies between successive observations
• Operator Notation: 1t tX XB
jr / department of mathematics and computer science 2IS55 TSA with R
t q tX ZB
backshift operator B
11 qq qB B Bwith:
2IS55 TSA with R 32
63
• Example: MA(2) Process
1 20.3 0.4t t t tX Z Z Z
23• Simulation:
t q tX ZB
• Operator Notation:
21 0.3 0.4q B B Bwith:
jr / department of mathematics and computer science 2IS55 TSA with R
ts.s
im0 20 40 60 80 100
-3-2
-10
1> ts.sim <- arima.sim(list(order=+ c(0,0,2), ma=c(-0.3,-0.4)),n=100) > plot(ts.sim)
ts.sim
02
64
• EDA-Summary: > tsdisplay(ts.sim)
0 20 40 60 80 100
-30
0.3
0.3
F
MA-specific
jr / department of mathematics and computer science 2IS55 TSA with R
5 10 15 20
-0.3
0.0
Lag
AC
F
5 10 15 20
-0.3
0.0
Lag
PA
CF
2IS55 TSA with R 33
65
• Autoregressive Process: AR(p)
1 1t t p t p tX X X Z
Interpretation: Present Value Moving Average of Past Values + Disturbance• Interpretation: Present Value = Moving Average of Past Values + Disturbance
• Process is mainly influenced by past values of the process !
• AR-models are often used to model time series which show longer term dependencies between successive observations
• Operator Notation: 1t tX XB
backshift operator B
jr / department of mathematics and computer science 2IS55 TSA with R
p t tX ZB
backshift operator B
11 pp pB B Bwith:
66
• Example: AR(2) Process
1 20.3 0.4t t t tX X X Z
23• Simulation:
p t tX ZB
• Operator Notation:
21 0.3 0.4p B B Bwith:
jr / department of mathematics and computer science 2IS55 TSA with R
Time
ts.s
im
0 20 40 60 80 100
-3-2
-10
1> ts.sim <- arima.sim(list(order=+ c(2,0,0), ar=c(0.3,0.4)),n=100) > plot(ts.sim)
2IS55 TSA with R 34
ts.sim
02
67
• EDA-Summary: > tsdisplay(ts.sim)
0 20 40 60 80 100
-30
.2 .2F
AR-specific
jr / department of mathematics and computer science 2IS55 TSA with R
5 10 15 20
-0.2
0
Lag
AC
F
5 10 15 20-0
.20
Lag
PA
CF
68
• Autoregressive Moving Average Process: ARMA(p,q)
1 1 1 1t t p t p t t q t qX X X Z Z Z
• Interpretation: Process is influenced both by levels and by disturbances fromthe past!
• Operator Notation: p t q tX ZB B
backshift operator B
jr / department of mathematics and computer science 2IS55 TSA with R
11 qq qB B B 11 p
p pB B B
backshift operator B
2IS55 TSA with R 35
69
• Example: ARMA(1,1) Process
1 10.6 0.4t t t tX X Z Z
23• Simulation:
• Operator Notation:
1 0.4q B B 11 0.6B B
p t q tX ZB B
jr / department of mathematics and computer science 2IS55 TSA with R
ts.s
im0 20 40 60 80 100
-3-2
-10
1
> ts.sim <- arima.sim(list(order=+ c(1,0,1),ar=c(0.6),ma=c(0.4)),n=100) > plot(ts.sim)
ts.sim
03
70
• EDA-Summary: > tsdisplay(ts.sim)
0 20 40 60 80 100
-30
0.6
0.6
F
AR-specific ?MA-specific ?
jr / department of mathematics and computer science 2IS55 TSA with R
5 10 15 20
-0.2
0.2
Lag
AC
F
5 10 15 20
-0.2
0.2
Lag
PA
CF
2IS55 TSA with R 36
71
ARMA Model Fittingstationary series without seasonality
Stochastic Processes:Stochastic Processes:
Individual stochasts, that might be dependent
1 2
, , , ,t tt X XX
StochasticProcess
1940 1950 1960 1970 1980 1990 2000 20105
6
7
8
9
10
11
12
13
14x 10
4
year
labo
rfor
ce -
dat
a
Realization:
Model Fit• Model Identification
• Parameter estimation
• Model Validation
jr / department of mathematics and computer science 2IS55 TSA with R
EDA - Finger Print
• Autocorrelation
• Partial Autocorrelation
Theoretical Finger Print
• Autocorrelation
• Partial Autocorrelation
72
> plot(volcano.ts); tsdisplay(volcano.ts)
• Example: Volcanic Dust Veil Index
60
0
0.4
F0.4
volc
an
o.t
s
1500 1600 1700 1800 1900
02
00
40
06 Stationary ?
AR(2) ?MA(3) ?
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 10 15 20 25
0.0
0
Lag
PA
CF
0 5 10 15 20 25
0.0
0
Lag
AC
F
ARMA(1,1) ?
2IS55 TSA with R 37
73
• Conjecture 1: MA(3)-model ?
> volcano.arma.0.3 <- Arima(volcano.ts,order=c(0,0,3))
> volcano.arma.0.3
Series: volcano.tsARIMA(0,0,3) with non-zero mean
Coefficients: ma1 ma2 ma3 intercept
0.7438 0.4513 0.1916 57.4559s.e. 0.0455 0.0502 0.0442 7.6534
sigma^2 estimated as 4852: log likelihood=-2661.69
AIC 5333 39 AIC 5333 52 BIC 5354 15
1 2 3
57.5
0.74 0.45 0.19
t t
t t t t t
X X
X Z Z Z Z
jr / department of mathematics and computer science 2IS55 TSA with R
AIC=5333.39 AICc=5333.52 BIC=5354.15
• Model Forecasts:
> volcano.arma.0.3.fore <-+ forecast(volcano.arma.0.3,h=19) > plot.forecast(volcano.arma.0.3.fore)
1500 1600 1700 1800 1900 2000
02
00
40
06
00
74
• In-Sample Accuracy:
ME RMSE MAE MPE MAPE MASE-0.2205417 69.7825327 37.8638920 -Inf Inf 0.9926308
• In-Sample Diagnostics: > tsdiag(volcano arma 0 3)
> accuracy(volcano.arma.0.3)
Standardized Residuals
Time
1500 1600 1700 1800 1900
08
0.0AC
F
ACF of Residuals
• In-Sample Diagnostics: > tsdiag(volcano.arma.0.3)
Constantover time?
Non-zero Autocorrelations?
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 10 15 20 25
0
Lag
2 4 6 8 10
0.0
p values for Ljung-Box statistic
lag
p va
lue
2
2 2#
1
2k
iBL N p
i
rX N N
N i
2IS55 TSA with R 38
75
• Conjecture 2: AR(2)-model ?
> volcano.arma.2.0 <- Arima(volcano.ts,order=c(2,0,0))
> volcano.arma.2.0
Series: volcano.tsARIMA(2,0,0) with non-zero mean
Coefficients: ar1 ar2 intercept0.7533 -0.1268 57.5274
s.e. 0.0457 0.0458 8.5958
sigma^2 estimated as 4870: log likelihood=-2662.54
AIC 5333 09 AIC 5333 17 BIC 5349 7
1 2
57.5
0.75 0.12
t t
t t t t
X X
X X X Z
jr / department of mathematics and computer science 2IS55 TSA with R
AIC=5333.09 AICc=5333.17 BIC=5349.7
• Model Forecasts:
> volcano.arma.2.0.fore <-+ forecast(volcano.arma.2.0,h=19) > plot.forecast(volcano.arma.2.0.fore)
1500 1600 1700 1800 1900 2000
02
00
40
06
00
76
• In-Sample Accuracy:
ME RMSE MAE MPE MAPE MASE-0.2205417 69.7825327 37.8638920 -Inf Inf 0.9926308
• In-Sample Diagnostics: > tsdiag(volcano arma 2 0)
> accuracy(volcano.arma.2.0)
Standardized Residuals
Time
1500 1600 1700 1800 1900
08
0.0AC
F
ACF of Residuals
• In-Sample Diagnostics: > tsdiag(volcano.arma.2.0)
Constantover time?
Non-zero Autocorrelations?
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 10 15 20 25
0
Lag
2 4 6 8 10
0.0
p values for Ljung-Box statistic
lag
p va
lue
2
2 2#
1
2k
iBL N p
i
rX N N
N i
2IS55 TSA with R 39
77
• Conjecture 3: ARMA(1,1)-model ?
> volcano.arma.1.1 <- Arima(volcano.ts,order=c(1,0,1))
> volcano.arma.1.1
Series: volcano.tsARIMA(1,0,1) with non-zero mean
Coefficients: ar1 ma1 intercept
0.5848 0.1555 57.5473 s.e. 0.0526 0.0613 8.9412
sigma^2 estimated as 4883: log likelihood=-2663.21
AIC 5334 42 AIC 5334 5 BIC 5351 03
1 2 3
57.5
0.3 0.4 0.4
t t
t t t t t
X X
X X X X Z
jr / department of mathematics and computer science 2IS55 TSA with R
AIC=5334.42 AICc=5334.5 BIC=5351.03
• Model Forecasts:
> volcano.arma.1.1.fore <-+ forecast(volcano.arma.1.1,h=19) > plot.forecast(volcano.arma.1.1.fore)
1500 1600 1700 1800 1900 2000
02
00
40
06
00
78
• In-Sample Accuracy:
ME RMSE MAE MPE MAPE MASE-0.2374427 69.8819979 37.7767429 -Inf Inf 0.9903461
• In-Sample Diagnostics: > tsdiag(volcano arma 1 1)
> accuracy(volcano.arma.1.1)
Standardized Residuals
Time
1500 1600 1700 1800 1900
08
0.0AC
F
ACF of Residuals
• In-Sample Diagnostics: > tsdiag(volcano.arma.1.1)
Constantover time?
Non-zero Autocorrelations?
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 10 15 20 25
0
Lag
2 4 6 8 10
0.0
p values for Ljung-Box statistic
lag
p va
lue
2
2 2#
1
2k
iBL N p
i
rX N N
N i
2IS55 TSA with R 40
79
ARIMA Model Fittingnon-stationary series
Time
kin
gs.
ts
0 10 20 30 40
20
40
60
80
Time
skir
ts.t
s
1870 1880 1890 1900 1910
60
08
00
10
00
Stationary ?• Level / Variation ?
R d
jr / department of mathematics and computer science 2IS55 TSA with R
• Remedy:• Remove dominant non-stationarity through finite differencing:
1 1t t t tX X X XB
• If successful then fit ARMA(p,q) on ▽Xt ARIMA(p,1,q) on Xt
80
ARMA(p,q) model on Wt:
p t q tW Z B B -200
-100
0
100
200
300
400
500
labo
rfo
rce
- A
R(a
ic)
nois
e
ARIMA(p,d,q)-model on Xt:
1940 1950 1960 1970 1980 1990 2000 2010-6000
-4000
-2000
0
2000
4000
6000
year
labo
rfor
ce
- de
tre
nd p
iece
wis
e dt tW X
1940 1950 1960 1970 1980 1990 2000 2010-400
-300
year
1940 1950 1960 1970 1980 1990 2000 20105
6
7
8
9
10
11
12
13
14x 10
4
year
labo
rfor
ce -
da
ta
jr / department of mathematics and computer science 2IS55 TSA with R
1d
p t q tX Z B B B 1940 1950 1960 1970 1980 1990 2000 2010
-400
-300
-200
-100
0
100
200
300
400
500
year
labo
rfo
rce
- A
R(a
ic)
nois
e
1940 1950 1960 1970 1980 1990 2000 20105
6
7
8
9
10
11
12
13
14x 10
4
year
labo
rfor
ce -
da
ta
2IS55 TSA with R 41
81
• Example: Annual Diameter of Women’s Skirts
> plot(skirts.ts)rts.
ts
80
01
00
0
p ( )
ski
1870 1880 1890 1900 1910
60
0
> skirts.d1 <- diff(skirts.ts,differences=1) > plot(skirts.d1)
skir
ts.d
if1
60
-20
20
60
jr / department of mathematics and computer science 2IS55 TSA with R
1870 1880 1890 1900 1910
-6
> skirts.d2 <- diff(skirts.ts,differences=2) > plot(skirts.d2) sk
irts
.dif2
1870 1880 1890 1900 1910
-40
-20
02
04
0
82
> tsdisplay(skirts.ts)
2 4 6 8 12
-0.4
0.2
0.8
AC
F
2 4 6 8 12
-0.4
0.2
0.8
PA
CF
• EDA-Summary:
2 4 6 8 12
Lag
2 4 6 8 12
Lag
> tsdisplay(skirts.d1)
2 4 6 8 12
-0.4
0.2
0.6
Lag
AC
F
2 4 6 8 12
-0.4
0.2
0.6
Lag
PA
CF
jr / department of mathematics and computer science 2IS55 TSA with R
Lag Lag
> tsdisplay(skirts.d2)
2 4 6 8 12
-0.4
0.0
0.4
Lag
AC
F
2 4 6 8 12
-0.4
0.0
0.4
Lag
PA
CF
2IS55 TSA with R 42
83
• Model: 21 1 22t t t t t t tW X X X X X X
> tsdisplay(skirts.d2)
-0.4
0.0
0.4
PA
CF
AR(?)
-0.4
0.0
0.4
AC
F
MA(?)
jr / department of mathematics and computer science 2IS55 TSA with R
2 4 6 8 12
Lag
2 4 6 8 12
Lag
ARMA(p,q) on WtARIMA(p,2,q) on Xt
84
• Conjecture: ARIMA(1,2,1)-model ?
> skirts.arima.1.2.1 <- Arima(skirts.ts,order=c(1,2,1))
> skirts.arima.1.2.1
Series: skirts.tsARIMA(1,2,1)
Coefficients: ar1 ma1
-0.3123 0.0139 s.e. 0.4265 0.4449
sigma^2 estimated as 388.7: log likelihood=-193.66
AIC 393 33 AIC 393 93 BIC 398 68
2
1 10.31 0.01t t
t t t t
W X
W W Z Z
jr / department of mathematics and computer science 2IS55 TSA with R
AIC=393.33 AICc=393.93 BIC=398.68
• Model Forecasts:
> skirts.arima.1.2.1.fore <-+ forecast(skirts.arima.1.2.1,h=19) > plot(skirts.arima.1.2.1.fore)
1870 1890 1910 1930-10
00
05
00
15
00
2IS55 TSA with R 43
85
SARIMA Model Fitting0
Time
sou
ven
ir.t
s
1987 1989 1991 19930e
+0
04
e+
04
8e
+0
4
Time
bir
ths.
ts
1946 1950 1954 1958
20
22
24
26
28
30
• Remedy:
Stationary ?• Level / Variation ?
Seasonality ?• Additive / Multiplicative ?
jr / department of mathematics and computer science 2IS55 TSA with R
y• Remove dominant non-stationarity through finite differencing:
1 1t t t tX X X XB
• Remove dominant seasonality through seasonal differencing:
1 sS t t t s tX X X XB
86
5000500
ARMA(p,q) for Wt:
Combined adjustment: 1 1 Ddd D s
t s t tW X XB B
1940 1950 1960 1970 1980 1990 2000 2010-4000
-3000
-2000
-1000
0
1000
2000
3000
4000
5000
year
labo
rfo
rce
- a
perio
dic
p t q tW ZB B1940 1950 1960 1970 1980 1990 2000 2010
-400
-300
-200
-100
0
100
200
300
400
500
year
labo
rfo
rce
- A
R(a
ic)
nois
e
Generalisation to account for changes in seasonality:
500
jr / department of mathematics and computer science 2IS55 TSA with R
1940 1950 1960 1970 1980 1990 2000 2010-4000
-3000
-2000
-1000
0
1000
2000
3000
4000
5000
year
labo
rfo
rce
- a
perio
dic
sp P t
sq Q t
W
Z
B B
B B1940 1950 1960 1970 1980 1990 2000 2010
-400
-300
-200
-100
0
100
200
300
400
500
year
labo
rfo
rce
- A
R(a
ic)
nois
e
2IS55 TSA with R 44
87
Equivalent:
sp P t
sq Q t
W
Z
B B
B B-300
-200
-100
0
100
200
300
400
500
labo
rfo
rce
- A
R(a
ic)
nois
e
1940 1950 1960 1970 1980 1990 2000 2010-6000
-4000
-2000
0
2000
4000
6000
year
labo
rfor
ce
- de
tre
nd p
iece
wis
e
d Dt s tW X
1940 1950 1960 1970 1980 1990 2000 20105
6
7
8
9
10
11
12
13
14x 10
4
year
labo
rfor
ce -
da
ta
q Q t1940 1950 1960 1970 1980 1990 2000 2010
-400
year
SARIMA (p,d,q)x(P,D,Q)s -model:
jr / department of mathematics and computer science 2IS55 TSA with R
1940 1950 1960 1970 1980 1990 2000 2010-400
-300
-200
-100
0
100
200
300
400
500
year
labo
rfo
rce
- A
R(a
ic)
nois
e
1940 1950 1960 1970 1980 1990 2000 20105
6
7
8
9
10
11
12
13
14x 10
4
year
labo
rfor
ce -
da
ta
s d Dp P s t
sq Q t
X
Z
B B
B B
88
28
30 > plot(births.ts); tsdisplay(births.ts)
• Example: Births per Month in New York City
5 10 15 20
-0.4
0.2
0.6
Lag
AC
F
5 10 15 20
-0.4
0.2
0.6
Lag
PA
CF
> births.d1.sd1<-diff(diff(births.ts,differences=1),lag=12,differences=1) > plot(births.d1.sd1); tsdisplay(births.d1.sd1)
bir
ths.
ts
1946 1950 1954 1958
20
22
24
26
jr / department of mathematics and computer science 2IS55 TSA with R
0 5 15 25 35
-0.4
-0.1
0.2
Lag
AC
F
0 5 15 25 35
-0.4
-0.1
0.2
Lag
PA
CF
bir
ths.
d1
.sd
1
1948 1952 1956 1960
-3-2
-10
12
2IS55 TSA with R 45
89
• Model: 12 121 1t t tW X XB B
tsdisplay(births.d1.sd1,lag.max=40)
-0.4
-0.1
0.2
PA
CF
AR(?), ARs(?)
-0.4
-0.1
0.2
AC
F
MA(?), MA s(?)
jr / department of mathematics and computer science 2IS55 TSA with R
0 10 20 30 40
Lag
0 10 20 30 40
Lag
ARMA(p,q) on WtSARIMA(p,1,q)x(P,1,Q)12 on Xt
90
• Conjecture: SARIMA(3,1,3)x(1,1,0)12-model ?
> births.sarima.3.1.3.1.1.0<-Arima(births.ts,order=c(3,1,3),+ seasonal=list(order=c(1,1,0),period=12))
> births.sarima.3.1.3.1.1.0
Series: births.tsARIMA(3,1,3)(1,1,0)[12]
Coefficients: ar1 ar2 ar3 ma1 ma2 ma3 sar1
-1.0567 -0.0302 0.3607 0.9311 -0.4072 -0.7804 -0.5139 s.e. 0.1441 0.2055 0.1497 0.1211 0.1565 0.1288 0.0767
sigma^2 estimated as 0.5566: log likelihood=-178.26
2
jr / department of mathematics and computer science 2IS55 TSA with R
AIC=372.51 AICc=373.5 BIC=396.86
• Model Forecasts:
> births.sarima.3.1.3.1.1.0.fore<-+ forecast(births.sarima.3.1.3.1.1.0,h=12) > plot(births.sarima.3.1.3.1.1.0.fore)
1950 1955 1960
20
22
24
26
28
30
32
2IS55 TSA with R 46
More Time Series Modeling…….
91
jr / department of mathematics and computer science 2IS55 TSA with R
Paul S. Cowpertwaith et al.,Introductory Time Series with R,
ISBN 978-0387886978
92
Journal of Statistical Forecasting, July 2008, Volume 27, Issue 3
jr / department of mathematics and computer science 2IS55 TSA with R
http://www.youtube.com/watch?v=1Lh1HlBUf8k
2IS55 TSA with R 47
93
R b J H d t l R b t M Sh t l
jr / department of mathematics and computer science 2IS55 TSA with R
Rob. J. Hyndman et al.,Forecasting with Exponential Smoothing,
ISBN 978-3540719168
Jonathan D. Cryer et al.,Time Series Analysis with Applications in R,
ISBN 978-1441926135
Robert M. Shumway et al.,Time Series Analysis and its Applications
with R examples, ISBN 978-1441978646
94
Suggestions for Improvements of TSAR
are welcomed:
j.j.m.rijpkema@tue.nl
jr / department of mathematics and computer science 2IS55 TSA with R
Recommended