131
Demographic forecasting using functional data analysis Rob J Hyndman Joint work with: Heather Booth, Han Lin Shang, Shahid Ullah, Farah Yasmeen. Demographic forecasting using functional data analysis 1

Demographic forecasting

Embed Size (px)

DESCRIPTION

Demographic forecasting using functional data analysis

Citation preview

Page 1: Demographic forecasting

Demographic forecasting

using functional data

analysis

Rob J Hyndman

Joint work with: Heather Booth, Han Lin Shang,Shahid Ullah, Farah Yasmeen.

Demographic forecasting using functional data analysis 1

Page 2: Demographic forecasting

Mortality rates

Demographic forecasting using functional data analysis 2

Page 3: Demographic forecasting

Fertility rates

Demographic forecasting using functional data analysis 3

Page 4: Demographic forecasting

Outline

1 A functional linear model

2 Bagplots, boxplots and outliers

3 Functional forecasting

4 Forecasting groups

5 Population forecasting

6 References

Demographic forecasting using functional data analysis 4

Page 5: Demographic forecasting

Outline

1 A functional linear model

2 Bagplots, boxplots and outliers

3 Functional forecasting

4 Forecasting groups

5 Population forecasting

6 References

Demographic forecasting using functional data analysis A functional linear model 5

Page 6: Demographic forecasting

Some notation

Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

Demographic forecasting using functional data analysis A functional linear model 6

Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.

εt ,xiid∼ N(0,1) and et(x)

iid∼ N(0,v(x)).

Page 7: Demographic forecasting

Some notation

Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

Demographic forecasting using functional data analysis A functional linear model 6

Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.

εt ,xiid∼ N(0,1) and et(x)

iid∼ N(0,v(x)).

Page 8: Demographic forecasting

Some notation

Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

Demographic forecasting using functional data analysis A functional linear model 6

Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.

εt ,xiid∼ N(0,1) and et(x)

iid∼ N(0,v(x)).

Page 9: Demographic forecasting

Some notation

Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

Demographic forecasting using functional data analysis A functional linear model 6

Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.

εt ,xiid∼ N(0,1) and et(x)

iid∼ N(0,v(x)).

Page 10: Demographic forecasting

Some notation

Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

Demographic forecasting using functional data analysis A functional linear model 6

Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.

εt ,xiid∼ N(0,1) and et(x)

iid∼ N(0,v(x)).

Page 11: Demographic forecasting

French mortality components

Demographic forecasting using functional data analysis A functional linear model 7

0 20 40 60 80 100

−6

−5

−4

−3

−2

−1

Age

µ(x)

0 20 40 60 80 100

0.00

0.05

0.10

0.15

0.20

Ageφ 1

(x)

t

β t1

1850 1900 1950 2000

−15

−10

−5

05

10

0 20 40 60 80 100

−0.

2−

0.1

0.0

0.1

0.2

Age

φ 2(x

)t

β t2

1850 1900 1950 2000

−2

02

46

8

Page 12: Demographic forecasting

French mortality components

Demographic forecasting using functional data analysis A functional linear model 7

1850 1900 1950 2000

020

4060

8010

0 Residuals

Year

Age

Page 13: Demographic forecasting

Outline

1 A functional linear model

2 Bagplots, boxplots and outliers

3 Functional forecasting

4 Forecasting groups

5 Population forecasting

6 References

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 8

Page 14: Demographic forecasting

French male mortality rates

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9

0 20 40 60 80 100

−8

−6

−4

−2

0France: male death rates (1900−2009)

Age

Log

deat

h ra

te

Page 15: Demographic forecasting

French male mortality rates

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9

0 20 40 60 80 100

−8

−6

−4

−2

0France: male death rates (1900−2009)

Age

Log

deat

h ra

te

War years

Page 16: Demographic forecasting

French male mortality rates

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9

0 20 40 60 80 100

−8

−6

−4

−2

0France: male death rates (1900−2009)

Age

Log

deat

h ra

te

War years

Aims

1 “Boxplots” for functional data2 Tools for detecting outliers in

functional data

Page 17: Demographic forecasting

Robust principal components

Let {ft(x)}, t = 1, . . . ,n , be a set of curves.

å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

1 Apply a robust principal component algorithm

ft(x) = µ(x)+n−1∑k=1

βt ,kφk(x)

µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores

2 Plot βi ,2 vs βi ,1

Page 18: Demographic forecasting

Robust principal components

Let {ft(x)}, t = 1, . . . ,n , be a set of curves.

å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

1 Apply a robust principal component algorithm

ft(x) = µ(x)+n−1∑k=1

βt ,kφk(x)

µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores

2 Plot βi ,2 vs βi ,1

Page 19: Demographic forecasting

Robust principal components

Let {ft(x)}, t = 1, . . . ,n , be a set of curves.

å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

1 Apply a robust principal component algorithm

ft(x) = µ(x)+n−1∑k=1

βt ,kφk(x)

µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores

2 Plot βi ,2 vs βi ,1

Page 20: Demographic forecasting

Robust principal components

Let {ft(x)}, t = 1, . . . ,n , be a set of curves.

å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

1 Apply a robust principal component algorithm

ft(x) = µ(x)+n−1∑k=1

βt ,kφk(x)

µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores

2 Plot βi ,2 vs βi ,1

Page 21: Demographic forecasting

Robust principal components

Let {ft(x)}, t = 1, . . . ,n , be a set of curves.

å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

1 Apply a robust principal component algorithm

ft(x) = µ(x)+n−1∑k=1

βt ,kφk(x)

µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores

2 Plot βi ,2 vs βi ,1

Page 22: Demographic forecasting

Robust principal components

Let {ft(x)}, t = 1, . . . ,n , be a set of curves.

å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

1 Apply a robust principal component algorithm

ft(x) = µ(x)+n−1∑k=1

βt ,kφk(x)

µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores

2 Plot βi ,2 vs βi ,1

Page 23: Demographic forecasting

Robust principal components

Let {ft(x)}, t = 1, . . . ,n , be a set of curves.

å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

1 Apply a robust principal component algorithm

ft(x) = µ(x)+n−1∑k=1

βt ,kφk(x)

µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores

2 Plot βi ,2 vs βi ,1

Page 24: Demographic forecasting

Robust principal components

Let {ft(x)}, t = 1, . . . ,n , be a set of curves.

å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

1 Apply a robust principal component algorithm

ft(x) = µ(x)+n−1∑k=1

βt ,kφk(x)

µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores

2 Plot βi ,2 vs βi ,1

Page 25: Demographic forecasting

Robust principal components

Let {ft(x)}, t = 1, . . . ,n , be a set of curves.

å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

1 Apply a robust principal component algorithm

ft(x) = µ(x)+n−1∑k=1

βt ,kφk(x)

µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores

2 Plot βi ,2 vs βi ,1

Page 26: Demographic forecasting

Robust principal components

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 11

●●●●●●●●●●

●●

●●

● ●

●●●●●●●●●

●●●●●●●●●●

●●●

●●●●●

●●●●

●●●

●●●

●●●●●●●●●●●●

●●●

●●●●●●●●●

●●●

●●●●●●●●●●●●

●●●●●●●

−10 −5 0 5 10 15

−2

02

46

Scatterplot of first two PC scores

PC score 1

PC

sco

re 2

Page 27: Demographic forecasting

Robust principal components

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 11

●●●●●●●●●●

●●

●●

● ●

●●●●●●●●●

●●●●●●●●●●

●●●

●●●●●

●●●●

●●●

●●●

●●●●●●●●●●●●

●●●

●●●●●●●●●

●●●

●●●●●●●●●●●●

●●●●●●●

−10 −5 0 5 10 15

−2

02

46

Scatterplot of first two PC scores

PC score 1

PC

sco

re 2

● ●

19141915

1916

1917

1918

1919

1940

1941

1942

1943

1944

1945

Page 28: Demographic forecasting

Functional bagplot

Bivariate bagplot due to Rousseeuw et al. (1999).Rank points by halfspace location depth.Display median, 50% convex hull and outerconvex hull (with 99% coverage if bivariatenormal).

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 12

−10 −5 0 5 10 15

−2

02

46

PC score 1

PC

sco

re 2

● ●

●●●●●●●●

●●

●●

● ●

●●

●●●

●●●●

●●●●●●●●

●●

●●●

●●

●●●

●●

●●

●●●●

●●

●●●●

●●●

●●●

●●●

●●●●

●●●●●

●●

●●●●

●●●

●●●

●●●●

● ●

19141915

1916

1917

1918

1919

19401943

1944

1945

Page 29: Demographic forecasting

Functional bagplot

Bivariate bagplot due to Rousseeuw et al. (1999).Rank points by halfspace location depth.Display median, 50% convex hull and outerconvex hull (with 99% coverage if bivariatenormal).Boundaries contain all curves inside bags.95% CI for median curve also shown.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 12

−10 −5 0 5 10 15

−2

02

46

PC score 1

PC

sco

re 2

● ●

●●●●●●●●

●●

●●

● ●

●●

●●●

●●●●

●●●●●●●●

●●

●●●

●●

●●●

●●

●●

●●●●

●●

●●●●

●●●

●●●

●●●

●●●●

●●●●●

●●

●●●●

●●●

●●●

●●●●

● ●

19141915

1916

1917

1918

1919

19401943

1944

1945

0 20 40 60 80 100

−8

−6

−4

−2

0

Age

Log

deat

h ra

te

1914191519161917

1918194019431944

Page 30: Demographic forecasting

Functional bagplot

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 13

0 20 40 60 80 100

−8

−6

−4

−2

0

Age

Log

deat

h ra

te

1914191519161917

1918194019431944

Page 31: Demographic forecasting

Functional HDR boxplot

Bivariate HDR boxplot due to Hyndman (1996).Rank points by value of kernel density estimate.Display mode, 50% and (usually) 99% highestdensity regions (HDRs) and mode.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14

−10 −5 0 5 10 15

−2

02

46

PC score 1

PC

sco

re 2

o●

●●●●●●●●●

●●

● ●

●●

●●●

●●●●

●●●●●●●●

●●

●●●

●●

●●●

●●

●●

●●●●

●●

●●●●

●●●

●●●

●●●

●●●●

●●●●●

●●

●●●●

●●●

●●●

●●●●

1919

1943

99% outer region

Page 32: Demographic forecasting

Functional HDR boxplot

Bivariate HDR boxplot due to Hyndman (1996).Rank points by value of kernel density estimate.Display mode, 50% and (usually) 99% highestdensity regions (HDRs) and mode.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14

−10 −5 0 5 10 15

−2

02

46

PC score 1

PC

sco

re 2

● ●

o●

●●●●●●●●●

●●

● ●

●●

●●●

●●●●

●●●●●●●●

●●

●●●

●●

●●●

●●

●●

●●●●

●●

●●●●

●●●

●●●

●●●

●●●●

●●●●●

●●

●●●●

●●●

●●●

●●●●

● ●

19141915

1916

1917

1918

1919

1940

1942

1943

1944

1945

90% outer region

Page 33: Demographic forecasting

Functional HDR boxplot

Bivariate HDR boxplot due to Hyndman (1996).Rank points by value of kernel density estimate.Display mode, 50% and (usually) 99% highestdensity regions (HDRs) and mode.Boundaries contain all curves inside HDRs.

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14

−10 −5 0 5 10 15

−2

02

46

PC score 1

PC

sco

re 2

● ●

o●

●●●●●●●●●

●●

● ●

●●

●●●

●●●●

●●●●●●●●

●●

●●●

●●

●●●

●●

●●

●●●●

●●

●●●●

●●●

●●●

●●●

●●●●

●●●●●

●●

●●●●

●●●

●●●

●●●●

● ●

19141915

1916

1917

1918

1919

1940

1942

1943

1944

1945

90% outer region

0 20 40 60 80 100

−8

−6

−4

−2

0

Age

Log

deat

h ra

te

191419151916191719181919

19401943194419451948

Page 34: Demographic forecasting

Functional HDR boxplot

Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 15

0 20 40 60 80 100

−8

−6

−4

−2

0

Age

Log

deat

h ra

te

191419151916191719181919

19401943194419451948

Page 35: Demographic forecasting

Outline

1 A functional linear model

2 Bagplots, boxplots and outliers

3 Functional forecasting

4 Forecasting groups

5 Population forecasting

6 References

Demographic forecasting using functional data analysis Functional forecasting 16

Page 36: Demographic forecasting

Functional time series model

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.

Demographic forecasting using functional data analysis Functional forecasting 17

Page 37: Demographic forecasting

Functional time series model

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.

Demographic forecasting using functional data analysis Functional forecasting 17

Page 38: Demographic forecasting

Functional time series model

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.

Demographic forecasting using functional data analysis Functional forecasting 17

Page 39: Demographic forecasting

Functional time series model

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.

Demographic forecasting using functional data analysis Functional forecasting 17

Page 40: Demographic forecasting

Functional time series model

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.

Demographic forecasting using functional data analysis Functional forecasting 17

Page 41: Demographic forecasting

Forecasts

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

Demographic forecasting using functional data analysis Functional forecasting 18

Page 42: Demographic forecasting

Forecasts

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

where vn+h ,k = Var(βn+h ,k | β1,k , . . . ,βn ,k)and y = [y1,x , . . . ,yn ,x ].Demographic forecasting using functional data analysis Functional forecasting 18

E[yn+h ,x | y] = µ̂(x)+K∑

k=1

β̂n+h ,k φ̂k(x)

Var[yn+h ,x | y] = σ̂2µ (x)+

K∑k=1

vn+h ,k φ̂2k(x)+ σ

2t (x)+ v(x)

Page 43: Demographic forecasting

Forecasting the PC scores

Demographic forecasting using functional data analysis Functional forecasting 19

0 20 40 60 80 100

−6

−4

−2

0

Main effects

Age

Mea

n

0 20 40 60 80 100

0.05

0.10

0.15

0.20

AgeB

asis

func

tion

1

Interaction

Year

Coe

ffici

ent 1

1900 1940 1980 2020

−20

−10

05

1015

0 20 40 60 80 100

−0.

2−

0.1

0.0

0.1

0.2

Age

Bas

is fu

nctio

n 2

YearC

oeffi

cien

t 2

1900 1940 1980 2020

−6

−4

−2

02

Page 44: Demographic forecasting

Forecasts of ft(x)

Demographic forecasting using functional data analysis Functional forecasting 20

0 20 40 60 80 100

−10

−8

−6

−4

−2

0France: male death rates (1900−2009)

Age

Log

deat

h ra

te

Page 45: Demographic forecasting

Forecasts of ft(x)

Demographic forecasting using functional data analysis Functional forecasting 20

0 20 40 60 80 100

−10

−8

−6

−4

−2

0France: male death rates (1900−2009)

Age

Log

deat

h ra

te

Page 46: Demographic forecasting

Forecasts of ft(x)

Demographic forecasting using functional data analysis Functional forecasting 20

0 20 40 60 80 100

−10

−8

−6

−4

−2

0France: male death forecasts (2010−2029)

Age

Log

deat

h ra

te

Page 47: Demographic forecasting

Forecasts of ft(x)

Demographic forecasting using functional data analysis Functional forecasting 20

0 20 40 60 80 100

−10

−8

−6

−4

−2

0France: male death forecasts (2010 & 2029)

Age

Log

deat

h ra

te

80% prediction intervals

Page 48: Demographic forecasting

Fertility application

Demographic forecasting using functional data analysis Functional forecasting 21

15 20 25 30 35 40 45 50

050

100

150

200

250

Australia fertility rates (1921−2009)

Age

Fer

tility

rat

e

Page 49: Demographic forecasting

Fertility model

Demographic forecasting using functional data analysis Functional forecasting 22

15 20 25 30 35 40 45 50

05

1015

Age

Μ

15 20 25 30 35 40 45 50

−0.

3−

0.1

0.0

0.1

0.2

AgeΦ

1(x)

t

Βt1

1970 1980 1990 2000 2010

−10

−5

05

10

15 20 25 30 35 40 45 50

0.05

0.15

0.25

Age

Φ2(

x)t

Βt2

1970 1980 1990 2000 2010

−4

−2

02

46

8

Page 50: Demographic forecasting

Fertility model

Demographic forecasting using functional data analysis Functional forecasting 23

1970 1980 1990 2000

1520

2530

3540

45Residuals

Year

Age

Page 51: Demographic forecasting

Fertility model

Demographic forecasting using functional data analysis Functional forecasting 24

15 20 25 30 35 40 45 50

05

1015

Main effects

Age

Mea

n

15 20 25 30 35 40 45 50

−0.

3−

0.1

0.0

0.1

0.2

AgeB

asis

func

tion

1

Interaction

Year

Coe

ffici

ent 1

1970 1990 2010 2030

−10

05

1015

2025

15 20 25 30 35 40 45 50

0.05

0.15

0.25

Age

Bas

is fu

nctio

n 2

YearC

oeffi

cien

t 2

1970 1990 2010 2030

−4

−2

02

46

8

Page 52: Demographic forecasting

Forecasts of ft(x)

Demographic forecasting using functional data analysis Functional forecasting 25

15 20 25 30 35 40 45 50

050

100

150

200

250

Australia fertility rates (1921−2009)

Age

Fer

tility

rat

e

Page 53: Demographic forecasting

Forecasts of ft(x)

Demographic forecasting using functional data analysis Functional forecasting 25

15 20 25 30 35 40 45 50

050

100

150

200

250

Australia fertility rates (1921−2009)

Age

Fer

tility

rat

e

Page 54: Demographic forecasting

Forecasts of ft(x)

Demographic forecasting using functional data analysis Functional forecasting 25

15 20 25 30 35 40 45 50

050

100

150

200

250

Australia fertility rates: 2010−2029

Age

Fer

tility

rat

e

Page 55: Demographic forecasting

Forecasts of ft(x)

Demographic forecasting using functional data analysis Functional forecasting 25

15 20 25 30 35 40 45 50

050

100

150

200

250

Australia fertility rates: 2010 and 2029

Age

Fer

tility

rat

e

80% prediction intervals

Page 56: Demographic forecasting

Outline

1 A functional linear model

2 Bagplots, boxplots and outliers

3 Functional forecasting

4 Forecasting groups

5 Population forecasting

6 References

Demographic forecasting using functional data analysis Forecasting groups 26

Page 57: Demographic forecasting

The problem

Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .

Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.

Demographic forecasting using functional data analysis Forecasting groups 27

Page 58: Demographic forecasting

The problem

Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .

Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.

Demographic forecasting using functional data analysis Forecasting groups 27

Page 59: Demographic forecasting

The problem

Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .

Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.

Demographic forecasting using functional data analysis Forecasting groups 27

Page 60: Demographic forecasting

The problem

Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .

Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.

Demographic forecasting using functional data analysis Forecasting groups 27

Page 61: Demographic forecasting

The problem

Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .

Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.

Demographic forecasting using functional data analysis Forecasting groups 27

Page 62: Demographic forecasting

The problem

Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .

Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.

Demographic forecasting using functional data analysis Forecasting groups 27

Page 63: Demographic forecasting

Forecasting the coefficients

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

We use ARIMA models for each coefficient{β1,j ,k , . . . ,βn ,j ,k }.The ARIMA models are non-stationary for thefirst few coefficients (k = 1,2)Non-stationary ARIMA forecasts will diverge.Hence the mortality forecasts are not coherent.

Demographic forecasting using functional data analysis Forecasting groups 28

Page 64: Demographic forecasting

Forecasting the coefficients

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

We use ARIMA models for each coefficient{β1,j ,k , . . . ,βn ,j ,k }.The ARIMA models are non-stationary for thefirst few coefficients (k = 1,2)Non-stationary ARIMA forecasts will diverge.Hence the mortality forecasts are not coherent.

Demographic forecasting using functional data analysis Forecasting groups 28

Page 65: Demographic forecasting

Forecasting the coefficients

yt ,x = ft(x)+ σt(x)εt ,x

ft(x) = µ(x)+K∑

k=1

βt ,k φk(x)+ et(x)

We use ARIMA models for each coefficient{β1,j ,k , . . . ,βn ,j ,k }.The ARIMA models are non-stationary for thefirst few coefficients (k = 1,2)Non-stationary ARIMA forecasts will diverge.Hence the mortality forecasts are not coherent.

Demographic forecasting using functional data analysis Forecasting groups 28

Page 66: Demographic forecasting

Male fts model

Demographic forecasting using functional data analysis Forecasting groups 29

0 20 40 60 80

−8

−6

−4

−2

Age

µ M(x

)

0 20 40 60 80

0.05

0.10

0.15

Age

φ 1(x

)

Year

β t1

1960 2000

−10

−5

05

0 20 40 60 80

−0.

3−

0.1

0.0

0.1

0.2

Age

φ 2(x

)

Year

β t2

1960 2000

−2.

5−

1.5

−0.

50.

5

0 20 40 60 80

−0.

10.

00.

10.

2

Age

φ 3(x

)

Year

β t3

1960 2000

−1.

0−

0.5

0.0

0.5

Australian male death rates

Page 67: Demographic forecasting

Female fts model

Demographic forecasting using functional data analysis Forecasting groups 30

0 20 40 60 80

−8

−6

−4

−2

Age

µ F(x

)

0 20 40 60 80

0.05

0.10

0.15

Age

φ 1(x

)

Year

β t1

1960 2000

−10

−5

05

0 20 40 60 80

−0.

10.

00.

10.

2

Age

φ 2(x

)

Year

β t2

1960 2000

−1.

0−

0.5

0.0

0.5

0 20 40 60 80

−0.

2−

0.1

0.0

0.1

Age

φ 3(x

)

Year

β t3

1960 2000

−0.

8−

0.4

0.0

0.4

Australian female death rates

Page 68: Demographic forecasting

Australian mortality forecasts

Demographic forecasting using functional data analysis Forecasting groups 31

0 20 40 60 80 100

−10

−8

−6

−4

−2

(a) Males

Age

Log

deat

h ra

te

0 20 40 60 80 100

−10

−8

−6

−4

−2

(b) Females

Age

Log

deat

h ra

te

Page 69: Demographic forecasting

Mortality product and ratios

Key idea

Model the geometric mean and the mortality ratioinstead of the individual rates for each sexseparately.

pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =

√ft ,M(x)

/ft ,F(x).

Product and ratio are approximatelyindependent

Ratio should be stationary (for coherence) butproduct can be non-stationary.

Demographic forecasting using functional data analysis Forecasting groups 32

Page 70: Demographic forecasting

Mortality product and ratios

Key idea

Model the geometric mean and the mortality ratioinstead of the individual rates for each sexseparately.

pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =

√ft ,M(x)

/ft ,F(x).

Product and ratio are approximatelyindependent

Ratio should be stationary (for coherence) butproduct can be non-stationary.

Demographic forecasting using functional data analysis Forecasting groups 32

Page 71: Demographic forecasting

Mortality product and ratios

Key idea

Model the geometric mean and the mortality ratioinstead of the individual rates for each sexseparately.

pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =

√ft ,M(x)

/ft ,F(x).

Product and ratio are approximatelyindependent

Ratio should be stationary (for coherence) butproduct can be non-stationary.

Demographic forecasting using functional data analysis Forecasting groups 32

Page 72: Demographic forecasting

Mortality product and ratios

Key idea

Model the geometric mean and the mortality ratioinstead of the individual rates for each sexseparately.

pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =

√ft ,M(x)

/ft ,F(x).

Product and ratio are approximatelyindependent

Ratio should be stationary (for coherence) butproduct can be non-stationary.

Demographic forecasting using functional data analysis Forecasting groups 32

Page 73: Demographic forecasting

Mortality rates

Demographic forecasting using functional data analysis Forecasting groups 33

Page 74: Demographic forecasting

Mortality rates

Demographic forecasting using functional data analysis Forecasting groups 34

Page 75: Demographic forecasting

Model product and ratios

pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =

√ft ,M(x)

/ft ,F(x).

log[pt(x)] = µp(x)+K∑

k=1

βt ,kφk(x)+ et(x)

log[rt(x)] = µr(x)+L∑`=1

γt ,`ψ`(x)+wt(x).

{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).No restrictions for βt ,1, . . . ,βt ,K .Forecasts: fn+h |n ,M(x) = pn+h |n(x)rn+h |n(x)

fn+h |n ,F(x) = pn+h |n(x)/rn+h |n(x).

Demographic forecasting using functional data analysis Forecasting groups 35

Page 76: Demographic forecasting

Model product and ratios

pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =

√ft ,M(x)

/ft ,F(x).

log[pt(x)] = µp(x)+K∑

k=1

βt ,kφk(x)+ et(x)

log[rt(x)] = µr(x)+L∑`=1

γt ,`ψ`(x)+wt(x).

{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).No restrictions for βt ,1, . . . ,βt ,K .Forecasts: fn+h |n ,M(x) = pn+h |n(x)rn+h |n(x)

fn+h |n ,F(x) = pn+h |n(x)/rn+h |n(x).

Demographic forecasting using functional data analysis Forecasting groups 35

Page 77: Demographic forecasting

Model product and ratios

pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =

√ft ,M(x)

/ft ,F(x).

log[pt(x)] = µp(x)+K∑

k=1

βt ,kφk(x)+ et(x)

log[rt(x)] = µr(x)+L∑`=1

γt ,`ψ`(x)+wt(x).

{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).No restrictions for βt ,1, . . . ,βt ,K .Forecasts: fn+h |n ,M(x) = pn+h |n(x)rn+h |n(x)

fn+h |n ,F(x) = pn+h |n(x)/rn+h |n(x).

Demographic forecasting using functional data analysis Forecasting groups 35

Page 78: Demographic forecasting

Model product and ratios

pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =

√ft ,M(x)

/ft ,F(x).

log[pt(x)] = µp(x)+K∑

k=1

βt ,kφk(x)+ et(x)

log[rt(x)] = µr(x)+L∑`=1

γt ,`ψ`(x)+wt(x).

{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).No restrictions for βt ,1, . . . ,βt ,K .Forecasts: fn+h |n ,M(x) = pn+h |n(x)rn+h |n(x)

fn+h |n ,F(x) = pn+h |n(x)/rn+h |n(x).

Demographic forecasting using functional data analysis Forecasting groups 35

Page 79: Demographic forecasting

Product model

Demographic forecasting using functional data analysis Forecasting groups 36

0 20 40 60 80

−8

−6

−4

−2

Age

µ P(x

)

0 20 40 60 80

0.05

0.10

0.15

Age

φ 1(x

)

Year

β t1

1960 2000

−10

−5

05

0 20 40 60 80

−0.

2−

0.1

0.0

0.1

0.2

Age

φ 2(x

)

Year

β t2

1960 2000−2.

5−

1.5

−0.

50.

5

0 20 40 60 80

−0.

2−

0.1

0.0

0.1

Age

φ 3(x

)

Year

β t3

1960 2000

−0.

6−

0.2

0.2

0.6

Page 80: Demographic forecasting

Ratio model

Demographic forecasting using functional data analysis Forecasting groups 37

0 20 40 60 80

0.1

0.2

0.3

0.4

0.5

Age

µ R(x

)

0 20 40 60 80

−0.

100.

000.

100.

20

Age

φ 1(x

)

Year

β t1

1960 2000

−0.

50.

00.

5

0 20 40 60 80

−0.

050.

050.

150.

25

Age

φ 2(x

)

Year

β t2

1960 2000

−0.

6−

0.2

0.0

0.2

0.4

0 20 40 60 80

−0.

20.

00.

10.

20.

30.

4

Age

φ 3(x

)

Year

β t3

1960 2000

−0.

3−

0.1

0.1

0.3

Page 81: Demographic forecasting

Product forecasts

Demographic forecasting using functional data analysis Forecasting groups 38

0 20 40 60 80 100

−10

−8

−6

−4

−2

Age

Log

of g

eom

etric

mea

n de

ath

rate

Page 82: Demographic forecasting

Ratio forecasts

Demographic forecasting using functional data analysis Forecasting groups 39

0 20 40 60 80 100

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Age

Sex

rat

io: M

/F

Page 83: Demographic forecasting

Coherent forecasts

Demographic forecasting using functional data analysis Forecasting groups 40

0 20 40 60 80 100

−10

−8

−6

−4

−2

(a) Males

Age

Log

deat

h ra

te

0 20 40 60 80 100

−10

−8

−6

−4

−2

(b) Females

Age

Log

deat

h ra

te

Page 84: Demographic forecasting

Ratio forecasts

Demographic forecasting using functional data analysis Forecasting groups 41

0 20 40 60 80 100

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Independent forecasts

Age

Sex

rat

io o

f rat

es: M

/F

0 20 40 60 80 100

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Coherent forecasts

Age

Sex

rat

io o

f rat

es: M

/F

Page 85: Demographic forecasting

Life expectancy forecasts

Demographic forecasting using functional data analysis Forecasting groups 42

Life expectancy forecasts

Year

Age

1960 1980 2000 2020

7075

8085

1960 1980 2000 2020

7075

8085Life expectancy difference: F−M

Year

Num

ber

of y

ears

1960 1980 2000 2020

24

68

Page 86: Demographic forecasting

Coherent forecasts for J groups

pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J

and rt ,j(x) = ft ,j(x)/pt(x),

log[pt(x)] = µp(x)+K∑

k=1

βt ,kφk(x)+ et(x)

log[rt ,j(x)] = µr ,j(x)+L∑

l=1

γt ,l ,jψl ,j(x)+wt ,j(x).

Demographic forecasting using functional data analysis Forecasting groups 43

pt(x) and all rt ,j(x)are approximatelyindependent.

Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.

log[ft ,j(x)] = log[pt(x)rt ,j(x)]

Page 87: Demographic forecasting

Coherent forecasts for J groups

pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J

and rt ,j(x) = ft ,j(x)/pt(x),

log[pt(x)] = µp(x)+K∑

k=1

βt ,kφk(x)+ et(x)

log[rt ,j(x)] = µr ,j(x)+L∑

l=1

γt ,l ,jψl ,j(x)+wt ,j(x).

Demographic forecasting using functional data analysis Forecasting groups 43

pt(x) and all rt ,j(x)are approximatelyindependent.

Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.

log[ft ,j(x)] = log[pt(x)rt ,j(x)]

Page 88: Demographic forecasting

Coherent forecasts for J groups

pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J

and rt ,j(x) = ft ,j(x)/pt(x),

log[pt(x)] = µp(x)+K∑

k=1

βt ,kφk(x)+ et(x)

log[rt ,j(x)] = µr ,j(x)+L∑

l=1

γt ,l ,jψl ,j(x)+wt ,j(x).

Demographic forecasting using functional data analysis Forecasting groups 43

pt(x) and all rt ,j(x)are approximatelyindependent.

Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.

log[ft ,j(x)] = log[pt(x)rt ,j(x)]

Page 89: Demographic forecasting

Coherent forecasts for J groups

pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J

and rt ,j(x) = ft ,j(x)/pt(x),

log[pt(x)] = µp(x)+K∑

k=1

βt ,kφk(x)+ et(x)

log[rt ,j(x)] = µr ,j(x)+L∑

l=1

γt ,l ,jψl ,j(x)+wt ,j(x).

Demographic forecasting using functional data analysis Forecasting groups 43

pt(x) and all rt ,j(x)are approximatelyindependent.

Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.

log[ft ,j(x)] = log[pt(x)rt ,j(x)]

Page 90: Demographic forecasting

Coherent forecasts for J groups

pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J

and rt ,j(x) = ft ,j(x)/pt(x),

log[pt(x)] = µp(x)+K∑

k=1

βt ,kφk(x)+ et(x)

log[rt ,j(x)] = µr ,j(x)+L∑

l=1

γt ,l ,jψl ,j(x)+wt ,j(x).

Demographic forecasting using functional data analysis Forecasting groups 43

pt(x) and all rt ,j(x)are approximatelyindependent.

Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.

log[ft ,j(x)] = log[pt(x)rt ,j(x)]

Page 91: Demographic forecasting

Coherent forecasts for J groups

µj(x) = µp(x)+µr ,j(x) is group mean

zt ,j(x) = et(x)+wt ,j(x) is error term.

{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).

No restrictions for βt ,1, . . . ,βt ,K .

Demographic forecasting using functional data analysis Forecasting groups 44

log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]

= µj(x)+K∑

k=1

βt ,kφk(x)+L∑`=1

γt ,`,jψ`,j(x)+ zt ,j(x)

Page 92: Demographic forecasting

Coherent forecasts for J groups

µj(x) = µp(x)+µr ,j(x) is group mean

zt ,j(x) = et(x)+wt ,j(x) is error term.

{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).

No restrictions for βt ,1, . . . ,βt ,K .

Demographic forecasting using functional data analysis Forecasting groups 44

log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]

= µj(x)+K∑

k=1

βt ,kφk(x)+L∑`=1

γt ,`,jψ`,j(x)+ zt ,j(x)

Page 93: Demographic forecasting

Coherent forecasts for J groups

µj(x) = µp(x)+µr ,j(x) is group mean

zt ,j(x) = et(x)+wt ,j(x) is error term.

{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).

No restrictions for βt ,1, . . . ,βt ,K .

Demographic forecasting using functional data analysis Forecasting groups 44

log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]

= µj(x)+K∑

k=1

βt ,kφk(x)+L∑`=1

γt ,`,jψ`,j(x)+ zt ,j(x)

Page 94: Demographic forecasting

Coherent forecasts for J groups

µj(x) = µp(x)+µr ,j(x) is group mean

zt ,j(x) = et(x)+wt ,j(x) is error term.

{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).

No restrictions for βt ,1, . . . ,βt ,K .

Demographic forecasting using functional data analysis Forecasting groups 44

log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]

= µj(x)+K∑

k=1

βt ,kφk(x)+L∑`=1

γt ,`,jψ`,j(x)+ zt ,j(x)

Page 95: Demographic forecasting

Coherent forecasts for J groups

µj(x) = µp(x)+µr ,j(x) is group mean

zt ,j(x) = et(x)+wt ,j(x) is error term.

{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).

No restrictions for βt ,1, . . . ,βt ,K .

Demographic forecasting using functional data analysis Forecasting groups 44

log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]

= µj(x)+K∑

k=1

βt ,kφk(x)+L∑`=1

γt ,`,jψ`,j(x)+ zt ,j(x)

Page 96: Demographic forecasting

Outline

1 A functional linear model

2 Bagplots, boxplots and outliers

3 Functional forecasting

4 Forecasting groups

5 Population forecasting

6 References

Demographic forecasting using functional data analysis Population forecasting 45

Page 97: Demographic forecasting

Demographic growth-balance equation

Demographic growth-balance equation

Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)

x = 0,1,2, . . . .

Pt(x) = population of age x at 1 January, year tBt = births in calendar year t

Dt(x ,x +1) = deaths in calendar year t of persons aged x atthe beginning of year t

Dt(B ,0) = infant deaths in calendar year tGt(x ,x +1) = net migrants in calendar year t of persons aged

x at the beginning of year tGt(B ,0) = net migrants of infants born in calendar year t

Demographic forecasting using functional data analysis Population forecasting 46

Page 98: Demographic forecasting

Demographic growth-balance equation

Demographic growth-balance equation

Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)

x = 0,1,2, . . . .

Pt(x) = population of age x at 1 January, year tBt = births in calendar year t

Dt(x ,x +1) = deaths in calendar year t of persons aged x atthe beginning of year t

Dt(B ,0) = infant deaths in calendar year tGt(x ,x +1) = net migrants in calendar year t of persons aged

x at the beginning of year tGt(B ,0) = net migrants of infants born in calendar year t

Demographic forecasting using functional data analysis Population forecasting 46

Page 99: Demographic forecasting

Key ideas

Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample

paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.

Demographic forecasting using functional data analysis Population forecasting 47

Page 100: Demographic forecasting

Key ideas

Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample

paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.

Demographic forecasting using functional data analysis Population forecasting 47

Page 101: Demographic forecasting

Key ideas

Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample

paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.

Demographic forecasting using functional data analysis Population forecasting 47

Page 102: Demographic forecasting

Key ideas

Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample

paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.

Demographic forecasting using functional data analysis Population forecasting 47

Page 103: Demographic forecasting

Key ideas

Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample

paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.

Demographic forecasting using functional data analysis Population forecasting 47

Page 104: Demographic forecasting

The available data

In most countries, the following data are

available:

Pt(x) = population of age x at 1 January, year tEt(x) = population of age x at 30 June, year tBt(x) = births in calendar year t to females of age xDt(x) = deaths in calendar year t of persons of age x

From these, we can estimate:

mt(x) = Dt(x)/Et(x) = central death rate incalendar year t ;ft(x) = Bt(x)/E F

t (x) = fertility rate for females ofage x in calendar year t .

Demographic forecasting using functional data analysis Population forecasting 48

Page 105: Demographic forecasting

The available data

In most countries, the following data are

available:

Pt(x) = population of age x at 1 January, year tEt(x) = population of age x at 30 June, year tBt(x) = births in calendar year t to females of age xDt(x) = deaths in calendar year t of persons of age x

From these, we can estimate:

mt(x) = Dt(x)/Et(x) = central death rate incalendar year t ;ft(x) = Bt(x)/E F

t (x) = fertility rate for females ofage x in calendar year t .

Demographic forecasting using functional data analysis Population forecasting 48

Page 106: Demographic forecasting

The available data

In most countries, the following data are

available:

Pt(x) = population of age x at 1 January, year tEt(x) = population of age x at 30 June, year tBt(x) = births in calendar year t to females of age xDt(x) = deaths in calendar year t of persons of age x

From these, we can estimate:

mt(x) = Dt(x)/Et(x) = central death rate incalendar year t ;ft(x) = Bt(x)/E F

t (x) = fertility rate for females ofage x in calendar year t .

Demographic forecasting using functional data analysis Population forecasting 48

Page 107: Demographic forecasting

Net migration

We need to estimatemigration data based ondifference in population numbers afteradjusting for births and deaths.

Demographic growth-balance equation

Gt(x ,x +1) = Pt+1(x +1)− Pt(x)+Dt(x ,x +1)Gt(B ,0) = Pt+1(0) −Bt +Dt(B ,0)

x = 0,1,2, . . . .

Note: “net migration” numbers also include errors

associated with all estimates. i.e., a “residual”.

Demographic forecasting using functional data analysis Population forecasting 49

Page 108: Demographic forecasting

Net migration

We need to estimatemigration data based ondifference in population numbers afteradjusting for births and deaths.

Demographic growth-balance equation

Gt(x ,x +1) = Pt+1(x +1)− Pt(x)+Dt(x ,x +1)Gt(B ,0) = Pt+1(0) −Bt +Dt(B ,0)

x = 0,1,2, . . . .

Note: “net migration” numbers also include errors

associated with all estimates. i.e., a “residual”.

Demographic forecasting using functional data analysis Population forecasting 49

Page 109: Demographic forecasting

Net migration

We need to estimatemigration data based ondifference in population numbers afteradjusting for births and deaths.

Demographic growth-balance equation

Gt(x ,x +1) = Pt+1(x +1)− Pt(x)+Dt(x ,x +1)Gt(B ,0) = Pt+1(0) −Bt +Dt(B ,0)

x = 0,1,2, . . . .

Note: “net migration” numbers also include errors

associated with all estimates. i.e., a “residual”.

Demographic forecasting using functional data analysis Population forecasting 49

Page 110: Demographic forecasting

Net migration

We need to estimatemigration data based ondifference in population numbers afteradjusting for births and deaths.

Demographic growth-balance equation

Gt(x ,x +1) = Pt+1(x +1)− Pt(x)+Dt(x ,x +1)Gt(B ,0) = Pt+1(0) −Bt +Dt(B ,0)

x = 0,1,2, . . . .

Note: “net migration” numbers also include errors

associated with all estimates. i.e., a “residual”.

Demographic forecasting using functional data analysis Population forecasting 49

Page 111: Demographic forecasting

Net migration

Demographic forecasting using functional data analysis Population forecasting 50

0 20 40 60 80 100

−20

000

2000

4000

6000

8000 Australia: male net migration (1973−2008)

Age

Net

mig

ratio

n

Page 112: Demographic forecasting

Net migration

Demographic forecasting using functional data analysis Population forecasting 50

0 20 40 60 80 100

−20

000

2000

4000

6000

8000 Australia: female net migration (1973−2008)

Age

Net

mig

ratio

n

Page 113: Demographic forecasting

Stochastic population forecasts

Component models

Data: age/sex-specific mortality rates, fertilityrates and net migration.Models: Functional time series models formortality (M/F), fertility and net migration (M/F)assuming independence between componentsand coherence between sexes.Generate random sample paths of eachcomponent conditional on observed data.Use simulated rates to generate Bt(x),D Ft (x ,x +1), DM

t (x ,x +1) for t = n +1, . . . ,n +h ,assuming deaths and births are Poisson.

Demographic forecasting using functional data analysis Population forecasting 51

Page 114: Demographic forecasting

Stochastic population forecasts

Component models

Data: age/sex-specific mortality rates, fertilityrates and net migration.Models: Functional time series models formortality (M/F), fertility and net migration (M/F)assuming independence between componentsand coherence between sexes.Generate random sample paths of eachcomponent conditional on observed data.Use simulated rates to generate Bt(x),D Ft (x ,x +1), DM

t (x ,x +1) for t = n +1, . . . ,n +h ,assuming deaths and births are Poisson.

Demographic forecasting using functional data analysis Population forecasting 51

Page 115: Demographic forecasting

Stochastic population forecasts

Component models

Data: age/sex-specific mortality rates, fertilityrates and net migration.Models: Functional time series models formortality (M/F), fertility and net migration (M/F)assuming independence between componentsand coherence between sexes.Generate random sample paths of eachcomponent conditional on observed data.Use simulated rates to generate Bt(x),D Ft (x ,x +1), DM

t (x ,x +1) for t = n +1, . . . ,n +h ,assuming deaths and births are Poisson.

Demographic forecasting using functional data analysis Population forecasting 51

Page 116: Demographic forecasting

Stochastic population forecasts

Component models

Data: age/sex-specific mortality rates, fertilityrates and net migration.Models: Functional time series models formortality (M/F), fertility and net migration (M/F)assuming independence between componentsand coherence between sexes.Generate random sample paths of eachcomponent conditional on observed data.Use simulated rates to generate Bt(x),D Ft (x ,x +1), DM

t (x ,x +1) for t = n +1, . . . ,n +h ,assuming deaths and births are Poisson.

Demographic forecasting using functional data analysis Population forecasting 51

Page 117: Demographic forecasting

Simulation

Demographic growth-balance equation used to getpopulation sample paths.

Demographic growth-balance equation

Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)

x = 0,1,2, . . . .

10000 sample paths of population Pt(x), deathsDt(x) and births Bt(x) generated fort = 2004, . . . ,2023 and x = 0,1,2, . . . ,.This allows the computation of the empiricalforecast distribution of any demographicquantity that is based on births, deaths andpopulation numbers.

Demographic forecasting using functional data analysis Population forecasting 52

Page 118: Demographic forecasting

Simulation

Demographic growth-balance equation used to getpopulation sample paths.

Demographic growth-balance equation

Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)

x = 0,1,2, . . . .

10000 sample paths of population Pt(x), deathsDt(x) and births Bt(x) generated fort = 2004, . . . ,2023 and x = 0,1,2, . . . ,.This allows the computation of the empiricalforecast distribution of any demographicquantity that is based on births, deaths andpopulation numbers.

Demographic forecasting using functional data analysis Population forecasting 52

Page 119: Demographic forecasting

Simulation

Demographic growth-balance equation used to getpopulation sample paths.

Demographic growth-balance equation

Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)

x = 0,1,2, . . . .

10000 sample paths of population Pt(x), deathsDt(x) and births Bt(x) generated fort = 2004, . . . ,2023 and x = 0,1,2, . . . ,.This allows the computation of the empiricalforecast distribution of any demographicquantity that is based on births, deaths andpopulation numbers.

Demographic forecasting using functional data analysis Population forecasting 52

Page 120: Demographic forecasting

Forecasts of TFR

Demographic forecasting using functional data analysis Population forecasting 53

Forecast Total Fertility Rate

Year

TF

R

1920 1940 1960 1980 2000 2020

2000

2500

3000

3500

Page 121: Demographic forecasting

Population forecastsForecast population: 2028

Population ('000)

Age

Male Female

150 100 50 0 50 100 150

010

3050

7090

010

3050

7090

020

4060

8010

0

020

4060

8010

0

Age

20092009

Forecast population pyramid for 2028, along with80% prediction intervals. Dashed: actual populationpyramid for 2009.

Demographic forecasting using functional data analysis Population forecasting 54

Page 122: Demographic forecasting

Population forecasts

Demographic forecasting using functional data analysis Population forecasting 55

● ● ●●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●●

●●

1980 1990 2000 2010 2020 2030

810

1214

16Total females

Year

Pop

ulat

ion

(mill

ions

)

Page 123: Demographic forecasting

Population forecasts

Demographic forecasting using functional data analysis Population forecasting 55

● ● ●●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

●●

●●

1980 1990 2000 2010 2020 2030

810

1214

16Total males

Year

Pop

ulat

ion

(mill

ions

)

Page 124: Demographic forecasting

Old-age dependency ratio

Demographic forecasting using functional data analysis Population forecasting 56

Old−age dependency ratio forecasts

Year

ratio

1920 1940 1960 1980 2000 2020

0.10

0.15

0.20

0.25

0.30

Page 125: Demographic forecasting

Advantages of stochastic simulation approach

Functional data analysis provides a way offorecasting age-specific mortality, fertility andnet migration.Stochastic age-specific cohort-componentsimulation provides a way of forecasting manydemographic quantities with predictionintervals.No need to select combinations of assumedrates.True prediction intervals with specifiedcoverage for population and all derivedvariables (TFR, life expectancy, old-agedependencies, etc.)

Demographic forecasting using functional data analysis Population forecasting 57

Page 126: Demographic forecasting

Advantages of stochastic simulation approach

Functional data analysis provides a way offorecasting age-specific mortality, fertility andnet migration.Stochastic age-specific cohort-componentsimulation provides a way of forecasting manydemographic quantities with predictionintervals.No need to select combinations of assumedrates.True prediction intervals with specifiedcoverage for population and all derivedvariables (TFR, life expectancy, old-agedependencies, etc.)

Demographic forecasting using functional data analysis Population forecasting 57

Page 127: Demographic forecasting

Advantages of stochastic simulation approach

Functional data analysis provides a way offorecasting age-specific mortality, fertility andnet migration.Stochastic age-specific cohort-componentsimulation provides a way of forecasting manydemographic quantities with predictionintervals.No need to select combinations of assumedrates.True prediction intervals with specifiedcoverage for population and all derivedvariables (TFR, life expectancy, old-agedependencies, etc.)

Demographic forecasting using functional data analysis Population forecasting 57

Page 128: Demographic forecasting

Advantages of stochastic simulation approach

Functional data analysis provides a way offorecasting age-specific mortality, fertility andnet migration.Stochastic age-specific cohort-componentsimulation provides a way of forecasting manydemographic quantities with predictionintervals.No need to select combinations of assumedrates.True prediction intervals with specifiedcoverage for population and all derivedvariables (TFR, life expectancy, old-agedependencies, etc.)

Demographic forecasting using functional data analysis Population forecasting 57

Page 129: Demographic forecasting

Outline

1 A functional linear model

2 Bagplots, boxplots and outliers

3 Functional forecasting

4 Forecasting groups

5 Population forecasting

6 References

Demographic forecasting using functional data analysis References 58

Page 130: Demographic forecasting

Selected referencesHyndman, Shang (2010). “Rainbow plots, bagplots and boxplots forfunctional data”. Journal of Computational and Graphical Statistics19(1), 29–45Hyndman, Ullah (2007). “Robust forecasting of mortality and fertilityrates: A functional data approach”. Computational Statistics andData Analysis 51(10), 4942–4956Hyndman, Shang (2009). “Forecasting functional time series (withdiscussion)”. Journal of the Korean Statistical Society 38(3),199–221Shang, Booth, Hyndman (2011). “Point and interval forecasts ofmortality rates and life expectancy : a comparison of ten principalcomponent methods”. Demographic Research 25(5), 173–214Hyndman, Booth (2008). “Stochastic population forecasts usingfunctional data models for mortality, fertility and migration”.International Journal of Forecasting 24(3), 323–342Hyndman, Booth, Yasmeen (2012). “Coherent mortality forecasting:the product-ratio method with functional time series models”.Demography, to appearHyndman (2012). demography: Forecasting mortality, fertility,migration and population data.cran.r-project.org/package=demography

Demographic forecasting using functional data analysis References 59

Page 131: Demographic forecasting

Selected referencesHyndman, Shang (2010). “Rainbow plots, bagplots and boxplots forfunctional data”. Journal of Computational and Graphical Statistics19(1), 29–45Hyndman, Ullah (2007). “Robust forecasting of mortality and fertilityrates: A functional data approach”. Computational Statistics andData Analysis 51(10), 4942–4956Hyndman, Shang (2009). “Forecasting functional time series (withdiscussion)”. Journal of the Korean Statistical Society 38(3),199–221Shang, Booth, Hyndman (2011). “Point and interval forecasts ofmortality rates and life expectancy : a comparison of ten principalcomponent methods”. Demographic Research 25(5), 173–214Hyndman, Booth (2008). “Stochastic population forecasts usingfunctional data models for mortality, fertility and migration”.International Journal of Forecasting 24(3), 323–342Hyndman, Booth, Yasmeen (2012). “Coherent mortality forecasting:the product-ratio method with functional time series models”.Demography, to appearHyndman (2012). demography: Forecasting mortality, fertility,migration and population data.cran.r-project.org/package=demography

Demographic forecasting using functional data analysis References 59

å Papers and R code:robjhyndman.com

å Email: [email protected]