Economic predictions with big data: The illusion of sparsity · 2018-07-04 · Economic predictions...

Economic predictions with big data:The illusion of sparsity

Giannone, Lenza, Primiceri The i l lusion of sparsity

Domenico GiannoneNew York Fed

Michele LenzaEuropean Central Bank

Giorgio PrimiceriNorthwestern University

10th ECB Workshop on Forecasting TechniquesFrankfurt, June 19, 2018

Predictive modeling with big data

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Big data: large 3

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Big data: large 3

n Standard inference (ML or flat prior) is a bad ideaØ Proliferation of parametersØ High estimation uncertaintyØ Overfitting and imprecise out-of-sample forecasting / poor external validity

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Big data: large 3

n Standard inference (ML or flat prior) is a bad idea

Ø Proliferation of parameters

Ø High estimation uncertainty

Ø Overfitting and imprecise out-of-sample forecasting / poor external validity

➥Methods to address curse of dimensionality (Ng, 2013, CHL, 2017)

Ø Sparse modeling e.g. hand picking, Lasso regression

Ø Dense modeling e.g. Ridge regression, Factor models

This paper: Sparse or dense modeling?

n Answer is an empirical matterØ Study a variety of predictive problems in macro, micro and finance

n Popular techniques not suitable to answer the questionØ Sparsity/density often assumedØ A small set of predictors might be selected simply to reduce estimation error,

even if the model is not sparse

n Popular techniques not suitable to answer the questionØ Sparsity/density often assumedØ A small set of predictors might be selected simply to reduce estimation error,

even if the model is not sparse

n Our predictive modelØ sparsity, without assuming itØ shrinkage, to give a chance to large modelsØ Bayesian inference on sparsity and shrinkage

Main results

� Inclusion probability and shrinkage are complements, but imperfect

1. No clear pattern of sparsityØ Posterior not concentrated on a single sparse model, but on a wide set

Main results

2. More sparsity emerges only if very tight prior favoring small models

Main results

The illusion of sparsity

Outline

n The predictive model

n Applications to macro, micro and finance

n Sparse or dense modeling? Ø Exploring the posterior

Outline

The predictive model

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

$-|12, 42, 5 ∼--. 6/ 0, 1242 with probability 5

0 with probability 1 − 5

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

Probability of inclusion, controls size

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

Variance of prior, controls shrinkage

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

n “Spike-and-slab” priorØ Mitchel and Beauchamp (1988)Ø Vast literature on Bayesian Model Averaging and Variable Selection

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

n “Spike-and-slab” priorØ Mitchel and Beauchamp (1988)Ø Vast literature on Bayesian Model Averaging and Variable SelectionØ This paper: inference on D and EF

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

$-|12, 42, 5 ∼

--.6/ 0, 1242 with probability 5

n Hyperpriors

5 ∼ ℬ E, F ,G ) HIJ(L) NO

G ) HIJ L NOP%≡ R2 ∼ ℬ S, T

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

$-|12, 42, 5 ∼

n Hyperpriors

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

$-|12, 42, 5 ∼

n Hyperpriors

∝ V WEX &"Y$ |5, 42, 12∝ V WEX !" |5, 4

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

$-|12, 42, 5 ∼

n Hyperpriors

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

$-|12, 42, 5 ∼

n Hyperpriors

5 ∼ ℬ 1, 1 ,E ) FGH(J) LM

E ) FGH(J) LMN%≡ P2 ∼ ℬ 1, 1

The implied joint prior on ! and "#

Outline

Economic applications

n MacroØ Forecasting industrial production with many macro predictorsØ The determinants of economic growth in a cross-section of countries

n FinanceØ Prediction of the US aggregate equity premium over timeØ Explaining the cross-section of equity returns across firms

n MicroØ Understanding the decline in crime rates in US states during the 1990sØ The determinants of government takings of private properties in US judicial

circuits

n Some references: Ø Stock-Watson (2002a and b), Barro-Lee (1994), Sala-i-Martin et al. (2004), Welch-Goyal (2008),

Freyberger et al. (2017), Donohue-Levitt (2001), Chen-Yeh (2012), Belloni et al. (2011, 2012, 2014).

Economic applications

Y X Sample

Macro 1 Growth rate of US Industrial Prod.

130 lagged macro and financial indicators

659 time-series obs.Feb. 60-Dec. 14

Macro 2 Countries average growth 1960-1985

60 country charact’socio-econ, inst. 90 cross-section obs.

Finance 1 US equity premium 16 lagged macro and financial indicators

58 time-series obs.1948-2015

Finance 2 Stock returns of US firms

144 dummies lagged characts’

≈1400k panel obs. Jul. 63–Dec. 15, ≈2k firms

Micro 1 Crime rate in US states

285 state characts’socio-econ, inst., law

476 panel obs. Jan. 86–Dec. 97, 48 states

Micro 2 Eminent domain judicial decisions

138 judges’ characts’socio-polit., profess

312 panel obs.1975-2008, circuits

Outline

Exploring the posterior

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

Variance of prior, controls shrinkage

Probability of inclusion and prior variance (! and "#)

Posterior probability of inclusion: p(#|%)

Patterns of sparsity:Probability of inclusion of each coefficient

Macro 1

20 40 60 80 100 120coefficients

1Macro 2

10 20 30 40 50 60coefficients

Finance 1

5 10 15coefficients

1Finance 2

20 40 60 80 100 120 140coefficients

Micro 1

50 100 150 200 250coefficients

1Micro 2

20 40 60 80 100 120coefficients

n Predictors rarely systematically excludedn Model uncertainty is pervasiven Best predictions not with single model, but mixture of many (BMA)

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

$-|12, 42, 5 ∼--.6/ 0, 1242 with probability 5

n Hyperpriors

5 ∼ ℬ 1, 1 ,E ) FGH I JK

E ) FGH I JKL%≡ N2 ∼ ℬ 1, 1

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

n Hyperpriors

5 ∼ ℬ 1, E ,F ) GHI J KL

F ) GHI J KLM%≡ O2 ∼ ℬ 1, 1

!" = $%&%" + ⋯+ $)&)" + *" , *" ∼--./ 0, 12

n Prior

n Hyperpriors

5 ∼ ℬ 1, E ,F ) GHI J KL

F ) GHI J KLM%≡ O2 ∼ ℬ 1, 1

Castillo et al. (2015, Annals)

Patterns of sparsity with a flat prior on !Probability of inclusion of each coefficient

Macro 1

20 40 60 80 100 120coefficients

1Macro 2

Finance 1

5 10 15coefficients

1Finance 2

20 40 60 80 100 120 140coefficients

Micro 1

1Micro 2

20 40 60 80 100 120coefficients

Patterns of sparsity with a tight prior on low !Probability of inclusion of each coefficient

Macro 1

20 40 60 80 100 120coefficients

1Macro 2

Finance 1

5 10 15coefficients

1Finance 2

20 40 60 80 100 120 140coefficients

Micro 1

1Micro 2

20 40 60 80 100 120coefficients

Patterns of sparsity with a flat prior on !Probability of inclusion of each coefficient, given !

Patterns of sparsity with a tight prior on !Probability of inclusion of each coefficient, given !

Baseline hyperprior: flat on !

0 0.2 0.4 0.6 0.8 1q

2Macro 1

Beta(1,1)

0 0.2 0.4 0.6 0.8 1q

2Macro2

0 0.2 0.4 0.6 0.8 1q

2Finance 1

0 0.2 0.4 0.6 0.8 1q

2Finance 2

0 0.2 0.4 0.6 0.8 1q

2Micro 1

0 0.2 0.4 0.6 0.8 1q

2Micro 2

Alternative hyperprior: tight on low !

0 0.2 0.4 0.6 0.8 1q

140Macro 1

Beta(1,1)Beta(1,k)

0 0.2 0.4 0.6 0.8 1q

60Macro2

0 0.2 0.4 0.6 0.8 1q

20Finance 1

0 0.2 0.4 0.6 0.8 1q

150Finance 2

0 0.2 0.4 0.6 0.8 1q

300Micro 1

0 0.2 0.4 0.6 0.8 1q

140Micro 2

Posterior of !" with a flat and a tight prior on #

Summing up

The illusion of sparsity

The prior distribution

n Alternative representation

!"|$%, '%, ( ∼""*+ 0, $%'%-" ,

-"|( ∼""*./0123445(()

n Relation with other popular shrinkage methodsØ Ridge: ( = 1

Ø Lasso: -" ∼""* ℰ;<21/1=5>4

Ø Lava: -" ∼""* ?ℎ5A=/B ℰ;<21/1=5>4

Ø Horse shoe: -" ∼""* C>4A D>3EℎF

Ø Elastic net: -" ∼""* =0>1GA20H>=521 2A > =031E>=/B I>HH>

n None admits a sparse representation of with positive probability

Bayesian interpretation of various shrinkage methods

! "|$ as a measure of predictive accuracy

n Posterior of q

! "|$ ∝ ! $|" & ! "

n Posterior of q

! "|$ ∝ ! $|" ='(

)! *(|*(+,, "

n Posterior of q

! "|$ ∝ ! $|" ='(

)! *(|*(+,, "

n Posterior of q

! "|$ ∝ ! $|" ='(

)! *(|*(+,, "

predictive score

n Posterior of q

! "|$ ∝ ! $|" ='(

)! *(|*(+,, "

➥Average log-predictive score

)log ! *(|*(+,, " = 1

/ 456 ! "|$ + 859:;<9;

Average log-predictive score, relative to best fitting model

0 0.2 0.4 0.6 0.8 1q

-0.015

-0.005

0Macro 1

0 0.2 0.4 0.6 0.8 1q

0Macro 2

0 0.2 0.4 0.6 0.8 1q

-0.015

-0.005

0Finance 1

0 0.2 0.4 0.6 0.8 1q

0 10-6 Finance 2

0 0.2 0.4 0.6 0.8 1q

-0.025

-0.015

-0.005

0Micro 1

0 0.2 0.4 0.6 0.8 1q

0Micro 2

Probability of inclusion (!) and "#

Economic predictions with big data: The illusion of sparsity · 2018-07-04 · Economic predictions...

Documents

Learning with Structured Sparsity - Rangerranger.uta.edu/~huang/papers/JMLR11_SS.pdfadvantage of structured sparsity over standard sparsity on some real applications. Keywords: structured

1 Convex Optimizationwith Sparsity-InducingNorms

Exploiting Sparsity for Wireless Communications

Economic Predictions with Big Data: The Illusion of …gep575/illusion4...ECONOMIC PREDICTIONS WITH BIG DATA: THE ILLUSION OF SPARSITY 2 ofsomeofthesemethodologiesineconomics). Attheoppositesideofthespectrum,dense

Sparsity Models - Tsinghuabigeye.au.tsinghua.edu.cn › DragonStar2012 › docs › sparsity.pdf · Sparsity Models Tong Zhang Rutgers University T. Zhang (Rutgers) Sparsity Models

Sparsity, the Lasso, and Friendsryantibs/statml/lectures/sparsity.pdf · So both interpretation and actual predictions are impossible with least squares when p>n, which is a pretty

Statistical Learning with Sparsity

Sparsity Based Regularization

Sparsity in Polynomial Optimization

Ted Adelson’s checkerboard illusion. Motion illusion, rotating snakes

Optical Illusion Puppet Generator - Semantic Scholar › 147c › 81cc5ceee6d5...3.2. Hollow-Face illusion The Hollow-Face illusion is an optical illusion in which the perception of

Marvels of illusion: illusion and perception in the art of ...smc.neuralcorrelate.com/files/publications/martinez-conde_et_al...Marvels of illusion: illusion and perception in the

FUNCTIONAL SPARSITY: GLOBAL VERSUS LOCALkaib.people.cofc.edu/research/Sinica_FunSparse.pdf · described two types of sparsity: global and local sparsity. In particular, if f is zero

A visual illusion - Revisioncbcpsychology.weebly.com › uploads › 8 › 1 › 4 › 3 › 8143681 › illusions-weebly.pdfThe Moon Illusion 0 The Moon illusion is an optical illusion

Economic Predictions with Big Data: The Illusion of Sparsity · 2018-04-20 · Economic Predictions with Big Data: The Illusion of Sparsity Domenico Giannone, Michele Lenza, and Giorgio

Strategic Foresight...Strategic foresight is not about better predictions of the future. As we noted, in a VUCA world accurate prediction is an illusion. Instead, strategic foresight

The Fraser illusion: Complex figures - Home - Springer · The Fraser illusion: Complex figures ... versions ofthe illusion, the "LIFE" figure and the spiral illusion, are shown in

Economic Predictions with Big Data: The Illusion of Sparsity · model. Without the possibility of shrinkage, the only way to improve prediction accuracy and avoid overﬁtting in

Sparsity-based Dynamic Hand Gesture Recognition Using Micro-Doppler Signaturesancortek.com/wp-content/uploads/2019/04/Sparsity-based... · 2019. 4. 20. · Sparsity-based Dynamic

Learning with sparsity-inducing norms