An empirical investigation - proceedings · an empirical investigation on the value of combined...

AN EMPIRICAL INVESTIGATION ON THE VALUE OF COMBINED JUDGMENTAL AND

STATISTICAL FORECASTING

PROF. DR. PHILIPPE BAECKEPROF. DR. KARLIEN VANDERHEYDEN

DRS. SHARI DE BAETS

CONTACT: SHARI.DEBAETS@VLERICK.COM

ISF 2014, The Netherlands

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

ON THE VALUE OF COMBINED JUDGMENTAL AND STATISTICAL

FORECASTING

Shari De Baets – ISF 20142

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

� “..the practice of forecasting does not however appear to have improved.” (p,33; Armstrong, Green &

Graefe, 2013)

� Ascher, 1978

� Allen, 1994

� McCarthy, Davis, Golicic & Mentzer, 2006

Shari De Baets – ISF 20143

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

An important task for researchers in our field:

Improving forecastingaccuracy in practice

(Sanders & Manrodt, 2003)

Shari De Baets – ISF 20144

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

�Call for more studies with real company data (Sanders, 2009)

�Going beyond artificial experiments to deriverules for practice

�Expert forecasters

�Context of forecasting task

�A new research model: JUD 3 session(room “Plate”, Wed., 10 AM)

Shari De Baets – ISF 20145

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

ON THE VALUE OF COMBINED JUDGMENTAL AND STATISTICAL

FORECASTING

Shari De Baets – ISF 20146

© Vlerick Business School

THE VALUE OF COMBINED FORECASTING

25%

25%

17%

33%

Forecasting method

judgment alone

statistical methods alone

average of statistical and

judgmental forecast

statistical forecast adjusted

judgmentally

Fildes & Goodwin, 2007

Shari De Baets – ISF 20147

© Vlerick Business School

THE VALUE OF COMBINED FORECASTING

� Potential of judgment in forecasting oftenundermined by unneccessary adjustments(Lawrence et al., 2006)

� Patterns in noise (Harvey, 1995)

� Illusion of control effect (Kotteman, Davis, & Remus,

1994)

� Persists despite warning (Lim & O’Connor, 1995)

Shari De Baets – ISF 20148

© Vlerick Business School

RESEARCH QUESTION

How can we counter damaging adjustments andreap the potential benefits from judgment, andthus, heighten forecasting accuracy?

Shari De Baets – ISF 20149

© Vlerick Business School

HYPOTHESES

� Data augmentation:

� Classic model + judgment:

Var1, Var2, .., Varn -> outcome -> judgmentaladjustment

� Judgment incorporated in model

Var1, Var2, VarJudgment , .., Varn -> outcome

Shari De Baets – ISF 201410

© Vlerick Business School

HYPOTHESES

�Data augmentation:

� H1:

� “integrated judgment”: judgment as part of the model

will outperform

� “restrictive judgment”: judgment as restriction on the model (judgmental adjustment)

Shari De Baets – ISF 201411

© Vlerick Business School

HYPOTHESES (SIMILAR TO FILDES ET AL., 2009)

� H1 applied to

�Direction of the adjustment

�Downward adjustments are beneficial, upwardadjustment are damaging

�Size of the adjustment

�Curvilinear (inverted U-shape) effect: both small and very large adjustments are damaging

�Volatility of the data series

� Judgment is beneficial in high volatility series (volatility due to special events)

Shari De Baets – ISF 201412

© Vlerick Business School

THE COMPANY

Shari De Baets – ISF 201413

Outlet store

Outlet store

Outlet store

International publishing company

Weekly and monthlymagazines

© Vlerick Business School

PROCEDURE

� Demand forecasting

� Predictive model: forecast of expected demandper store

� Optimisation model: profit optimisation – finalnumber for supply per store

Input

Shari De Baets – ISF 201414

© Vlerick Business School

THE COMPANY

� Profit optimization model

Overstock

Stockout

Shari De Baets – ISF 201415

© Vlerick Business School

PROCEDURE

�Predictive model: forecast of expected demandper store

�Optimisation model: profit optimisation – finalnumber for supply per store

�Judgmental adjustment: according to insight

Shari De Baets – ISF 201416

© Vlerick Business School

PROCEDURE

Predictive Optimisation Judgmentalmodel model adjustment

Restrictive judgment

Shari De Baets – ISF 201417

© Vlerick Business School

PROCEDURE – DATA AUGMENTATION

Predictive Optimisation Judgmentalmodel model adjustment

Integrative judgment

Shari De Baets – ISF 201418

© Vlerick Business School

PROCEDURE – DATA AUGMENTATION

Predictive model Optimisation model(incl. judgmentparameter)

Integrative judgment

Shari De Baets – ISF 201419

© Vlerick Business School

GENERAL COMPARISON N = 1223

19%

20%

21%

22%

23%

24%

25%

26%

27%

Basic model Restrictive judgment Integrative judgment

MAPE

Shari De Baets – ISF 201420

© Vlerick Business School

GENERAL COMPARISON

-1,5%

-1,0%

-0,5%

0,0%

0,5%

1,0%

1,5%

2,0%

2,5%

3,0%

3,5%

Restrictive Integrative

FCIMP

Shari De Baets – ISF 201421

© Vlerick Business School

DIRECTION OF ADJUSTMENTS

Direction of adj Restrictive Integrative

Downward 587 464

No adjustment 28 375

Upward 608 384

Countering of ‘tinkering’ withforecasts

Shari De Baets – ISF 201422

© Vlerick Business School

DIRECTION OF ADJUSTMENT

-25%

-20%

-15%

-10%

-5%

0%

5%

10%

15%

20%

25%

Restrictive Integrative

Downward

Upward

Shari De Baets – ISF 201423

© Vlerick Business School

SIZE OF ADJUSTMENT

�Restrictive judgment: 98% adjustments

�Four quantiles of 1192 adjustments (298 per Q)

-10%

-8%

-6%

-4%

-2%

0%

2%

4%

6%

8%

1 2 3 4

Shari De Baets – ISF 201424

© Vlerick Business School

SIZE OF ADJUSTMENT

�Integrative judgment: 70% adjustments

�Four quantiles of 848 adjustments (212 per Q)

-15%

-10%

-5%

0%

5%

10%

15%

20%

1 2 3 4

Restrictive

Integrative

Shari De Baets – ISF 201425

© Vlerick Business School

VOLATILITY (SD)

Shari De Baets – ISF 2014

-0,35

-0,3

-0,25

-0,2

-0,15

-0,1

-0,05

0

0,05

0,1

0,15

1 2 3 4

Restrictive

Integrative

26

© Vlerick Business School

VOLATILITY (CATEGORY)

�Low volatility: n = 850

�High volatility: n 373

or

Shari De Baets – ISF 201427

DVD, CD

© Vlerick Business School

VOLATILITY (CATEGORY)

-15%

-10%

-5%

0%

5%

10%

15%

20%

25%

30%

Restrictive Integrative

Low volatility

High volatility

Shari De Baets – ISF 201428

© Vlerick Business School

CONCLUSION

�General: integrative > restrictive

�Counters harmful adjustments

�Upward

�Small

�Too big

�Low volatility

�But can also limit benefits of restrictive judgment

�Downward

�Big

�High volatility

Shari De Baets – ISF 201429

© Vlerick Business School

CONCLUSION

�Profitability?

74%

75%

76%

77%

78%

79%

80%

81%

82%

83%

84%

Basic model Restrictive Integrative

Profit (% of max profit)

Profit

Shari De Baets – ISF 201430

THANK YOU

CONTACT: SHARI.DEBAETS@VLERICK.COM

ISF 2014, The Netherlands