An empirical investigation - proceedings · an empirical investigation on the value of combined judgmental and statistical forecasting prof. dr. philippe baecke prof. dr. karlien

AN EMPIRICAL INVESTIGATION ON THE VALUE OF COMBINED JUDGMENTAL AND

STATISTICAL FORECASTING

PROF. DR. PHILIPPE BAECKEPROF. DR. KARLIEN VANDERHEYDEN

DRS. SHARI DE BAETS

CONTACT: [email protected]

ISF 2014, The Netherlands

© Vlerick Business School

AN EMPIRICAL INVESTIGATION

ON THE VALUE OF COMBINED JUDGMENTAL AND STATISTICAL

FORECASTING

Shari De Baets – ISF 20142



� “..the practice of forecasting does not however appear to have improved.” (p,33; Armstrong, Green &

Graefe, 2013)

� Ascher, 1978

� Allen, 1994

� McCarthy, Davis, Golicic & Mentzer, 2006




An important task for researchers in our field:

Improving forecastingaccuracy in practice

(Sanders & Manrodt, 2003)




�Call for more studies with real company data (Sanders, 2009)

�Going beyond artificial experiments to deriverules for practice

�Expert forecasters

�Context of forecasting task

�A new research model: JUD 3 session(room “Plate”, Wed., 10 AM)




ON THE VALUE OF COMBINED JUDGMENTAL AND STATISTICAL

FORECASTING



THE VALUE OF COMBINED FORECASTING

25%

25%

17%

33%

Forecasting method

judgment alone

statistical methods alone

average of statistical and

judgmental forecast

statistical forecast adjusted

judgmentally

Fildes & Goodwin, 2007



THE VALUE OF COMBINED FORECASTING

� Potential of judgment in forecasting oftenundermined by unneccessary adjustments(Lawrence et al., 2006)

� Patterns in noise (Harvey, 1995)

� Illusion of control effect (Kotteman, Davis, & Remus,

1994)

� Persists despite warning (Lim & O’Connor, 1995)



RESEARCH QUESTION

How can we counter damaging adjustments andreap the potential benefits from judgment, andthus, heighten forecasting accuracy?



HYPOTHESES

� Data augmentation:

� Classic model + judgment:

Var1, Var2, .., Varn -> outcome -> judgmentaladjustment

� Judgment incorporated in model

Var1, Var2, VarJudgment , .., Varn -> outcome



HYPOTHESES

�Data augmentation:

� H1:

� “integrated judgment”: judgment as part of the model

will outperform

� “restrictive judgment”: judgment as restriction on the model (judgmental adjustment)



HYPOTHESES (SIMILAR TO FILDES ET AL., 2009)

� H1 applied to

�Direction of the adjustment

�Downward adjustments are beneficial, upwardadjustment are damaging

�Size of the adjustment

�Curvilinear (inverted U-shape) effect: both small and very large adjustments are damaging

�Volatility of the data series

� Judgment is beneficial in high volatility series (volatility due to special events)



THE COMPANY


Outlet store

Outlet store

Outlet store

International publishing company

Weekly and monthlymagazines


PROCEDURE

� Demand forecasting

� Predictive model: forecast of expected demandper store

� Optimisation model: profit optimisation – finalnumber for supply per store

Input



THE COMPANY

� Profit optimization model

Overstock

Stockout



PROCEDURE

�Predictive model: forecast of expected demandper store

�Optimisation model: profit optimisation – finalnumber for supply per store

�Judgmental adjustment: according to insight



PROCEDURE

Predictive Optimisation Judgmentalmodel model adjustment

Restrictive judgment



PROCEDURE – DATA AUGMENTATION

Predictive Optimisation Judgmentalmodel model adjustment

Integrative judgment



PROCEDURE – DATA AUGMENTATION

Predictive model Optimisation model(incl. judgmentparameter)

Integrative judgment



GENERAL COMPARISON N = 1223

19%

20%

21%

22%

23%

24%

25%

26%

27%

Basic model Restrictive judgment Integrative judgment

MAPE



GENERAL COMPARISON

-1,5%

-1,0%

-0,5%

0,0%

0,5%

1,0%

1,5%

2,0%

2,5%

3,0%

3,5%

Restrictive Integrative

FCIMP



DIRECTION OF ADJUSTMENTS

Direction of adj Restrictive Integrative

Downward 587 464

No adjustment 28 375

Upward 608 384

Countering of ‘tinkering’ withforecasts



DIRECTION OF ADJUSTMENT

-25%

-20%

-15%

-10%

-5%

0%

5%

10%

15%

20%

25%


Downward

Upward



SIZE OF ADJUSTMENT

�Restrictive judgment: 98% adjustments

�Four quantiles of 1192 adjustments (298 per Q)

-10%

-8%

-6%

-4%

-2%

0%

2%

4%

6%

8%

1 2 3 4



SIZE OF ADJUSTMENT

�Integrative judgment: 70% adjustments

�Four quantiles of 848 adjustments (212 per Q)

-15%

-10%

-5%

0%

5%

10%

15%

20%

1 2 3 4

Restrictive

Integrative



VOLATILITY (SD)


-0,35

-0,3

-0,25

-0,2

-0,15

-0,1

-0,05

0

0,05

0,1

0,15

1 2 3 4

Restrictive

Integrative

26


VOLATILITY (CATEGORY)

�Low volatility: n = 850

�High volatility: n 373

or


DVD, CD


VOLATILITY (CATEGORY)

-15%

-10%

-5%

0%

5%

10%

15%

20%

25%

30%


Low volatility

High volatility



CONCLUSION

�General: integrative > restrictive

�Counters harmful adjustments

�Upward

�Small

�Too big

�Low volatility

�But can also limit benefits of restrictive judgment

�Downward

�Big

�High volatility



CONCLUSION

�Profitability?

74%

75%

76%

77%

78%

79%

80%

81%

82%

83%

84%

Basic model Restrictive Integrative

Profit (% of max profit)

Profit


THANK YOU

CONTACT: [email protected]

ISF 2014, The Netherlands

Documents

An empirical investigation - proceedings · an empirical investigation on the value of combined judgmental and statistical forecasting prof. dr. philippe baecke prof. dr. karlien