Forecasting decisions in conflicts:Best methods for supply chain, competition, union-

management, and takeover strategy problems

Ehrenberg Centre for Research in MarketingMonday 15th November 2010 at 5:00 PM

London South Bank University

Kesten C. GreenEhrenberg-Bass Institute for Marketing Science

International Graduate School of Business University of South Australia

kestencgreen.com ForPrin.com AdPrin.com

2

Scientific* forecasting methodsProcedures for making predictions about matters currently unknown, based on:

- empirical comparisons of proper alternatives- ex ante tests of accuracy given stated conditions

(e.g. level of knowledge about the situation)

*Used interchangeably with ”evidence-based”

3

Evidence-based methods Knowledge advances when multiple hypotheses are tested, especially if hypotheses challenge accepted wisdom, e.g.:• Market-share objectives harm profits.• Minimum-wage laws harm low-skilled workers• Regulation harms consumers• Pre-announced satisfaction surveys harm satisfaction• Anti-inflammatory drugs harm head injury patients

4

Principles for the selection and application of forecasting methods

Mid-1990s: Wharton School’s Scott Armstrong started the “principles of forecasting project” to summarize all knowledge about forecasting in the form of scientific principles.

A principle is a condition-action statement.

This project led to 139 principles as described in the Principles of Forecasting handbook.

39 authors & 123 reviewers

The principles (currently 140 in number) can guide the selection and application of methods

5

ForPrin.com features

• Descriptions of 140 forecasting principles • The Forecasting Canon with 9 key rules• Answers to Frequently Asked Questions • The Forecasting Dictionary• Forecasting Methodology & Selection Trees• Forecasting Audit Software• Resources for practitioners, educators, researchers• Special Interest Groups (SIGs)

– E.g. ConflictForecasting.com

6

Why not just ask an expert what will happen?

Most decisions in business based on managers’, or other experts’, judgments about what will happen, but… Tetlock (2005): evaluated

• 82,361 forecasts • made over 20 years

• by 284 professional commentators and advisors on politics and economics

Expertise did not lead to better forecasts… (but their excuses are better than novices’!)Tetlock’s finding is consistent with other findings from

research on forecasting by unaided experts.

7

Today’s issue: “predicting decisions in

conflicts”Predicting decisions of

– parties with– divergent interests, – who interact

Note the focus here is on predicting decisions, not outcomes.

8

Examples of conflict situations • What reactions to a first strike in Iran?• What offer to make in a union/management

negotiation?• How to respond to demands by mob protesters?• How to best resolve legal conflicts?• How will proposed water sharing regimes work?• How to design policy for…

• benefits?• tax? • job security?• market regulation?

…all require predictions of how people will respond.

9

Forecasting decisions in conflicts:

Research problems Artists protest: Artists stage sit-in & demand funding

Nurses dispute: Demand same pay increase

Distribution channel: Novel proposal to market appliances

Telco takeover bid: Bid for all after rejecting mobile offer

55% pay plan: NFL players demand 55% of revenue

Water dispute: Troops mass & threat to bomb dam

Employee grievance: Mediation when job down-graded

Zenith investment: Investment decision with politics

10

Water Dispute Syria is filling a new dam on the Euphrates River, thereby slowing the flow of water into Iraq in Spring 1975.

Syria and Iraq mass troops on their border, both threatening invasion.

Saudi Arabia offers mediation.

11

Telco takeover bidCenturyTel approaches AllTel with an offer to sell its mobile business.AllTel rejects the offer, offers to pay a 50%+ premium for the whole business.CenturyTel rejects counter offer, and AllTel pursues takover bid.

12

Methods for forecasting decisions in conflicts

Novices Experts

Guessing 28% 28%

Unaided judgmentRole thinkingGame theoryStructured analogiesSimulated interaction

13

Unaided forecasts from experts on…• conflict management• political science• industrial relations• marketing• judgement & decision making• forecasting• game theory

14

Unaided judgment methodThe most commonly used method for predicting decisions in conflict situations.Appropriate when:

• experts are unbiased• large changes are unlikely• relationships are known• experts get useful feedback from many similar cases

Expert and novice subjects read descriptions and made predictions.

15

Unaided judgment accuracy

3228Averages (unweighted)7333Nurses Dispute5033Water Dispute3125Personal Grievance3633Zenith Investment182555% Pay Plan025Telco Takeover

3833Distribution Channel1017 Artists Protest

Bold = more accurate

Chance UJ-ExpertsPercent Accurate Forecasts

UJ-Naive

55

10

2729

444568

29

16

Effect of experience & time-spent on accuracy

Percent correct forecasts

<5 yrs 5 yrs+Experience 36 29

<30 min30 min+

Time spent 32 35

17

Do game theorists recommend game theory for forecasting?

Yes, judging from• textbooks• papers• consultants’ advertisements

Google searches: “game theory” & “prediction” or “forecasting” . . .2,170,000 as of 15 November, 2010

adding “conflicts” to the search, yielded 789,000

18

Game theorists’ forecasts

Game theorists made predictions in response to a request that read “Using game theory to predict the outcomes of conflicts”

Hundreds invited; 23 participated

Respondents selected only some of the situations, yielding 101 forecasts

19

Game theorists’ accuracy

Telco Takeover Bid 0 0Artists’ Protest 10 655% Pay Plan 18 29Employee Grievance 31 43Zenith Investment 36 22Distribution Channel 38 23Water Dispute 50 75Nurses Dispute 73 50Averages (unweighted) 32 31

UJ GT

Percent Accurate Forecasts

20

Use of analogies in forecasting Analogical reasoning commonly used (informally) for prediction

58% of respondents said their organizations used analogies to forecast competitor actions (Armstrong, Brodie & McIntyre 1987)

Neustadt & May (1986). Thinking in time: The uses of history for decision makers.Kahneman & Lovallo (1993) example

21

Structured Analogies Domain experts individually:

1. list similar situations2. rate similarity to target situation3. match outcome to target situation

An administrator mechanically derives forecast (e.g., select outcome of the most similar situation as forecast).

Little prior evidence on structured analogies

22

Structured analogies method International Water Dispute

1) (A) In the table below, please briefly describe (i) your analogies, (ii) their source (e.g. your own experience, media reports, history, literature, etc.), and (iii) the main similarities and differences between your analogies and this situation. (B) Rate analogies out of 10 (0 = no similarity… 5 = similar… 10 = high similarity). (C) Enter the responses from question 2 (below) closest to the outcomes of your analogies.

(A) (i) description, (ii) source, (iii) similarities & differences

(B) Rate

(C) Q2

a.

b.

c.

d.

e.

2) The gist of the statement issued at the end of the meeting was? (check one , or %)

a. Midistan has decided to release additional water in order to meet the needs of the Deltalandish people [__] b. Deltaland has ordered the bombing of the dam at Mididam to release water for the needy Deltalandish people [__] c. Deltaland has declared war on Midistan [__]

23

Structured analogies accuracy

Telco Takeover Bid 0 14Artists’ Protest 10 5055% Pay Plan 18 80Employee Grievance 31 50Zenith Investment 36 50Distribution Channel 38 67Water Dispute 50 88Nurses Dispute 73 50Averages (unweighted) 32 56

UJ SA2+

Percent Accurate Forecasts

24

Use of structured analogies Percent

Correct

Unaided experts’ judgment 32

Structured analogies with experts’ predictions

42

Structured analogies withmechanical predictions

46

Structured analogies withexperts who “knew better”

25

25

Number of analogies and familiarity

% CorrectNumber of analogies (forecasts)

one 38 (53)two or more

56 (44)

Familiarity with analogiesindirect 37 (45)direct 49 (50)

+ two or more 60 (23)

ConclusionUse experts who have direct experience with analogous conflicts

26

Collaborating for analogies Collaborators were more experienced and spent more time than solo forecasters. % correct (forecasts)Solo 44 (75)Collaborative 42 (22)

Conclusions: • Collaboration is not effective when using analogies. • But, there is little harm if people want to discuss analogies.

27

Role thinking process

In the Fog of War, Robert McNamara concluded that one should put oneself in the shoes of an opponent.

Subjects received information about the situations along with the information about roles and were asked to think about the roles before making a prediction

28

Role thinking processInternational Water Dispute

1) A person’s role can make a big difference to how he or she views a situation, so it can be difficult to predict

what decisions will be made when people interact with each other. In the following table, please indicate which decision you think each party in the situation just described would prefer to be made and assess how likely it is that each party’s preferred decision will actually occur.

For each party in the conflict, please use your judgment to:

(A) (i) select from the following list the decision or decisions the party would prefer to see emerge from today’s meeting:

a. Midistan has decided to release additional water in order to meet the needs of the Deltalandish people b. Deltaland has ordered the bombing of the dam at Mididam to release water for the needy Deltalandish people c. Deltaland has declared war on Midistan (ii) explain why you think the party prefers that decision or those decisions

(B) (i) explain how you think the party would go about trying to achieve its most-preferred decision (ii) rate the chances that the decision will be made, out of 10 (0 = almost no chance…10 = almost certain)

Party

(A)(i) Party’s preferred decision(s) from a-c, above

(ii) Why preferred

(B)(i) How party would try to achieve most-preferred decision

(ii) Chances that most-preferred decision will be made(0-10)

Midistan (i) [______] (ii) (i) (ii) [___]

Deltaland (i) [______] (ii) (i) (ii) [___]

Concordia (i) [______] (ii) (i) (ii) [___]

2) Given your analysis in Q1, which of the decisions listed in (A)(i) above is most likely? [____] a-c 3) Why will that (Q2) decision occur, and why might it not occur?

29

Role thinking accuracy

Distribution Plan 38 0 0Artists Protest 10 8 1355% Pay Plan 18 13 8Telco Takeover 0 13 18Journal Negotiations - 25 30Personal Grievance 31 - 36Zenith Investment 36 46 55Water Dispute 50 75 56Nurses Dispute 73 82 73Averages (unweighted) 32 33 31

Unaided Judgment (Experts)

Role Thinking (Novices)

Percent Accurate ForecastsRole Thinking (Experts)

30

Simulated interaction (SI) process

Brief role description, name badge (2+ roles)

One-page situation description

Simulate interactions (i.e., role play realistic interactions)

Interactions take less than an hour

Outcome (decision) of the simulation used as a forecast

31

Simulated interaction accuracy(Naïve Subjects)

Telco Takeover Bid 0 40Artists’ Protest 10 2955% Pay Plan 18 60Employee Grievance 31 60Zenith Investment 36 59Distribution Channel 38 75Water Dispute 50 90Nurses Dispute 73 82Averages (unweighted) 32 62

UJ

Percent Correct Predictions

SI (N)

32

Summary of accuracy

Telco Takeover Bid 0 (8) 0 (7) 14 (7) 40 (10)Artists’ Protest 10 (20) 6 (17) 50 (4) 29 (14)55% Pay Plan 18 (11) 29 (17) 80 (5) 60 (10)Employee Grievance 31 (13) 43 (7) 50 (6) 60 (10)Zenith Investment 36 (14) 22 (18) 50 (4) 59 (17)Distribution Channel 38 (17) 23 (13) 67 (6) 75 (12)Water Dispute 50 (8) 75 (8) 88 (8) 90 (10)Nurses Dispute 73 (15) 50 (14) 50 (4) 82 (22)Averages (unweighted) 32 (106) 31 (101) 56 (44) 62 (105)

UJ (E) SA2+

Percent Correct Predictions

SI (N)GT

_______Experts________ Non-experts

33

Combine forecasts “mechanically”:Avoid face-to-face meetings

• A meta analysis of 30 studies found 12% error reduction (3% to 24%) with combinations always more accurate than typical individual forecasts.

• With favourable conditions (election forecasts)*, error reductions averaged between 42% and 50% compared to typical individual forecasts.

• Each component must contain some information.*Several valid forecasting methods using different information sources allowing combining within and between methods.ReferenceGraefe, et al. (2010).

34

The logic of combining

How bad can a second forecast (F2) be and still give an average that is no worse than F1?

------------------------------A--------F1----

35

The logic of combining 2How bad can a second forecast (F2) be and

still give an average that is no worse than F1?

Answer: The error can be the same size, or up to three-times bigger if it is the opposite sign.

------- F2 -----------------------A--------F1----

36

Accuracy of combined forecasts(across 8 situations)

Game theorists 31 38 10

Structured analogies (SA) 46 63 31

Simulated interaction (SI) 62 88 68

SA & SI [63 + 88]/2 =75.5 - 88 51

Percent correctIndividual Combined

Approx. %Error

Reduction

37

Conclusions1. Unaided experts’ judgments little better than

those of college students or guessing. 2. Forecasts by game theorists no better than

unaided experts’ judgments.3. Structured analogies method provides

substantial improvements in accuracy.4. Forecasts from simulated interaction were

most accurate and the method most flexible.5. Combining further improves accuracy.

Possible applications of SA and SI for business and government

Forecast responses to alternative strategies in conflict situations:• Labour-management disputes• Competitor, supplier, and distributor behavior• Customer reactions to major changes in product,

price or service (e.g., “New Coke”)• Behavior in response to new laws or regulations• Diplomatic and national security problems

38

39

Summary• Do not rely on experts’ unaided

judgmental forecasts• Do require forecasts:

– That are scientific (i.e. from evidence-based methods – see ForPrin.com)

– For alternative policies or strategies– Of all effects– Of all costs and benefits

ReferencesArmstrong, J. S. (2001). Principles of Forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer.Graefe, A., Armstrong, J. S., Jones, R. J. & Cuzan, A. (2010). Combining forecasts: An application to U.S. Presidential Elections.

Working paper. Available at: http://dl.dropbox.com/u/3662406/Articles/Graefe_et_al_Combining.pdf Green, K. C. (2005). Game theory, simulated interaction, and unaided judgement for forecasting decisions in conflicts: Further

evidence. International Journal of Forecasting, 21, 463-472. Available at http://www.kestencgreen.com/gt_update_in_IJF21.pdf

Green, K. C. (2002). Forecasting decisions in conflict situations: a comparison of game theory, role-playing, and unaided judgement. International Journal of Forecasting, 18, 321-344. http://www.forecastingprinciples.com/paperpdf/Greenforecastinginconflict.pdf

Green, K. C. & Armstrong, J. S. (2007). The value of expertise for forecasting decisions in conflicts. Interfaces, 37, 287-299. Available at http://kestencgreen.com/green&armstrong2007-expertise.pdf

Green, K. C. & Armstrong J. S. (2007). Structured Analogies in Forecasting, International Journal of Forecasting, 23, 365-376. Available at http://www.forecastingprinciples.com/files/pdf/INTFOR3581_Publication15.pdf

Green, K. C. & Armstrong, J. S. (2011). Role thinking: Standing in other people’s shoes to forecast decisions in conflicts. International Journal of Forecasting, 27, 69-80. Available at http://kestencgreen.com/group_shoes-2009.pdf

Green, K. C., Graefe, A. & Armstrong, J. S. (2010). Forecasting principles. In Lovric, M. (ed.), International Encyclopedia of Statistical Science. Springer [In press].

Tetlock, P. E. (2005). Expert political judgment: How good is it? How can we know? Princeton, NJ: Princeton.

40