MAD 2015 2016 T01 Introduction to decision analysis · sensiVvity’analysis’ ’ ’ ’ Requisite’Decision’Models

16/09/15

1

Introduc*on to Decision Analysis

Carlos Bana e Costa

1st SEMESTER, 2015/2016 DECISION SUPPORT MODELS, DEPARTMENT OF ENGINEERING AND MANAGEMENT

h7p://web.ist.utl.pt/carlosbana/

DECISION SUPPORT MODELS, DEPARTMENT OF ENGINEERING AND MANAGEMENT 2 1st SEMESTER, 2015/2016

16/09/15

2

Decisionmaking and organisa*onal management

Decision makers in all organisa/ons con/nually face the difficult task of balancing benefits against costs and the risks of realising the benefits.

1st SEMESTER, 2015/2016 DECISION SUPPORT MODELS, DEPARTMENT OF ENGINEERING AND MANAGEMENT 3

The task is more difficult in face of high levels of complexity and uncertainty


Complexity

• Many aspects are present (costs, beneIts, risks, etc.)…

• … which are interrelated and evolve quickly with Vme…

• … thus making difficult the idenVficaVn of the key-‐issues for decisionmaking

• What are the expected consequences?

•  Different sources:

-‐ Unclear objecVves -‐ Scarce informaVon -‐ Rough data -‐ Non-‐control over interrelated decisons areas -‐ Lack of coordenaVon, …

Uncertainty

16/09/15

3

Cogni*ve map of the key survival factor of SMS tex*le firms of the State of Santa Catarina, Brazil


One view of decisionmaking…


Decision making is a job that lies at the very heart of leadership...

… above all else leaders are made or broken by the quality of their decisions.

D. Garvin & M. Roberto What you don't know about making decisions

Harvard Business Review, 2001

Nothing is more difficult, and therefore more precious, than to be able to decide.

Napoleon Maxims, 1804

16/09/15

4

An opposite view of decisionmaking…


Nothing good ever came from a management decision. Avoid making decisions whenever possible. They can only get you in trouble.

•  Act confused •  Form a task force of people who are too

busy to meet •  Send employees to find more data •  Lose documents submiUed for your

approval •  Say you are wai/ng for some other

manager to “get up to speed”

•  Make illegible margin scrawls on the documents requiring your decision

Dogbert 1996

Taking a wrong approach to decisionmaking…


Indeed, our research over the past several years strongly suggests that, simply put, most leaders get decision making all wrong.

D. Garvin & M. Roberto What you don't know about making decisions

Harvard Business Review, 2001

A major study of the behavior of 165 top execu/ves in six companies reveals decision-‐making weaknesses which all management groups have in some degree

Chris Argyris “Interpersonal barriers to decision making”

Harvard Business Review on Decision Making, 2001

16/09/15

5

Decisionmaking strategies…


 Intui3ve decisionmaking

 Analy3cal process

 Process consulta3on

Intui*ve decisionmaking: Subject to inconsistent judgement


Simon French, 1988 Decision Theory:

An Introduc/on to the Mathema/cs of Ra/onality

Despite our natural inclina/on to believe in the ability of the human

mind to make well-‐considered judgements and decisions, much

evidence has been accumulated by many psychologists to make such a

belief untenable. It appears that unguided, intui/ve decision making is

suscep/ble to many forms of inconsistency.

16/09/15

6

People's preferences may be dictated by the presenta*on of a problem and not by its underlying structure


A group of 152 students were to imagine that the US was preparing for an epidemic which was expected to kill 600 (thousands) people. They had to choose between two health programmes to combat the epidemic

Ø  Programme A would save 200 people.

Ø  Programme B would give a 1/3 probability of saving all 600 lives and a 2/3 probability that no one would be saved.

72% of the students preferred programme A. In a second test, 155 different students were presented with the same situaVon. However, they were offered the choice between the following programmes

Ø Programme C would lead to 400 dying.

Ø Programme D would give 1/3 probability that no one would die and 2/3 probability that 600 would die.

78% of the students preferred programme D. Amos Tversky & Daniel Kahneman “The framing of decisions and the psychology of choice”

Science, 1981

Which of the two lines, the blue or the red, is the longest?


16/09/15

7

Which of the two lines, the blue or the red, is the longest?


Müller-‐Lyer illusion

Analy*cal process: Subject to quan*ta*ve meaningfulness


High risk of using

theoreVcal inconsistent

and / or inadequate

analyVc procedures

16/09/15

8

Example of inconsistent quan*ta*ve method...


Ranking Crit.1 Crit.2 Crit.3 Crit.4 Crit.5 Crit.6 Crit.7 Bid A 1st 4th 3rd 1st 4th 3rd 1st

Bid B 2nd 1st 4th 2nd 1st 4th 2nd

Bid C 3rd 2nd 1st 3rd 2nd 1st 3rd

Bid D 4th 3rd 2nd 4th 3rd 2nd 4th

Scoring Crit.1 Crit.2 Crit.3 Crit.4 Crit.5 Crit.6 Crit.7 Total Bid A 3 0 1 3 0 1 3 11 Bid B 2 3 0 2 3 0 2 12 Bid C 1 2 3 1 2 3 1 13 Bid D 0 1 2 0 1 2 0 6

… then, to score the bids, assigning to each bid in each criterion a score equal to the number of other bids that it outranks…

Procedure used to evaluate four bids (A, B, C, D) against seven criteria. Firstly, to rank the bids against each criterion…

0 points 4th 1 point 3rd 2 points 2nd 3 points 1st

… finally, choose the bid with the higher score in all criteria together:

Example of inconsistent quan*ta*ve method...


Ranking Crit.1 Crit.2 Crit.3 Crit.4 Crit.5 Crit.6 Crit.7 Bid A 1st 4th 3rd 1st 4th 3rd 1st

Bid B 2nd 1st 4th 2nd 1st 4th 2nd

Bid C 3rd 2nd 1st 3rd 2nd 1st 3rd

Bid D 4th 3rd 2nd 4th 3rd 2nd 4th

Scoring Crit.1 Crit.2 Crit.3 Crit.4 Crit.5 Crit.6 Crit.7 Total Bid A 2 0 1 2 0 1 2 Bid B 1 2 0 1 2 0 1 Bid C 0 1 2 0 1 2 0

… the same procedure applied to bids A, B and C only… … would give rise to rank reversal!

0 points 3rd 1 point 2nd 2 points 1st

Meanwhile, it was found out the bid D should have been eliminated during the screening phase, because it does not respect some acceptability requisites… It was noted that C dominates D. However…

3rd 2nd 3rd 2nd

3rd 3rd

6 7 8

16/09/15

9

Where is the problem? How to overcome it?


B 3 B 2

A 0

C 2 C 1

D 1

A 0

1 1

2 1

Dependance of irrelevant alternaVves (only ordinal judgements are present)

The difference of ahracVveness (value)

between B and C is bigger, equal or

smaller than the difference of

ahracVvenes between C e A ?

Ask for judgements about differences of value

(cardinal preference informaVon)

Weigh*ng criteria...


The most common cri/cal mistake

Ralph L. Keeney Value-‐Focused Thinking, 1992

16/09/15

10

B ► 20×0.625 + 40×0.375 = 27.5

A ► 0×0.625 + 100×0.375 = 37.5

€100,000 A

€50,000 C

C 10 months

€90,000 B B 8 months

5 months A

Cost Deadline

Cri*cal mistake – Importance weigh*ng

100

20

0

100

40

0

Scores

Importance grades

1 2 3 4 5 1 2 3 4 5

3/8 5/8

Scenario 1 (3 bids accepted A, B, C): Worst deadline = 10 months

C ► 100×0.625 + 0×0.375 = 62.5


100

0

100

0

B ► 100×0.625 + 0×0.375 = 62.5

A ► 0×0.625 + 100×0.375 = 37.5

C

B

A C 10 months

B 8 months

5 months A

Scores

1 2 3 4 5 1 2 3 4 5

3/8 5/8

Scenario 2 (C rejected ⇒ 2 bids accepted A, B): Worst deadline = 8 months

Cost Deadline

Importance grades

Cri*cal mistake – Importance weigh*ng

€100,000 A

€50,000 C

C 10 months

€90,000 B B 8 months


16/09/15

11

Weigh*ng criteria...


Plausible worst

Construc3on cost (millions of euros)

Construc3on deadline

(months) Plausible best

At the end of the day, which is more important, cost or deadline of construcCon?

Worst

Best

100 40

75 35

100 39

75 34

Traps in priori*sing projects for resource alloca*on


16/09/15

12

How to improve decisionmaking in organisa*onal management?

Help clients structure and simplify the task of making a complex decision as well, and as easily, at the nature of the decision permits, with the professional support of

Decision Analysis and Decision Conferencing


Process consulta3on

Dear Sir,

In the affair of so much importance to you,

wherein you ask my advice, I cannot …

advise you what to determine, but if you

please I will tell you how.

Benjamin Franklin leUer to Joseph Priestly, 1772 Moral and Pruden/al Algebra

Decision Analysis:


 Development and use of logical methods for the improvement of decision-‐making

in public and private enterprise.

Such methods include: ü  models for decision-‐making under condi*ons of uncertainty or mul*ple

objec*ves

ü  techniques of risk analysis and risk assessment

ü  experimental and descrip*ve studies of decision-‐making behavior

ü  economic analysis of compe**ve and strategic decisions ü  techniques for facilita*ng decision-‐making by groups

ü  computer modeling soWware and expert systems for decision support

16/09/15

13

25 1st SEMESTER, 2015/2016 DECISION SUPPORT MODELS, DEPARTMENT OF ENGINEERING AND MANAGEMENT

Schools in the founda3ons of Decision Analysis


Harvard

mulV-‐ahribute

uVlity analysis

Stanford model stages

sensiVvity analysis USC small models

judgmental raVngs Requisite Decision Models ‘sufficient in form and content to resolve the issues of concern’

LSE groups

small models constant feedback iteraVve approach

generaVve, construcVve

16/09/15

14


A Taxonomy of Decision Models (In Decision Analysis in the 1990s -‐ L.D. Phillips)

Problem dominated by

REVISE opinion • Bayesian nets

EXTEND conversaVon • Event tree • Fault tree • Influence diagram

SEPARATE into components • Credence decomposiVon • Risk analysis

EVALUATE opVons

• MulV-‐criteria decision analysis

ALLOCATE resources

• MulV-‐criteria commons dilemma

NEGOTIATE

• MulV-‐criteria bargaining analysis

CHOOSE opVon • Payoff matrix • Decision tree

Uncertainty MulVple ObjecVves

27

What kind of decision-‐aid approach?


→ a socio-‐technical approach

NormaVve PrescripVve ConstrucVve

ParVcipaVon

•  Sound theoreVcal basis •  Group learning •  InteracVvity; facilitaVon •  IntuiVve holisVc preferences vs. model outputs

•  The problem and the soluVon belongs to the decision-‐maker not to the consultant

•  The facilitator guides the process interfering in the context not in the contents

16/09/15

15


Professional decision-‐aiding requires a sound theore*cal basis

…in the same way that we rely so firmly upon the natural sciences for our technological advances.

Elliot Jaques Requisite Organization,

1988


Theore*cal founda*ons of Decision Analysis

Probability theory

•  Origins: Pascal, De Fermat, 1654; Bayes, 1763

•  AxiomaVc foundaVon : Ramsey, 1931; de Finet, 1937

•  Origins: Bernoulli, 1738 •  AxiomaVc foundaVon: von Neumann e Morgenstern, 1947 : Savage, 1951

U3lity theory

•  Origins: Savage, 1954 (“Sure thing” principle)

•  AxiomaVc foundaVon : Ranking, transiVvity, dominance

Axioms of preference

EUi = pijj∑ uij and max

i(EUi )

i opVons and j consequences

•  Probability exists •  UVlity exists

•  Choose opVon that maximizes EU

Expected u3lity (EU)

UwU ijkk

kij ∑= '

•  k independent criteria

Mul3-‐AVribute U3lity Theory (MAUT)

30

16/09/15

16

Requisite Decision Modelling


q  DefiniVon ◦ Model is requisite when its form and content are sufficient to resolve the issues of concern.

q  GeneraVon ◦  Through iteraVve and consultaVve interacVon amongst specialists and key players, facilitated by an imparVal decision analyst.


Consulta*on frameworks Model of

consulta3on Approach Goal Learning provided Decision Analysis

Schools

EXPERT model

NormaVve Fix client’s problem

AdapVve, Single loop

Client

Consultant

Howard (STANFORD)

DOCTOR model

PrescripVve Fix client’s problem

together with the client

More adapVve than generaVve

Keeney & Raiffa (HARVARD)

HELPER model

ConstrucVve Increase client’s

capacity of learning

GeneraVve, Double loop

Client

Consultant

Roy (LAMSADE) Edwards (USC) Phillips (LSE) Bana e Costa (IST)

16/09/15

17

General helper model

Edgar Schein, 1999 Process Consulta/on Revisited:

Building the Helping Rela/onship

q  Always try to be helpful

q  Always stay in touch with the current reality

q  Access your ignorance

q  Everything you do is an intervenVon

q  It is the client who owns the problem and the soluVon

33

Process Consulta/on is the crea/on of a rela/onship with the client that permits the client to perceive, understand, and act on the process events that occur in the client’s internal and external environment in order to improve the situa/on as defined by the client


Intersection (Bana e Costa) MACBETH Decision-‐aid Conferencing

Process consulta*on mul*criteria approaches

MulVcriteria Decision Aiding (Bernard Roy)

Decision aid studies

MulVcriteria Decision Analysis Conferencing

(Larry Phillips) Decision conferences


16/09/15

18


The Decision Conference Process

Awareness of issue

Ac/ons

Key Players

Explore Issues

Build Model

Explore Model

Shared Understanding Commitment

Prepare -‐objecVves -‐parVcipants -‐calling note

Compare: Gut⇔Model

What is a decision conference? Ø A two-‐ or three-‐day meeVng

Ø To resolve important issues of concern

Ø Ahended by key players who represent the diversity of perspecVves on the issues

Ø  Facilitated by an imparVal specialist in group processes & decision analysis

Ø Using a requisite model created on-‐the-‐spot to help provide structure to thinking

© 2009 Larry Phillips

36 36 1st SEMESTER, 2015/2016 DECISION SUPPORT MODELS, DEPARTMENT OF ENGINEERING AND MANAGEMENT

16/09/15

19

There are some generic objec*ves

Ø To generate shared understanding of the issues (not necessarily consensus)

Ø To develop a sense of common purpose (allowing individual differences of opinion)

Ø To agree about the way forward (commitment to the direcVon, not the individual paths)




Why do they work?

Ø Three condiVons for group to outperform its members →

Regan-‐Cirincione, P. (1994). Organiza/onal Behavior and Human Decision Processes 58: 246-‐70.

Ø Process gains in group allow ‘many heads to be beher than one’

Ø  Social and technical

Group FacilitaVon

Judgement Modelling

InformaVon Technology


16/09/15

20

The new Decision Conferencing

Ø  Sustained, engaged working with a client Ø  Use of workshops, decision conferences, and off-‐line

data gathering

Ø  Focus on strategy: what & why, not how and when Ø  Less concern with decisions, more on how groups can

contribute to decision processes


© 2009 Larry Phillips & Carlos Bana e Costa


Social component: Methodological guidelines Sociotechnical process design Cherns, A. (1976). The Principles of sociotechnical design. Human RelaVons, 29, 8, 783-‐792.

Requisite Decision Modelling Phillips, L.D. (1984). A theory of requisite decision models. Acta Psychologica, 56, 29-‐48.

q  DefiniVon: Model is requisite when its form and content are sufficient to resolve the issues of concern. Model generaVon: Through iteraVve and consultaVve interacVon amongst specialists and key players, facilitated by an imparVal decision analyst.

Process Consulta3on Schein, E. (1999). Process ConsultaVon Revisited: Building the Helping RelaVonship.

q  The problem and the soluVon belong to the client not to the consultant.

q  “OrganizaVonal objecVves are best met [...] by the joint opVmizaVon of the technical and the social aspects, thus exploiVng the adaptability and innovaVveness of people in ahaining goals instead of over-‐determining technically the maher in which these goals should be ahained.”

16/09/15

21


Design of the social process

MACBETH decision-‐aid conferencing…


MACBETH Approach

Analyzing the problem context and structuring the decision-‐aiding interven3on process

Structuring the evalua3on elements

Building a mul3criteria evalua3on model

Sensi3vity and robustness analyses and elabora3on of recommenda3ons

Documents

MAD 2015 2016 T01 Introduction to decision analysis · sensiVvity’analysis’ ’ ’ ’ Requisite’Decision’Models