Risk-Based Decision Making - DAU Home Decision Making Richard Sugarman Air Force Institute of...

Risk-Based Decision Making

Richard Sugarman<NAMES>

Air Force Institute of TechnologySchool of Systems and Logistics

Department of Systems and Software Engineering Management

Question:

What is a “good” decision?

Graphic adapted from “Decision Analysis for the Professional” by Peter McNamee and John Celona (4th Ed, 2008)

RM is about Decisions

Outcomes● ● ●DecisionAlternatives

Information

Logic

Objectives

A “good decision” is one that is logically consistent with the objectives, alternatives,

and information brought to the decision.

Note: Outputs = measureable objectives

…which starts with knowing organizational objectives!

RM is about Decisions…

I-P-O for Individual Organizations

Organization AProcesses

Inputs Outputs

Organization BProcesses

Organization nProcesses

NOTE:Organization =

- Squadron- Division- Section- Project- Team- Etc.

Inputs

Inputs

Outputs

Outputs

I-P-O for Enterprise

Sub-Org A“P”

“I” “O”

Sub-Org B“P”

“I” “O”

Sub-Org n“P”

“I” “O”

INPUTS OUTPUTS

PROCESSES

Do you know your organization’s measureable objectives?

A. Yes!

B. No, but I know where to get them

C. No, & I don’t know where to get them Yes!

No, but I

know

where

to ge...

No, & I

don’t know

where

..

0% 0%0%

Outputs = measureable objectives

QUESTION

Tip #1

Know your org’s measureable objectives

OrganizationalProcesses

Inputs Outputs

What is Risk?

“Uncertainty That Matters”

* Definition from Dr. David Hillson, www.risk-doctor.com

What is Risk Management?

Getting sufficient, relevant, timely information about uncertainty to

make better decisions

Outcomes● ● ●DecisionAlternatives

Information

Logic

Objectives

The “Value” of Information

Adapted from Douglas Hubbard’s book “How to Measure Anything” (2nd Ed, 2010)

A Measurement Question:

When you have a lot of uncertainty and don’t know anything,

you need a lot of relevant data to tell you something useful.

True or False?

“Val

ue”

of

info

rmat

ion

to

war

ds

red

uci

ng

un

cert

ain

ty

None

Theoretical “Perfect”

Info

Resource expenditure to get information None ∞From Douglas Hubbard’s book “How to Measure Anything” (2nd Ed, 2010)

The “Value” of Information

How much RM do I need?

Like

lihoo

d of

Ris

k E

vent

Occ

urrin

g

Negative Impact of Event

More tolerant

Less tolerant

Do you think about risk tolerance when making a decision?

A. Yes, quite often

B. Sometimes

C. Rarely or not at allYes,

quite o

ften

Sometim

es

Rarely o

r not a

t all

0% 0%0%

QUESTION

Tip #2

Think about tolerance to the uncertainty that matters

OrganizationalProcesses

Inputs Outputs

Do you have trouble identifying risks?

A. Yes, always – it’s hard!

B. Sometimes

C. I can often ID things that can go wrong

Yes, alw

ays – it

’s hard

!

Sometim

es

I can ofte

n ID th

ings t

hat c..

0% 0%0%

QUESTION

Tip #3

What could go wrong here… that would negatively

impact these

Start with objectives and work back

Tip #3

Start with objectives and work back…in practice…

How do you most often think about the magnitude of risks?

A. Quantitatively

B. Qualitatively

C. I don’t really think about the magnitude of risks

Quantitativ

ely

Qualitativ

ely

I don’t

really

thin

k about t

..

0% 0%0%

QUESTION

Risk Analysis Spectrum

Easier HarderEasiness to Create

Quicker LongerTime to Create

Less MoreValue of Information about Uncertainty

Subjective/Qualitative Objective/QuantitativeType

• SME “guesstimates”• Ordinal Scales• Risk Matrices• Risk “Levels”

• Confidence Ranges• Parametric Analysis• Monte Carlo Simulation• Expected Loss / EMV• Probability Distributions

Accuracy and Uncertainty

Range of values

Lower Bound

Upper Bound

“True” Value

Range of values

Lower Bound

Upper Bound

“True” Value

Probability of Risk Event Confidence

100%1% 99%

<100%20% 80%

<<100%45% 65%

~0%53.785%

Probability Range vs. Confidence

Tip #4

Think about ranges and confidence…not ordinal levels or red/yellow/green

NO!! ↓ YES!! ↓

Risk Matrices are Reporting Tools, NOT Decision Tools

Bottom line: Plotting on the 5x5 is NOT analyzing risks or performing “risk management”!

Corollary Tip #4a

The Big Picture of RM

AFPAM 63-128: “At its core, effective program management and oversight isrisk management: the proactive management of future uncertainties…”

“It must be a core responsibility of every team member performed on a daily basis.”

Time →Start

Decision

Complete tasks,gather info

Complete tasks,gather info

Complete tasks,gather info

Complete tasks,gather info

Complete tasks,gather info

Decision Decision DecisionEnd

Reminder: Four Tips

1. Know your organization’s measureable objectives (outputs)

2. Think about tolerance to the uncertainty that matters

3. To ID risks, start with objectives and work back

4. Think about ranges and confidence…not ordinal levels or red/yellow/green4a. Risk Matrices are Reporting Tools, NOT Decision Tools

Do you think you can now make better risk-based decisions?

A. Yes!

B. Not really

C. I think so, but need additional education Yes!

Not really

I thin

k so, b

ut need a

dditio...

0% 0%0%

QUESTION

Good risk mgmt gets us…

An Air Force that doesn’t get off the ground

Poor risk mgmt gets us…

…Or at least, not very far!!

E d u c a t i o n - R e s e a r c h - C o n s u l t a t i o n

Air Force Institute of Technology

Thank you!