18
AMERICA’S ARMY: THE STRENGTH OF THE NATION Mort Anvari [email protected] 1 Cost Risk and Uncertainty Analysis MORS Special Meeting | 19-22 September 2011 Sheraton Premiere at Tysons Corner, Vienna, VA

Mort Anvari [email protected]

Embed Size (px)

DESCRIPTION

Mort Anvari [email protected]. Cost Risk and Uncertainty Analysis. MORS Special Meeting | 19-22 September 2011 Sheraton Premiere at Tysons Corner, Vienna, VA. Cost Risk and Uncertainty Analysis. The Need For: - PowerPoint PPT Presentation

Citation preview

Page 1: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Mort [email protected]

1

Cost Risk and Uncertainty Analysis

MORS Special Meeting | 19-22 September 2011 Sheraton Premiere at Tysons Corner, Vienna, VA

Page 2: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

AS OF: 04/20/23 03:55

The Need For: Without risk analysis, a cost estimate will usually be a point estimate, which may not account for all

of the uncertainties inherent in the effort Not accounting for potential uncertainties may lead to underfunding, cost overruns, and the

potential for a program to be reduced in scope in the future

Definitions: Cost risk and uncertainty analysis identifies the cost, in terms of dollars, time, and materials that

should be added to a point estimate to increase the probability of meeting the desired outcome Risk is the occurrence of an outcome subject to a known pattern of random variation (i.e., Known-

Unknown). Changes in those technical parameters that are captured in the estimating methodology Uncertainty is the occurrence of an outcome subject to an unknown random fluctuations (i.e.,

Unknown-Unknown). Changes in those parameters that are not part of the estimating methodology

Analysis Methodology:1. Identify Areas of Uncertainty2. Address the Uncertainty within the Cost Estimate3. Quantify the Risk within the Cost Estimate4. Presentation of Results

2

Cost Risk and Uncertainty Analysis

Page 3: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Technical Programmatic Cost Schedule

Physical Properties Material Availability Sensitivity to Assumptions

Degree of Concurrency

Material properties Skill Requirement Sensitivity to Technical Risk

Sensitivity to Technical Risk

Software Complexity

Environmental Impact

Sensitivity to Programmatic Risk

Sensitivity to Programmatic Risk

Integration Interface Contractors Stability Sensitivity to Schedule Risk

Sensitivity to Schedule Risk

Requirement Changes

Funding Profile Estimating Errors Number of Critical Path

Operational Environment

Political Advocacy Estimating Errors

Reasons for RiskReasons for RiskReasons for RiskReasons for Risk

3

Cost Risk and Uncertainty Analysis

Page 4: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Risk ConceptsRisk ConceptsRisk ConceptsRisk Concepts

Severity of Consequence

Pro

babi

lity

of O

ccur

renc

e High RiskLow Risk

Low RiskIncreasing

Moderate Risk

4

Cost Risk and Uncertainty Analysis

Page 5: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Risk Analysis MethodsRisk Analysis MethodsRisk Analysis MethodsRisk Analysis Methods

Qualitative Methods (e.g., subjective assessments of Low, Medium, or High Risk) are of most use when there is little or no historical data available or firm requirements have not yet been established

Quantitative Methods are considered where probability distribution on cost elements or drivers can be estimated from historical data or deduced from expert opinion

Analytical Method involves the mathematical determination of a total cost distribution from its components cost distributions

Simulation Method involves the computer generation of random costs from component distributions and aggregation into a total cost distribution

5

Cost Risk and Uncertainty Analysis

Page 6: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

AS OF: 04/20/23 03:556

Cost Risk and Uncertainty Analysis

Page 7: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

AS OF: 04/20/23 03:55

2. Address the Uncertainty within the Cost Estimate:

Need to determine which identified uncertainties to model (i.e., which are cost-sensitive) Cost methodology highly dependant, reliant upon data available Application of uncertainty dependant upon cost methodology

Uncertainty assessment is best done at the lowest-level variables possibleImportant to identify uncertainty associated with cost drivers at a minimum

Selection of appropriate probability distribution for each risk element is critical

7

Figure 1: Common Risk Distributions

Cost Risk and Uncertainty Analysis

Page 8: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

AS OF: 04/20/23 03:55

3. Quantify the Risk within the Cost Estimate:

ODASA-CE utilizes the Automated Cost Estimating Integrated Tools (ACEIT)Integrated within ACEIT is a simulation-based risk analysis capability (RI$K) that allows

the analyst to perform cost, schedule, and technical risk and uncertainty analysisProvides the capability to calculate risk results for various confidence levels using Latin-

Hypercube sampling Input: Specification of risk within ACEIT

Output: ACEIT risk statistics reportDisplays statistical results in Base Year (BY) dollars for each WBS element at the

specified confidence level

8

Figure 2: Specifying a Risk Distribution within ACEIT

Figure 3: Example Statistics Report within ACEIT

Cost Risk and Uncertainty Analysis

Page 9: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

AS OF: 04/20/23 03:55

4. Presentation of Results:

ACEIT can also be used to formulate a Cumulative Density Function (CDF) or S-CurveEach point on the S-Curve identifies the cumulative probability that the associated cost on

the x-axis will not be exceeded. This is referred to as the level of confidence in a particular estimate.

AoA cost estimates are presented as a point estimate at the 50% confidence level accompanied by a risk-informed cost range (±σ)

9

Figure 4: Example S-Curve

0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

3.0%

$40.0 $60.0 $80.0 $100.0 $120.0 $140.0 $160.0 $180.0 $200.0

Prob

abili

ty D

ensi

tyDollars

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

$40.0 $60.0 $80.0 $100.0 $120.0 $140.0 $160.0 $180.0 $200.0

Cum

ulati

ve P

roba

bilit

y

Dollars

Point Estimate 50% Confidence Level 80% Confidence Level

Risk Dollars to 80% “Confidence Level” = $127.4 - $90.0 = $37.4)

Probability Density Function (pdf)

Cumulative Distribution Function (cdf)

“S-Curve”

*Cumulative Probability is often referred to as “Confidence Level”**P.E. = Point Estimate

Allocating Risk Dollars: If the decision is made to add Risk Dollars to an estimate, then those dollars may be allocated across lower level elements if it makes sense to report them as such. Risk Dollars can be allocated to lower level elements in a variety of ways. Confidence for lower level elements often will be different than for higher levels. Risk Dollars must add up to the total Risk Dollars needed to reach the desired Confidence at the higher level .

Phasing Risk Dollars: After Risk Dollars are allocated according to the desired approach, it may be necessary to phase the dollars over time. Risk Dollars can be spread over time using a variety of techniques, including pro-rating dollars according to the profile of the point estimate, defining a curve to match a particular profile, front-loading dollars to early years, back-loading dollars to later years, or manually inputting dollars to hedge against risks in specific years.

Cost Risk and Uncertainty Analysis

Page 10: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Risk Analysis ApproachesRisk Analysis ApproachesRisk Analysis ApproachesRisk Analysis Approaches

Detail

Effo

rt

• Detailed Network & Stochastic

• Detailed Monte Carlo Simulation (each WBS)

• Bottom Line Monte Carlo Simulation

• Add a Risk Factor/Percentage

• Discrete Technical, Schedule, and Estimating Risks

10

Cost Risk and Uncertainty Analysis

Page 11: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Probability DistributionsProbability DistributionsProbability DistributionsProbability Distributions

WBS Cost Distribution Total Cost

P.E.

P.E.

P.E.

+

+

=

Sum PE MLC (Mode)

PDF

Fre

quen

cy

CDF

Con

fide

nce

50%

20%

Sum PE MLC

11

Cost Risk and Uncertainty Analysis

Many More+

Page 12: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Probability Density FunctionProbability Density FunctionProbability Density FunctionProbability Density Function

Cell B28 Frequency Chart

Mean = $2,198

4,978 Trials Shown

.000

.006

.012

.018

.024

0

29.2

58.5

87.7

117

$1,250 $1,750 $2,250 $2,750 $3,250

Forecast: Results=

12

Cost Risk and Uncertainty Analysis

Page 13: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Cumulative Distribution FunctionsCumulative Distribution FunctionsCumulative Distribution FunctionsCumulative Distribution Functions

Cell B28 Cumulative Chart 4,977 Trials Shown

.000

.249

.498

.747

.995

0

4977

$1,250 $1,750 $2,250 $2,750 $3,250

Forecast: Results=

13

Cost Risk and Uncertainty Analysis

Page 14: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

AS OF: 04/20/23 03:55

Cost Schedule Curve Cost Schedule Curve Cost Schedule Curve Cost Schedule Curve

Development Schedule

Lif

e C

ycle

Cos

t

Min Optimal

Typ

ical

Parallel EffortMore ECPLess Mature Design

Fixed CostTechnology Outdate

14

Cost Risk and Uncertainty Analysis

Page 15: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Schedule GoalsSchedule GoalsSchedule GoalsSchedule Goals

Defense

Development Schedule

Cos

t

Commercial Drivers•Technology Drives Schedule•Constraint Schedule•Goal is ROI Maximization

DoD Drivers•Funding Drives Schedule•Unconstrained Schedule•Goal is Cost Reduction

Commercial

15

Cost Risk and Uncertainty Analysis

Page 16: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

AS OF: 04/20/23 03:55

ConceptConceptRefinementRefinement

System Development System Development & Demonstration& Demonstration

Production & DeploymentProduction & Deployment Operations & SupportOperations & SupportTechnology Technology DevelopmentDevelopment

Concept Decision

Design Readiness Review

FRP Decision Review

A B C FOCIOC

MDAP Process

Rang

e of

Unc

erta

inty

Estimating Accuracy Trumpet

Technical and Program Maturity

RO

M -

30%

to

+75

%

Par

amet

ric

-10%

to

+20

%

Ana

logy

-15

% t

o +

30%

Eng

inee

ring

-5%

to

+15

%

Act

ual

-3%

to +

10%

Most Likely Cost

Target Cost

Point estimates are the most likely within the range estimate

Point estimates are the most likely within the range estimate

Upper bound

Lower bound

+75%

-30%

Range estimates provide risk and uncertainty as well as most likely(more information)

Range estimates provide risk and uncertainty as well as most likely(more information)

16

Cost Risk and Uncertainty Analysis

Range Cost EstimatesRange Cost EstimatesRange Cost EstimatesRange Cost Estimates

Page 17: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

COA-1 highest benefit COA-2 best value COA-3 lowest cost

Cost = $ quantifiable cost – $ quantifiable benefit or saving

Benefit = $ non-quantifiable benefit and $ non-quantifiable risk

CBA provide framework for making resource informed decisions

CBA StepsCBA Steps

Cost Risk and Uncertainty Analysis

17

Page 18: Mort Anvari Mort.Anvari@us.army.mil

AMERICA’S ARMY:THE STRENGTH OF THE NATION

Risk analysis reduces the uncertainty between requirements and funding

Cost Risk and Uncertainty Analysis

Risk ReductionRisk Reduction Cost Risk Analysis will not reduce the risk inherent in a program; however

it helps PM to understand the nature of the risk involved, and the uncertainty associated with the cost estimates

More frequent Cost Risk Analysis result in a more realistic assessment of the cost and schedule and the development of effective risk mitigation plan