Upload
corey-wade
View
219
Download
0
Embed Size (px)
DESCRIPTION
Results You Can Trust 3 Interactive Exercise
Citation preview
Futron Corporation • 2021 Cunningham Drive, Suite 303 • Hampton, Virginia 23666Phone 757-262-2074 • Fax 757-262-2173 • www.futron.com
Results You Can Trust
AssessingAssessingUncertaintyUncertainty
HRA – INCOSE November 8 & 9 2007
• 2 Results You Can Trust
OverviewOverview• Interactive Exercise• Definitions• Why Assess Uncertainty• Interactive Exercise• Types of Uncertainty• Sources of Uncertainty• Assessing (Bounding) Uncertainty• Interactive Exercise• Summary
• 3 Results You Can Trust
Interactive ExerciseInteractive Exercise
• 4 Results You Can Trust
Definitions: UncertaintyDefinitions: Uncertainty
• Uncertainty: may range from a falling short of certainty to an almost complete lack of conviction or knowledge especially about an outcome or result. Lack of clear definition
• 5 Results You Can Trust
Definitions: RiskDefinitions: Risk
• Risk: possibility (chance) of loss or injury (Likelihood, Consequence. Ordered Pair )
• Probability: (1) : the ratio of the number of outcomes in an exhaustive set of equally likely outcomes that produce a given event to the total number of possible outcomes (2) : the chance that a given event will occur (3) A numerical measure of the likelihood of an event relative to a set of alternative events.
• Likelihood: probability
Consequence
54321
1
2
3
4
5
Likelihood
Consequence
54321
1
2
3
4
5
Likelihood
Generic Risk Matrix
• 6 Results You Can Trust
Definitions: Confidence IntervalDefinitions: Confidence Interval
• Confidence Interval: a group of continuous or discrete adjacent values that is used to estimate a statistical parameter (as a mean or variance) and that tends to include the true value of the parameter a predetermined proportion of the time if the process of finding the group of values is repeated a number of times
• Used to quantify/communicate amount of certainty
• 90% confidence interval (limit/bounds)
mean
• 7 Results You Can Trust
Why Assess Uncertainty?Why Assess Uncertainty?• We manage risks to ensure we meet our objectives
• We have tasks, projects, and programs to accomplish They all have objectives We all make decisions and commit resources to meet these objectives Decision are choices between alternatives and involve uncertainty This uncertainty leads to risk
• Managing uncertainty is a part of our job Involves understanding our decisions Managing the uncertainty in those decisions Identifying the risks to meeting our objectives that come from uncertainty Make better decisions / select better mitigation strategies
• 8 Results You Can Trust
Interactive ExerciseInteractive Exercise
• 9 Results You Can Trust
• Aleatory Randomness or variability that remains when the system is well known Inherent Variability You can calculate the odds Example: Drawing a queen of diamonds from a standard deck of 52
playing cards on the first draw. Not reducible
Types of Uncertainty: AleatoryTypes of Uncertainty: Aleatory
• 10 Results You Can Trust
• Epistemic Stems from limitations of fundamental
knowledge about a phenomenon of interest. Model Uncertainty Can be revealed by divergences in the opinions
of experts who may disagree. Example: Drawing a queen of diamonds from a
deck of playing cards on the first draw when you have no idea how many cards are in the deck
This type of uncertainty is reducible:• Improvements of current knowledge• Improvements of corresponding models• Collection of additional data (test results)
Types of Uncertainty: EpistemicTypes of Uncertainty: Epistemic
• 11 Results You Can Trust
Sources of UncertaintySources of Uncertainty
• Insufficient data Measurement errors State not directly observed (inferred) Imperfect understanding of processes Not enough data to be statistically significant
• Natural variability Varies spatially (with space) Varies temporally (with time)
• Model imperfections
• Underestimating uncertainty and the illusion of control - We tend to underestimate future uncertainty because we tend to believe we have more control over events than we really do. We believe we have control to minimize potential problems in our decisions.
• 12 Results You Can Trust
Six levels of “sophistication”• Simply establish the existence of uncertainty
(risk)• Identifying a “worst-case” scenario
(unbiased bounds – one tailed or two tailed)• Computing the effects of the quasi-worst
case scenario or plausible upper bound• Computing a maximum-likelihood estimate
of the failure probability of losses• Modeling the probability distribution of
potential outcomes (risk curve) • Modeling explicitly the uncertainties about
the risk, separating epistemic from aleatory
Bounding UncertaintyBounding UncertaintyUniform(-0.17000, 0.20000)
X <= 0.1815095.0%
X <= -0.151505.0%
0
0.5
1
1.5
2
2.5
3
-0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0.25
Triang(-.17, 0, .2)X <= 0.13917
95.0%X <= -0.11392
5.0%
0
1
2
3
4
5
6
-0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0.25
Normal(0, .113) Trunc(-.4,+inf) Shift=+.015X <= 0.20088
95.0%X <= -0.17066
5.0%
0
0.5
1
1.5
2
2.5
3
3.5
4
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4
• 13 Results You Can Trust
Interactive ExerciseInteractive Exercise
• 14 Results You Can Trust
SummarySummary
• It is not easy• We think we know more than we really do• Understand, assess and communicate your own uncertainty• Understand how others assess and communicate their
uncertainty• Focus Mitigation Strategies
• 15 Results You Can Trust
BibliographyBibliography
• Ang, Alfred H-S. and Tang, Wilson H. “Probability Concepts in Engineering Planning and Design, Volume I – Basic Principles”
• Ang, Alfred H-S. and Tang, Wilson H. “Probability Concepts in Engineering Planning and Design, Volume II – Decision, Risk and Reliability.” John Wiley, New York 1984
• Roberds, William J. (1990). “Methods for Developing Defensible Subjective Probability Assessments,” Transportation Research Record No. 1288, 183-190.
• CE311S Elementary Probability and Statistics for Civil Engineers class notes, University of Texas – Austin
• Futron Technical University (FTU) Fundamentals of Risk Management course notes
• Decision Analysis Training for ESMD Risk Management Officers, by Dr. Douglas Stanley, Georgia Institute of Technology, National Institute of Aerospace
• The Epistimic Uncertainty Project, Sandia National Laboratories http://www.sandia.gov/epistemic/ • Wikipedia (interactive exercises) http://en.wikipedia.org/wiki/Main_Page• Encyclopedia Britannica Online http://www.britannica.com/