RPIC Edmonton 2015

David Wilson, Stantec Consulting Ltd.

June 4, 2015

Advancements in Managing Uncertainty in Remedial Options Analysis and Remedial Action Plan Development for Northern Sites

Agenda

1 Uncertainty in ROA and RAP Development

2 Project Impacts due to Uncertainty

3 Identifying and Characterizing Uncertainty

4 Uncertainty and Risk: Mitigation and Management

5 Case Examples

Uncertainty in ROA and RAP Development

“We demand rigidly defined areas of doubt and uncertainty!” - Douglas Adams, The Hitchhiker's Guide to the Galaxy

1

Definitions

Risk •  Treasury Board (2010b): “the effect of uncertainty

on objectives” or “the expression of likelihood and impact of an event”

Uncertainty •  Treasury Board (2010a): “the state, even partial, of

deficiency of information related to understanding or knowledge of an event, its consequence, or likelihood”

•  Environment Canada (2012): the “state of having limited knowledge where it is impossible to exactly describe an existing state or future outcome”

Types of Uncertainty

Aleatory or Exogenous Uncertainty •  statistical variability and

heterogeneity of the system (e.g., standard deviation of sample results)

Epistemic Uncertainty •  model and parameter

uncertainty (e.g., infiltration rate)

Deep Uncertainty (CDMU, 2013) •  uncertainty about

fundamental processes or assumptions

Often forgotten: also includes scenario and decision-rule uncertainty

ROA and RAP Development •  Federal 10-step process – Step 7: Develop Remedial/

Risk Management Strategy •  Consider project approach (remediate or risk

manage; generic or site-specific) •  Determine remedial/risk management objectives •  Then consider remedial/risk management options

Sources of Uncertainty SOURCE e.g.s: •  Background conditions

poorly defined •  CSM not fully developed •  Some COCs not identified •  Impacts not delineated •  Decision criteria not

defined

Uncertainty is generally identified within a phase but is often not carried to a subsequent phase (e.g., from ESA to ROA/RAP; from RA to ROA/RAP; from ROA to RAP; from ROA/RAP to Cost)

NORTHERN SITE ‘MAGNIFIERS’ •  Limited time/samples •  Limited site-specific physical

system data •  Limited historical data •  Risk assessment exposure

scenarios not ‘typical’ •  Identifying/engaging

stakeholders challenging

Project Impacts Due to Uncertainty

"No matter how good the team or how efficient the methodology, if we’re not solving the right problem, the project fails“ - Woody Williams

2

Impacts of Uncertainty

Uncertainty in CSM •  Uncertainty in remedy performance/project risk

of remedy failure •  Errors in identifying appropriate technologies for

actual site conditions and fate and transport mechanisms

Uncertainty in Project Objectives •  Inappropriate clean-up goals/drivers Northern/Remote Site Challenges •  Use/evaluation of emerging technologies difficult •  Impact of short field season and seasonal demob

After ITRC, 2011

Impacts of Uncertainty

Quality •  DQOs not met

•  Range of impact (COCs, volumes, risk y/n)

Schedule •  Range of task durations

•  Ill-defined go/no go criteria

Cost •  Range of task costs

Quality

Schedule Cost

Quality

Schedule Cost

Quantifying Project Impacts

Quality, e.g. •  Significant risks not

identified •  Remediation over-scoped Schedule, e.g. •  Assessment extended/

revisited •  Remediation extended Cost, e.g. •  Exceed estimates •  Not optimized Every uncertainty will impact all three project dimensions – they are inextricably interlinked

Projects reside here

Identifying and Characterizing Uncertainty

“Madness is the result not of uncertainty but of certainty” - Friedrich Nietzsche

3

Identifying Uncertainty

Phase I/II/III ESAs •  Site history known and documented?

•  How mature is the CSM?

•  Contaminant concentrations compared to background?

RAs •  Model and parameter uncertainty (e.g., contaminant

fate + transport; exposure factors) defined?

•  COCs and pathways supported by CSM?

Remediation/Risk Management Objectives •  Future use decided? R/RM scenario and decision-rule

uncertainty defined?

Characterizing Uncertainty

Aleatory or Exogenous Uncertainty •  Measures of statistical variability:

o  standard deviation (parametric and non-parametric)/ standard error; variogram

•  Bayesian probabilities

Epistemic Uncertainty •  Model sensitivity analysis

•  Monte Carlo simulation

Deep Uncertainty •  Expert judgment

•  Pairwise comparison of significance

Quantifying Potential Impacts Source Soil impacts not fully delineated Background not characterized COCs not identified Model uncertainty Decision-rule uncertainty

Characterization # samples/m2 or m3

# background samples by media Predict likelihood – compare to analogy sites Range of results from family of models Objectives for site defined?

Quantification Impacted soil volume range: x m3 ± y m3

Parameter inclusion as COC: Y/N/Unk. Parameter inclusion as COC: Y/N/Unk. Level of confidence in model result Identify reliance of decisions on site objectives

Uncertainty and Risk: Mitigation and Management

“Fail to plan, plan to fail”

4

Project Risk Management

ITRC, 2011

The Project Risk Register

What is it? Tool for estimating the impact of project risk on project outcomes How should it be developed? List identified and characterized hazards (including uncertainties), determine likelihood of occurrence and consequence (i.e., risk score) How can it be used? Identify, implement and monitor risk mitigation and management measures

Methods: Mitigating Uncertainty

Exogenous Uncertainty •  Variability is due to spatial, temporal, or individual randomness and

cannot be decreased by further data collection: it’s impact can only be (and should be) managed

Epistemic and Deep Uncertainties •  Rank first by risk significance (e.g., hazardous vs. non-hazardous,

COC HQs), then by magnitude •  Work down ranking, and answer the questions:

•  Does the uncertainty span a decision threshold? [e.g., remediate or risk manage; on-site or off-site disposal]

•  Worth investing in reduction of uncertainty (mitigate), or better to manage? [Cost-benefit analysis]

Decision Threshold Examples

Ex1 - Risk Assessment Scenario: HHRA HQ for primary

COC, based on UCLM, yields a value of 0.3, exceeding threshold of 0.2. ‘Expected’ HQ is <0.2

Decision: to remediate or risk manage

Options: 1) accept conservative case and remediate; 2) invest in additional analysis to reduce epistemic uncertainty

Ex2 - CSM Scenario: discontinuous

permafrost site, groundwater shows metals impacts, one season of sampling, nearest surface water body 25 m

Decision: to include a gw-sw pathway

Options: 1) accept conservative case of a pathway; 2) invest in additional field work to reduce uncertainty

Cost-Benefit Example

Situation •  Phase III ESAs have been completed on several abandoned

mine sites grouped into a project. Significant areas of PHC and metals-contaminated soils have been delineated to varying degrees of certainty – there is high confidence that all COCs have been identified, but low confidence in quantity estimates. Is investment in additional delineation warranted?

Analysis Scenarios •  Scenario 1: additional season of assessment, followed by

remediation

•  Scenario 2: remediation

Analysis Method •  Comparison of expected costs including uncertainty impact

Cost-Benefit Example cont’d

Scenario 1 •  Cost of additional season

of assessment

•  Cost of remediation, using expected lower uncertainty impacted soil quantity

Scenario 2

•  Cost of remediation, using higher uncertainty impacted soil quantity

Investment not Warranted

Cost ($M): Assessment $0.45 Remediation $25.0

Cost ($M): Remediation $23.5

References and Tools References •  Environment Canada. 2012. Federal Contaminated Sites Action Plan (FCSAP)

Environmental Risk Assessment Guidance. ISBN no. 978-1-100-22282-0. Cat. no. En14-19/1-2013E-PDF.

•  Committee on Decision Making Under Uncertainty (CDMU). 2013. Environmental Decisions in the Face of Uncertainty. Board on Population Health and Public Health Practice. National Academy of Sciences. ISBN 978-0-309-13034-9.

•  Interstate Technology & Regulatory Council (ITRC). 2011. Project Risk Management for Site Remediation. RRM-1. Washington, D.C.: Interstate Technology & Regulatory Council, Remediation Risk Management Team. www.itrcweb.org.

•  TBS. 2010a. Framework for the Management of Risk. •  TBS. 2010b. Guide to Integrated Risk Management. Distribution Fitting •  ProUCL. www.epa.gov/OSP/hstl/tsc/software.htm. •  Visual Sample Plan (VSP). vsp.pnnl.gov. Monte Carlo Simulation Tools •  Oracle Crystal Ball.

www.oracle.com/technetwork/middleware/crystalball/overview/index.html. •  Palisades @Risk. www.palisade.com.

Questions?

David Wilson, M.A.Sc., P.Eng. Senior Associate Stantec Ottawa (613) 738-6091 david.wilson@stantec.com