International Atomic Energy Agency Definition and overview of required safety documentation (e.g., safety case and safety assessment) Phil Metcalf Workshop

International Atomic Energy Agency

Definition and overview of required safety documentation (e.g., safety case

and safety assessment)

Phil MetcalfWorkshop on Strategy and Methodologies for the Development

of Near Surface Disposal Facilities

April 7-11, 2014, Amman Jordan


OVERVIEWOVERVIEW

• Safety Assessment• Assessment context

• Development and justification of scenarios

• Models

• Data

• Uncertainties

• Analysis of results

• Use of safety case and safety assessment results

• Documentation of safety case and safety assessment

2



System DescriptionSite and waste characteristics, Safety Functions, Design Options

Safety Case Context• Safety objectives• Safety principles • Regulations

Safety Strategy• Isolation, Containment•Passive systems, robustness•Defence in depth, demonstrability

Integration of Safety ArgumentsDemonstration of robustness, defence in depth

system understanding, monitoring, etc

Non-radiological

Environ

men

tal Impact

Ope

rationalS

afety

Site

/ Eng

ineerin

g

Safety Assessment

Stakeholder &

Regulatory Involvem

ent

Limits, controls and conditions

Man

ag

em

ent

Syste

m

Managem

en

t of U

nce

rtain

ty

Itera

tion

& D

esig

n O

ptim

izatio

n M

anagem

en

t Syste

m

Post Closure Radiological Impact

Scenarios

Models

Calculations

4CNSC Workshop 2010 - Canada


SpecifyAssessment Context

Disposal SystemDescription

Develop and Justify Scenarios

Formulate and Implement Models for

Consequence Analysis

Interpretationof Results

AcceptanceMake Decision

Assess Further Information

Needs

Effective to Improve Assessment

Components

RejectionMake Decision

PerformConsequence Analysis

Compare AgainstAssessment Criteria

Yes

Yes

No

No

Review andModify

AdequateSafety Case?


SPECIFICATION OF ASSESSMENT CONTEXTSPECIFICATION OF ASSESSMENT CONTEXT

• Assessment context sets the scope and content of the safety assessment

• Assessment context provides clear statement as to: • what is being assessed

• why it is being assessed

• what decisions are to be made

• Strongly influenced by: • stage of development of the facility

• regulatory and stakeholder requirements

6


SPECIFICATION OF ASSESSMENT CONTEXTSPECIFICATION OF ASSESSMENT CONTEXT

• Provides information concerning:

• purpose of the assessment

• regulatory framework

• assessment end points

• assessment philosophy

• waste characteristics

• disposal system characteristics

• assessment timeframes

7


DESCRIPTION OF DISPOSAL SYSTEMDESCRIPTION OF DISPOSAL SYSTEM

• Includes information relevant to assessment context on:• near field - e.g. waste types, waste forms, waste

inventories, disposal practices, engineered barriers (chemical and physical characteristics), facility dimensions

• geosphere - e.g. lithologies, flow and transport characteristics

• biosphere - e.g. exposure pathways, climate characteristics, human habits and behaviour

8


DEVELOPMENT AND JUSTIFICATION OF DEVELOPMENT AND JUSTIFICATION OF SCENARIOSSCENARIOS

• Safety assessment needs to consider the performance of the disposal facility under current and future conditions

• Scenarios describe alternative evolutions of the disposal system based on information from the assessment context and system description

• What-if scenarios may be used to assess robustness and defence in depth

• A range of safety indicators can be considered (e.g. dose, risk, concentrations, fluxes)

9


DEVELOPMENT AND JUSTIFICATION OF DEVELOPMENT AND JUSTIFICATION OF SCENARIOSSCENARIOS

• Several scenario generation techniques (expert judgment, fault tree analysis, generic scenarios)

• No one technique is the best; the technique used should be fit for purpose

• Any technique should ensure appropriate level of transparency and comprehensiveness

• Common features:• initial construction/audit of list of features, events and

processes (FEPs) influencing disposal system and migration/fate of radionuclides within it

• scenarios screened in light of assessment context and system description

10


NEA INTERNATIONAL FEP LISTNEA INTERNATIONAL FEP LIST


CONCEPTUAL MODELS DEVELOPMENTCONCEPTUAL MODELS DEVELOPMENT- BACKGROUND -- BACKGROUND -

• Once scenarios have been developed, need to analyze their consequences

• Do this by developing conceptual model(s) of:• disposal system

• associated release, transport and exposure mechanisms and media

• Use information from the assessment context, system description and scenario development steps of the assessment approach

12


CONCEPTUAL MODELS DEVELOPMENT CONCEPTUAL MODELS DEVELOPMENT - BACKGROUND -- BACKGROUND -

• Conceptual models:• describe model’s basic FEPs

• consider relationship between FEPs

• consider model’s application in spatial and temporal terms

• Mathematical models and computer tools:• algebraic and differential equations with empirical and/or

physical basis

• solved using computer tools using analytic and/or numerical techniques

13


CONCEPTUAL MODELS DEVELOPMENTCONCEPTUAL MODELS DEVELOPMENT- BACKGROUND -- BACKGROUND -

• Model data:

• disposal system parameters (e.g. facility dimensions, flow path lengths)

• human exposure parameters (e.g. food produce consumption rates, occupancy rates)

• radionuclide/element dependent parameters (e.g. sorption coefficients, transfer factors, dose coefficients)

14


CONCEPTUAL MODEL DEVELOPMENT (1) CONCEPTUAL MODEL DEVELOPMENT (1)

• Need to be aware of conceptual model uncertainties:• data are sparse

• models are incomplete

• projections are into the unknown future

• Need to consider a range of credible models

• Models need to be fit for purpose

15


CONCEPTUAL MODEL DEVELOPMENT (2)CONCEPTUAL MODEL DEVELOPMENT (2)

• Interaction Matrix:• Divide system into

constituent components

• Main components go into leading diagonal elements of matrix

• Interactions are noted in off diagonal elements

Workshop Islamabad 2014

16

A AC

B

CB C

DA D

BE

E



Example Interaction Matrix



• Influence Diagram• Define system

barriers

• Select relevant FEPs

• Represent on diagram

• Identify influences between FEPs

• Document FEPs and influences

18


MATHEMATICAL MODELS DEVELOPMENTMATHEMATICAL MODELS DEVELOPMENT- BACKGROUND -- BACKGROUND -

• Mathematical models are required for two primary purposes:

• to describe evolution of disposal system (e.g. chemical evolution in near-field, impact of climate change on disposal system)

• to describe transfer of radionuclides through the evolving disposal system

19


MATHEMATICAL MODELS DEVELOPMENTMATHEMATICAL MODELS DEVELOPMENT- BACKGROUND -- BACKGROUND -

• Translate assumptions of conceptual models into sets of coupled algebraic, differential and/or integral equations with appropriate initial and boundary conditions in a specified domain

• Equations are solved to give the temporal and spatial dependence of the quantities of interest (e.g. radionuclide concentrations and doses to humans)

20


MATHEMATICAL MODELS DEVELOPMENTMATHEMATICAL MODELS DEVELOPMENT- COMPLEXITY -- COMPLEXITY -

• Particular mathematical representation of a conceptual model depends on assessment context and on process-level understanding of the ways in which FEPs can be represented

• As the understanding of the system is developed, more detailed models may be needed to adequately represent the system

• BUT models should be simple enough to be compatible with available data

21


COMPLEXITY OF MODELS (1) COMPLEXITY OF MODELS (1)

Factors affecting model complexity

22

MODEL

Nature of the problem

Purpose of theAssessment

Data availabilityUncertainties

Disposal regulations(end points)


COMPLEXITY OF MODELS (2)COMPLEXITY OF MODELS (2)

• Some simplification is generally required to translate the concepts of conceptual model into mathematical terms

• Can take several forms:

• simplification of geometry or structure (e.g. considering transport in 1D and homogeneous and isotropic medium)

• omission/simplification of processes (e.g. neglecting kinetic terms in chemical reactions)

23


ANALYTICAL METHODSANALYTICAL METHODS

• Can provide exact solutions to the flow and transport equations

• Useful for:

• screening level assessments when site data are sparse and uncertain

• sensitivity and uncertainty analysis

• verifying more complex models

• BUT only developed for simple cases with homogeneous spatial domain, steady flow and 1-D advection/dispersion

24


NUMERICAL METHODSNUMERICAL METHODS

• Discretize the spatial domain into cells and the resulting set of algebraic equations is solved by iteration, matrix methods or a combination of the two

• Advantages include:

• Easy handling of spatial and temporal variability and complex geometry and boundary conditions

• 2-D and 3-D transport problems can be solved

• BUT resource (time, money and data) intensive

25


COMPUTER CODES (1)COMPUTER CODES (1)

• Solution of mathematical models is usually achieved by implementing one or more computer codes

• Need to consider software design of code(s) - should be conducted within an appropriate software quality assurance system

26


COMPUTER CODES (2)COMPUTER CODES (2)

• Codes may be:

• proprietary codes - advantage of being previously developed and checked, history of application to a range of cases, but not necessarily appropriate to the problem

• modified codes - need to be developed and checked, however tailored to the needs of the specific problem

• specifically developed for implementation of the chosen mathematical models – same advantages/disadvantages as modified codes

27


SELECTION OF COMPUTER CODES (1)SELECTION OF COMPUTER CODES (1)

• Need to ensure that the code(s) used are fit for purpose

• Factors to consider:

• Assessment context (scoping vs detailed calculations)

• Resource availability (time, money and data)

• Nature of the processes to be modelled (e.g. fractured vs porous medium)

• Relative importance of the processes

28


SELECTION OF COMPUTER CODES (2) SELECTION OF COMPUTER CODES (2)

• Ideally the codes should be chosen to be consistent with the conceptual and mathematical models, and not vice versa

• However, for practical reasons, the conceptual and mathematical models are often developed with the code already selected

• If this happens then it is important to document the constraints that the pre-selection of the code places on the conceptual and mathematical models

29


VERIFICATIONVERIFICATION

• ‘Test problems’

• show that equations are solved satisfactorily in the computer codes

• Verify calculation methods

• Can be feasible and should be used for confidence building

• BUT a model is only verified for the specific problems which have been considered

30


CALIBRATIONCALIBRATION

• Compare model estimates with site specific field observations. Can include modifying:

• input data and boundary conditions

• conceptual and mathematical models

• Limitations:

• A model can only be calibrated for the same temporal and spatial scales as the field measurements

• Different conceptual models may show similar agreement with observed data

31


VALIDATIONVALIDATION

• Production of credible results under a range of conditions

• Validation over limited timescales is achievable

• Not possible for long term evolution of a specific site, since there are insufficient data

• Natural analogues and other assessments/ experience may be useful

32


DATA COLLECTION (1) DATA COLLECTION (1)

• Many kinds of data needed during the modelling process:• Waste inventory and form

• Design data

• Site data (geosphere, biosphere)

• Flow and transport data

• Data are often unavailable:• Many kinds of data cannot be collected or are very costly

• Data relate to small space and time scales

• Information about the future unavailable

33


DATA COLLECTION (2)DATA COLLECTION (2)

• Collection of all data is impossible and any attempt would lead to an enormous expenditure of resources

• Recognition that safety assessment has unique characteristics:

• Goal of safety assessment is to make decisions, not to predict the future

• The decision is about reasonable assurance of safety

• Predictive capability is unnecessary/impossible

34


SOURCES OF UNCERTAINTIESSOURCES OF UNCERTAINTIES

35


DATA/PARAMETER UNCERTAINTYDATA/PARAMETER UNCERTAINTY

• Uncertainty indata and parameters used as inputs in modelling

• Lack of specific data

• Natural variability (spatial and/or temporal) variability in some parameters

36



MODEL UNCERTAINTYMODEL UNCERTAINTY

• Uncertainty in conceptual, mathematical and computer modelsused to simulate the disposal systembehaviour andevolution

37



FUTURE UNCERTAINTYFUTURE UNCERTAINTYUncertainty in evolution of disposal system over timescales of interest

1.00E-12

1.00E-11

1.00E-10

1.00E-09

1.00E-08

1.00E-07

1.00E-06

1.00E-05

1.00E-04

1.00E-03

1.00E-02

1.00E-01

1.00E+00

1.00E+00 1.00E+01 1.00E+02 1.00E+03 1.00E+04 1.00E+05 1.00E+06 1.00E+07 1.00E+08

Time after closure (Years)

An

nal

In

div

idu

al E

ffec

tive

Do

se (

Sv

y-1)

Normal - Liquid

Normal - Gas

Normal - Solid

Variant A - Intruder

Variant A - Site Dweller

Variant B - Bathtubbing

Variant C - Rapid Transport

38Workshop Islamabad

2014


SUBJECTIVE UNCERTAINTYSUBJECTIVE UNCERTAINTY

• Uncertainty due to reliance on expert judgment

39



40

APPROACHES FOR UNCERTAINTY APPROACHES FOR UNCERTAINTY MANAGEMENTMANAGEMENT

• Awareness – be aware of all major locations of uncertainty

• Importance – determine relative importance of various sources of uncertainty using sensitivity analysis

• Reduction – reduce uncertainties, e.g. through further data collation

• Quantification – quantify effects of uncertainties on model output using sensitivity analysis



MANAGEMENT OF SCENARIO AND MODEL MANAGEMENT OF SCENARIO AND MODEL UNCERTAINTY UNCERTAINTY

• Scenario analysis: describes alternative futures and allows for a mixture of quantitative analysis and qualitative judgements

• Conceptual model: consider alternative conceptual models and collect further data

• Mathematical/computer model: use model verification, calibration and validation, and range of models

41


MANAGEMENT OF MODEL UNCERTAINTYMANAGEMENT OF MODEL UNCERTAINTY

• Example Results from BIOMOVS II

42



MANAGEMENT OF DATA/PARAMETER MANAGEMENT OF DATA/PARAMETER UNCERTAINTYUNCERTAINTY

• Approaches can be used:

• conservative/worse case approach

• best estimate and what if

• sensitivity analysis

• probabilistic

• Can also be used to address future and model uncertainties

43



PROPAGATION OF UNCERTAINTIESPROPAGATION OF UNCERTAINTIES

44

00.5

11.5

22.5

33.5

44.5

0.05 0.1625 0.275 0.3875 0.5

00.020.040.060.08

0.10.120.140.16

85 90 95 100 105 110

0

1

2

3

4

5

6

7

0 125 250 375 500

Input Parameter

Endpoint = F(Input, Parameter)

Endpoint



CONSERVATIVE / WORST CASE APPROACHCONSERVATIVE / WORST CASE APPROACH

• Use pessimistic parameter values to overestimate impact

• Danger of being so pessimistic as to be worthless and misleading

• Difficult to define the worst value, and prove that this is the worst one.

• Not always obvious what is conservative for a particular combination of parameters, exposure pathways and radionuclides

45



MODELS – SUMMARY AND CONCLUSION MODELS – SUMMARY AND CONCLUSION

• Need to make process of formulating and developing models formal, defensible, and transparent to independent review

• Process consists of:• generation of conceptual models using information

from the assessment context, system description and scenario generation

• representation of conceptual models and associated processes in mathematical models

• implementation and solution of mathematical models in computer codes


46


MODELS – SUMMARY AND CONCLUSIONMODELS – SUMMARY AND CONCLUSION

• Models should be:

• as simple and easy to use as possible whilst including enough detail to represent the system’s behaviour adequately for the purpose of the assessment

• consistent with assessment context

• consistent with data availability

• A simple modelling approach is likely to be more efficient, easily understandable and justified


47


ANALYSIS OF RESULTSANALYSIS OF RESULTS

• Compare results with criteria defined in assessment context

• Analyze the extent and implications of uncertainties

• Confidence can be built using various approaches (e.g. transparent, logical and well documented reasoning, multiple lines of reasoning, treatment of uncertainties)


48


ITERATIONITERATION

• The safety assessment process is iterative, although iteration need only proceed until the assessment is judged fit for purpose

• Iteration promotes:

• consideration of improvements to and optimisation of the disposal system

• confidence in the understanding of the main safety related parameters and the robustness of the disposal system under the assumed scenarios

• collection of relevant data


49


DOCUMENTING THE SAFETY CASEDOCUMENTING THE SAFETY CASE

• Executive summary• Introduction and the safety case context• Safety strategy• Safety assessment• Synthesis and conclusions• Follow-up programmes and actions• Public involvement• Requirements on the documentation of

safety assessment• Traceability


50


USE OF THE SAFETY CASEUSE OF THE SAFETY CASE

• Comparison of Options

• Prioritization of Site Characterization and R&D

• Facility Design and Operation

• Licensing

• Derivation of Limits and Conditions

• Monitoring

• Staff Training


51


REGULATORY REVIEW PROCESSREGULATORY REVIEW PROCESS

• Objectives and attributes of the regulatory review process

• Managing the review process

• Conducting the review and reporting review findings


52


Documents

International Atomic Energy Agency Definition and overview of required safety documentation (e.g., safety case and safety assessment) Phil Metcalf Workshop