Copernicus Institute Interfaces between Science & Society, Milano, 27-28 November 2003 Quicksandy Knowledge Bases The need for guidance for dealing with

Copernicus InstituteInterfaces between Science & Society, Milano, 27-28 November 2003

Quicksandy Knowledge Bases

The need for guidance for dealing with uncertainty, assumptions and value

commitments in environmental assessment

Jeroen van der Sluijs (UU Copernicus Institute)

[email protected]

Copernicus Institute Interfaces between Science & Society, Milano 27-11-2003 More info: www.nusap.net

Quicksand

• Quicksand is only dangerous if you misunderstand its nature and treat it as if it is not different from the rest of the beach

• Survival handbook: It is basically just sand mixed with water, which makes it behave different from sand “The worst thing to do is to thrash around and move your arms and legs through the mixture, because you'll only force yourself further into it”

• Quicksandy knowledge bases are basically a mixture of knowledge and ignorance and you need to understand its nature and need some guidance to handle it


Complex environmental problemsTypical characteristics (Funtowicz & Ravetz):• Decisions will need to be made before conclusive scientific

evidence is available;• Decision stakes are high: potential error costs of wrong

decisions can be huge • Values are in dispute • Knowledge base is mixture of knowledge and ignorance:

large (partly irreducible) uncertainties, knowledge gaps, and imperfect understanding;

• Assessment dominated by models, scenarios, and assumptions

• Many (hidden) value loadings in problem frames, indicators, assumptions

Uncertainty management is essential


Omitting uncertainty management can lead to crisis and loss of trust

• Keepin & Wynne, 1984– IIASA global Energy Scenarios are assumption laden,

highly unstable, and based on informal guesswork• De Kwaadsteniet, 1999

– RIVM over-exact prognoses based on virtual reality of computer models

• BSE in UK– if Government’s position on BSE had reflected more

accurately scientific uncertainty, both the public and the authorities would have been better prepared for the shift in scientific consensus which prompted the notorious change of policy (House of Lords, 2000)


Once environmental numbers are thrown over the disciplinary fence, important caveats tend to be ignored, uncertainties compressed and numbers used at face value

e.g. Climate Sensitivity, see Van der Sluijs, Wynne, Shackley, 1998:

1.5-4.5 °C ?!

Crossing the disciplinary boundaries

Resulting misconception:

Worst case = 4.5°C


The uncertainty problemNow widely held that uncertaintyassessment is essentialBut in practice…• Little appreciation for multi dimensional

nature of uncertainty• Lack of understanding of characteristics

and relative importance• State of the Art Methodology not widely

disseminated• Need for guidance and training


RIVM Uncertainty Guidance

Structure uncertainty management in environmental assessment

Promote awareness & critical (self)reflection May not reduce uncertainties, but provides means to

assess their potential consequences and avoid pitfalls associated with ignoring or ignorance of uncertainties

Assist in documenting uncertainty, assumptions, and value-loadings

Guides the use and helps against misuse of uncertainty tools

Promote the adoption of uncertainty methods in common practice at RIVM/MNP

Facilitate design of effective strategies for communicating uncertainty


Project phases:• 25 October 2001 Multidisciplinary Expert

workshop• 22 November 2001 User workshop• Expert review• User review

Development of a portal:Quick Scan & Mini Checklist

• Testing phase 2002• Implementation (2002/2003; MV6)


RIVM/MNP Guidance for Uncertainty Assessment and Communication

All five documents can be downloaded from www.nusap.net

I. Mini-Checklist

III. Tool Catalogue

Uncertainty Assessment

II. Quickscan Questionnaire

III. Detailed Guidance

(incl. Glossary)

II. Quickscan Hints & Actions List


Sections of the checklist

• Problem framing• Involvement of stakeholders• Indicators• Appraisal of knowledge base• Mapping, assessment and prioritization

of relevant uncertainties• Reporting of uncertainty information


Outputs Quickscan• Rival problem framings, awareness of what is left out• List of salient uncertainties based on problem structure• Scoring of maturity of problem in the policy process & role of

assessment in this process• List of relevant stakeholders and indication of whether, when

and how to involve them– Problem co-framer, knowledge co-producer, extended peer review

• Identification of areas of agreement and disagreement among stakeholders on value dimensions of the problem

• Gauge of how well assessment tools and indicators address the problem

• Prioritized list of salient uncertainties (using typology)• Selection of tools for addressing each uncertainty type• Assessment of (likely) robustness of results



Tool catalogue• Quantitative tools (Sensitivity Analysis, Monte Carlo)• NUSAP• Expert Elicitation• Scenario analysis• PRIMA• Extended Quality Assurance (pedigree scheme)• Checklist model quality assistance• Assumption analysis• Actor analysis• …...


Tool catalogueFor each tool:

• What sorts and locations of uncertainty does this tool address?

• What resources are required to use it?• Strengths and limitations• guidance on application & complementarity • Typical pitfalls of each tool• References to handbooks, example case

studies, web-sites, experts etc.


Triggers that increase policy relevance of uncertainty• Close to a norm or target• Near a threshold of severe impact• Possibility of serious irreversible effects

(precautionary principle)• On steep part of cost-curve or impact curve• Low pedigree• High (potential) value-ladenness• ‘temperatute’ of scientific or societal

controversies


Reporting

• Make uncertainties, key assumptions and value loadings explicit

• Assess robustness of results• Discuss implications of uncertainty for

different settings of burden of proof• Relevance of results to the problem• Progressive disclosure of information ->

traceability and backing


ConclusionsThe guidance:• Structures the tasks of uncertainty

management• Can be used flexibly

– Quick&dirty, Quick-scan, full-mode– Before/during/after

• Promotes reflection and forces deliberate choice on how uncertainties are handled

• Helps to avoid pitfalls• Its development and introduction at RIVM

constitutes an institutional innovation


Conclusions (2)Similar to a patient information leaflet

alerting the patient to risks and unsuitable uses of a medicine, the guidance enables the delivery of policy-relevant quantitative information together with the essential warnings on its nature, limitations and pitfalls. It thereby promotes responsible and effective use of knowledge and ignorance in policy processes.


Text contributions by:Jeroen van der SluijsJames RisbeyArthur PetersenPeter JansenBruna de MarchiSerafin CorralSilvio FuntowiczRob HoppeSimone HuijsPenny KloproggeAngela Pereira Jerry RavetzWillemijn TuinstraMarjolein van Asselt

Further Workshop participants and or reviewersPoul HarremoesPeter HeubergerAndrea SaltelliStefano TarantolaWim TurkenburgJohn van AardenneAnton van der Giessen Mark van Oorschot,Ad Seebregts Hans VisserWarren Walker >40 RIVM employees…….

Acknowledgements

Documents

Copernicus Institute Interfaces between Science & Society, Milano, 27-28 November 2003 Quicksandy Knowledge Bases The need for guidance for dealing with