Embed Size (px)
Citation preview
Communicating Uncertainty with Probability Phrases
David V. Budescu Fordham University, NYC, USA
IPCC Communication MeetingOslo, Feb 9-10, 2016
I
Translation Table for IPCC Authors
Phrase Likelihood Conveyed
Virtually certain > 99%
Very likely > 90%
Likely > 66%
About as likely as not 33% to 66%
Unlikely < 33%
Very unlikely < 10%
Exceptionally unlikely < 1%5
Reservations
• The ranges of the terms are not mutually exclusive
• The ranges associated with the terms are arbitrary and are not related to people's “natural” understanding of the language
• People prefer to receive numerical information if available
• Large variability in interpretation of the terms
The 2014 Intl Study
• Read 8 statements extracted from the IPCC executive summary containing 4 terms: very likely, likely, unlikely, and very unlikely (2 statements for each term)
• Provide Best estimate and Lower and Upper bounds for each term (as intended by authors)
• Presentation format: Translation or Dual (VN)
• 27 samples in 25 countries and 17 languages (~400 /country) • All materials were translated and back translated by native
speakers• Analyze n=10,239 in 25 samples (51% men; mean age = 41)
7
Translation Screen shot
8
Translation screen shot with guidelines
9
Verbal-Numerical screen shot
10
Estimates are Regressive, but the VN Presentation Makes Them More Extreme
12
T VN T VN T VN T VNVery Unlikely
<10%Unlikely
<33%Likely>66%
Very Likely>90%
0
20
40
60
80
100
Bes
t Est
imat
es
Consistency with IPCC Guidelines Improves (especially for extremes)
13
Very Unlikely Unlikely Likely Very Likely0
10
20
30
40
50
16.13
36.62 37.64
18.95
31.76
43.1447.59
36.65
Overall TranslationOverall VN
% o
f Est
imat
es C
ompl
ying
with
G
uide
lines
IndiaNetherland
UKBrazil
Hong KongItaly
SpainCanadaQuebecSweden
AustraliaOverall
GermanyJapan
TaiwanRussiaPolandFrance
SlovakiaUSA
ChileChina
RSATurkeyKoreaIsrael
21%24%25%25%25%26%27%27%27%27%27%27%27%28%28%28%28%28%29%30%31%31%31%32%33%35%
28%32%
43%36%
41%39%
37%46%
38%41%
37%40%
43%35%
43%39%40%
38%44%45%
43%40%
48%46%
38%54%
% Compliance Translation Series2
14
The Benefits of Multiple Modalities
• The dual scale was superior in several senses: – Better differentiation and discrimination, – Higher agreement with the prescribed meaning, – More uniform interpretations across various groups– It works for people with different preferences
• It makes communicators more mindful of the terms’ intended meaning
• It can be used in more creative and flexible ways to accommodate, and signal, different levels of (im)precision in various cases
The Evidence Based Approach• Identify the target audience• Identify a subset of terms that are reasonable
candidates for inclusion in the lexicon• Establish empirically how the target population
understands, interprets and uses these terms (using single estimates, ranges, MFs)
• Use statistical methods to Identify optimal cutoff points that archive the highest consensus in interpretation in the target population
Evidence-Based Lexicons
Group Membership FunctionsDistribution of Estimates
Application to the IPCC
Sample Lexicon Consistency (%)UK IPCC 27
Peak 44Membership 50
Australia IPCC 25Peak 41Membership 45
Evidence-Based Lexicons
• There is a clear advantage to deriving evidence - based lexicons over “intuitive” ones
• Such lexicons are constructed to take full advantage of the common norms and meanings shared by a majority of participants in the target group
• As such they are less sensitive to idiosyncratic preferences of the committees that construct them and they are easier to implement and maintain
• There are many possible methods one can use in this context.
Evidence-Based Lexicons
• There is no single universal lexicon that serves all purposes
• It makes sense to choose the size and composition of the lexicon to – Fit the needs of the target population – Be sensitive to the precision and discrimination of
the information prevalent in the decision environment
– Be sensitive to the probabilities of interest
