Upload
frank-catanzaro
View
2.153
Download
1
Embed Size (px)
DESCRIPTION
A presentation of the Real-Time Delphi Futures Methodology
Citation preview
The Millennium ProjectOctober, 2007
Conference room problems
Now the “classic” means for combining expert opinions
Anonymity
Feedback
The 1964 S&T Study
Developed in a DARPA study Web based Does not use “rounds” as in conventional
Delphi Permits revisits during the study Some applications to date
Energy MP Priorities Education and learning Decision making
Asynchronous, round based Pencil and paper: mail or fax back On line sequential questionnaires
Synchronous, round based Consensor Specialized machines, laptops
Real Time Delphi, roundless
An on-screen form presents a series of questions to each participant, and for each: The average (or median) response of the group
so far (and possibly the distribution of responses) The number of responses made so far A button that opens a window showing reasons
that others have given for their responses. A button that opens a window that allows the
respondent to type in justifications for their own answer.
A space for the new numerical estimate, answering the question.
If an answer is beyond a pre-specified distance from the average an indicator flags the question
When the flag is “up” reasons become particularly important
There is no explicit second round When the respondent comes back to the
study in a minute or a day, the input form is presented with prior answers
By then others may have contributed judgments, the averages or medians may have changed and other questions may be flagged
The respondents are now not shown the group average until after they had inserted an answer
The answers can be submitted cell by cell rather than the entire document at once.
To avoid crowding, the reasons can appear on a separate page.
A notation appears after the respondent submits an answer indicating that the response has been received and entered into the database.
One, two, or four columns of questions
Inclusion of extensive hyperlink reference material permits the respondents to retrieve data on line while completing the questionnaire.
Reference material can appear directly on the questionnaire
No Questions Your Responses Group Reasons
1 When will Israel be recognized as a nation by the Palestinian Authority?
Link go
Your estimate: 2008
Num Responding: 16
Average: 2010
Your reasons: Conciliation is mandatory
Things will happen fast when the suicide bombing stops
Conciliation is mandatory
2 When will cell phones replace 75% of standard phones?
Link go
Your estimate: 2010
Num Responding: 15
Average:2010
Your reasons: Cells phones leap frog infrastructure
Cells phones leap frog infrastructure
Too many towers required
3 When will a method be found to denature spent uranium rods?
Link go
Your estimate: 2050
Num Responding:5
Average:2100
Your reasons
DIFFERENT THAN AVERAGE
Immensely valuable
Tough to do
Number
Possible Project Importance to the MP
.
1
Promotion of 2007 SOF (translations, launchings, reviews, Foreign Affairs ad, mailings, etc.) Are there any new techniques that we have not tried that may prove effective?Should the Nodes have responsibility for selling in their countries?Should we print free copies for the General Assembly?
Your answer has been entered, and you may change it at any time. The average group answer 6.1 Respondents: 8 Submit only this cell Reasons click here
2
Complete and issue a new CD ROM: Futures Research Methodology 3.0 (Jerry) We have the raw materials already in hand but not the time to complete.What else can we do to finish this project?
The average group answer will appear after you have entered a value above Respondents: 5 Submit only this cell Reasons click here
go
go
Number Possible Development Likelihood by 2030 (%)
1
National programs for improving collective intelligence
Some richer as well as lower income countries have (by this year of 2030) made improving collective intelligence a national goal; this includes improving individual as well as intelligence for their nations-as-whole. Click here to see references
Likelihood by 2030 The average group answer: 61,7Respondents: 195
Comments on this possibility (page 2) click here
2
Just in time knowledge and learning
Rote learning has diminished in importance. With ubiquitous computing and education for life-long learning, 'just in time knowledge' has become the norm. Reasoning, problem solving, and learning strategies form the core focus of public educational systems. Click here to see references
Likelihood by 2030 The average group answer: 71.1 Respondents: 185
Comments on this possibility (page 2) click here
72
65
Submit
Submit
Variables
What is the best plausible value this
variable can achieve in 2016?
What is the worst plausible value this
variable can achieve in 2016?
How important is this variable to the future of the world over the next
10 years (10= essential)
How important is this variable to the future of your country over the
next 10 years (10= essential)
1
Infant mortality (deaths per 1,000 births)World (1983)= 89.1World (2005)= 50.0Singapore (2005)= 2.29Angola (2005)= 187.5Click here to see references
Best plausible global value in 2016? 38The average group answer: 40.5Respondents: 17 Reasons click here (after submitting a numerical answer above.)
Worst plausible global value in 2016? 90.0The average group answer: 100Respondents: 11 Reasons click here (after submitting a numerical answer above.)
Its importance to the future of the world? 7.0The average group answer: 8.0 Respondents: 6 Reasons click here (after submitting a numerical answer above.)
Its importance to the future of your country? 5.0Respondents: 4 Reasons click here (after submitting a numerical answer above.)
2
Food availability (Cal/cap)World (1980)= 2549World (2002)= 2798Egypt (2002)= 3300Eritrea (2002)= 1500Click here to see reference
Best plausible global value in 2016? 3000The average group answer: 3,166.5 Respondents: 21Reasons click here (after submitting a numerical answer above.)
Worst plausible global value in 2016? 2300The average group answer: 2,400.0 Respondents: 11Reasons click here (after submitting a numerical answer above.)
Its importance to the future of the world? 6The average group answer: 4.5 Respondents: 21Reasons click here (after submitting a numerical answer above.)
Its importance to the future of your country? 7Respondents: 11Reasons click here (after submitting a numerical answer above.)
Go GoGo Go
Go Go Go Go
.No serious negative effects
Likely to be effective
Implement quickly
PlausibleProvides
useful feedback
Reasonable cost
Weights
Avg.: 4Responses:
5Reasons.Input 4Justify
Avg.: 6Responses:
5Reasons.Input 5 Justify
Avg.: 7Responses:
5Reasons.Input 6Justify
Avg.: 8 Responses:
5Reasons.Input 7Justify
Avg.: 3 Responses:
5Reasons.Input 4Justify
Avg.: 6Responses: 5Reasons.Input 5Justify
Governments modify school curricula to remove cultural biases
Avg.: 10Responses:
5Reasons.Input 10Justify
Avg.: 6Responses:
5Reasons.Input 1Justify
Avg.: 4Responses:
5Reasons.Input 6Justify
Avg.: 5 Responses:
5Reasons.Input 4Justify
Avg.: 9 Responses:
5Reasons.Input 8Justify
Avg.: 9Responses: 5Reasons.Input 10Justify
The world implements a vastly improved disease early warning system
Avg.: 2Responses:
5Reasons.Input 1Justify
Avg.: 3Responses:
5Reasons.Input 2 Justify
Avg.: 1Responses:
5Reasons.Input 2Justify
Avg.: 5 Responses:
5Reasons.Input 7Justify
Avg.: 9 Responses:
5Reasons.Input 8Justify
Avg.: 2Responses: 5Reasons.Input 1Justify
URL: www.realtimedelphi.com Just sign in and use one of the
following codes: mp08 edu xsofix
Average, median, standard deviation
Distribution of opinions
Summaries by region
Reasons could be identified and associated with a particular numerical response,
Analysis techniques had also progressed. In the case of the later priorities study the analysis included not only computation of average scores, but also the distribution of responses, so that it was possible to identify distribution of responses, and thus proposed projects about which there was high agreement and those which may have shown similar average scores, but were in fact bi-modal.
Client PriceNode initiated studies; limit 1 study per year per node, total 5 node initiated studies max per year.
Free; but if external funding is available for the node’s study then, $1,500 or 10%
whichever is greaterConsulting firms for internal use, limit two studies per year for evaluation and familiarity
$15,000 per year/first year
Sponsors for internal use, limit two studies per year for evaluation and familiarity
$10,000 per year/first year
Consulting firms for external use with clients, limit 500 participants per external study.
$50,000 per study
Universities for teaching purposes $8,000/yearCorporations, NGO’s, government agencies, limit 500 participants per study. $35,000 per study
Consulting firms for external use with clients, limit 500 participants per external study.
$50,000 per study