Ted GordonSenior Research FellowThe Millennium ProjectWorld Federation of United Nations Associations

First developed at RAND in the 1960’s A means of eliciting and combining expert

judgments while avoiding the pitfalls of conference room confrontations.

Used a sequential questionnaires, each building on the previous round.

Required features: Participants guaranteed anonymity Feedback of reason for extreme opinions

Has been used in thousands of studies

Economically useful desalination of sea water 1970 1964-1980

Effective fertility control by oral contraceptive 1970 1970-1983

Development of new synthetic ultra light materials 1971 1970-1978

Automated language translators 1972 1968-1976

New organs through transplanting or prosthesis 1972 1968-1982

Reliable weather forecasts 1975 1972-1988

Operation of a central data storage facility 1980 1971-1991

Reformation of quantum relativity 1980 1975-1993

Implanted artificial organs made of plastic and electronic components 1981 1975-1988

Widespread and socially acceptable use of psychotropic drugs 1983 1980-2000

Creation of a form of artificial life (at least in the form of self replicating molecules) 1989 1979-2000

Economically useful exploitation of the ocean bottom 1980 1980-2000

Feasibility of limited weather control 1990 1987-2000

Commercial generation of protein for food 1990 1985-2003

Drug treatment for psychotic cases 1992 1983-2017

Biochemical general immunization 1994 1983-2000 From Gordon and Helmer, A Long Range Forecasting Study, RAND, 1964

Sociologist Economist Technologist

Environment

Person 1 XXX XXX

Person 2 XXX

Person 3 XXX

Person 4 XXX

Person 5 XXX XXX

Person 6 XXX

Person 7 XXX XXX

Person 8 XXX

TOTAL 3 2 2 3

First developed in a Defense Advanced Research Projects Agency (DARPA) study by Gordon and Pease (2004).

The object was to invent a technique by which a commander could rapidly collect and synthesize opinions of field personnel, using Delphi, to help him reach a decision.

Work was published in Technological Forecasting and Social Change.

Since then, many improvements and adaptations.

An on-screen form presents a series of questions to each participant, and for each: The average (or median) response of the group

so far (and possibly the distribution of responses) The number of responses made so far A button that opens a window showing reasons

that others have given for their responses. A button that opens a window that allows the

respondent to type in justifications for their own answer.

A space for the new numerical estimate, answering the question.

There is only one round; no explicit second round is necessary as in a conventional Delphi.

When a respondent revisits the study in a minute or a day, the original input form is presented with their prior answers in place and the new average or median of the group, and if desired, the distribution of responses.

By then others may have contributed judgments, so the averages or medians may have changed since the original answers were provided.

If an answer is beyond a pre-specified distance from the average an indicator “flag” can be used to show the respondents the questions in which their answers differ significantly from the group.

When the flag is “up” reasons become particularly important

The respondents are not shown the group average until after they have inserted an answer and a selected number of responses have been received or a given time has elapse since the study has started.

The answers can be submitted cell by cell or as an entire document.

To avoid crowding, the reasons can be submitted and appear on a separate page.

A notation appears after the respondent has submitted an answer indicating that the response has been received and entered into the database.

Notations also appear if individual responses are outside of predetermined plausible ranges.

Extensive hyperlink reference material can be included to permit the respondents to retrieve data on line while completing the questionnaire.

Reference material is included directly on the questionnaire

Split samples can be used to reduce the effect of bias introduced by question placement on the questionnaire.

A permanent record is maintained of responses including both quantitative and narrative responses.

Collection of judgments about when certain developments might occur, their importance, and impacts and reasons for extreme opinions

Choosing from among alternative decisions based on clear cut criteria

Providing information for and sharpening of scenarios

Input/output and cross impact matrixes Environmental monitoring

Decision models:  Real Time Decision Making. (DARPA)  Choosing a Configuration. (auto mfg)

 Priority studies:   Assessing Priorities of Proposed Millennium Project Study

Topic, 2006. Assessing Priorities of Proposed Millennium Project Study

Topic. 2007 Assessing Priorities of Proposed Millennium Project Study

Topic. 2008 SOFI studies: 

Variables for the Global State of the Future Index Variables for the Republic of Korea State of the Future Index Variables for the South Africa Global State of the Future

Index

Substantive studies:  Future education and learning possibilities Future Energy Developments

  Studies in support of UNESCO:

Judgments on acceptability of a Delphi process.  Scenario Drivers. Report construction

Resource allocation (WB subcontractor) Performance Biodiversity Climate Change

Possible Development Likelihood by 2030 (%)

1. National programs for improving collective intelligence

Some richer as well as lower income countries have (by this year of 2030) made improving collective intelligence a national goal; this includes improving individual as well as intelligence for their nations-as-whole. Click here to see references

Likelihood by 2030 The average group answer: 61.7Respondents: 195

Comments on this possibility (page 2) click here

2. Just in time knowledge and learning

Rote learning has diminished in importance. With ubiquitous computing and education for life-long learning, 'just in time knowledge' has become the norm. Reasoning, problem solving, and learning strategies form the core focus of public educational systems. Click here to see references

Likelihood by 2030 The average group answer: 71.1 Respondents: 185

Comments on this possibility (page 2) click here

1. What might encourage this possibility? What positive consequences might follow? Please enter positive developments and consequences. If you wish to change your prior input, type it here:

There is a great need for this I believe

***1. In ouropinion, knowledge is the result of the epistemological process of human brain triggered by information from the surroundings. What ipossible is that a group of persons, preferably comprehensivelydeveloped, in a community with shared goals work together, a synegenerated yielding results greater than the sum of its parts, oftenawesome. We firmly believe that this cooperation will be widely spraround the world by the year 2030.***I believe that public and private investments in social aspects anin education of the members of the society they can contribute.***complex and urgent national and international problems and disa

Average, median, standard deviation

Distribution of opinions

Summaries by region, occupation, gender

Correlation of qualitative responses with quantitative responses (e.g. “those who answered with a low estimate, had these reasons…”)