Health policy-makers' perceptions of their use of evidence: a systematic review

Health policy-makersrsquo perceptions of their useof evidence a systematic review

Simon Innvtimesr Gunn Vist Mari Trommald Andrew OxmanHealth Services Research Unit National Institute of Public Health Oslo Norway

Objectives The empirical basis for theories and common wisdom regarding how to improve appropriate use ofresearch evidence in policy decisions is unclear One source of empirical evidence is interview studies with policy-makers The aim of this systematic review was to summarise the evidence from interview studies of facilitators ofand barriers to the use of research evidence by health policy-makers

Methods We searched multiple databases including Medline Embase Socio le PsychLit PAIS IBSS IPSA andHealthStar in June 2000 hand-searched key journals and personally contacted investigators We included interviewstudies with health policy-makers that covered their perceptions of the use of research evidence in health policydecisions at a national regional or organisational level Two reviewers independently assessed the relevance ofretrieved articles described the methods of included studies and extracted data that were summarised in tables andanalysed qualitatively

Results We identi ed 24 studies that met our inclusion criteria These studies included a total of 2041 interviewswith health policy-makers Assessments of the use of evidence were largely descriptive and qualitative focusing onhypothetical scenarios or retrospective perceptions of the use of evidence in relation to speci c cases Perceivedfacilitators of and barriers to the use of evidence varied The most commonly reported facilitators were personalcontact (1324) timely relevance (1324) and the inclusion of summaries with policy recommendations (1124)The most commonly reported barriers were absence of personal contact (1124) lack of timeliness or relevance ofresearch (924) mutual mistrust (824) and power and budget struggles (724)

Conclusions Interview studies with health policy-makers provide only limited support for commonly held beliefsabout facilitators of and barriers to their use of evidence and raise questions about commonsense proposals forimproving the use of research for policy decisions Two-way personal communication the most common suggestionmay improve the appropriate use of research evidence but it might also promote selective (inappropriate) use ofresearch evidence

Journal of Health Services Research amp Policy Vol 7 No 4 2002 239ndash244 The Royal Society of Medicine Press Ltd 2002

Introduction

Weiss and Weiss argue that lsquoboth decision-makers andsocial scientists behave as though social science researchmakes a genuine contribution to public policyrsquo1 At thesame time many researchers are sceptical about theextent to which research is used and many policy-makersare sceptical about the usefulness of research AsLindblom and Cohen suggest lsquo in public policymaking many suppliers and users of social research aredissatis ed the former because they are not listened tothe latter because they do not hear much they want tolisten torsquo2

Several theories have been put forward to explain therole of research in policy-making and common wisdom

about how to improve the appropriate use of research isnot hard to nd although empirical evidence tosupport it is dif cult to nd3 One source of suchevidence is the perceptions of those involved in policy-making The objective of this systematic review was tosummarise the evidence from interview studies withhealth policy decision-makers that cover their use ofresearch evidence in health policy decisions We wereparticularly interested in identifying facilitators of andbarriers to the use of evidence by health policy decision-makers as a basis for developing strategies to improvethe appropriate use of research

We found three previous non-systematic reviews thataddressed policy-makersrsquo use of evidence4ndash6 The rstevaluated evidence regarding research utilisation inrelationship to the lsquotwo-communitiesrsquo metaphor and sixmodels of research utilisation3 The second reviewincluded 27 empirical studies with data relevant to thequestion of how to improve the utilisation of organisa-tional research5 The third review was similar to thesecond but less comprehensive6 In contrast to these

J Health Serv Res Policy Vol 7 No 4 October 2002 239

Review article

Simon Innvaeligr PhD Researcher Gunn E Vist PhD Researcher MariTrommald MD Researcher Andrew D Oxman MD Head of SectionHealth Services Research Unit National Institute of Public HealthPostboks 4404 Nydalen 0403 Oslo Norway

Correspondence to SI

earlier reviews we have used a systematic approach toidentify select and synthesise relevant research We havealso focused the review on interview studies involvinghealth policy decision-makers and their perceptions ofthe enablers of and barriers to the use of researchevidence for policy decisions

Methods

Search strategy

We searched PubMed using an iterative strategy begin-ning with relevant articles that we had in our les andsearching for related articles Relevant articles fromthese searches were identied and new searches forrelated articles were performed Additional articles wereidenti ed by hand-searching journals (n ˆ 2) personalcommunication (n ˆ 3) tracking references in articlesand books (n ˆ 7) and searching in other electronicdatabases

The following databases were searched in June 2000using combinations of index terms and text wordsderived from relevant articles that we had previouslyidentied Medline from 1966 EMBASE from 1980Socio le (Sociological Abstracts) from 1963 PsychLITfrom 1987 PAIS from 1972 IBSS (International Biblio-graphy of the Social Sciences) from 1981 IPSA (Interna-tional Political Science Abstracts) from 1989 HealthStarfrom 1992 ISI Web of Science from 1981 and the NHSNational Research Register

Knowledge Creation Diffusion Utilization was hand-searched Due to lack of availability only 1979 1980(vols 1 2 and 4) 1981ndash1984 1985 (vols 3 and 4) 1987ndash1993 1994 (vols 3 and 4) were hand-searched Knowledgeand Policy was hand-searched for 1995ndash1997 when itcontinued as Knowledge and Policy The InternationalJournal of Knowledge Transfer and Utilization which washand-searched for 1998ndash2000

Selection criteria

We included interview studies and surveys with healthpolicy decision-makers We only included studies ofhealth policy-makers responsible for decisions on behalfof a large organisation or jurisdiction If others were alsointerviewed ndash most often researchers ndash decision-makershad to be explicitly de ned as a subgroup within thestudy Studies of clinical decision-making for individualpatients were excluded

The studies had to address decision-makersrsquo use ofresearch evidence in health policy decisions or on abroader range of policy decisions if these includedhealth policy decisions

Data collection and analysis

Articles were retrieved if it appeared likely that theycontained empirical data from interviews or if they other-wise appeared highly relevant Three people reviewed asample of 200 citations and abstracts following which

one person reviewed all of the citations and abstractsthat were found and the reference lists of all of theretrieved articles

The retrieved articles were assessed for inclusion byat least two people When disagreements could not beresolved by discussion a third person assessed the articleStudies that appeared relevant but which were not areincluded in a list of excluded studies (46 studies availablefrom the authors)

Two people extracted data from the included studiesusing a data collection form The extracted data weresummarised in three tables with the following informa-tion

Table 1 (characteristics) the studyrsquos objective thetype of interviews used characteristics of the partici-pants the types of research evidence and the types ofpolicy decisions that were considered

Table 2 (methods) the sampling frame and strategyresponse rate how the use of evidence was measuredor assessed how the determinants of use weremeasured or assessed

Table 3 (main results) identied the facilitators ofand barriers to the use of research

Tables 1 2 and 3 are included only in the onlineversion of the journal and may be viewed free of chargeat httpwwwcatchwordcomrpsvcatchwordrsm13558196v7n4s8p239

Results

More than 3000 references were found Seventy studiesappeared relevant on the basis of title keywords andabstracts Twenty-four studies reported in 26 articles metthe inclusion criteria Although non-English languagepapers were not excluded from our search all theincluded papers were in English The studies wereundertaken in the USA (n ˆ 10) both the USA and theUK (n ˆ 1) the UK (n ˆ 3) Canada (n ˆ 3) Australia(n ˆ 1) Burkina Faso (n ˆ 1) Mexico (n ˆ 1) TheNetherlands (n ˆ 1) Pakistan (n ˆ 1) South Africa(n ˆ 1) and Sweden (n ˆ 1)

Search strategy

The study designs varied Face-to-face interviews tele-phone interviews and postal questionnaires were usedwith open-ended semi-structured and structured inter-views (Table 1) Most of the studies focused on one or afew cases with widely varying policy questions and typesof evidence Generally the participants were poorlydescribed but most of the studies interviewed peoplewith apparent responsibility for making important policydecisions such as people in lsquoupper level positionsrsquo andlsquosenior managersrsquo Staff members who appeared to haveless responsibility for decisions were also included inseveral studies Twenty-one of the 24 studies examinedthe role of evidence in actual decision-making processesThe other three studies examined hypothetical decisionsAll studies measured perceptions of use or hypothetical

Review article Health policy-makersrsquo perceptions of their use of evidence

240 J Health Serv Res Policy Vol 7 No 4 October 2002

use rather than actual use of research evidence Five ofthe included studies did not report how many policy-makers were interviewed The remaining 19 studiesinterviewed a total of 2041 policy-makers (median 58range 16ndash479)

Study methods

The included studies were all limited with respect to thegeneralisability of their results Three studies intervieweda representative sample of health policy decision-makersdrawn from a clearly described sampling frame171834

Most studies reported how many decision-makers wereincluded and their positions but the relation betweenthe included policy-makers and the system from whichthey were drawn was rarely described Three of the 24included studies met all of our methodologicalcriteria171834 ie clearly described the sampling frameand the relationship of the included policy-makers tothis had a response rate of at least 60 and clearlydescribed how the use of evidence and the determinantsof how evidence was used were measured or assessed(Table 2) Seven studies partially met our criteria(Table 2last column) The other 14 studies did not adequatelydescribe the methods that were used

Measurement or assessment of the use and determi-nants of use of evidence was mostly descriptive Moststudies included a list of what the investigators perceivedto be barriers to or facilitators of the use of evidence Itis unclear which barriers and facilitators the decisions-makers found most important and which barriers andfacilitators were considered Six of the 24 studies ratedvariables that appeared to determine the use of evidence(Table 2)914171835

Facilitators and barriers

Many of the studies used open-ended questions Therespondents and investigators used different words todescribe facilitators of and barriers to the use ofresearch evidence However several common factorswere described across studies These are summarised inTable 3 A detailed version of this table including theexact words used in each of the included studies isavailable from the authors The most commonlymentioned facilitators of the use of research evidencein policy-making were

Personal contact between researchers and policy-makers (1324)

Timeliness and relevance of the research (1324)

Research that included a summary with clear recom-mendations (1124)

Good quality research (624)

Research that con rmed current policy or endorsedself-interest (624)

Community pressure or client demand for research(424)

Research that included effectiveness data (324)

The most commonly mentioned barriers to the use ofresearch evidence in policy-making were

Absence of personal contact between researchers andpolicy-makers (1124)

Lack of timeliness or relevance of research (924)

Mutual mistrust including perceived political naivetyof scientists and scienti c naivety of policy-makers(824)

Power and budget struggles (724)

Poor quality of research (624)

Political instability or high turnover of policy-makingstaff (524)

Based on the ndings of these studies personal two-way communication between researchers and decision-makers should be used to facilitate the use of researchThis can reduce mutual mistrust and promote a betterunderstanding of policy-making by researchers andresearch by policy-makers It can inform researchersabout what the decision-makers consider timely relevantquestions and policy-makers about how to obtain validanswers to these questions However the frequentlyidentied facilitators including personal two-way commu-nication may not be easy to establish ndash for examplebecause of political instability or high turnover of policy-making staff

Discussion

The strengths of this review include an extensive andsystematic literature search explicit inclusion criteriaand a systematic and transparent approach to collectingand presenting data from the included studies Everyincluded study was read and appraised by at least two ofthe authors The limitations of our review largely re ectthe limitations of the literature we reviewed

Because much research in the social sciences is poorlyindexed in electronic databases we may have missedrelevant studies Personal communication with investiga-tors in the eld yielded only three additional studies andalthough we attempted to contact the authors of all of theincluded studies only three responded and no additionalreferences were identi ed through these contacts

The limitations of the included studies with respect tosampling and generalisability are not surprising in lightof the fact that most of the studies were qualitative andwere not necessarily intended to include representativesamples Moreover given the diversity of contexts inwhich health policy decision-making occurs it is notpossible to obtain a generalisable sample Nonethelessinadequate descriptions of the participants and contextsfor many of the included studies make it dif cult tointerpret the results

No factor was mentioned in more than 13 of the 24studies as a facilitator or barrier even when similar factorswere grouped together This variation can in part beexplained by the fact that some of the studies focusedon speci c factors that were identied a priori andtherefore may have excluded other factors Anothersource of variation is the fact that most studies focused

Health policy-makersrsquo perceptions of their use of evidence Review article


on speci c cases and factors were considered only inrelationship to those cases For example in a studyfocusing on the use of an economic evaluation it ismore likely that nancial constraints would be consid-ered as a barrier In a developing country study it is morelikely that international support would be considered as afacilitator In a study focusing on an issue on whichinterest groups have a strong position it would be morelikely that decision-makers would consider alliances withinterest groups as a facilitator or a barrier

Two theoretical perspectives were dominant in theliterature The rst perspective draws analogies betweenthe relationship of researchers and policy-makers andthe relationship between the natural sciences and thehumanities From this perspective problems with com-munication between researchers and policy-makers arelarge and dif cult to overcome936 This is often referredto as the lsquotwo-communities thesisrsquo or the lsquotwo culturesrsquoThe other perspective focuses on the concept of the lsquouseof researchrsquo It addresses how the word lsquousersquo may havefundamentally different meanings

The `two-communities thesisrsquo

The lsquotwo-communities thesisrsquo postulates the existence oftwo camps that lack the ability to take into account therealities or perspectives of one another Caplan et alfound that social scientists see themselves as rationalobjective and open to new ideas9 they see decision-makers as action- and interest-oriented indifferent toevidence and new ideas Decision-makers on the otherhand see themselves as responsible action-oriented andpragmatic they see scientists as naive jargon-ridden andirresponsible in relationship to practical realities Two-way communication between the two camps can facilitatea mutual understanding of a policy question and thekind of knowledge that is needed This requires thatresearchers and decision-makers agree on which ques-tions can be answered on the basis of research evidenceand which require political judgement This does notimply that research alone can answer policy questionssince these will always require some political judgementThe aim of two-way communication is to help ensurethat research appropriately informs judgements forwhich policy-makers are accountable not for researchersto assume the role of policy-makers

The results of this review support the lsquotwo communitiesthesisrsquo since the most commonly identied facilitatorof the use of evidence was personal contact betweenresearchers and decision-makers It is further supportedby the most commonly mentioned barriers to the use ofevidence absence of personal contact and mutual mistrustbetween decision-makers and researchers Personal two-way communication may also be a necessary preconditionfor other facilitators For example without personaltwo-way communication it may be dif cult for researchersto understand what decision-makers regard as timelyrelevant or good quality research

On the other hand this review is mostly based oninformation from one side of the two camps the decision-

makers It is not surprising that decision-makers nd iteasier to use evidence that they have had an opportunityto in uence through two-way communication with theresearchers Additionally the included studies do notclarify whether decision-makers use only the researchthey want to use Decision-makers may use evidence thatsupports their own ideology or their own politicalprogramme

The lsquotwo-communitiesrsquo thesis sheds light on animportant possible paradox inherent in the results thefactors that facilitate use of research may not necessarilybe factors that researchers should seek to enhance Ifwhat is required for research to be used is that researchersdo what the policy-maker wants them to do then researchmay fail to ful l one of its most important functionsnamely to be objective reliable and unbiased

What is ùsersquo of evidence

Along with the lsquotwo-communitiesrsquo thesis the question ofwhat is meant by the concept of the lsquousersquo of evidence is themost commonly discussed theoretical issue in the litera-ture on knowledge utilisation The most frequent categor-isation of different types of use in this review is direct(lsquoinstrumentalrsquo or lsquoengineeringrsquo) selective (lsquosymbolicrsquo orlsquolegitimatingrsquo) and enlightening (lsquoconceptualrsquo) use ofevidence Direct use of evidence refers to speci c use ofresearch results It indicates that if research results arerelevant for a solution the results should directly affectthe solution without much adjustment Enlighteninguse of evidence refers to research that helps to lsquoestablishnew goals and bench marks of the attainablersquo37 and helpsto lsquoenrich and deepen understanding of the complexityof problems and the unintended consequences ofactionrsquo38 Selective use is strategic involving use lsquotolegitimate and sustain predetermined positionsrsquo5

The different de nitions of lsquousersquo contribute to thedif culty of interpreting the results of the studies includedin this review A health policy decision-maker who de nesuse as direct use is likely to report less use of researchevidence than decision-makers who also include selectiveor enlightening use of evidence Many of the studies haveaddressed different kinds of use and found that healthpolicy decision-makers refer to use in all three of theabove ways

The degree to which evidence is used directly in anenlightening way or selectively may vary in relation to

Different types of decision-makers upper middle andlower level

Different types of policy questions vague andcomplex or focused and simple

Different issues adoption versus implementation ordecision versus action

The degree to which evidence was directly used variedacross studies One study found that 40 of the use ofevidence was direct de ned as the lsquoprimary source inthe policy formulation processrsquo9 Another study foundonly 7 direct use212 The study that found 40 directuse was in a context where 94 of all the research used



had been commissioned The study with only 7 directuse did not cover examples of commissioned researchThis suggests that expecting direct use frequently maybe unreasonable particularly for research that is notcommissioned A third study not included in thisreview found that local administrators close to pro-gramme operations reported more direct use ofevidence than federal-level decision-makers39

Enlightening use of evidence is more dif cult to assessthan direct use Decision-makers may interpret this asno more than a question of whether they considerresearch to be of value One study found that 60 ofdecision-makers lsquousedrsquo evidence in this way9 Self-reportedselective use of research on the other hand may bedif cult to elicit in interviews because it puts decision-makers in a bad light although it is commonly perceivedas a problem by researchers and others Research thatcon rms current policy or supports community pressuremay facilitate selective use Personal contact may alsofacilitate selective use of research

Conclusions

From the results of this review of 24 studies that met ourinclusion criteria based on 2041 interviews with policy-makers researchers who wish to increase the use of theresults of their research should have personal and closetwo-way communication with decision-makers providedecision-makers with a brief summary of their researchwith clear policy recommendations ensure that theirresearch is perceived as timely relevant and of highquality include effectiveness data argue that the resultsof their research are relevant to current policy anddemands from the community They should avoid gettinginvolved in power and budget struggles and be aware ofthe high turnover of policy-making staff Of course if theaim is to increase appropriate (direct or enlightening)use of research rather than selective use these strategieswill often not be effective as well as being dif cult toimplement

If the results from the interview studies with decision-makers are taken literally use of evidence will befacilitated by timely and relevant research that givesdecision-makers the answers that they want Howeverthey may not always be able to get what they want sinceresearchers should and it is to be hoped will insist onusing methods that protect against bias If decision-makers and researchers start to talk together about thebarriers and facilitators found in this review a morere ective appropriate and cooperative way of workingtogether which values both relevance and validity mayevolve

The studies included in this review address decision-makersrsquo perceptions of their use of evidence Some studiessuggested document analysis as a way to check whetherthese perceptions correspond to what was done1215192133

but none of the included studies did this system-atically Future research should combine interviews withdocument analysis focus on commissioned research andclearly de ne what is meant by lsquousersquo of research

The methods that were used in the included studiesand their diversity limit the extent to which any rmconclusion can be drawn from this review In fact one ofthe main conclusions of this review is that there is atbest only limited support for any of the many opinionsput forward in the literature on the use of researchevidence by policy-makers At the same time given thediversity of the included studies the policy-makers whowere interviewed and the contexts in which they workedit is striking that some factors were identied as fre-quently as they were

Advice about how to improve the use of evidence bypolicy-makers is typically based on personal experiencesupported by anecdotes The results of this review providea context in which to consider such advice They providea menu of factors to consider but not a basis for anystrong recommendations Similarly the two theoriesthat are commonly used in these studies provide usefulinsights for understanding the perceptions summarised inthis review and other opinions about the use of evidenceby policy-makers but there are limited empirical data tosupport these or other theories

AcknowledgementWe thank Janette Boynton for help with literature searches

References1 Weiss JA Weiss CH Social scientists and decision makers

look at the usefulness of mental health research InLorion RP et al eds Psychology and public policybalancing public service and professional needWashington DC American Psychological Association1996 165ndash181

2 Lindblom CE Cohen DK Usable knowledge socialscience and social problem solving New Haven CT YaleUniversity Press 1979

3 Granados A Jonsson E Banta HD Bero L Bonair ACochet C et al EUR-Assess Project Subgroup Report onDissemination and Impact International Journal ofTechnology Assessment in Health Care 1997 13 220ndash286

4 Dunn WN The two-communities metaphor and models ofknowledge use an explanatory case study KnowledgeCreation Diffusion Utilization 1980 1 515ndash536

5 Beyer JM Trice HM The utilization process a conceptualframework and synthesis of empirical ndings Adminis-trative Science Quarterly 1982 27 591ndash622

6 Nelson CE Roberts J Maederer CM Wertheimer BJohnson B The utilization of social science informationby policy makers American Behavioral Scientist 1987 30569ndash577

7 van de Vall M Bolas C Using social policy research forreducing social problems an empirical analysis of struc-ture and functions Journal of Applied Behavioral Science1982 18 49ndash67

8 Glaser EM Taylor SH Factors in uencing the success ofapplied research American Psychologist 1973 28 140ndash146

9 Caplan N Morrison A Stambaugh RJ The use of socialscience knowledge in policy decisions at the national levela report to respondents Ann Arbor MI The University ofMichigan 1975 1ndash63

10 Caplan N A minimal set of conditions necessary for theutilization of social science knowledge in policy formula-tion at the national level In Weiss CH ed Using social



research in public policy making Toronto LexingtonBooks 1977 183ndash198

11 Patton MQ Grimes PS Guthrie KM Breanan NJ et al Insearch for impact an analysis of the utilization of federalhealth evaluation research In Weiss CH ed Using socialresearch in public policy making Toronto LexingtonBooks 1977 141ndash164

12 Weiss CH Bucuvalas MJ Truth test and utility testdecision makersrsquo frames of reference for social scienceresearch American Sociological Review 1980 45 302ndash313

13 Weiss JA Coping with complexity An experimental studyof public policy decision-making In Kinder DD PalfreyTR eds Experimental foundations of political scienceAnn Arbor MI University of Michigan Press 1993 185ndash208

14 McNeece CA DiNitto DM Johnson PJ The utility ofevaluation research for administrative decision-makingAdministration in Social Work 1983 7 77ndash87

15 Sunesson S Nilsson K Explaining research utilizationbeyond lsquofunctionsrsquo Knowledge Creation Diffusion Util-ization 1989 10 140ndash155

16 Florio E DeMartini JR The use of information bypolicymakers at the local community level Knowledge1993 15 106ndash123

17 Boyer JF Langbein LI Factors in uencing the use ofhealth evaluation research in congress Evaluation Review1991 15 507ndash532

18 Ibbotson SL Long AF Sheldon TA Mason J An initialevaluation of Effective Health Care bulletins as instru-ments of effective dissemination Journal of Managementin Medicine 1993 7 48ndash57

19 Soumerai SB Ross-Degnan D Fortess EE Walser BLDeterminants of change in Medicaid pharmaceutical costsharing does evidence affect policy Milbank Quarterly1997 75 11ndash34

20 Harries U Elliott H Higgins A Evidence-based policy-making in the NHS exploring the interface betweenresearch and the commissioning process Journal of PublicHealth Medicine 1999 21 29ndash36

21 Elliott H Popay J How are policy makers using evidenceModels of research utilisation and local NHS policymaking Journal of Epidemiology and CommunityHealth 2000 54 461ndash468

22 Trostle J Bronfman M Langer A How do researchersin uence decision-makers Case studies of Mexicanpolicies Health Policy and Planning 1999 14 103ndash114

23 Ross J The use of economic evaluation in health careAustralian decision makersrsquo perceptions Health Policy1995 31 103ndash110

24 Oh CH Explaining the impact of policy information onpolicy-making Knowledge and Policy The InternationalJournal of Knowledge Transfer and Utilization 1997 1022ndash55

25 Oh CH Rich RF Explaining use of information in publicpolicymaking Knowledge and Policy The InternationalJournal of Knowledge Transfer and Utilization 1996 9 3ndash35

26 Hilderbrand M Simon J Hyder A The role of research inchild health policy and programs in Pakistan Geneva TheCouncil of Health Research and Development (COHRED)Working Group on Research to Action and Policy 200077ndash85

27 Rudat K Evaluation of the Effective Health Care BulletinLondon Of ce For Public Management 1998 1ndash46

28 Burns A Charlwood P Darling H Fox DM Greeneld LHamlyn L et al Better information better outcomes Theuse of health technology assessment and clinical effective-ness data in health care purchasing decisions in theUnited Kingdom and the United States New YorkMilbank Memorial Fund 2000 1ndash26

29 Eyles J Stoddart G Lavis J Pranger T Molyneaux-Smith LMcMullan C Making resource shifts supportive of thebroad determinants of health The PEI experienceHamilton McMaster Institute of Environment and Health2000 3ndash37

30 Gerhardus A Kielmann K Sanou A Lessons in research toaction and policy case studies from seven differentcountries Geneva The Council of Health Research andDevelopment (COHRED) Working Group on Research toAction and Policy 2000 19ndash27

31 Moodley J Jacobs M Research to action and policycombating vitamin a deciencies in South Africa GenevaThe Council of Health Research and Development(COHRED) Working Group on Research to Action andPolicy 2000 54ndash66

32 Lavis JN Farrant MSR Stoddard GL Barriers to employ-ment-related healthy public policy Toronto McMasterUniversity Centre for Health Economics and PolicyAnalysis 2000 00ndash03 1ndash28

33 Lavis JN Ross SE Hohenadel J Hurley J Stoddard GLWoodward C et al The role of health services research inCanadian provincial policy-making CHSRF Report 1997-021 Hamilton McMaster University 2000 1ndash44



36 Snow CP The two cultures and the scientic revolutionCambridge Cambridge University Press 1959

37 Merton RK The role of applied social science in theformation of policy a research memorandum Philosophyof Science 1949 16 161ndash181

38 Aaron HJ Politics and the professors the great society inperspective Washington DC Brookings Institution 1978

39 Alkin MC Kosecoff J Fitzgibbon C Seligman R Evalua-tion and decision-making the title vii experience LosAngeles Center for the Study of Evaluation University ofCalifornia 1974




244a J Health Serv Res Policy Vol 7 No 4 October 2002

Tab

le1

Cha

ract

eris

tics

ofin

clud

edst

udie

s

Co

un

try

year

cond

uct

ed(

year

pu

blis

hed

)re

fere

nce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fev

iden

ce(f

rom

whi

chd

ecis

ion

sw

ere

mad

e)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

The

Net

herla

nds

1969

^197

17 Th

eco

nditi

ons

and

func

tions

influ

enci

ngut

ilisa

tion

ofso

cial

pol

icy

rese

arch

(p4

9)

Sem

i-stru

ctur

edsu

rvey

s(p

50)

Res

earc

hers

and

polic

y-m

aker

sth

atw

ere

resp

onsi

ble

for

inco

rpor

atin

gth

ere

sear

chre

sults

(p5

0)N

ˆno

tst

ated

120

rese

arch

proj

ects

on`c

lient

-orie

nted

rsquosoc

ial

rese

arch

with

inin

dust

rial

and

labo

urre

latio

ns

regi

onal

and

urb

anpl

anni

ngs

ocia

lwel

fare

and

publ

iche

alth

(p5

0)

Per

ceiv

ed(r

etro

spec

tive)

dec

isio

ns(p

50)

Rev

ised

vers

ion

ofpa

per

pres

ente

din

1979

(p4

9)

40ou

toft

he12

0pr

ojec

tsw

ere

onso

cial

wel

fare

and

publ

iche

alth

US

A19

738

Wha

twas

done

wha

the

lped

and

wha

thin

dere

dth

esu

cces

sfo

rten

rese

arch

proj

ects

unde

rthe

Nat

iona

lIns

titut

eof

Men

tal

Hea

lth

(p1

41)

Pers

onal

inte

rvie

ws

(p1

41)

Prin

cipa

linv

estig

ator

oth

erst

affm

emb

ers

adm

inis

trato

rsan

dpe

rson

sm

ostl

ikel

yto

be

utili

sers

(p1

41)

Nˆ

notg

iven

Five

proj

ects

rate

dlo

wan

dfiv

epr

ojec

tsra

ted

high

for

`suc

cess

rsquo(p

140

)

Per

ceiv

edus

eT

hein

terv

iew

sw

ere

don

ein

eith

er19

67or

1968

(pp

140^

141)

Su

cces

sde

fined

aspr

ojec

ttha

tw

asdi

rect

edto

the

orig

inal

obje

ctiv

ese

mbo

died

ina

clea

ran

dco

gent

rep

ort

cand

iden

ough

tobe

repl

icab

lea

ndth

atth

efin

din

gsw

ere

diss

emin

ated

adeq

uate

ly(p

140

)U

SA

1973

^197

49(p

vii)

and

USA

1973

^197

410So

cial

scie

nce

rese

arch

utili

satio

nan

dpo

licy

form

atio

n(p

vii)

Aw

aren

ess

ofre

sear

chk

now

ledg

eof

sour

ces

info

rmat

ion

retri

eval

pra

ctic

ein

tere

stin

soci

alin

dica

tors

eva

luat

ion

ofw

orth

and

obje

ctiv

ity

met

hods

and

mea

sure

men

tpr

oced

ures

deg

ree

ofco

nfid

ence

onfin

ding

sat

titud

esab

outu

sef

acto

rsth

atin

fluen

ceus

e(p

ix)

Pers

onal

inte

rvie

ws

open

-en

ded

and

stru

ctur

ed(p

pv

ix)

Upp

er-le

veld

ecis

ion-

mak

ers

inth

eex

ecut

ive

bran

chof

the

US

fede

ral

gove

rnm

ent(

ppv

viii)

31

depu

tyun

ders

ecre

tarie

s54

inst

itute

dire

ctor

s67

depu

tyas

sist

antd

irect

ors

52ag

ency

pers

onne

lN

ˆ20

4(p

v)

575

repo

rted

inst

ance

sof

use

whe

re38

5in

volv

edpr

imar

yre

sear

chso

urce

s(p

8)

94

wer

eei

ther

fund

edby

the

gove

rnm

ent

cond

ucte

dby

the

gove

rnm

ento

rbot

h(p

8)

Per

ceiv

ed(s

elf-r

epor

ted)

use

with

spec

ific

follo

w-u

pq

uest

ions

whe

nus

ew

asre

porte

d(p

1)

15

ofth

eus

eof

know

ledg

ew

ason

heal

thor

wel

fare

prob

lem

s(p

5)

`Mos

toft

heex

ampl

esre

spon

dent

sof

fere

dto

illust

rate

know

ledg

eap

plic

atio

nsre

ally

invo

lved

the

appl

icat

ion

ofse

cond

ary

sour

cein

form

atio

nrsquo(p

187

)

US

A19

77

11`T

oex

amin

eth

ena

ture

and

degr

eeof

utiliz

atio

nof

fede

rale

valu

atio

nre

sear

chrsquo

(p1

43)

Op

en-e

nded

inte

rvie

ws

with

diffe

rent

ques

tions

for

the

deci

sion

-mak

eran

dth

eev

alua

tor(

p1

43)

Thre

eke

yin

form

ants

from

each

ofth

e20

stud

ies

wer

ein

terv

iew

eda

proj

ect

offic

ert

hed

ecis

ion-

mak

er(g

over

nmen

tlev

el)a

ndth

eev

alua

tor(

p14

3)N

ˆ60

(p1

43)

20na

tiona

lhea

lthca

reev

alua

tions

(cas

est

udie

s)w

ithso

me

syst

emat

icd

ata

colle

ctio

nm

ade

betw

een

1971

and

1974

bas

edon

11fa

ctor

sid

entif

ied

byth

eau

thor

s(p

p14

2^14

3)

Per

ceiv

edan

dse

lf-d

efin

edus

e(p

144

)H

ealth

train

ing

lab

orat

ory

prof

icie

ncy

neig

hbou

rhoo

dhe

alth

cent

res

heal

thse

rvic

esd

eliv

ery

syst

ems

alco

holis

ml

oan

forg

iven

ess

and

train

ing

wor

ksho

p(p

p14

2^14

3)U

SA

1980

2 an

dU

SA19

8012

Polic

y-m

aker

srsquope

rcep

tions

ofth

eus

eful

ness

ofso

cial

scie

nces

rese

arch

(pp

165

302)

Pers

onal

inte

rvie

ws

both

open

-end

edan

dst

ruct

ured

(pp

168

171

303)

51fe

dera

l52

stat

e(fr

omte

nst

ates

)and

52lo

cal

offic

iald

ecis

ion-

mak

ers

inup

per-

leve

lpos

ition

s(p

p16

730

3)N

ˆ15

5

Each

dec

isio

n-m

aker

read

two

abst

ract

s(tw

o-pa

gers

)ou

tof5

0fo

ract

uals

ocia

lsc

ienc

ere

sear

chre

ports

(p

p1

683

03)

Hyp

othe

tical

deci

sion

s(i

e`re

leva

ntto

the

issu

esyo

urof

fice

deal

sw

ithrsquo)

onm

enta

lhe

alth

alc

ohol

and

drug

abus

e

(pp

166

303

)

Not

stat

edb

utpu

blis

hed

Ap

ril19

80R

eprin

ted

from

Am

eric

anP

sych

olog

ist 1

981

Bas

edon

the

sam

ere

sear

chp

roje

ctas

US

A19

8012

(but

par

tlyon

othe

rpar

tsof

the

data

)

The

deci

sion

-mak

ers

wer

eg

iven

diffe

rent

èm

piri

calrsquo

inte

rven

tions

(ie

two-

page

rs)a

ndar

eth

eref

ore

diffi

cult

toco

mpa

reon

the

effe

ctva

riabl

esco

ntin

ued


244b J Health Serv Res Policy Vol 7 No 4 October 2002

Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1982

13

Toob

serv

eth

eim

pac

ttha

tde

cisi

onco

mp

lexi

tya

ndam

ount

ofin

form

atio

nav

aila

ble

had

onth

eq

ualit

yof

the

deci

sion

proc

ess

(p1

87)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edqu

estio

n-na

ires

com

bine

dw

itha

cont

rolle

dex

perim

ent

Con

seq

uenc

esof

com

plex

ityw

ere

exam

ined

byef

fect

sof

info

rmat

ion

load

and

num

bero

fde

cisi

onal

tern

ativ

es(p

p1

871

931

94)

Exp

erie

nced

pub

lican

dp

rivat

ese

ctor

man

ager

sfro

msi

xm

id-c

aree

rp

rog

ram

mes

atH

arva

rd

MIT

Sta

nfor

dan

dth

eW

ashi

ngto

nP

ublic

Affa

irsC

ente

r(pp

187

188

)N

ˆ13

2(p

188

)

Each

dec

isio

n-m

aker

rece

ived

alo

ngm

emor

and

umde

scrib

ing

apr

oble

mp

rese

ntin

gev

iden

ceus

eful

inan

alys

ing

the

prob

lem

and

pose

da

dec

isio

nto

bem

ade

base

don

real

peo

ple

and

even

ts(p

188

)

Two

hypo

thet

ical

dec

isio

ns

one

onth

est

atersquo

sD

epar

t-m

ento

fHea

lthan

dR

ehab

i-lit

ativ

ese

rvic

esan

don

eon

am

etha

done

mai

nten

ance

pro

gram

me

inN

ewYo

rkC

ity(p

p1

88^1

89)

Dec

isio

nha

dto

bem

ade

from

avai

labl

ein

form

atio

nw

ithin

1ho

ur45

min

utes

(p1

93)

Rep

rinte

dfro

mth

eJo

urna

lof

Pol

icy

Ana

lysi

san

dM

anag

emen

t2

66^8

7(p

185

)

The

exp

erim

entw

asb

ased

onsi

xdi

ffere

ntve

rsio

nsof

each

ofth

etw

oca

ses

The

six

diff

eren

tver

sion

sof

the

case

sva

ried

inin

form

atio

nco

mpl

exity

Eac

hde

cisi

on-m

aker

had

only

one

deci

sion

tom

ake

aton

lyon

ele

velo

fcom

plex

ity(p

p18

9^19

019

3)U

SA19

83

14To

dete

rmin

eso

me

ofth

efa

ctor

sth

atin

fluen

cead

min

istra

tors

rsquouse

ofev

alua

tive

data

inde

cisi

on-

mak

ing

and

toex

amin

eth

ety

pes

ofde

cisi

ons

(p7

9)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p7

9^80

)

Dire

ctor

sin

com

mun

itym

enta

lhea

lthce

ntre

sin

Flor

ida

and

Nor

thC

arol

ina

(p7

9)N

ˆ42

Con

duct

edst

udie

son

prog

ram

me

effe

ctiv

enes

sor

imp

actd

urin

gth

ela

st12

-m

onth

per

iod

(p8

1)

Per

ceiv

ed(i

ere

por

ted

)us

e(p

p80

^81)

Swed

en19

84^1

98515

(p1

52)

Diff

eren

tway

sof

usin

gso

cial

rese

arch

in

stru

men

tal

polit

ical

co

nflic

ten

light

enm

ent

inte

ract

ive

and

tact

ical

(pp

141

143^

144)

Inte

rvie

ws

(p1

52)

Soc

ialw

orke

rs

man

agem

entd

irect

ors

loca

land

natio

nalp

oliti

cian

sin

15Sw

edis

hci

ties

and

com

mun

es(p

152

)N

ˆ90

(p1

52)

Doc

umen

tsth

atth

ere

spon

dent

sre

ferr

edto

Th

ese

docu

men

tsw

ere

also

anal

ysed

by

the

auth

ors

(p1

52)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

52)

USA

1990

16To

dete

rmin

ew

hatt

ype

ofin

form

atio

nd

ecis

ion-

mak

ers

use

and

how

ideo

log

yan

din

tere

sts

influ

ence

the

use

ofin

form

atio

n(p

109

)

Pers

onal

sem

i-stru

ctur

edin

terv

iew

s(p

112

)Lo

calp

lann

ing

com

mitt

ees

and

gove

rnm

ente

mp

loye

esin

two

rura

lcom

mun

ities

inth

eP

acifi

cN

orth

wes

t(p

110

)N

ˆ27

(p1

10)

Sour

ces

ofin

form

atio

nus

edin

com

mun

ityhe

alth

care

plan

ning

and

the

asse

ssm

ento

fthe

stra

tegi

cpl

anni

ngp

roce

ss(p

112

)

Per

ceiv

edus

e(p

112

)

USA

1991

17

Wha

tfac

tors

influ

ence

the

use

ofev

alua

tion

rese

arch

(on

heal

than

dhu

man

serv

ices

)in

cong

ress

(p5

08)

Stru

ctur

edte

lep

hone

inte

rvie

ws

(p5

13)

C

ongr

ess

mem

bers

and

thei

rsta

ff(p

512

)N

ˆ10

0Ea

chde

cisi

on-m

aker

chos

eon

est

udy

they

wer

efa

mili

arw

ith(p

515

)

Per

ceiv

edus

e(p

515

)E

valu

atio

nst

udie

sfro

mth

eD

epar

tmen

tofH

ealth

and

Hum

anSe

rvic

esth

atd

eal

with

natio

nalh

ealth

pro

gram

mes

(pp

507

515

)

Use

isde

fined

here

asbo

thin

stru

men

tal(

dire

ctim

ple

men

tatio

n)c

once

ptua

l(it

chan

ged

the

way

ofth

inki

ng)a

ndpe

rsua

sive

(use

dto

conv

ince

othe

rsp

516

)

The

inte

rvie

ws

last

ed15

^40

min

utes

(p5

14)

UK

1991

18U

sean

dre

ason

sfo

ruse

not

use

ofth

eE

iexclect

ive

Hea

lthC

are

Bul

letin

(pp

51^

54)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p5

0^51

)

Dis

trict

gen

eral

man

ager

sd

irect

ors

ofpu

blic

heal

than

dd

irect

ors

ofpl

anni

ng

com

mis

sion

ing

inU

Kd

istri

cthe

alth

auth

oriti

esor

com

mis

sion

ing

cons

ortia

(p5

0)N

ˆ17

2(p

51)

Eiexcle

ctiv

eH

ealth

Car

eB

ulle

tinN

o1

1991

on

scre

enin

gfo

rost

eop

oros

isto

prev

entf

ract

ures

(p5

0)

Per

ceiv

edus

eof

info

rmat

ion

and

hypo

thet

ical

dec

isio

ns

chan

geof

opin

ion

and

conf

iden

ceto

acto

nin

form

atio

nin

the

bulle

tin(p

p52

^54)

Res

pon

dent

sw

ere

que

stio

ned

abou

tabu

lletin

that

had

been

prev

ious

lypo

sted

toth

em(p

p51

52)

cont

inue

d


244c J Health Serv Res Policy Vol 7 No 4 October 2002

Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1993

^199

419Th

ero

leof

rese

arch

info

rmat

ion

inp

olic

yde

velo

pmen

t(p

29)

Que

stio

nnai

res

(p1

3)

open

-end

edte

leph

one

inte

rvie

ws

(p1

4)an

dse

mi-

stru

ctur

edte

leph

one

inte

rvie

ws

(p1

4)

Key

info

rman

ts

inst

ates

that

chan

ged

cost

-sha

ring

pol

icie

sfo

rpre

scrip

tion

reim

burs

emen

tlim

itsan

dco

-pay

men

ts(p

14)

N

în

adeq

uate

lyd

escr

ibed

Res

earc

hp

ublic

atio

nsan

dco

mm

ents

(p2

5)P

erce

ived

retro

spec

tive

dec

isio

ns(p

25)

Onl

y2^

3pa

ges

outo

f21

wer

ere

leva

nt(p

25f

f)I

nfor

man

tsin

clud

edM

edic

aid

drug

prog

ram

me

adm

inis

trato

rs

dire

ctor

sor

polic

yan

alys

ts

phar

mac

yco

nsul

tant

san

dle

gisl

ator

s(p

15)

UK

1994

^199

5202

1`T

oid

entif

yfa

ctor

sw

hich

faci

litat

eor

imp

ede

evid

ence

-bas

edp

olic

y-m

akin

grsquo(

p2

9)

In-d

epth

inte

rvie

ws

(usi

nga

topi

cgu

ide)

(p2

9an

dre

spon

seto

ale

tter)

Lead

polic

y-m

aker

sG

Psan

dre

sear

cher

sin

nine

diff

eren

tcas

est

udie

sat

alo

call

evel

inth

eN

HS

(pp

293

1)N

ˆ28

Soci

alre

sear

ch(p

42

inth

epr

ojec

trep

ort)

Per

ceiv

edde

cisi

ons

Polic

yon

the

inte

rface

bet

wee

nR

ampD

and

com

mis

sion

ing

atth

ehe

alth

auth

ority

leve

l(p

30)

Mex

ico

1994

^199

522`T

here

latio

nshi

pbe

twee

nhe

alth

rese

arch

(and

rese

arch

ers)

and

heal

thpo

licie

s(a

ndpo

licy-

mak

ers)

info

urve

rtica

lhe

alth

prog

ram

mes

inM

exic

orsquo(p

104

)

Ope

n-en

ded

(in-

dept

hrsquo)

inte

rvie

ws

with

aco

mm

onin

terv

iew

prot

ocol

(p1

06)

`Res

earc

hers

and

polic

y-m

aker

sfro

mdi

ffere

ntin

stitu

tions

and

leve

lsof

resp

onsi

bilit

yrsquo(p

103

)N

ˆ67

Use

ofre

sear

chpu

blic

atio

ns`w

here

we

knew

that

ther

ew

asat

leas

tso

me

inte

ract

ion

bet

wee

nre

sear

cher

san

dde

cisi

on-

mak

ersrsquo

(p1

05)

Per

ceiv

ed

use

ofev

iden

ceco

ncer

ning

AID

Sch

oler

afa

mily

pla

nnin

gan

dim

mun

isat

ion

ing

over

nmen

talp

rogr

amm

es(p

103

)

Per

sona

lcom

mun

icat

ion

gav

ead

ditio

nali

nfor

mat

ion

onth

isis

sue

It

isd

iffic

ultt

od

isce

rnw

heth

erth

ede

cisi

ons

wer

ep

erce

ived

orhy

pot

hetic

als

ince

ther

ear

eno

actu

alde

cisi

ons

men

tione

d

Aus

tralia

1995

23

Dec

isio

n-m

aker

srsquoaw

aren

ess

and

use

ofec

onom

icev

alua

tions

inhe

alth

care

(p1

03)

Inte

rvie

ws

open

-end

edan

dst

ruct

ured

(not

stat

edif

post

alt

elep

hone

orfa

ce-to

-fa

ce)(

p10

5)

Sen

iorm

anag

ers

inth

eFe

der

alD

epar

tmen

tof

Hea

lthH

ousi

ngan

dC

omm

unity

Serv

ices

New

Sou

thW

ales

Stat

eth

eH

ealth

Dep

artm

ent(

p10

4)

Nˆ

34(p

105

)

Use

ofec

onom

icev

alua

tions

forp

ricin

gon

drug

ssc

reen

ing

radi

olog

ytra

nspl

ants

and

scan

ning

(p1

05)

`Per

ceiv

edus

eof

econ

omic

eval

uatio

nsrsquo(

p1

04)

USA

1996

24

and

USA

1997

25

Tode

term

ine

whe

ther

the

proc

ess

of(s

ocia

lsci

ence

)in

form

atio

nut

ilisa

tion

varie

sac

ross

diff

eren

tare

as(i

ese

rvic

epr

ovis

ion

and

finan

cing

)(pp

16^

17)

Stru

ctur

edin

terv

iew

s(p

16)

Fe

der

als

tate

and

loca

lp

olic

y-m

aker

sat

diff

eren

tle

vels

deal

ing

with

finan

cean

dse

rvic

es(p

16)

N

ˆ47

9(p

16)

No

spec

ific

repo

rtb

utin

foty

persquor

efer

topr

ogra

mm

eev

alua

tion

dem

onst

ratio

np

roje

ct

stat

istic

ald

ata

and

polic

yan

alys

is(p

34)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

7)O

pen-

ende

din

terv

iew

sw

ere

used

toob

tain

basi

cb

ackg

roun

din

form

atio

nab

outt

hep

olic

ypr

oces

sTh

est

ruct

ured

inte

rvie

ws

built

onth

eop

en-e

nded

inte

rvie

ws

(p3

3)Pa

kist

an19

9826

The

role

that

rese

arch

pla

ysin

expe

rienc

esin

linki

ngre

sear

chw

ithp

olic

y(p

77)

Ope

n-en

ded

in-d

epth

inte

rvie

ws

(p7

7)D

ecis

ion-

mak

ers

from

gov

ernm

ent

the

cent

ral

min

istry

ofhe

alth

the

pla

nnin

gco

mm

issi

onan

dp

rovi

ncia

lhea

lthau

thor

ities

(p7

7)N

ˆ16

(p7

7)

Not

stat

edG

ener

alpe

rcep

tions

C

lose

stfo

cus

onch

ildhe

alth

prob

lem

ssu

chas

co-tr

imox

azol

em

ater

nal

mor

talit

yvi

tam

inA

b

reas

tfeed

ing

and

pol

io(p

p77

79)

UK

1998

27

The

imp

actt

hatt

heE

iexclect

ive

Hea

lthca

reB

ulle

tinha

sin

side

the

NH

Sor

gan

isat

ion

(p1

)

Tele

phon

ein

terv

iew

sst

ruct

ure

notm

entio

ned

(p1

)

Hea

lthau

thor

ities

(p

2)

Nˆ

34Tw

osp

ecifi

cbu

lletin

sJu

ne19

97an

dO

ctob

er19

97(p

1)

Hyp

othe

tical

deci

sion

s(e

g

itin

fluen

ced

the

dev

elop

men

tofo

urpo

licy

alth

oug

hth

isha

sbe

eng

oing

onfo

rsom

etim

ersquo)

(p1

0)

The

date

isno

tcle

arly

stat

edb

utit

is19

98or

1999

Tr

usts

and

GPs

are

incl

uded

inth

est

udy

buta

reno

trel

evan

tin

this

revi

ew

cont

inue

d


244d J Health Serv Res Policy Vol 7 No 4 October 2002

Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent


244e J Health Serv Res Policy Vol 7 No 4 October 2002

Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d


244f J Health Serv Res Policy Vol 7 No 4 October 2002

Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated


244g J Health Serv Res Policy Vol 7 No 4 October 2002

Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d


244h J Health Serv Res Policy Vol 7 No 4 October 2002

Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated

earlier reviews we have used a systematic approach toidentify select and synthesise relevant research We havealso focused the review on interview studies involvinghealth policy decision-makers and their perceptions ofthe enablers of and barriers to the use of researchevidence for policy decisions

Methods

Search strategy

We searched PubMed using an iterative strategy begin-ning with relevant articles that we had in our les andsearching for related articles Relevant articles fromthese searches were identied and new searches forrelated articles were performed Additional articles wereidenti ed by hand-searching journals (n ˆ 2) personalcommunication (n ˆ 3) tracking references in articlesand books (n ˆ 7) and searching in other electronicdatabases

The following databases were searched in June 2000using combinations of index terms and text wordsderived from relevant articles that we had previouslyidentied Medline from 1966 EMBASE from 1980Socio le (Sociological Abstracts) from 1963 PsychLITfrom 1987 PAIS from 1972 IBSS (International Biblio-graphy of the Social Sciences) from 1981 IPSA (Interna-tional Political Science Abstracts) from 1989 HealthStarfrom 1992 ISI Web of Science from 1981 and the NHSNational Research Register

Knowledge Creation Diffusion Utilization was hand-searched Due to lack of availability only 1979 1980(vols 1 2 and 4) 1981ndash1984 1985 (vols 3 and 4) 1987ndash1993 1994 (vols 3 and 4) were hand-searched Knowledgeand Policy was hand-searched for 1995ndash1997 when itcontinued as Knowledge and Policy The InternationalJournal of Knowledge Transfer and Utilization which washand-searched for 1998ndash2000

Selection criteria

We included interview studies and surveys with healthpolicy decision-makers We only included studies ofhealth policy-makers responsible for decisions on behalfof a large organisation or jurisdiction If others were alsointerviewed ndash most often researchers ndash decision-makershad to be explicitly de ned as a subgroup within thestudy Studies of clinical decision-making for individualpatients were excluded

The studies had to address decision-makersrsquo use ofresearch evidence in health policy decisions or on abroader range of policy decisions if these includedhealth policy decisions

Data collection and analysis

Articles were retrieved if it appeared likely that theycontained empirical data from interviews or if they other-wise appeared highly relevant Three people reviewed asample of 200 citations and abstracts following which

one person reviewed all of the citations and abstractsthat were found and the reference lists of all of theretrieved articles

The retrieved articles were assessed for inclusion byat least two people When disagreements could not beresolved by discussion a third person assessed the articleStudies that appeared relevant but which were not areincluded in a list of excluded studies (46 studies availablefrom the authors)

Two people extracted data from the included studiesusing a data collection form The extracted data weresummarised in three tables with the following informa-tion

Table 1 (characteristics) the studyrsquos objective thetype of interviews used characteristics of the partici-pants the types of research evidence and the types ofpolicy decisions that were considered

Table 2 (methods) the sampling frame and strategyresponse rate how the use of evidence was measuredor assessed how the determinants of use weremeasured or assessed

Table 3 (main results) identied the facilitators ofand barriers to the use of research

Tables 1 2 and 3 are included only in the onlineversion of the journal and may be viewed free of chargeat httpwwwcatchwordcomrpsvcatchwordrsm13558196v7n4s8p239

Results

More than 3000 references were found Seventy studiesappeared relevant on the basis of title keywords andabstracts Twenty-four studies reported in 26 articles metthe inclusion criteria Although non-English languagepapers were not excluded from our search all theincluded papers were in English The studies wereundertaken in the USA (n ˆ 10) both the USA and theUK (n ˆ 1) the UK (n ˆ 3) Canada (n ˆ 3) Australia(n ˆ 1) Burkina Faso (n ˆ 1) Mexico (n ˆ 1) TheNetherlands (n ˆ 1) Pakistan (n ˆ 1) South Africa(n ˆ 1) and Sweden (n ˆ 1)

Search strategy

The study designs varied Face-to-face interviews tele-phone interviews and postal questionnaires were usedwith open-ended semi-structured and structured inter-views (Table 1) Most of the studies focused on one or afew cases with widely varying policy questions and typesof evidence Generally the participants were poorlydescribed but most of the studies interviewed peoplewith apparent responsibility for making important policydecisions such as people in lsquoupper level positionsrsquo andlsquosenior managersrsquo Staff members who appeared to haveless responsibility for decisions were also included inseveral studies Twenty-one of the 24 studies examinedthe role of evidence in actual decision-making processesThe other three studies examined hypothetical decisionsAll studies measured perceptions of use or hypothetical




Study methods





















Discussion



























Conclusions























































Tab

le1

Cha

ract

eris

tics

ofin

clud

edst

udie

s

Co

un

try

year

cond

uct

ed(

year

pu

blis

hed

)re

fere

nce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fev

iden

ce(f

rom

whi

chd

ecis

ion

sw

ere

mad

e)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

The

Net

herla

nds

1969

^197

17 Th

eco

nditi

ons

and

func

tions

influ

enci

ngut

ilisa

tion

ofso

cial

pol

icy

rese

arch

(p4

9)

Sem

i-stru

ctur

edsu

rvey

s(p

50)

Res

earc

hers

and

polic

y-m

aker

sth

atw

ere

resp

onsi

ble

for

inco

rpor

atin

gth

ere

sear

chre

sults

(p5

0)N

ˆno

tst

ated

120

rese

arch

proj

ects

on`c

lient

-orie

nted

rsquosoc

ial

rese

arch

with

inin

dust

rial

and

labo

urre

latio

ns

regi

onal

and

urb

anpl

anni

ngs

ocia

lwel

fare

and

publ

iche

alth

(p5

0)

Per

ceiv

ed(r

etro

spec

tive)

dec

isio

ns(p

50)

Rev

ised

vers

ion

ofpa

per

pres

ente

din

1979

(p4

9)

40ou

toft

he12

0pr

ojec

tsw

ere

onso

cial

wel

fare

and

publ

iche

alth

US

A19

738

Wha

twas

done

wha

the

lped

and

wha

thin

dere

dth

esu

cces

sfo

rten

rese

arch

proj

ects

unde

rthe

Nat

iona

lIns

titut

eof

Men

tal

Hea

lth

(p1

41)

Pers

onal

inte

rvie

ws

(p1

41)

Prin

cipa

linv

estig

ator

oth

erst

affm

emb

ers

adm

inis

trato

rsan

dpe

rson

sm

ostl

ikel

yto

be

utili

sers

(p1

41)

Nˆ

notg

iven

Five

proj

ects

rate

dlo

wan

dfiv

epr

ojec

tsra

ted

high

for

`suc

cess

rsquo(p

140

)

Per

ceiv

edus

eT

hein

terv

iew

sw

ere

don

ein

eith

er19

67or

1968

(pp

140^

141)

Su

cces

sde

fined

aspr

ojec

ttha

tw

asdi

rect

edto

the

orig

inal

obje

ctiv

ese

mbo

died

ina

clea

ran

dco

gent

rep

ort

cand

iden

ough

tobe

repl

icab

lea

ndth

atth

efin

din

gsw

ere

diss

emin

ated

adeq

uate

ly(p

140

)U

SA

1973

^197

49(p

vii)

and

USA

1973

^197

410So

cial

scie

nce

rese

arch

utili

satio

nan

dpo

licy

form

atio

n(p

vii)

Aw

aren

ess

ofre

sear

chk

now

ledg

eof

sour

ces

info

rmat

ion

retri

eval

pra

ctic

ein

tere

stin

soci

alin

dica

tors

eva

luat

ion

ofw

orth

and

obje

ctiv

ity

met

hods

and

mea

sure

men

tpr

oced

ures

deg

ree

ofco

nfid

ence

onfin

ding

sat

titud

esab

outu

sef

acto

rsth

atin

fluen

ceus

e(p

ix)

Pers

onal

inte

rvie

ws

open

-en

ded

and

stru

ctur

ed(p

pv

ix)

Upp

er-le

veld

ecis

ion-

mak

ers

inth

eex

ecut

ive

bran

chof

the

US

fede

ral

gove

rnm

ent(

ppv

viii)

31

depu

tyun

ders

ecre

tarie

s54

inst

itute

dire

ctor

s67

depu

tyas

sist

antd

irect

ors

52ag

ency

pers

onne

lN

ˆ20

4(p

v)

575

repo

rted

inst

ance

sof

use

whe

re38

5in

volv

edpr

imar

yre

sear

chso

urce

s(p

8)

94

wer

eei

ther

fund

edby

the

gove

rnm

ent

cond

ucte

dby

the

gove

rnm

ento

rbot

h(p

8)

Per

ceiv

ed(s

elf-r

epor

ted)

use

with

spec

ific

follo

w-u

pq

uest

ions

whe

nus

ew

asre

porte

d(p

1)

15

ofth

eus

eof

know

ledg

ew

ason

heal

thor

wel

fare

prob

lem

s(p

5)

`Mos

toft

heex

ampl

esre

spon

dent

sof

fere

dto

illust

rate

know

ledg

eap

plic

atio

nsre

ally

invo

lved

the

appl

icat

ion

ofse

cond

ary

sour

cein

form

atio

nrsquo(p

187

)

US

A19

77

11`T

oex

amin

eth

ena

ture

and

degr

eeof

utiliz

atio

nof

fede

rale

valu

atio

nre

sear

chrsquo

(p1

43)

Op

en-e

nded

inte

rvie

ws

with

diffe

rent

ques

tions

for

the

deci

sion

-mak

eran

dth

eev

alua

tor(

p1

43)

Thre

eke

yin

form

ants

from

each

ofth

e20

stud

ies

wer

ein

terv

iew

eda

proj

ect

offic

ert

hed

ecis

ion-

mak

er(g

over

nmen

tlev

el)a

ndth

eev

alua

tor(

p14

3)N

ˆ60

(p1

43)

20na

tiona

lhea

lthca

reev

alua

tions

(cas

est

udie

s)w

ithso

me

syst

emat

icd

ata

colle

ctio

nm

ade

betw

een

1971

and

1974

bas

edon

11fa

ctor

sid

entif

ied

byth

eau

thor

s(p

p14

2^14

3)

Per

ceiv

edan

dse

lf-d

efin

edus

e(p

144

)H

ealth

train

ing

lab

orat

ory

prof

icie

ncy

neig

hbou

rhoo

dhe

alth

cent

res

heal

thse

rvic

esd

eliv

ery

syst

ems

alco

holis

ml

oan

forg

iven

ess

and

train

ing

wor

ksho

p(p

p14

2^14

3)U

SA

1980

2 an

dU

SA19

8012

Polic

y-m

aker

srsquope

rcep

tions

ofth

eus

eful

ness

ofso

cial

scie

nces

rese

arch

(pp

165

302)

Pers

onal

inte

rvie

ws

both

open

-end

edan

dst

ruct

ured

(pp

168

171

303)

51fe

dera

l52

stat

e(fr

omte

nst

ates

)and

52lo

cal

offic

iald

ecis

ion-

mak

ers

inup

per-

leve

lpos

ition

s(p

p16

730

3)N

ˆ15

5

Each

dec

isio

n-m

aker

read

two

abst

ract

s(tw

o-pa

gers

)ou

tof5

0fo

ract

uals

ocia

lsc

ienc

ere

sear

chre

ports

(p

p1

683

03)

Hyp

othe

tical

deci

sion

s(i

e`re

leva

ntto

the

issu

esyo

urof

fice

deal

sw

ithrsquo)

onm

enta

lhe

alth

alc

ohol

and

drug

abus

e

(pp

166

303

)

Not

stat

edb

utpu

blis

hed

Ap

ril19

80R

eprin

ted

from

Am

eric

anP

sych

olog

ist 1

981

Bas

edon

the

sam

ere

sear

chp

roje

ctas

US

A19

8012

(but

par

tlyon

othe

rpar

tsof

the

data

)

The

deci

sion

-mak

ers

wer

eg

iven

diffe

rent

èm

piri

calrsquo

inte

rven

tions

(ie

two-

page

rs)a

ndar

eth

eref

ore

diffi

cult

toco

mpa

reon

the

effe

ctva

riabl

esco

ntin

ued



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1982

13

Toob

serv

eth

eim

pac

ttha

tde

cisi

onco

mp

lexi

tya

ndam

ount

ofin

form

atio

nav

aila

ble

had

onth

eq

ualit

yof

the

deci

sion

proc

ess

(p1

87)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edqu

estio

n-na

ires

com

bine

dw

itha

cont

rolle

dex

perim

ent

Con

seq

uenc

esof

com

plex

ityw

ere

exam

ined

byef

fect

sof

info

rmat

ion

load

and

num

bero

fde

cisi

onal

tern

ativ

es(p

p1

871

931

94)

Exp

erie

nced

pub

lican

dp

rivat

ese

ctor

man

ager

sfro

msi

xm

id-c

aree

rp

rog

ram

mes

atH

arva

rd

MIT

Sta

nfor

dan

dth

eW

ashi

ngto

nP

ublic

Affa

irsC

ente

r(pp

187

188

)N

ˆ13

2(p

188

)

Each

dec

isio

n-m

aker

rece

ived

alo

ngm

emor

and

umde

scrib

ing

apr

oble

mp

rese

ntin

gev

iden

ceus

eful

inan

alys

ing

the

prob

lem

and

pose

da

dec

isio

nto

bem

ade

base

don

real

peo

ple

and

even

ts(p

188

)

Two

hypo

thet

ical

dec

isio

ns

one

onth

est

atersquo

sD

epar

t-m

ento

fHea

lthan

dR

ehab

i-lit

ativ

ese

rvic

esan

don

eon

am

etha

done

mai

nten

ance

pro

gram

me

inN

ewYo

rkC

ity(p

p1

88^1

89)

Dec

isio

nha

dto

bem

ade

from

avai

labl

ein

form

atio

nw

ithin

1ho

ur45

min

utes

(p1

93)

Rep

rinte

dfro

mth

eJo

urna

lof

Pol

icy

Ana

lysi

san

dM

anag

emen

t2

66^8

7(p

185

)

The

exp

erim

entw

asb

ased

onsi

xdi

ffere

ntve

rsio

nsof

each

ofth

etw

oca

ses

The

six

diff

eren

tver

sion

sof

the

case

sva

ried

inin

form

atio

nco

mpl

exity

Eac

hde

cisi

on-m

aker

had

only

one

deci

sion

tom

ake

aton

lyon

ele

velo

fcom

plex

ity(p

p18

9^19

019

3)U

SA19

83

14To

dete

rmin

eso

me

ofth

efa

ctor

sth

atin

fluen

cead

min

istra

tors

rsquouse

ofev

alua

tive

data

inde

cisi

on-

mak

ing

and

toex

amin

eth

ety

pes

ofde

cisi

ons

(p7

9)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p7

9^80

)

Dire

ctor

sin

com

mun

itym

enta

lhea

lthce

ntre

sin

Flor

ida

and

Nor

thC

arol

ina

(p7

9)N

ˆ42

Con

duct

edst

udie

son

prog

ram

me

effe

ctiv

enes

sor

imp

actd

urin

gth

ela

st12

-m

onth

per

iod

(p8

1)

Per

ceiv

ed(i

ere

por

ted

)us

e(p

p80

^81)

Swed

en19

84^1

98515

(p1

52)

Diff

eren

tway

sof

usin

gso

cial

rese

arch

in

stru

men

tal

polit

ical

co

nflic

ten

light

enm

ent

inte

ract

ive

and

tact

ical

(pp

141

143^

144)

Inte

rvie

ws

(p1

52)

Soc

ialw

orke

rs

man

agem

entd

irect

ors

loca

land

natio

nalp

oliti

cian

sin

15Sw

edis

hci

ties

and

com

mun

es(p

152

)N

ˆ90

(p1

52)

Doc

umen

tsth

atth

ere

spon

dent

sre

ferr

edto

Th

ese

docu

men

tsw

ere

also

anal

ysed

by

the

auth

ors

(p1

52)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

52)

USA

1990

16To

dete

rmin

ew

hatt

ype

ofin

form

atio

nd

ecis

ion-

mak

ers

use

and

how

ideo

log

yan

din

tere

sts

influ

ence

the

use

ofin

form

atio

n(p

109

)

Pers

onal

sem

i-stru

ctur

edin

terv

iew

s(p

112

)Lo

calp

lann

ing

com

mitt

ees

and

gove

rnm

ente

mp

loye

esin

two

rura

lcom

mun

ities

inth

eP

acifi

cN

orth

wes

t(p

110

)N

ˆ27

(p1

10)

Sour

ces

ofin

form

atio

nus

edin

com

mun

ityhe

alth

care

plan

ning

and

the

asse

ssm

ento

fthe

stra

tegi

cpl

anni

ngp

roce

ss(p

112

)

Per

ceiv

edus

e(p

112

)

USA

1991

17

Wha

tfac

tors

influ

ence

the

use

ofev

alua

tion

rese

arch

(on

heal

than

dhu

man

serv

ices

)in

cong

ress

(p5

08)

Stru

ctur

edte

lep

hone

inte

rvie

ws

(p5

13)

C

ongr

ess

mem

bers

and

thei

rsta

ff(p

512

)N

ˆ10

0Ea

chde

cisi

on-m

aker

chos

eon

est

udy

they

wer

efa

mili

arw

ith(p

515

)

Per

ceiv

edus

e(p

515

)E

valu

atio

nst

udie

sfro

mth

eD

epar

tmen

tofH

ealth

and

Hum

anSe

rvic

esth

atd

eal

with

natio

nalh

ealth

pro

gram

mes

(pp

507

515

)

Use

isde

fined

here

asbo

thin

stru

men

tal(

dire

ctim

ple

men

tatio

n)c

once

ptua

l(it

chan

ged

the

way

ofth

inki

ng)a

ndpe

rsua

sive

(use

dto

conv

ince

othe

rsp

516

)

The

inte

rvie

ws

last

ed15

^40

min

utes

(p5

14)

UK

1991

18U

sean

dre

ason

sfo

ruse

not

use

ofth

eE

iexclect

ive

Hea

lthC

are

Bul

letin

(pp

51^

54)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p5

0^51

)

Dis

trict

gen

eral

man

ager

sd

irect

ors

ofpu

blic

heal

than

dd

irect

ors

ofpl

anni

ng

com

mis

sion

ing

inU

Kd

istri

cthe

alth

auth

oriti

esor

com

mis

sion

ing

cons

ortia

(p5

0)N

ˆ17

2(p

51)

Eiexcle

ctiv

eH

ealth

Car

eB

ulle

tinN

o1

1991

on

scre

enin

gfo

rost

eop

oros

isto

prev

entf

ract

ures

(p5

0)

Per

ceiv

edus

eof

info

rmat

ion

and

hypo

thet

ical

dec

isio

ns

chan

geof

opin

ion

and

conf

iden

ceto

acto

nin

form

atio

nin

the

bulle

tin(p

p52

^54)

Res

pon

dent

sw

ere

que

stio

ned

abou

tabu

lletin

that

had

been

prev

ious

lypo

sted

toth

em(p

p51

52)

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1993

^199

419Th

ero

leof

rese

arch

info

rmat

ion

inp

olic

yde

velo

pmen

t(p

29)

Que

stio

nnai

res

(p1

3)

open

-end

edte

leph

one

inte

rvie

ws

(p1

4)an

dse

mi-

stru

ctur

edte

leph

one

inte

rvie

ws

(p1

4)

Key

info

rman

ts

inst

ates

that

chan

ged

cost

-sha

ring

pol

icie

sfo

rpre

scrip

tion

reim

burs

emen

tlim

itsan

dco

-pay

men

ts(p

14)

N

în

adeq

uate

lyd

escr

ibed

Res

earc

hp

ublic

atio

nsan

dco

mm

ents

(p2

5)P

erce

ived

retro

spec

tive

dec

isio

ns(p

25)

Onl

y2^

3pa

ges

outo

f21

wer

ere

leva

nt(p

25f

f)I

nfor

man

tsin

clud

edM

edic

aid

drug

prog

ram

me

adm

inis

trato

rs

dire

ctor

sor

polic

yan

alys

ts

phar

mac

yco

nsul

tant

san

dle

gisl

ator

s(p

15)

UK

1994

^199

5202

1`T

oid

entif

yfa

ctor

sw

hich

faci

litat

eor

imp

ede

evid

ence

-bas

edp

olic

y-m

akin

grsquo(

p2

9)

In-d

epth

inte

rvie

ws

(usi

nga

topi

cgu

ide)

(p2

9an

dre

spon

seto

ale

tter)

Lead

polic

y-m

aker

sG

Psan

dre

sear

cher

sin

nine

diff

eren

tcas

est

udie

sat

alo

call

evel

inth

eN

HS

(pp

293

1)N

ˆ28

Soci

alre

sear

ch(p

42

inth

epr

ojec

trep

ort)

Per

ceiv

edde

cisi

ons

Polic

yon

the

inte

rface

bet

wee

nR

ampD

and

com

mis

sion

ing

atth

ehe

alth

auth

ority

leve

l(p

30)

Mex

ico

1994

^199

522`T

here

latio

nshi

pbe

twee

nhe

alth

rese

arch

(and

rese

arch

ers)

and

heal

thpo

licie

s(a

ndpo

licy-

mak

ers)

info

urve

rtica

lhe

alth

prog

ram

mes

inM

exic

orsquo(p

104

)

Ope

n-en

ded

(in-

dept

hrsquo)

inte

rvie

ws

with

aco

mm

onin

terv

iew

prot

ocol

(p1

06)

`Res

earc

hers

and

polic

y-m

aker

sfro

mdi

ffere

ntin

stitu

tions

and

leve

lsof

resp

onsi

bilit

yrsquo(p

103

)N

ˆ67

Use

ofre

sear

chpu

blic

atio

ns`w

here

we

knew

that

ther

ew

asat

leas

tso

me

inte

ract

ion

bet

wee

nre

sear

cher

san

dde

cisi

on-

mak

ersrsquo

(p1

05)

Per

ceiv

ed

use

ofev

iden

ceco

ncer

ning

AID

Sch

oler

afa

mily

pla

nnin

gan

dim

mun

isat

ion

ing

over

nmen

talp

rogr

amm

es(p

103

)

Per

sona

lcom

mun

icat

ion

gav

ead

ditio

nali

nfor

mat

ion

onth

isis

sue

It

isd

iffic

ultt

od

isce

rnw

heth

erth

ede

cisi

ons

wer

ep

erce

ived

orhy

pot

hetic

als

ince

ther

ear

eno

actu

alde

cisi

ons

men

tione

d

Aus

tralia

1995

23

Dec

isio

n-m

aker

srsquoaw

aren

ess

and

use

ofec

onom

icev

alua

tions

inhe

alth

care

(p1

03)

Inte

rvie

ws

open

-end

edan

dst

ruct

ured

(not

stat

edif

post

alt

elep

hone

orfa

ce-to

-fa

ce)(

p10

5)

Sen

iorm

anag

ers

inth

eFe

der

alD

epar

tmen

tof

Hea

lthH

ousi

ngan

dC

omm

unity

Serv

ices

New

Sou

thW

ales

Stat

eth

eH

ealth

Dep

artm

ent(

p10

4)

Nˆ

34(p

105

)

Use

ofec

onom

icev

alua

tions

forp

ricin

gon

drug

ssc

reen

ing

radi

olog

ytra

nspl

ants

and

scan

ning

(p1

05)

`Per

ceiv

edus

eof

econ

omic

eval

uatio

nsrsquo(

p1

04)

USA

1996

24

and

USA

1997

25

Tode

term

ine

whe

ther

the

proc

ess

of(s

ocia

lsci

ence

)in

form

atio

nut

ilisa

tion

varie

sac

ross

diff

eren

tare

as(i

ese

rvic

epr

ovis

ion

and

finan

cing

)(pp

16^

17)

Stru

ctur

edin

terv

iew

s(p

16)

Fe

der

als

tate

and

loca

lp

olic

y-m

aker

sat

diff

eren

tle

vels

deal

ing

with

finan

cean

dse

rvic

es(p

16)

N

ˆ47

9(p

16)

No

spec

ific

repo

rtb

utin

foty

persquor

efer

topr

ogra

mm

eev

alua

tion

dem

onst

ratio

np

roje

ct

stat

istic

ald

ata

and

polic

yan

alys

is(p

34)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

7)O

pen-

ende

din

terv

iew

sw

ere

used

toob

tain

basi

cb

ackg

roun

din

form

atio

nab

outt

hep

olic

ypr

oces

sTh

est

ruct

ured

inte

rvie

ws

built

onth

eop

en-e

nded

inte

rvie

ws

(p3

3)Pa

kist

an19

9826

The

role

that

rese

arch

pla

ysin

expe

rienc

esin

linki

ngre

sear

chw

ithp

olic

y(p

77)

Ope

n-en

ded

in-d

epth

inte

rvie

ws

(p7

7)D

ecis

ion-

mak

ers

from

gov

ernm

ent

the

cent

ral

min

istry

ofhe

alth

the

pla

nnin

gco

mm

issi

onan

dp

rovi

ncia

lhea

lthau

thor

ities

(p7

7)N

ˆ16

(p7

7)

Not

stat

edG

ener

alpe

rcep

tions

C

lose

stfo

cus

onch

ildhe

alth

prob

lem

ssu

chas

co-tr

imox

azol

em

ater

nal

mor

talit

yvi

tam

inA

b

reas

tfeed

ing

and

pol

io(p

p77

79)

UK

1998

27

The

imp

actt

hatt

heE

iexclect

ive

Hea

lthca

reB

ulle

tinha

sin

side

the

NH

Sor

gan

isat

ion

(p1

)

Tele

phon

ein

terv

iew

sst

ruct

ure

notm

entio

ned

(p1

)

Hea

lthau

thor

ities

(p

2)

Nˆ

34Tw

osp

ecifi

cbu

lletin

sJu

ne19

97an

dO

ctob

er19

97(p

1)

Hyp

othe

tical

deci

sion

s(e

g

itin

fluen

ced

the

dev

elop

men

tofo

urpo

licy

alth

oug

hth

isha

sbe

eng

oing

onfo

rsom

etim

ersquo)

(p1

0)

The

date

isno

tcle

arly

stat

edb

utit

is19

98or

1999

Tr

usts

and

GPs

are

incl

uded

inth

est

udy

buta

reno

trel

evan

tin

this

revi

ew

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated


Study methods





















Discussion



























Conclusions























































Tab

le1

Cha

ract

eris

tics

ofin

clud

edst

udie

s

Co

un

try

year

cond

uct

ed(

year

pu

blis

hed

)re

fere

nce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fev

iden

ce(f

rom

whi

chd

ecis

ion

sw

ere

mad

e)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

The

Net

herla

nds

1969

^197

17 Th

eco

nditi

ons

and

func

tions

influ

enci

ngut

ilisa

tion

ofso

cial

pol

icy

rese

arch

(p4

9)

Sem

i-stru

ctur

edsu

rvey

s(p

50)

Res

earc

hers

and

polic

y-m

aker

sth

atw

ere

resp

onsi

ble

for

inco

rpor

atin

gth

ere

sear

chre

sults

(p5

0)N

ˆno

tst

ated

120

rese

arch

proj

ects

on`c

lient

-orie

nted

rsquosoc

ial

rese

arch

with

inin

dust

rial

and

labo

urre

latio

ns

regi

onal

and

urb

anpl

anni

ngs

ocia

lwel

fare

and

publ

iche

alth

(p5

0)

Per

ceiv

ed(r

etro

spec

tive)

dec

isio

ns(p

50)

Rev

ised

vers

ion

ofpa

per

pres

ente

din

1979

(p4

9)

40ou

toft

he12

0pr

ojec

tsw

ere

onso

cial

wel

fare

and

publ

iche

alth

US

A19

738

Wha

twas

done

wha

the

lped

and

wha

thin

dere

dth

esu

cces

sfo

rten

rese

arch

proj

ects

unde

rthe

Nat

iona

lIns

titut

eof

Men

tal

Hea

lth

(p1

41)

Pers

onal

inte

rvie

ws

(p1

41)

Prin

cipa

linv

estig

ator

oth

erst

affm

emb

ers

adm

inis

trato

rsan

dpe

rson

sm

ostl

ikel

yto

be

utili

sers

(p1

41)

Nˆ

notg

iven

Five

proj

ects

rate

dlo

wan

dfiv

epr

ojec

tsra

ted

high

for

`suc

cess

rsquo(p

140

)

Per

ceiv

edus

eT

hein

terv

iew

sw

ere

don

ein

eith

er19

67or

1968

(pp

140^

141)

Su

cces

sde

fined

aspr

ojec

ttha

tw

asdi

rect

edto

the

orig

inal

obje

ctiv

ese

mbo

died

ina

clea

ran

dco

gent

rep

ort

cand

iden

ough

tobe

repl

icab

lea

ndth

atth

efin

din

gsw

ere

diss

emin

ated

adeq

uate

ly(p

140

)U

SA

1973

^197

49(p

vii)

and

USA

1973

^197

410So

cial

scie

nce

rese

arch

utili

satio

nan

dpo

licy

form

atio

n(p

vii)

Aw

aren

ess

ofre

sear

chk

now

ledg

eof

sour

ces

info

rmat

ion

retri

eval

pra

ctic

ein

tere

stin

soci

alin

dica

tors

eva

luat

ion

ofw

orth

and

obje

ctiv

ity

met

hods

and

mea

sure

men

tpr

oced

ures

deg

ree

ofco

nfid

ence

onfin

ding

sat

titud

esab

outu

sef

acto

rsth

atin

fluen

ceus

e(p

ix)

Pers

onal

inte

rvie

ws

open

-en

ded

and

stru

ctur

ed(p

pv

ix)

Upp

er-le

veld

ecis

ion-

mak

ers

inth

eex

ecut

ive

bran

chof

the

US

fede

ral

gove

rnm

ent(

ppv

viii)

31

depu

tyun

ders

ecre

tarie

s54

inst

itute

dire

ctor

s67

depu

tyas

sist

antd

irect

ors

52ag

ency

pers

onne

lN

ˆ20

4(p

v)

575

repo

rted

inst

ance

sof

use

whe

re38

5in

volv

edpr

imar

yre

sear

chso

urce

s(p

8)

94

wer

eei

ther

fund

edby

the

gove

rnm

ent

cond

ucte

dby

the

gove

rnm

ento

rbot

h(p

8)

Per

ceiv

ed(s

elf-r

epor

ted)

use

with

spec

ific

follo

w-u

pq

uest

ions

whe

nus

ew

asre

porte

d(p

1)

15

ofth

eus

eof

know

ledg

ew

ason

heal

thor

wel

fare

prob

lem

s(p

5)

`Mos

toft

heex

ampl

esre

spon

dent

sof

fere

dto

illust

rate

know

ledg

eap

plic

atio

nsre

ally

invo

lved

the

appl

icat

ion

ofse

cond

ary

sour

cein

form

atio

nrsquo(p

187

)

US

A19

77

11`T

oex

amin

eth

ena

ture

and

degr

eeof

utiliz

atio

nof

fede

rale

valu

atio

nre

sear

chrsquo

(p1

43)

Op

en-e

nded

inte

rvie

ws

with

diffe

rent

ques

tions

for

the

deci

sion

-mak

eran

dth

eev

alua

tor(

p1

43)

Thre

eke

yin

form

ants

from

each

ofth

e20

stud

ies

wer

ein

terv

iew

eda

proj

ect

offic

ert

hed

ecis

ion-

mak

er(g

over

nmen

tlev

el)a

ndth

eev

alua

tor(

p14

3)N

ˆ60

(p1

43)

20na

tiona

lhea

lthca

reev

alua

tions

(cas

est

udie

s)w

ithso

me

syst

emat

icd

ata

colle

ctio

nm

ade

betw

een

1971

and

1974

bas

edon

11fa

ctor

sid

entif

ied

byth

eau

thor

s(p

p14

2^14

3)

Per

ceiv

edan

dse

lf-d

efin

edus

e(p

144

)H

ealth

train

ing

lab

orat

ory

prof

icie

ncy

neig

hbou

rhoo

dhe

alth

cent

res

heal

thse

rvic

esd

eliv

ery

syst

ems

alco

holis

ml

oan

forg

iven

ess

and

train

ing

wor

ksho

p(p

p14

2^14

3)U

SA

1980

2 an

dU

SA19

8012

Polic

y-m

aker

srsquope

rcep

tions

ofth

eus

eful

ness

ofso

cial

scie

nces

rese

arch

(pp

165

302)

Pers

onal

inte

rvie

ws

both

open

-end

edan

dst

ruct

ured

(pp

168

171

303)

51fe

dera

l52

stat

e(fr

omte

nst

ates

)and

52lo

cal

offic

iald

ecis

ion-

mak

ers

inup

per-

leve

lpos

ition

s(p

p16

730

3)N

ˆ15

5

Each

dec

isio

n-m

aker

read

two

abst

ract

s(tw

o-pa

gers

)ou

tof5

0fo

ract

uals

ocia

lsc

ienc

ere

sear

chre

ports

(p

p1

683

03)

Hyp

othe

tical

deci

sion

s(i

e`re

leva

ntto

the

issu

esyo

urof

fice

deal

sw

ithrsquo)

onm

enta

lhe

alth

alc

ohol

and

drug

abus

e

(pp

166

303

)

Not

stat

edb

utpu

blis

hed

Ap

ril19

80R

eprin

ted

from

Am

eric

anP

sych

olog

ist 1

981

Bas

edon

the

sam

ere

sear

chp

roje

ctas

US

A19

8012

(but

par

tlyon

othe

rpar

tsof

the

data

)

The

deci

sion

-mak

ers

wer

eg

iven

diffe

rent

èm

piri

calrsquo

inte

rven

tions

(ie

two-

page

rs)a

ndar

eth

eref

ore

diffi

cult

toco

mpa

reon

the

effe

ctva

riabl

esco

ntin

ued



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1982

13

Toob

serv

eth

eim

pac

ttha

tde

cisi

onco

mp

lexi

tya

ndam

ount

ofin

form

atio

nav

aila

ble

had

onth

eq

ualit

yof

the

deci

sion

proc

ess

(p1

87)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edqu

estio

n-na

ires

com

bine

dw

itha

cont

rolle

dex

perim

ent

Con

seq

uenc

esof

com

plex

ityw

ere

exam

ined

byef

fect

sof

info

rmat

ion

load

and

num

bero

fde

cisi

onal

tern

ativ

es(p

p1

871

931

94)

Exp

erie

nced

pub

lican

dp

rivat

ese

ctor

man

ager

sfro

msi

xm

id-c

aree

rp

rog

ram

mes

atH

arva

rd

MIT

Sta

nfor

dan

dth

eW

ashi

ngto

nP

ublic

Affa

irsC

ente

r(pp

187

188

)N

ˆ13

2(p

188

)

Each

dec

isio

n-m

aker

rece

ived

alo

ngm

emor

and

umde

scrib

ing

apr

oble

mp

rese

ntin

gev

iden

ceus

eful

inan

alys

ing

the

prob

lem

and

pose

da

dec

isio

nto

bem

ade

base

don

real

peo

ple

and

even

ts(p

188

)

Two

hypo

thet

ical

dec

isio

ns

one

onth

est

atersquo

sD

epar

t-m

ento

fHea

lthan

dR

ehab

i-lit

ativ

ese

rvic

esan

don

eon

am

etha

done

mai

nten

ance

pro

gram

me

inN

ewYo

rkC

ity(p

p1

88^1

89)

Dec

isio

nha

dto

bem

ade

from

avai

labl

ein

form

atio

nw

ithin

1ho

ur45

min

utes

(p1

93)

Rep

rinte

dfro

mth

eJo

urna

lof

Pol

icy

Ana

lysi

san

dM

anag

emen

t2

66^8

7(p

185

)

The

exp

erim

entw

asb

ased

onsi

xdi

ffere

ntve

rsio

nsof

each

ofth

etw

oca

ses

The

six

diff

eren

tver

sion

sof

the

case

sva

ried

inin

form

atio

nco

mpl

exity

Eac

hde

cisi

on-m

aker

had

only

one

deci

sion

tom

ake

aton

lyon

ele

velo

fcom

plex

ity(p

p18

9^19

019

3)U

SA19

83

14To

dete

rmin

eso

me

ofth

efa

ctor

sth

atin

fluen

cead

min

istra

tors

rsquouse

ofev

alua

tive

data

inde

cisi

on-

mak

ing

and

toex

amin

eth

ety

pes

ofde

cisi

ons

(p7

9)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p7

9^80

)

Dire

ctor

sin

com

mun

itym

enta

lhea

lthce

ntre

sin

Flor

ida

and

Nor

thC

arol

ina

(p7

9)N

ˆ42

Con

duct

edst

udie

son

prog

ram

me

effe

ctiv

enes

sor

imp

actd

urin

gth

ela

st12

-m

onth

per

iod

(p8

1)

Per

ceiv

ed(i

ere

por

ted

)us

e(p

p80

^81)

Swed

en19

84^1

98515

(p1

52)

Diff

eren

tway

sof

usin

gso

cial

rese

arch

in

stru

men

tal

polit

ical

co

nflic

ten

light

enm

ent

inte

ract

ive

and

tact

ical

(pp

141

143^

144)

Inte

rvie

ws

(p1

52)

Soc

ialw

orke

rs

man

agem

entd

irect

ors

loca

land

natio

nalp

oliti

cian

sin

15Sw

edis

hci

ties

and

com

mun

es(p

152

)N

ˆ90

(p1

52)

Doc

umen

tsth

atth

ere

spon

dent

sre

ferr

edto

Th

ese

docu

men

tsw

ere

also

anal

ysed

by

the

auth

ors

(p1

52)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

52)

USA

1990

16To

dete

rmin

ew

hatt

ype

ofin

form

atio

nd

ecis

ion-

mak

ers

use

and

how

ideo

log

yan

din

tere

sts

influ

ence

the

use

ofin

form

atio

n(p

109

)

Pers

onal

sem

i-stru

ctur

edin

terv

iew

s(p

112

)Lo

calp

lann

ing

com

mitt

ees

and

gove

rnm

ente

mp

loye

esin

two

rura

lcom

mun

ities

inth

eP

acifi

cN

orth

wes

t(p

110

)N

ˆ27

(p1

10)

Sour

ces

ofin

form

atio

nus

edin

com

mun

ityhe

alth

care

plan

ning

and

the

asse

ssm

ento

fthe

stra

tegi

cpl

anni

ngp

roce

ss(p

112

)

Per

ceiv

edus

e(p

112

)

USA

1991

17

Wha

tfac

tors

influ

ence

the

use

ofev

alua

tion

rese

arch

(on

heal

than

dhu

man

serv

ices

)in

cong

ress

(p5

08)

Stru

ctur

edte

lep

hone

inte

rvie

ws

(p5

13)

C

ongr

ess

mem

bers

and

thei

rsta

ff(p

512

)N

ˆ10

0Ea

chde

cisi

on-m

aker

chos

eon

est

udy

they

wer

efa

mili

arw

ith(p

515

)

Per

ceiv

edus

e(p

515

)E

valu

atio

nst

udie

sfro

mth

eD

epar

tmen

tofH

ealth

and

Hum

anSe

rvic

esth

atd

eal

with

natio

nalh

ealth

pro

gram

mes

(pp

507

515

)

Use

isde

fined

here

asbo

thin

stru

men

tal(

dire

ctim

ple

men

tatio

n)c

once

ptua

l(it

chan

ged

the

way

ofth

inki

ng)a

ndpe

rsua

sive

(use

dto

conv

ince

othe

rsp

516

)

The

inte

rvie

ws

last

ed15

^40

min

utes

(p5

14)

UK

1991

18U

sean

dre

ason

sfo

ruse

not

use

ofth

eE

iexclect

ive

Hea

lthC

are

Bul

letin

(pp

51^

54)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p5

0^51

)

Dis

trict

gen

eral

man

ager

sd

irect

ors

ofpu

blic

heal

than

dd

irect

ors

ofpl

anni

ng

com

mis

sion

ing

inU

Kd

istri

cthe

alth

auth

oriti

esor

com

mis

sion

ing

cons

ortia

(p5

0)N

ˆ17

2(p

51)

Eiexcle

ctiv

eH

ealth

Car

eB

ulle

tinN

o1

1991

on

scre

enin

gfo

rost

eop

oros

isto

prev

entf

ract

ures

(p5

0)

Per

ceiv

edus

eof

info

rmat

ion

and

hypo

thet

ical

dec

isio

ns

chan

geof

opin

ion

and

conf

iden

ceto

acto

nin

form

atio

nin

the

bulle

tin(p

p52

^54)

Res

pon

dent

sw

ere

que

stio

ned

abou

tabu

lletin

that

had

been

prev

ious

lypo

sted

toth

em(p

p51

52)

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1993

^199

419Th

ero

leof

rese

arch

info

rmat

ion

inp

olic

yde

velo

pmen

t(p

29)

Que

stio

nnai

res

(p1

3)

open

-end

edte

leph

one

inte

rvie

ws

(p1

4)an

dse

mi-

stru

ctur

edte

leph

one

inte

rvie

ws

(p1

4)

Key

info

rman

ts

inst

ates

that

chan

ged

cost

-sha

ring

pol

icie

sfo

rpre

scrip

tion

reim

burs

emen

tlim

itsan

dco

-pay

men

ts(p

14)

N

în

adeq

uate

lyd

escr

ibed

Res

earc

hp

ublic

atio

nsan

dco

mm

ents

(p2

5)P

erce

ived

retro

spec

tive

dec

isio

ns(p

25)

Onl

y2^

3pa

ges

outo

f21

wer

ere

leva

nt(p

25f

f)I

nfor

man

tsin

clud

edM

edic

aid

drug

prog

ram

me

adm

inis

trato

rs

dire

ctor

sor

polic

yan

alys

ts

phar

mac

yco

nsul

tant

san

dle

gisl

ator

s(p

15)

UK

1994

^199

5202

1`T

oid

entif

yfa

ctor

sw

hich

faci

litat

eor

imp

ede

evid

ence

-bas

edp

olic

y-m

akin

grsquo(

p2

9)

In-d

epth

inte

rvie

ws

(usi

nga

topi

cgu

ide)

(p2

9an

dre

spon

seto

ale

tter)

Lead

polic

y-m

aker

sG

Psan

dre

sear

cher

sin

nine

diff

eren

tcas

est

udie

sat

alo

call

evel

inth

eN

HS

(pp

293

1)N

ˆ28

Soci

alre

sear

ch(p

42

inth

epr

ojec

trep

ort)

Per

ceiv

edde

cisi

ons

Polic

yon

the

inte

rface

bet

wee

nR

ampD

and

com

mis

sion

ing

atth

ehe

alth

auth

ority

leve

l(p

30)

Mex

ico

1994

^199

522`T

here

latio

nshi

pbe

twee

nhe

alth

rese

arch

(and

rese

arch

ers)

and

heal

thpo

licie

s(a

ndpo

licy-

mak

ers)

info

urve

rtica

lhe

alth

prog

ram

mes

inM

exic

orsquo(p

104

)

Ope

n-en

ded

(in-

dept

hrsquo)

inte

rvie

ws

with

aco

mm

onin

terv

iew

prot

ocol

(p1

06)

`Res

earc

hers

and

polic

y-m

aker

sfro

mdi

ffere

ntin

stitu

tions

and

leve

lsof

resp

onsi

bilit

yrsquo(p

103

)N

ˆ67

Use

ofre

sear

chpu

blic

atio

ns`w

here

we

knew

that

ther

ew

asat

leas

tso

me

inte

ract

ion

bet

wee

nre

sear

cher

san

dde

cisi

on-

mak

ersrsquo

(p1

05)

Per

ceiv

ed

use

ofev

iden

ceco

ncer

ning

AID

Sch

oler

afa

mily

pla

nnin

gan

dim

mun

isat

ion

ing

over

nmen

talp

rogr

amm

es(p

103

)

Per

sona

lcom

mun

icat

ion

gav

ead

ditio

nali

nfor

mat

ion

onth

isis

sue

It

isd

iffic

ultt

od

isce

rnw

heth

erth

ede

cisi

ons

wer

ep

erce

ived

orhy

pot

hetic

als

ince

ther

ear

eno

actu

alde

cisi

ons

men

tione

d

Aus

tralia

1995

23

Dec

isio

n-m

aker

srsquoaw

aren

ess

and

use

ofec

onom

icev

alua

tions

inhe

alth

care

(p1

03)

Inte

rvie

ws

open

-end

edan

dst

ruct

ured

(not

stat

edif

post

alt

elep

hone

orfa

ce-to

-fa

ce)(

p10

5)

Sen

iorm

anag

ers

inth

eFe

der

alD

epar

tmen

tof

Hea

lthH

ousi

ngan

dC

omm

unity

Serv

ices

New

Sou

thW

ales

Stat

eth

eH

ealth

Dep

artm

ent(

p10

4)

Nˆ

34(p

105

)

Use

ofec

onom

icev

alua

tions

forp

ricin

gon

drug

ssc

reen

ing

radi

olog

ytra

nspl

ants

and

scan

ning

(p1

05)

`Per

ceiv

edus

eof

econ

omic

eval

uatio

nsrsquo(

p1

04)

USA

1996

24

and

USA

1997

25

Tode

term

ine

whe

ther

the

proc

ess

of(s

ocia

lsci

ence

)in

form

atio

nut

ilisa

tion

varie

sac

ross

diff

eren

tare

as(i

ese

rvic

epr

ovis

ion

and

finan

cing

)(pp

16^

17)

Stru

ctur

edin

terv

iew

s(p

16)

Fe

der

als

tate

and

loca

lp

olic

y-m

aker

sat

diff

eren

tle

vels

deal

ing

with

finan

cean

dse

rvic

es(p

16)

N

ˆ47

9(p

16)

No

spec

ific

repo

rtb

utin

foty

persquor

efer

topr

ogra

mm

eev

alua

tion

dem

onst

ratio

np

roje

ct

stat

istic

ald

ata

and

polic

yan

alys

is(p

34)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

7)O

pen-

ende

din

terv

iew

sw

ere

used

toob

tain

basi

cb

ackg

roun

din

form

atio

nab

outt

hep

olic

ypr

oces

sTh

est

ruct

ured

inte

rvie

ws

built

onth

eop

en-e

nded

inte

rvie

ws

(p3

3)Pa

kist

an19

9826

The

role

that

rese

arch

pla

ysin

expe

rienc

esin

linki

ngre

sear

chw

ithp

olic

y(p

77)

Ope

n-en

ded

in-d

epth

inte

rvie

ws

(p7

7)D

ecis

ion-

mak

ers

from

gov

ernm

ent

the

cent

ral

min

istry

ofhe

alth

the

pla

nnin

gco

mm

issi

onan

dp

rovi

ncia

lhea

lthau

thor

ities

(p7

7)N

ˆ16

(p7

7)

Not

stat

edG

ener

alpe

rcep

tions

C

lose

stfo

cus

onch

ildhe

alth

prob

lem

ssu

chas

co-tr

imox

azol

em

ater

nal

mor

talit

yvi

tam

inA

b

reas

tfeed

ing

and

pol

io(p

p77

79)

UK

1998

27

The

imp

actt

hatt

heE

iexclect

ive

Hea

lthca

reB

ulle

tinha

sin

side

the

NH

Sor

gan

isat

ion

(p1

)

Tele

phon

ein

terv

iew

sst

ruct

ure

notm

entio

ned

(p1

)

Hea

lthau

thor

ities

(p

2)

Nˆ

34Tw

osp

ecifi

cbu

lletin

sJu

ne19

97an

dO

ctob

er19

97(p

1)

Hyp

othe

tical

deci

sion

s(e

g

itin

fluen

ced

the

dev

elop

men

tofo

urpo

licy

alth

oug

hth

isha

sbe

eng

oing

onfo

rsom

etim

ersquo)

(p1

0)

The

date

isno

tcle

arly

stat

edb

utit

is19

98or

1999

Tr

usts

and

GPs

are

incl

uded

inth

est

udy

buta

reno

trel

evan

tin

this

revi

ew

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated





















Conclusions























































Tab

le1

Cha

ract

eris

tics

ofin

clud

edst

udie

s

Co

un

try

year

cond

uct

ed(

year

pu

blis

hed

)re

fere

nce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fev

iden

ce(f

rom

whi

chd

ecis

ion

sw

ere

mad

e)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

The

Net

herla

nds

1969

^197

17 Th

eco

nditi

ons

and

func

tions

influ

enci

ngut

ilisa

tion

ofso

cial

pol

icy

rese

arch

(p4

9)

Sem

i-stru

ctur

edsu

rvey

s(p

50)

Res

earc

hers

and

polic

y-m

aker

sth

atw

ere

resp

onsi

ble

for

inco

rpor

atin

gth

ere

sear

chre

sults

(p5

0)N

ˆno

tst

ated

120

rese

arch

proj

ects

on`c

lient

-orie

nted

rsquosoc

ial

rese

arch

with

inin

dust

rial

and

labo

urre

latio

ns

regi

onal

and

urb

anpl

anni

ngs

ocia

lwel

fare

and

publ

iche

alth

(p5

0)

Per

ceiv

ed(r

etro

spec

tive)

dec

isio

ns(p

50)

Rev

ised

vers

ion

ofpa

per

pres

ente

din

1979

(p4

9)

40ou

toft

he12

0pr

ojec

tsw

ere

onso

cial

wel

fare

and

publ

iche

alth

US

A19

738

Wha

twas

done

wha

the

lped

and

wha

thin

dere

dth

esu

cces

sfo

rten

rese

arch

proj

ects

unde

rthe

Nat

iona

lIns

titut

eof

Men

tal

Hea

lth

(p1

41)

Pers

onal

inte

rvie

ws

(p1

41)

Prin

cipa

linv

estig

ator

oth

erst

affm

emb

ers

adm

inis

trato

rsan

dpe

rson

sm

ostl

ikel

yto

be

utili

sers

(p1

41)

Nˆ

notg

iven

Five

proj

ects

rate

dlo

wan

dfiv

epr

ojec

tsra

ted

high

for

`suc

cess

rsquo(p

140

)

Per

ceiv

edus

eT

hein

terv

iew

sw

ere

don

ein

eith

er19

67or

1968

(pp

140^

141)

Su

cces

sde

fined

aspr

ojec

ttha

tw

asdi

rect

edto

the

orig

inal

obje

ctiv

ese

mbo

died

ina

clea

ran

dco

gent

rep

ort

cand

iden

ough

tobe

repl

icab

lea

ndth

atth

efin

din

gsw

ere

diss

emin

ated

adeq

uate

ly(p

140

)U

SA

1973

^197

49(p

vii)

and

USA

1973

^197

410So

cial

scie

nce

rese

arch

utili

satio

nan

dpo

licy

form

atio

n(p

vii)

Aw

aren

ess

ofre

sear

chk

now

ledg

eof

sour

ces

info

rmat

ion

retri

eval

pra

ctic

ein

tere

stin

soci

alin

dica

tors

eva

luat

ion

ofw

orth

and

obje

ctiv

ity

met

hods

and

mea

sure

men

tpr

oced

ures

deg

ree

ofco

nfid

ence

onfin

ding

sat

titud

esab

outu

sef

acto

rsth

atin

fluen

ceus

e(p

ix)

Pers

onal

inte

rvie

ws

open

-en

ded

and

stru

ctur

ed(p

pv

ix)

Upp

er-le

veld

ecis

ion-

mak

ers

inth

eex

ecut

ive

bran

chof

the

US

fede

ral

gove

rnm

ent(

ppv

viii)

31

depu

tyun

ders

ecre

tarie

s54

inst

itute

dire

ctor

s67

depu

tyas

sist

antd

irect

ors

52ag

ency

pers

onne

lN

ˆ20

4(p

v)

575

repo

rted

inst

ance

sof

use

whe

re38

5in

volv

edpr

imar

yre

sear

chso

urce

s(p

8)

94

wer

eei

ther

fund

edby

the

gove

rnm

ent

cond

ucte

dby

the

gove

rnm

ento

rbot

h(p

8)

Per

ceiv

ed(s

elf-r

epor

ted)

use

with

spec

ific

follo

w-u

pq

uest

ions

whe

nus

ew

asre

porte

d(p

1)

15

ofth

eus

eof

know

ledg

ew

ason

heal

thor

wel

fare

prob

lem

s(p

5)

`Mos

toft

heex

ampl

esre

spon

dent

sof

fere

dto

illust

rate

know

ledg

eap

plic

atio

nsre

ally

invo

lved

the

appl

icat

ion

ofse

cond

ary

sour

cein

form

atio

nrsquo(p

187

)

US

A19

77

11`T

oex

amin

eth

ena

ture

and

degr

eeof

utiliz

atio

nof

fede

rale

valu

atio

nre

sear

chrsquo

(p1

43)

Op

en-e

nded

inte

rvie

ws

with

diffe

rent

ques

tions

for

the

deci

sion

-mak

eran

dth

eev

alua

tor(

p1

43)

Thre

eke

yin

form

ants

from

each

ofth

e20

stud

ies

wer

ein

terv

iew

eda

proj

ect

offic

ert

hed

ecis

ion-

mak

er(g

over

nmen

tlev

el)a

ndth

eev

alua

tor(

p14

3)N

ˆ60

(p1

43)

20na

tiona

lhea

lthca

reev

alua

tions

(cas

est

udie

s)w

ithso

me

syst

emat

icd

ata

colle

ctio

nm

ade

betw

een

1971

and

1974

bas

edon

11fa

ctor

sid

entif

ied

byth

eau

thor

s(p

p14

2^14

3)

Per

ceiv

edan

dse

lf-d

efin

edus

e(p

144

)H

ealth

train

ing

lab

orat

ory

prof

icie

ncy

neig

hbou

rhoo

dhe

alth

cent

res

heal

thse

rvic

esd

eliv

ery

syst

ems

alco

holis

ml

oan

forg

iven

ess

and

train

ing

wor

ksho

p(p

p14

2^14

3)U

SA

1980

2 an

dU

SA19

8012

Polic

y-m

aker

srsquope

rcep

tions

ofth

eus

eful

ness

ofso

cial

scie

nces

rese

arch

(pp

165

302)

Pers

onal

inte

rvie

ws

both

open

-end

edan

dst

ruct

ured

(pp

168

171

303)

51fe

dera

l52

stat

e(fr

omte

nst

ates

)and

52lo

cal

offic

iald

ecis

ion-

mak

ers

inup

per-

leve

lpos

ition

s(p

p16

730

3)N

ˆ15

5

Each

dec

isio

n-m

aker

read

two

abst

ract

s(tw

o-pa

gers

)ou

tof5

0fo

ract

uals

ocia

lsc

ienc

ere

sear

chre

ports

(p

p1

683

03)

Hyp

othe

tical

deci

sion

s(i

e`re

leva

ntto

the

issu

esyo

urof

fice

deal

sw

ithrsquo)

onm

enta

lhe

alth

alc

ohol

and

drug

abus

e

(pp

166

303

)

Not

stat

edb

utpu

blis

hed

Ap

ril19

80R

eprin

ted

from

Am

eric

anP

sych

olog

ist 1

981

Bas

edon

the

sam

ere

sear

chp

roje

ctas

US

A19

8012

(but

par

tlyon

othe

rpar

tsof

the

data

)

The

deci

sion

-mak

ers

wer

eg

iven

diffe

rent

èm

piri

calrsquo

inte

rven

tions

(ie

two-

page

rs)a

ndar

eth

eref

ore

diffi

cult

toco

mpa

reon

the

effe

ctva

riabl

esco

ntin

ued



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1982

13

Toob

serv

eth

eim

pac

ttha

tde

cisi

onco

mp

lexi

tya

ndam

ount

ofin

form

atio

nav

aila

ble

had

onth

eq

ualit

yof

the

deci

sion

proc

ess

(p1

87)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edqu

estio

n-na

ires

com

bine

dw

itha

cont

rolle

dex

perim

ent

Con

seq

uenc

esof

com

plex

ityw

ere

exam

ined

byef

fect

sof

info

rmat

ion

load

and

num

bero

fde

cisi

onal

tern

ativ

es(p

p1

871

931

94)

Exp

erie

nced

pub

lican

dp

rivat

ese

ctor

man

ager

sfro

msi

xm

id-c

aree

rp

rog

ram

mes

atH

arva

rd

MIT

Sta

nfor

dan

dth

eW

ashi

ngto

nP

ublic

Affa

irsC

ente

r(pp

187

188

)N

ˆ13

2(p

188

)

Each

dec

isio

n-m

aker

rece

ived

alo

ngm

emor

and

umde

scrib

ing

apr

oble

mp

rese

ntin

gev

iden

ceus

eful

inan

alys

ing

the

prob

lem

and

pose

da

dec

isio

nto

bem

ade

base

don

real

peo

ple

and

even

ts(p

188

)

Two

hypo

thet

ical

dec

isio

ns

one

onth

est

atersquo

sD

epar

t-m

ento

fHea

lthan

dR

ehab

i-lit

ativ

ese

rvic

esan

don

eon

am

etha

done

mai

nten

ance

pro

gram

me

inN

ewYo

rkC

ity(p

p1

88^1

89)

Dec

isio

nha

dto

bem

ade

from

avai

labl

ein

form

atio

nw

ithin

1ho

ur45

min

utes

(p1

93)

Rep

rinte

dfro

mth

eJo

urna

lof

Pol

icy

Ana

lysi

san

dM

anag

emen

t2

66^8

7(p

185

)

The

exp

erim

entw

asb

ased

onsi

xdi

ffere

ntve

rsio

nsof

each

ofth

etw

oca

ses

The

six

diff

eren

tver

sion

sof

the

case

sva

ried

inin

form

atio

nco

mpl

exity

Eac

hde

cisi

on-m

aker

had

only

one

deci

sion

tom

ake

aton

lyon

ele

velo

fcom

plex

ity(p

p18

9^19

019

3)U

SA19

83

14To

dete

rmin

eso

me

ofth

efa

ctor

sth

atin

fluen

cead

min

istra

tors

rsquouse

ofev

alua

tive

data

inde

cisi

on-

mak

ing

and

toex

amin

eth

ety

pes

ofde

cisi

ons

(p7

9)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p7

9^80

)

Dire

ctor

sin

com

mun

itym

enta

lhea

lthce

ntre

sin

Flor

ida

and

Nor

thC

arol

ina

(p7

9)N

ˆ42

Con

duct

edst

udie

son

prog

ram

me

effe

ctiv

enes

sor

imp

actd

urin

gth

ela

st12

-m

onth

per

iod

(p8

1)

Per

ceiv

ed(i

ere

por

ted

)us

e(p

p80

^81)

Swed

en19

84^1

98515

(p1

52)

Diff

eren

tway

sof

usin

gso

cial

rese

arch

in

stru

men

tal

polit

ical

co

nflic

ten

light

enm

ent

inte

ract

ive

and

tact

ical

(pp

141

143^

144)

Inte

rvie

ws

(p1

52)

Soc

ialw

orke

rs

man

agem

entd

irect

ors

loca

land

natio

nalp

oliti

cian

sin

15Sw

edis

hci

ties

and

com

mun

es(p

152

)N

ˆ90

(p1

52)

Doc

umen

tsth

atth

ere

spon

dent

sre

ferr

edto

Th

ese

docu

men

tsw

ere

also

anal

ysed

by

the

auth

ors

(p1

52)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

52)

USA

1990

16To

dete

rmin

ew

hatt

ype

ofin

form

atio

nd

ecis

ion-

mak

ers

use

and

how

ideo

log

yan

din

tere

sts

influ

ence

the

use

ofin

form

atio

n(p

109

)

Pers

onal

sem

i-stru

ctur

edin

terv

iew

s(p

112

)Lo

calp

lann

ing

com

mitt

ees

and

gove

rnm

ente

mp

loye

esin

two

rura

lcom

mun

ities

inth

eP

acifi

cN

orth

wes

t(p

110

)N

ˆ27

(p1

10)

Sour

ces

ofin

form

atio

nus

edin

com

mun

ityhe

alth

care

plan

ning

and

the

asse

ssm

ento

fthe

stra

tegi

cpl

anni

ngp

roce

ss(p

112

)

Per

ceiv

edus

e(p

112

)

USA

1991

17

Wha

tfac

tors

influ

ence

the

use

ofev

alua

tion

rese

arch

(on

heal

than

dhu

man

serv

ices

)in

cong

ress

(p5

08)

Stru

ctur

edte

lep

hone

inte

rvie

ws

(p5

13)

C

ongr

ess

mem

bers

and

thei

rsta

ff(p

512

)N

ˆ10

0Ea

chde

cisi

on-m

aker

chos

eon

est

udy

they

wer

efa

mili

arw

ith(p

515

)

Per

ceiv

edus

e(p

515

)E

valu

atio

nst

udie

sfro

mth

eD

epar

tmen

tofH

ealth

and

Hum

anSe

rvic

esth

atd

eal

with

natio

nalh

ealth

pro

gram

mes

(pp

507

515

)

Use

isde

fined

here

asbo

thin

stru

men

tal(

dire

ctim

ple

men

tatio

n)c

once

ptua

l(it

chan

ged

the

way

ofth

inki

ng)a

ndpe

rsua

sive

(use

dto

conv

ince

othe

rsp

516

)

The

inte

rvie

ws

last

ed15

^40

min

utes

(p5

14)

UK

1991

18U

sean

dre

ason

sfo

ruse

not

use

ofth

eE

iexclect

ive

Hea

lthC

are

Bul

letin

(pp

51^

54)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p5

0^51

)

Dis

trict

gen

eral

man

ager

sd

irect

ors

ofpu

blic

heal

than

dd

irect

ors

ofpl

anni

ng

com

mis

sion

ing

inU

Kd

istri

cthe

alth

auth

oriti

esor

com

mis

sion

ing

cons

ortia

(p5

0)N

ˆ17

2(p

51)

Eiexcle

ctiv

eH

ealth

Car

eB

ulle

tinN

o1

1991

on

scre

enin

gfo

rost

eop

oros

isto

prev

entf

ract

ures

(p5

0)

Per

ceiv

edus

eof

info

rmat

ion

and

hypo

thet

ical

dec

isio

ns

chan

geof

opin

ion

and

conf

iden

ceto

acto

nin

form

atio

nin

the

bulle

tin(p

p52

^54)

Res

pon

dent

sw

ere

que

stio

ned

abou

tabu

lletin

that

had

been

prev

ious

lypo

sted

toth

em(p

p51

52)

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1993

^199

419Th

ero

leof

rese

arch

info

rmat

ion

inp

olic

yde

velo

pmen

t(p

29)

Que

stio

nnai

res

(p1

3)

open

-end

edte

leph

one

inte

rvie

ws

(p1

4)an

dse

mi-

stru

ctur

edte

leph

one

inte

rvie

ws

(p1

4)

Key

info

rman

ts

inst

ates

that

chan

ged

cost

-sha

ring

pol

icie

sfo

rpre

scrip

tion

reim

burs

emen

tlim

itsan

dco

-pay

men

ts(p

14)

N

în

adeq

uate

lyd

escr

ibed

Res

earc

hp

ublic

atio

nsan

dco

mm

ents

(p2

5)P

erce

ived

retro

spec

tive

dec

isio

ns(p

25)

Onl

y2^

3pa

ges

outo

f21

wer

ere

leva

nt(p

25f

f)I

nfor

man

tsin

clud

edM

edic

aid

drug

prog

ram

me

adm

inis

trato

rs

dire

ctor

sor

polic

yan

alys

ts

phar

mac

yco

nsul

tant

san

dle

gisl

ator

s(p

15)

UK

1994

^199

5202

1`T

oid

entif

yfa

ctor

sw

hich

faci

litat

eor

imp

ede

evid

ence

-bas

edp

olic

y-m

akin

grsquo(

p2

9)

In-d

epth

inte

rvie

ws

(usi

nga

topi

cgu

ide)

(p2

9an

dre

spon

seto

ale

tter)

Lead

polic

y-m

aker

sG

Psan

dre

sear

cher

sin

nine

diff

eren

tcas

est

udie

sat

alo

call

evel

inth

eN

HS

(pp

293

1)N

ˆ28

Soci

alre

sear

ch(p

42

inth

epr

ojec

trep

ort)

Per

ceiv

edde

cisi

ons

Polic

yon

the

inte

rface

bet

wee

nR

ampD

and

com

mis

sion

ing

atth

ehe

alth

auth

ority

leve

l(p

30)

Mex

ico

1994

^199

522`T

here

latio

nshi

pbe

twee

nhe

alth

rese

arch

(and

rese

arch

ers)

and

heal

thpo

licie

s(a

ndpo

licy-

mak

ers)

info

urve

rtica

lhe

alth

prog

ram

mes

inM

exic

orsquo(p

104

)

Ope

n-en

ded

(in-

dept

hrsquo)

inte

rvie

ws

with

aco

mm

onin

terv

iew

prot

ocol

(p1

06)

`Res

earc

hers

and

polic

y-m

aker

sfro

mdi

ffere

ntin

stitu

tions

and

leve

lsof

resp

onsi

bilit

yrsquo(p

103

)N

ˆ67

Use

ofre

sear

chpu

blic

atio

ns`w

here

we

knew

that

ther

ew

asat

leas

tso

me

inte

ract

ion

bet

wee

nre

sear

cher

san

dde

cisi

on-

mak

ersrsquo

(p1

05)

Per

ceiv

ed

use

ofev

iden

ceco

ncer

ning

AID

Sch

oler

afa

mily

pla

nnin

gan

dim

mun

isat

ion

ing

over

nmen

talp

rogr

amm

es(p

103

)

Per

sona

lcom

mun

icat

ion

gav

ead

ditio

nali

nfor

mat

ion

onth

isis

sue

It

isd

iffic

ultt

od

isce

rnw

heth

erth

ede

cisi

ons

wer

ep

erce

ived

orhy

pot

hetic

als

ince

ther

ear

eno

actu

alde

cisi

ons

men

tione

d

Aus

tralia

1995

23

Dec

isio

n-m

aker

srsquoaw

aren

ess

and

use

ofec

onom

icev

alua

tions

inhe

alth

care

(p1

03)

Inte

rvie

ws

open

-end

edan

dst

ruct

ured

(not

stat

edif

post

alt

elep

hone

orfa

ce-to

-fa

ce)(

p10

5)

Sen

iorm

anag

ers

inth

eFe

der

alD

epar

tmen

tof

Hea

lthH

ousi

ngan

dC

omm

unity

Serv

ices

New

Sou

thW

ales

Stat

eth

eH

ealth

Dep

artm

ent(

p10

4)

Nˆ

34(p

105

)

Use

ofec

onom

icev

alua

tions

forp

ricin

gon

drug

ssc

reen

ing

radi

olog

ytra

nspl

ants

and

scan

ning

(p1

05)

`Per

ceiv

edus

eof

econ

omic

eval

uatio

nsrsquo(

p1

04)

USA

1996

24

and

USA

1997

25

Tode

term

ine

whe

ther

the

proc

ess

of(s

ocia

lsci

ence

)in

form

atio

nut

ilisa

tion

varie

sac

ross

diff

eren

tare

as(i

ese

rvic

epr

ovis

ion

and

finan

cing

)(pp

16^

17)

Stru

ctur

edin

terv

iew

s(p

16)

Fe

der

als

tate

and

loca

lp

olic

y-m

aker

sat

diff

eren

tle

vels

deal

ing

with

finan

cean

dse

rvic

es(p

16)

N

ˆ47

9(p

16)

No

spec

ific

repo

rtb

utin

foty

persquor

efer

topr

ogra

mm

eev

alua

tion

dem

onst

ratio

np

roje

ct

stat

istic

ald

ata

and

polic

yan

alys

is(p

34)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

7)O

pen-

ende

din

terv

iew

sw

ere

used

toob

tain

basi

cb

ackg

roun

din

form

atio

nab

outt

hep

olic

ypr

oces

sTh

est

ruct

ured

inte

rvie

ws

built

onth

eop

en-e

nded

inte

rvie

ws

(p3

3)Pa

kist

an19

9826

The

role

that

rese

arch

pla

ysin

expe

rienc

esin

linki

ngre

sear

chw

ithp

olic

y(p

77)

Ope

n-en

ded

in-d

epth

inte

rvie

ws

(p7

7)D

ecis

ion-

mak

ers

from

gov

ernm

ent

the

cent

ral

min

istry

ofhe

alth

the

pla

nnin

gco

mm

issi

onan

dp

rovi

ncia

lhea

lthau

thor

ities

(p7

7)N

ˆ16

(p7

7)

Not

stat

edG

ener

alpe

rcep

tions

C

lose

stfo

cus

onch

ildhe

alth

prob

lem

ssu

chas

co-tr

imox

azol

em

ater

nal

mor

talit

yvi

tam

inA

b

reas

tfeed

ing

and

pol

io(p

p77

79)

UK

1998

27

The

imp

actt

hatt

heE

iexclect

ive

Hea

lthca

reB

ulle

tinha

sin

side

the

NH

Sor

gan

isat

ion

(p1

)

Tele

phon

ein

terv

iew

sst

ruct

ure

notm

entio

ned

(p1

)

Hea

lthau

thor

ities

(p

2)

Nˆ

34Tw

osp

ecifi

cbu

lletin

sJu

ne19

97an

dO

ctob

er19

97(p

1)

Hyp

othe

tical

deci

sion

s(e

g

itin

fluen

ced

the

dev

elop

men

tofo

urpo

licy

alth

oug

hth

isha

sbe

eng

oing

onfo

rsom

etim

ersquo)

(p1

0)

The

date

isno

tcle

arly

stat

edb

utit

is19

98or

1999

Tr

usts

and

GPs

are

incl

uded

inth

est

udy

buta

reno

trel

evan

tin

this

revi

ew

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



Conclusions























































Tab

le1

Cha

ract

eris

tics

ofin

clud

edst

udie

s

Co

un

try

year

cond

uct

ed(

year

pu

blis

hed

)re

fere

nce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fev

iden

ce(f

rom

whi

chd

ecis

ion

sw

ere

mad

e)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

The

Net

herla

nds

1969

^197

17 Th

eco

nditi

ons

and

func

tions

influ

enci

ngut

ilisa

tion

ofso

cial

pol

icy

rese

arch

(p4

9)

Sem

i-stru

ctur

edsu

rvey

s(p

50)

Res

earc

hers

and

polic

y-m

aker

sth

atw

ere

resp

onsi

ble

for

inco

rpor

atin

gth

ere

sear

chre

sults

(p5

0)N

ˆno

tst

ated

120

rese

arch

proj

ects

on`c

lient

-orie

nted

rsquosoc

ial

rese

arch

with

inin

dust

rial

and

labo

urre

latio

ns

regi

onal

and

urb

anpl

anni

ngs

ocia

lwel

fare

and

publ

iche

alth

(p5

0)

Per

ceiv

ed(r

etro

spec

tive)

dec

isio

ns(p

50)

Rev

ised

vers

ion

ofpa

per

pres

ente

din

1979

(p4

9)

40ou

toft

he12

0pr

ojec

tsw

ere

onso

cial

wel

fare

and

publ

iche

alth

US

A19

738

Wha

twas

done

wha

the

lped

and

wha

thin

dere

dth

esu

cces

sfo

rten

rese

arch

proj

ects

unde

rthe

Nat

iona

lIns

titut

eof

Men

tal

Hea

lth

(p1

41)

Pers

onal

inte

rvie

ws

(p1

41)

Prin

cipa

linv

estig

ator

oth

erst

affm

emb

ers

adm

inis

trato

rsan

dpe

rson

sm

ostl

ikel

yto

be

utili

sers

(p1

41)

Nˆ

notg

iven

Five

proj

ects

rate

dlo

wan

dfiv

epr

ojec

tsra

ted

high

for

`suc

cess

rsquo(p

140

)

Per

ceiv

edus

eT

hein

terv

iew

sw

ere

don

ein

eith

er19

67or

1968

(pp

140^

141)

Su

cces

sde

fined

aspr

ojec

ttha

tw

asdi

rect

edto

the

orig

inal

obje

ctiv

ese

mbo

died

ina

clea

ran

dco

gent

rep

ort

cand

iden

ough

tobe

repl

icab

lea

ndth

atth

efin

din

gsw

ere

diss

emin

ated

adeq

uate

ly(p

140

)U

SA

1973

^197

49(p

vii)

and

USA

1973

^197

410So

cial

scie

nce

rese

arch

utili

satio

nan

dpo

licy

form

atio

n(p

vii)

Aw

aren

ess

ofre

sear

chk

now

ledg

eof

sour

ces

info

rmat

ion

retri

eval

pra

ctic

ein

tere

stin

soci

alin

dica

tors

eva

luat

ion

ofw

orth

and

obje

ctiv

ity

met

hods

and

mea

sure

men

tpr

oced

ures

deg

ree

ofco

nfid

ence

onfin

ding

sat

titud

esab

outu

sef

acto

rsth

atin

fluen

ceus

e(p

ix)

Pers

onal

inte

rvie

ws

open

-en

ded

and

stru

ctur

ed(p

pv

ix)

Upp

er-le

veld

ecis

ion-

mak

ers

inth

eex

ecut

ive

bran

chof

the

US

fede

ral

gove

rnm

ent(

ppv

viii)

31

depu

tyun

ders

ecre

tarie

s54

inst

itute

dire

ctor

s67

depu

tyas

sist

antd

irect

ors

52ag

ency

pers

onne

lN

ˆ20

4(p

v)

575

repo

rted

inst

ance

sof

use

whe

re38

5in

volv

edpr

imar

yre

sear

chso

urce

s(p

8)

94

wer

eei

ther

fund

edby

the

gove

rnm

ent

cond

ucte

dby

the

gove

rnm

ento

rbot

h(p

8)

Per

ceiv

ed(s

elf-r

epor

ted)

use

with

spec

ific

follo

w-u

pq

uest

ions

whe

nus

ew

asre

porte

d(p

1)

15

ofth

eus

eof

know

ledg

ew

ason

heal

thor

wel

fare

prob

lem

s(p

5)

`Mos

toft

heex

ampl

esre

spon

dent

sof

fere

dto

illust

rate

know

ledg

eap

plic

atio

nsre

ally

invo

lved

the

appl

icat

ion

ofse

cond

ary

sour

cein

form

atio

nrsquo(p

187

)

US

A19

77

11`T

oex

amin

eth

ena

ture

and

degr

eeof

utiliz

atio

nof

fede

rale

valu

atio

nre

sear

chrsquo

(p1

43)

Op

en-e

nded

inte

rvie

ws

with

diffe

rent

ques

tions

for

the

deci

sion

-mak

eran

dth

eev

alua

tor(

p1

43)

Thre

eke

yin

form

ants

from

each

ofth

e20

stud

ies

wer

ein

terv

iew

eda

proj

ect

offic

ert

hed

ecis

ion-

mak

er(g

over

nmen

tlev

el)a

ndth

eev

alua

tor(

p14

3)N

ˆ60

(p1

43)

20na

tiona

lhea

lthca

reev

alua

tions

(cas

est

udie

s)w

ithso

me

syst

emat

icd

ata

colle

ctio

nm

ade

betw

een

1971

and

1974

bas

edon

11fa

ctor

sid

entif

ied

byth

eau

thor

s(p

p14

2^14

3)

Per

ceiv

edan

dse

lf-d

efin

edus

e(p

144

)H

ealth

train

ing

lab

orat

ory

prof

icie

ncy

neig

hbou

rhoo

dhe

alth

cent

res

heal

thse

rvic

esd

eliv

ery

syst

ems

alco

holis

ml

oan

forg

iven

ess

and

train

ing

wor

ksho

p(p

p14

2^14

3)U

SA

1980

2 an

dU

SA19

8012

Polic

y-m

aker

srsquope

rcep

tions

ofth

eus

eful

ness

ofso

cial

scie

nces

rese

arch

(pp

165

302)

Pers

onal

inte

rvie

ws

both

open

-end

edan

dst

ruct

ured

(pp

168

171

303)

51fe

dera

l52

stat

e(fr

omte

nst

ates

)and

52lo

cal

offic

iald

ecis

ion-

mak

ers

inup

per-

leve

lpos

ition

s(p

p16

730

3)N

ˆ15

5

Each

dec

isio

n-m

aker

read

two

abst

ract

s(tw

o-pa

gers

)ou

tof5

0fo

ract

uals

ocia

lsc

ienc

ere

sear

chre

ports

(p

p1

683

03)

Hyp

othe

tical

deci

sion

s(i

e`re

leva

ntto

the

issu

esyo

urof

fice

deal

sw

ithrsquo)

onm

enta

lhe

alth

alc

ohol

and

drug

abus

e

(pp

166

303

)

Not

stat

edb

utpu

blis

hed

Ap

ril19

80R

eprin

ted

from

Am

eric

anP

sych

olog

ist 1

981

Bas

edon

the

sam

ere

sear

chp

roje

ctas

US

A19

8012

(but

par

tlyon

othe

rpar

tsof

the

data

)

The

deci

sion

-mak

ers

wer

eg

iven

diffe

rent

èm

piri

calrsquo

inte

rven

tions

(ie

two-

page

rs)a

ndar

eth

eref

ore

diffi

cult

toco

mpa

reon

the

effe

ctva

riabl

esco

ntin

ued



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1982

13

Toob

serv

eth

eim

pac

ttha

tde

cisi

onco

mp

lexi

tya

ndam

ount

ofin

form

atio

nav

aila

ble

had

onth

eq

ualit

yof

the

deci

sion

proc

ess

(p1

87)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edqu

estio

n-na

ires

com

bine

dw

itha

cont

rolle

dex

perim

ent

Con

seq

uenc

esof

com

plex

ityw

ere

exam

ined

byef

fect

sof

info

rmat

ion

load

and

num

bero

fde

cisi

onal

tern

ativ

es(p

p1

871

931

94)

Exp

erie

nced

pub

lican

dp

rivat

ese

ctor

man

ager

sfro

msi

xm

id-c

aree

rp

rog

ram

mes

atH

arva

rd

MIT

Sta

nfor

dan

dth

eW

ashi

ngto

nP

ublic

Affa

irsC

ente

r(pp

187

188

)N

ˆ13

2(p

188

)

Each

dec

isio

n-m

aker

rece

ived

alo

ngm

emor

and

umde

scrib

ing

apr

oble

mp

rese

ntin

gev

iden

ceus

eful

inan

alys

ing

the

prob

lem

and

pose

da

dec

isio

nto

bem

ade

base

don

real

peo

ple

and

even

ts(p

188

)

Two

hypo

thet

ical

dec

isio

ns

one

onth

est

atersquo

sD

epar

t-m

ento

fHea

lthan

dR

ehab

i-lit

ativ

ese

rvic

esan

don

eon

am

etha

done

mai

nten

ance

pro

gram

me

inN

ewYo

rkC

ity(p

p1

88^1

89)

Dec

isio

nha

dto

bem

ade

from

avai

labl

ein

form

atio

nw

ithin

1ho

ur45

min

utes

(p1

93)

Rep

rinte

dfro

mth

eJo

urna

lof

Pol

icy

Ana

lysi

san

dM

anag

emen

t2

66^8

7(p

185

)

The

exp

erim

entw

asb

ased

onsi

xdi

ffere

ntve

rsio

nsof

each

ofth

etw

oca

ses

The

six

diff

eren

tver

sion

sof

the

case

sva

ried

inin

form

atio

nco

mpl

exity

Eac

hde

cisi

on-m

aker

had

only

one

deci

sion

tom

ake

aton

lyon

ele

velo

fcom

plex

ity(p

p18

9^19

019

3)U

SA19

83

14To

dete

rmin

eso

me

ofth

efa

ctor

sth

atin

fluen

cead

min

istra

tors

rsquouse

ofev

alua

tive

data

inde

cisi

on-

mak

ing

and

toex

amin

eth

ety

pes

ofde

cisi

ons

(p7

9)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p7

9^80

)

Dire

ctor

sin

com

mun

itym

enta

lhea

lthce

ntre

sin

Flor

ida

and

Nor

thC

arol

ina

(p7

9)N

ˆ42

Con

duct

edst

udie

son

prog

ram

me

effe

ctiv

enes

sor

imp

actd

urin

gth

ela

st12

-m

onth

per

iod

(p8

1)

Per

ceiv

ed(i

ere

por

ted

)us

e(p

p80

^81)

Swed

en19

84^1

98515

(p1

52)

Diff

eren

tway

sof

usin

gso

cial

rese

arch

in

stru

men

tal

polit

ical

co

nflic

ten

light

enm

ent

inte

ract

ive

and

tact

ical

(pp

141

143^

144)

Inte

rvie

ws

(p1

52)

Soc

ialw

orke

rs

man

agem

entd

irect

ors

loca

land

natio

nalp

oliti

cian

sin

15Sw

edis

hci

ties

and

com

mun

es(p

152

)N

ˆ90

(p1

52)

Doc

umen

tsth

atth

ere

spon

dent

sre

ferr

edto

Th

ese

docu

men

tsw

ere

also

anal

ysed

by

the

auth

ors

(p1

52)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

52)

USA

1990

16To

dete

rmin

ew

hatt

ype

ofin

form

atio

nd

ecis

ion-

mak

ers

use

and

how

ideo

log

yan

din

tere

sts

influ

ence

the

use

ofin

form

atio

n(p

109

)

Pers

onal

sem

i-stru

ctur

edin

terv

iew

s(p

112

)Lo

calp

lann

ing

com

mitt

ees

and

gove

rnm

ente

mp

loye

esin

two

rura

lcom

mun

ities

inth

eP

acifi

cN

orth

wes

t(p

110

)N

ˆ27

(p1

10)

Sour

ces

ofin

form

atio

nus

edin

com

mun

ityhe

alth

care

plan

ning

and

the

asse

ssm

ento

fthe

stra

tegi

cpl

anni

ngp

roce

ss(p

112

)

Per

ceiv

edus

e(p

112

)

USA

1991

17

Wha

tfac

tors

influ

ence

the

use

ofev

alua

tion

rese

arch

(on

heal

than

dhu

man

serv

ices

)in

cong

ress

(p5

08)

Stru

ctur

edte

lep

hone

inte

rvie

ws

(p5

13)

C

ongr

ess

mem

bers

and

thei

rsta

ff(p

512

)N

ˆ10

0Ea

chde

cisi

on-m

aker

chos

eon

est

udy

they

wer

efa

mili

arw

ith(p

515

)

Per

ceiv

edus

e(p

515

)E

valu

atio

nst

udie

sfro

mth

eD

epar

tmen

tofH

ealth

and

Hum

anSe

rvic

esth

atd

eal

with

natio

nalh

ealth

pro

gram

mes

(pp

507

515

)

Use

isde

fined

here

asbo

thin

stru

men

tal(

dire

ctim

ple

men

tatio

n)c

once

ptua

l(it

chan

ged

the

way

ofth

inki

ng)a

ndpe

rsua

sive

(use

dto

conv

ince

othe

rsp

516

)

The

inte

rvie

ws

last

ed15

^40

min

utes

(p5

14)

UK

1991

18U

sean

dre

ason

sfo

ruse

not

use

ofth

eE

iexclect

ive

Hea

lthC

are

Bul

letin

(pp

51^

54)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p5

0^51

)

Dis

trict

gen

eral

man

ager

sd

irect

ors

ofpu

blic

heal

than

dd

irect

ors

ofpl

anni

ng

com

mis

sion

ing

inU

Kd

istri

cthe

alth

auth

oriti

esor

com

mis

sion

ing

cons

ortia

(p5

0)N

ˆ17

2(p

51)

Eiexcle

ctiv

eH

ealth

Car

eB

ulle

tinN

o1

1991

on

scre

enin

gfo

rost

eop

oros

isto

prev

entf

ract

ures

(p5

0)

Per

ceiv

edus

eof

info

rmat

ion

and

hypo

thet

ical

dec

isio

ns

chan

geof

opin

ion

and

conf

iden

ceto

acto

nin

form

atio

nin

the

bulle

tin(p

p52

^54)

Res

pon

dent

sw

ere

que

stio

ned

abou

tabu

lletin

that

had

been

prev

ious

lypo

sted

toth

em(p

p51

52)

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1993

^199

419Th

ero

leof

rese

arch

info

rmat

ion

inp

olic

yde

velo

pmen

t(p

29)

Que

stio

nnai

res

(p1

3)

open

-end

edte

leph

one

inte

rvie

ws

(p1

4)an

dse

mi-

stru

ctur

edte

leph

one

inte

rvie

ws

(p1

4)

Key

info

rman

ts

inst

ates

that

chan

ged

cost

-sha

ring

pol

icie

sfo

rpre

scrip

tion

reim

burs

emen

tlim

itsan

dco

-pay

men

ts(p

14)

N

în

adeq

uate

lyd

escr

ibed

Res

earc

hp

ublic

atio

nsan

dco

mm

ents

(p2

5)P

erce

ived

retro

spec

tive

dec

isio

ns(p

25)

Onl

y2^

3pa

ges

outo

f21

wer

ere

leva

nt(p

25f

f)I

nfor

man

tsin

clud

edM

edic

aid

drug

prog

ram

me

adm

inis

trato

rs

dire

ctor

sor

polic

yan

alys

ts

phar

mac

yco

nsul

tant

san

dle

gisl

ator

s(p

15)

UK

1994

^199

5202

1`T

oid

entif

yfa

ctor

sw

hich

faci

litat

eor

imp

ede

evid

ence

-bas

edp

olic

y-m

akin

grsquo(

p2

9)

In-d

epth

inte

rvie

ws

(usi

nga

topi

cgu

ide)

(p2

9an

dre

spon

seto

ale

tter)

Lead

polic

y-m

aker

sG

Psan

dre

sear

cher

sin

nine

diff

eren

tcas

est

udie

sat

alo

call

evel

inth

eN

HS

(pp

293

1)N

ˆ28

Soci

alre

sear

ch(p

42

inth

epr

ojec

trep

ort)

Per

ceiv

edde

cisi

ons

Polic

yon

the

inte

rface

bet

wee

nR

ampD

and

com

mis

sion

ing

atth

ehe

alth

auth

ority

leve

l(p

30)

Mex

ico

1994

^199

522`T

here

latio

nshi

pbe

twee

nhe

alth

rese

arch

(and

rese

arch

ers)

and

heal

thpo

licie

s(a

ndpo

licy-

mak

ers)

info

urve

rtica

lhe

alth

prog

ram

mes

inM

exic

orsquo(p

104

)

Ope

n-en

ded

(in-

dept

hrsquo)

inte

rvie

ws

with

aco

mm

onin

terv

iew

prot

ocol

(p1

06)

`Res

earc

hers

and

polic

y-m

aker

sfro

mdi

ffere

ntin

stitu

tions

and

leve

lsof

resp

onsi

bilit

yrsquo(p

103

)N

ˆ67

Use

ofre

sear

chpu

blic

atio

ns`w

here

we

knew

that

ther

ew

asat

leas

tso

me

inte

ract

ion

bet

wee

nre

sear

cher

san

dde

cisi

on-

mak

ersrsquo

(p1

05)

Per

ceiv

ed

use

ofev

iden

ceco

ncer

ning

AID

Sch

oler

afa

mily

pla

nnin

gan

dim

mun

isat

ion

ing

over

nmen

talp

rogr

amm

es(p

103

)

Per

sona

lcom

mun

icat

ion

gav

ead

ditio

nali

nfor

mat

ion

onth

isis

sue

It

isd

iffic

ultt

od

isce

rnw

heth

erth

ede

cisi

ons

wer

ep

erce

ived

orhy

pot

hetic

als

ince

ther

ear

eno

actu

alde

cisi

ons

men

tione

d

Aus

tralia

1995

23

Dec

isio

n-m

aker

srsquoaw

aren

ess

and

use

ofec

onom

icev

alua

tions

inhe

alth

care

(p1

03)

Inte

rvie

ws

open

-end

edan

dst

ruct

ured

(not

stat

edif

post

alt

elep

hone

orfa

ce-to

-fa

ce)(

p10

5)

Sen

iorm

anag

ers

inth

eFe

der

alD

epar

tmen

tof

Hea

lthH

ousi

ngan

dC

omm

unity

Serv

ices

New

Sou

thW

ales

Stat

eth

eH

ealth

Dep

artm

ent(

p10

4)

Nˆ

34(p

105

)

Use

ofec

onom

icev

alua

tions

forp

ricin

gon

drug

ssc

reen

ing

radi

olog

ytra

nspl

ants

and

scan

ning

(p1

05)

`Per

ceiv

edus

eof

econ

omic

eval

uatio

nsrsquo(

p1

04)

USA

1996

24

and

USA

1997

25

Tode

term

ine

whe

ther

the

proc

ess

of(s

ocia

lsci

ence

)in

form

atio

nut

ilisa

tion

varie

sac

ross

diff

eren

tare

as(i

ese

rvic

epr

ovis

ion

and

finan

cing

)(pp

16^

17)

Stru

ctur

edin

terv

iew

s(p

16)

Fe

der

als

tate

and

loca

lp

olic

y-m

aker

sat

diff

eren

tle

vels

deal

ing

with

finan

cean

dse

rvic

es(p

16)

N

ˆ47

9(p

16)

No

spec

ific

repo

rtb

utin

foty

persquor

efer

topr

ogra

mm

eev

alua

tion

dem

onst

ratio

np

roje

ct

stat

istic

ald

ata

and

polic

yan

alys

is(p

34)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

7)O

pen-

ende

din

terv

iew

sw

ere

used

toob

tain

basi

cb

ackg

roun

din

form

atio

nab

outt

hep

olic

ypr

oces

sTh

est

ruct

ured

inte

rvie

ws

built

onth

eop

en-e

nded

inte

rvie

ws

(p3

3)Pa

kist

an19

9826

The

role

that

rese

arch

pla

ysin

expe

rienc

esin

linki

ngre

sear

chw

ithp

olic

y(p

77)

Ope

n-en

ded

in-d

epth

inte

rvie

ws

(p7

7)D

ecis

ion-

mak

ers

from

gov

ernm

ent

the

cent

ral

min

istry

ofhe

alth

the

pla

nnin

gco

mm

issi

onan

dp

rovi

ncia

lhea

lthau

thor

ities

(p7

7)N

ˆ16

(p7

7)

Not

stat

edG

ener

alpe

rcep

tions

C

lose

stfo

cus

onch

ildhe

alth

prob

lem

ssu

chas

co-tr

imox

azol

em

ater

nal

mor

talit

yvi

tam

inA

b

reas

tfeed

ing

and

pol

io(p

p77

79)

UK

1998

27

The

imp

actt

hatt

heE

iexclect

ive

Hea

lthca

reB

ulle

tinha

sin

side

the

NH

Sor

gan

isat

ion

(p1

)

Tele

phon

ein

terv

iew

sst

ruct

ure

notm

entio

ned

(p1

)

Hea

lthau

thor

ities

(p

2)

Nˆ

34Tw

osp

ecifi

cbu

lletin

sJu

ne19

97an

dO

ctob

er19

97(p

1)

Hyp

othe

tical

deci

sion

s(e

g

itin

fluen

ced

the

dev

elop

men

tofo

urpo

licy

alth

oug

hth

isha

sbe

eng

oing

onfo

rsom

etim

ersquo)

(p1

0)

The

date

isno

tcle

arly

stat

edb

utit

is19

98or

1999

Tr

usts

and

GPs

are

incl

uded

inth

est

udy

buta

reno

trel

evan

tin

this

revi

ew

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



































Tab

le1

Cha

ract

eris

tics

ofin

clud

edst

udie

s

Co

un

try

year

cond

uct

ed(

year

pu

blis

hed

)re

fere

nce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fev

iden

ce(f

rom

whi

chd

ecis

ion

sw

ere

mad

e)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

The

Net

herla

nds

1969

^197

17 Th

eco

nditi

ons

and

func

tions

influ

enci

ngut

ilisa

tion

ofso

cial

pol

icy

rese

arch

(p4

9)

Sem

i-stru

ctur

edsu

rvey

s(p

50)

Res

earc

hers

and

polic

y-m

aker

sth

atw

ere

resp

onsi

ble

for

inco

rpor

atin

gth

ere

sear

chre

sults

(p5

0)N

ˆno

tst

ated

120

rese

arch

proj

ects

on`c

lient

-orie

nted

rsquosoc

ial

rese

arch

with

inin

dust

rial

and

labo

urre

latio

ns

regi

onal

and

urb

anpl

anni

ngs

ocia

lwel

fare

and

publ

iche

alth

(p5

0)

Per

ceiv

ed(r

etro

spec

tive)

dec

isio

ns(p

50)

Rev

ised

vers

ion

ofpa

per

pres

ente

din

1979

(p4

9)

40ou

toft

he12

0pr

ojec

tsw

ere

onso

cial

wel

fare

and

publ

iche

alth

US

A19

738

Wha

twas

done

wha

the

lped

and

wha

thin

dere

dth

esu

cces

sfo

rten

rese

arch

proj

ects

unde

rthe

Nat

iona

lIns

titut

eof

Men

tal

Hea

lth

(p1

41)

Pers

onal

inte

rvie

ws

(p1

41)

Prin

cipa

linv

estig

ator

oth

erst

affm

emb

ers

adm

inis

trato

rsan

dpe

rson

sm

ostl

ikel

yto

be

utili

sers

(p1

41)

Nˆ

notg

iven

Five

proj

ects

rate

dlo

wan

dfiv

epr

ojec

tsra

ted

high

for

`suc

cess

rsquo(p

140

)

Per

ceiv

edus

eT

hein

terv

iew

sw

ere

don

ein

eith

er19

67or

1968

(pp

140^

141)

Su

cces

sde

fined

aspr

ojec

ttha

tw

asdi

rect

edto

the

orig

inal

obje

ctiv

ese

mbo

died

ina

clea

ran

dco

gent

rep

ort

cand

iden

ough

tobe

repl

icab

lea

ndth

atth

efin

din

gsw

ere

diss

emin

ated

adeq

uate

ly(p

140

)U

SA

1973

^197

49(p

vii)

and

USA

1973

^197

410So

cial

scie

nce

rese

arch

utili

satio

nan

dpo

licy

form

atio

n(p

vii)

Aw

aren

ess

ofre

sear

chk

now

ledg

eof

sour

ces

info

rmat

ion

retri

eval

pra

ctic

ein

tere

stin

soci

alin

dica

tors

eva

luat

ion

ofw

orth

and

obje

ctiv

ity

met

hods

and

mea

sure

men

tpr

oced

ures

deg

ree

ofco

nfid

ence

onfin

ding

sat

titud

esab

outu

sef

acto

rsth

atin

fluen

ceus

e(p

ix)

Pers

onal

inte

rvie

ws

open

-en

ded

and

stru

ctur

ed(p

pv

ix)

Upp

er-le

veld

ecis

ion-

mak

ers

inth

eex

ecut

ive

bran

chof

the

US

fede

ral

gove

rnm

ent(

ppv

viii)

31

depu

tyun

ders

ecre

tarie

s54

inst

itute

dire

ctor

s67

depu

tyas

sist

antd

irect

ors

52ag

ency

pers

onne

lN

ˆ20

4(p

v)

575

repo

rted

inst

ance

sof

use

whe

re38

5in

volv

edpr

imar

yre

sear

chso

urce

s(p

8)

94

wer

eei

ther

fund

edby

the

gove

rnm

ent

cond

ucte

dby

the

gove

rnm

ento

rbot

h(p

8)

Per

ceiv

ed(s

elf-r

epor

ted)

use

with

spec

ific

follo

w-u

pq

uest

ions

whe

nus

ew

asre

porte

d(p

1)

15

ofth

eus

eof

know

ledg

ew

ason

heal

thor

wel

fare

prob

lem

s(p

5)

`Mos

toft

heex

ampl

esre

spon

dent

sof

fere

dto

illust

rate

know

ledg

eap

plic

atio

nsre

ally

invo

lved

the

appl

icat

ion

ofse

cond

ary

sour

cein

form

atio

nrsquo(p

187

)

US

A19

77

11`T

oex

amin

eth

ena

ture

and

degr

eeof

utiliz

atio

nof

fede

rale

valu

atio

nre

sear

chrsquo

(p1

43)

Op

en-e

nded

inte

rvie

ws

with

diffe

rent

ques

tions

for

the

deci

sion

-mak

eran

dth

eev

alua

tor(

p1

43)

Thre

eke

yin

form

ants

from

each

ofth

e20

stud

ies

wer

ein

terv

iew

eda

proj

ect

offic

ert

hed

ecis

ion-

mak

er(g

over

nmen

tlev

el)a

ndth

eev

alua

tor(

p14

3)N

ˆ60

(p1

43)

20na

tiona

lhea

lthca

reev

alua

tions

(cas

est

udie

s)w

ithso

me

syst

emat

icd

ata

colle

ctio

nm

ade

betw

een

1971

and

1974

bas

edon

11fa

ctor

sid

entif

ied

byth

eau

thor

s(p

p14

2^14

3)

Per

ceiv

edan

dse

lf-d

efin

edus

e(p

144

)H

ealth

train

ing

lab

orat

ory

prof

icie

ncy

neig

hbou

rhoo

dhe

alth

cent

res

heal

thse

rvic

esd

eliv

ery

syst

ems

alco

holis

ml

oan

forg

iven

ess

and

train

ing

wor

ksho

p(p

p14

2^14

3)U

SA

1980

2 an

dU

SA19

8012

Polic

y-m

aker

srsquope

rcep

tions

ofth

eus

eful

ness

ofso

cial

scie

nces

rese

arch

(pp

165

302)

Pers

onal

inte

rvie

ws

both

open

-end

edan

dst

ruct

ured

(pp

168

171

303)

51fe

dera

l52

stat

e(fr

omte

nst

ates

)and

52lo

cal

offic

iald

ecis

ion-

mak

ers

inup

per-

leve

lpos

ition

s(p

p16

730

3)N

ˆ15

5

Each

dec

isio

n-m

aker

read

two

abst

ract

s(tw

o-pa

gers

)ou

tof5

0fo

ract

uals

ocia

lsc

ienc

ere

sear

chre

ports

(p

p1

683

03)

Hyp

othe

tical

deci

sion

s(i

e`re

leva

ntto

the

issu

esyo

urof

fice

deal

sw

ithrsquo)

onm

enta

lhe

alth

alc

ohol

and

drug

abus

e

(pp

166

303

)

Not

stat

edb

utpu

blis

hed

Ap

ril19

80R

eprin

ted

from

Am

eric

anP

sych

olog

ist 1

981

Bas

edon

the

sam

ere

sear

chp

roje

ctas

US

A19

8012

(but

par

tlyon

othe

rpar

tsof

the

data

)

The

deci

sion

-mak

ers

wer

eg

iven

diffe

rent

èm

piri

calrsquo

inte

rven

tions

(ie

two-

page

rs)a

ndar

eth

eref

ore

diffi

cult

toco

mpa

reon

the

effe

ctva

riabl

esco

ntin

ued



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1982

13

Toob

serv

eth

eim

pac

ttha

tde

cisi

onco

mp

lexi

tya

ndam

ount

ofin

form

atio

nav

aila

ble

had

onth

eq

ualit

yof

the

deci

sion

proc

ess

(p1

87)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edqu

estio

n-na

ires

com

bine

dw

itha

cont

rolle

dex

perim

ent

Con

seq

uenc

esof

com

plex

ityw

ere

exam

ined

byef

fect

sof

info

rmat

ion

load

and

num

bero

fde

cisi

onal

tern

ativ

es(p

p1

871

931

94)

Exp

erie

nced

pub

lican

dp

rivat

ese

ctor

man

ager

sfro

msi

xm

id-c

aree

rp

rog

ram

mes

atH

arva

rd

MIT

Sta

nfor

dan

dth

eW

ashi

ngto

nP

ublic

Affa

irsC

ente

r(pp

187

188

)N

ˆ13

2(p

188

)

Each

dec

isio

n-m

aker

rece

ived

alo

ngm

emor

and

umde

scrib

ing

apr

oble

mp

rese

ntin

gev

iden

ceus

eful

inan

alys

ing

the

prob

lem

and

pose

da

dec

isio

nto

bem

ade

base

don

real

peo

ple

and

even

ts(p

188

)

Two

hypo

thet

ical

dec

isio

ns

one

onth

est

atersquo

sD

epar

t-m

ento

fHea

lthan

dR

ehab

i-lit

ativ

ese

rvic

esan

don

eon

am

etha

done

mai

nten

ance

pro

gram

me

inN

ewYo

rkC

ity(p

p1

88^1

89)

Dec

isio

nha

dto

bem

ade

from

avai

labl

ein

form

atio

nw

ithin

1ho

ur45

min

utes

(p1

93)

Rep

rinte

dfro

mth

eJo

urna

lof

Pol

icy

Ana

lysi

san

dM

anag

emen

t2

66^8

7(p

185

)

The

exp

erim

entw

asb

ased

onsi

xdi

ffere

ntve

rsio

nsof

each

ofth

etw

oca

ses

The

six

diff

eren

tver

sion

sof

the

case

sva

ried

inin

form

atio

nco

mpl

exity

Eac

hde

cisi

on-m

aker

had

only

one

deci

sion

tom

ake

aton

lyon

ele

velo

fcom

plex

ity(p

p18

9^19

019

3)U

SA19

83

14To

dete

rmin

eso

me

ofth

efa

ctor

sth

atin

fluen

cead

min

istra

tors

rsquouse

ofev

alua

tive

data

inde

cisi

on-

mak

ing

and

toex

amin

eth

ety

pes

ofde

cisi

ons

(p7

9)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p7

9^80

)

Dire

ctor

sin

com

mun

itym

enta

lhea

lthce

ntre

sin

Flor

ida

and

Nor

thC

arol

ina

(p7

9)N

ˆ42

Con

duct

edst

udie

son

prog

ram

me

effe

ctiv

enes

sor

imp

actd

urin

gth

ela

st12

-m

onth

per

iod

(p8

1)

Per

ceiv

ed(i

ere

por

ted

)us

e(p

p80

^81)

Swed

en19

84^1

98515

(p1

52)

Diff

eren

tway

sof

usin

gso

cial

rese

arch

in

stru

men

tal

polit

ical

co

nflic

ten

light

enm

ent

inte

ract

ive

and

tact

ical

(pp

141

143^

144)

Inte

rvie

ws

(p1

52)

Soc

ialw

orke

rs

man

agem

entd

irect

ors

loca

land

natio

nalp

oliti

cian

sin

15Sw

edis

hci

ties

and

com

mun

es(p

152

)N

ˆ90

(p1

52)

Doc

umen

tsth

atth

ere

spon

dent

sre

ferr

edto

Th

ese

docu

men

tsw

ere

also

anal

ysed

by

the

auth

ors

(p1

52)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

52)

USA

1990

16To

dete

rmin

ew

hatt

ype

ofin

form

atio

nd

ecis

ion-

mak

ers

use

and

how

ideo

log

yan

din

tere

sts

influ

ence

the

use

ofin

form

atio

n(p

109

)

Pers

onal

sem

i-stru

ctur

edin

terv

iew

s(p

112

)Lo

calp

lann

ing

com

mitt

ees

and

gove

rnm

ente

mp

loye

esin

two

rura

lcom

mun

ities

inth

eP

acifi

cN

orth

wes

t(p

110

)N

ˆ27

(p1

10)

Sour

ces

ofin

form

atio

nus

edin

com

mun

ityhe

alth

care

plan

ning

and

the

asse

ssm

ento

fthe

stra

tegi

cpl

anni

ngp

roce

ss(p

112

)

Per

ceiv

edus

e(p

112

)

USA

1991

17

Wha

tfac

tors

influ

ence

the

use

ofev

alua

tion

rese

arch

(on

heal

than

dhu

man

serv

ices

)in

cong

ress

(p5

08)

Stru

ctur

edte

lep

hone

inte

rvie

ws

(p5

13)

C

ongr

ess

mem

bers

and

thei

rsta

ff(p

512

)N

ˆ10

0Ea

chde

cisi

on-m

aker

chos

eon

est

udy

they

wer

efa

mili

arw

ith(p

515

)

Per

ceiv

edus

e(p

515

)E

valu

atio

nst

udie

sfro

mth

eD

epar

tmen

tofH

ealth

and

Hum

anSe

rvic

esth

atd

eal

with

natio

nalh

ealth

pro

gram

mes

(pp

507

515

)

Use

isde

fined

here

asbo

thin

stru

men

tal(

dire

ctim

ple

men

tatio

n)c

once

ptua

l(it

chan

ged

the

way

ofth

inki

ng)a

ndpe

rsua

sive

(use

dto

conv

ince

othe

rsp

516

)

The

inte

rvie

ws

last

ed15

^40

min

utes

(p5

14)

UK

1991

18U

sean

dre

ason

sfo

ruse

not

use

ofth

eE

iexclect

ive

Hea

lthC

are

Bul

letin

(pp

51^

54)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p5

0^51

)

Dis

trict

gen

eral

man

ager

sd

irect

ors

ofpu

blic

heal

than

dd

irect

ors

ofpl

anni

ng

com

mis

sion

ing

inU

Kd

istri

cthe

alth

auth

oriti

esor

com

mis

sion

ing

cons

ortia

(p5

0)N

ˆ17

2(p

51)

Eiexcle

ctiv

eH

ealth

Car

eB

ulle

tinN

o1

1991

on

scre

enin

gfo

rost

eop

oros

isto

prev

entf

ract

ures

(p5

0)

Per

ceiv

edus

eof

info

rmat

ion

and

hypo

thet

ical

dec

isio

ns

chan

geof

opin

ion

and

conf

iden

ceto

acto

nin

form

atio

nin

the

bulle

tin(p

p52

^54)

Res

pon

dent

sw

ere

que

stio

ned

abou

tabu

lletin

that

had

been

prev

ious

lypo

sted

toth

em(p

p51

52)

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1993

^199

419Th

ero

leof

rese

arch

info

rmat

ion

inp

olic

yde

velo

pmen

t(p

29)

Que

stio

nnai

res

(p1

3)

open

-end

edte

leph

one

inte

rvie

ws

(p1

4)an

dse

mi-

stru

ctur

edte

leph

one

inte

rvie

ws

(p1

4)

Key

info

rman

ts

inst

ates

that

chan

ged

cost

-sha

ring

pol

icie

sfo

rpre

scrip

tion

reim

burs

emen

tlim

itsan

dco

-pay

men

ts(p

14)

N

în

adeq

uate

lyd

escr

ibed

Res

earc

hp

ublic

atio

nsan

dco

mm

ents

(p2

5)P

erce

ived

retro

spec

tive

dec

isio

ns(p

25)

Onl

y2^

3pa

ges

outo

f21

wer

ere

leva

nt(p

25f

f)I

nfor

man

tsin

clud

edM

edic

aid

drug

prog

ram

me

adm

inis

trato

rs

dire

ctor

sor

polic

yan

alys

ts

phar

mac

yco

nsul

tant

san

dle

gisl

ator

s(p

15)

UK

1994

^199

5202

1`T

oid

entif

yfa

ctor

sw

hich

faci

litat

eor

imp

ede

evid

ence

-bas

edp

olic

y-m

akin

grsquo(

p2

9)

In-d

epth

inte

rvie

ws

(usi

nga

topi

cgu

ide)

(p2

9an

dre

spon

seto

ale

tter)

Lead

polic

y-m

aker

sG

Psan

dre

sear

cher

sin

nine

diff

eren

tcas

est

udie

sat

alo

call

evel

inth

eN

HS

(pp

293

1)N

ˆ28

Soci

alre

sear

ch(p

42

inth

epr

ojec

trep

ort)

Per

ceiv

edde

cisi

ons

Polic

yon

the

inte

rface

bet

wee

nR

ampD

and

com

mis

sion

ing

atth

ehe

alth

auth

ority

leve

l(p

30)

Mex

ico

1994

^199

522`T

here

latio

nshi

pbe

twee

nhe

alth

rese

arch

(and

rese

arch

ers)

and

heal

thpo

licie

s(a

ndpo

licy-

mak

ers)

info

urve

rtica

lhe

alth

prog

ram

mes

inM

exic

orsquo(p

104

)

Ope

n-en

ded

(in-

dept

hrsquo)

inte

rvie

ws

with

aco

mm

onin

terv

iew

prot

ocol

(p1

06)

`Res

earc

hers

and

polic

y-m

aker

sfro

mdi

ffere

ntin

stitu

tions

and

leve

lsof

resp

onsi

bilit

yrsquo(p

103

)N

ˆ67

Use

ofre

sear

chpu

blic

atio

ns`w

here

we

knew

that

ther

ew

asat

leas

tso

me

inte

ract

ion

bet

wee

nre

sear

cher

san

dde

cisi

on-

mak

ersrsquo

(p1

05)

Per

ceiv

ed

use

ofev

iden

ceco

ncer

ning

AID

Sch

oler

afa

mily

pla

nnin

gan

dim

mun

isat

ion

ing

over

nmen

talp

rogr

amm

es(p

103

)

Per

sona

lcom

mun

icat

ion

gav

ead

ditio

nali

nfor

mat

ion

onth

isis

sue

It

isd

iffic

ultt

od

isce

rnw

heth

erth

ede

cisi

ons

wer

ep

erce

ived

orhy

pot

hetic

als

ince

ther

ear

eno

actu

alde

cisi

ons

men

tione

d

Aus

tralia

1995

23

Dec

isio

n-m

aker

srsquoaw

aren

ess

and

use

ofec

onom

icev

alua

tions

inhe

alth

care

(p1

03)

Inte

rvie

ws

open

-end

edan

dst

ruct

ured

(not

stat

edif

post

alt

elep

hone

orfa

ce-to

-fa

ce)(

p10

5)

Sen

iorm

anag

ers

inth

eFe

der

alD

epar

tmen

tof

Hea

lthH

ousi

ngan

dC

omm

unity

Serv

ices

New

Sou

thW

ales

Stat

eth

eH

ealth

Dep

artm

ent(

p10

4)

Nˆ

34(p

105

)

Use

ofec

onom

icev

alua

tions

forp

ricin

gon

drug

ssc

reen

ing

radi

olog

ytra

nspl

ants

and

scan

ning

(p1

05)

`Per

ceiv

edus

eof

econ

omic

eval

uatio

nsrsquo(

p1

04)

USA

1996

24

and

USA

1997

25

Tode

term

ine

whe

ther

the

proc

ess

of(s

ocia

lsci

ence

)in

form

atio

nut

ilisa

tion

varie

sac

ross

diff

eren

tare

as(i

ese

rvic

epr

ovis

ion

and

finan

cing

)(pp

16^

17)

Stru

ctur

edin

terv

iew

s(p

16)

Fe

der

als

tate

and

loca

lp

olic

y-m

aker

sat

diff

eren

tle

vels

deal

ing

with

finan

cean

dse

rvic

es(p

16)

N

ˆ47

9(p

16)

No

spec

ific

repo

rtb

utin

foty

persquor

efer

topr

ogra

mm

eev

alua

tion

dem

onst

ratio

np

roje

ct

stat

istic

ald

ata

and

polic

yan

alys

is(p

34)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

7)O

pen-

ende

din

terv

iew

sw

ere

used

toob

tain

basi

cb

ackg

roun

din

form

atio

nab

outt

hep

olic

ypr

oces

sTh

est

ruct

ured

inte

rvie

ws

built

onth

eop

en-e

nded

inte

rvie

ws

(p3

3)Pa

kist

an19

9826

The

role

that

rese

arch

pla

ysin

expe

rienc

esin

linki

ngre

sear

chw

ithp

olic

y(p

77)

Ope

n-en

ded

in-d

epth

inte

rvie

ws

(p7

7)D

ecis

ion-

mak

ers

from

gov

ernm

ent

the

cent

ral

min

istry

ofhe

alth

the

pla

nnin

gco

mm

issi

onan

dp

rovi

ncia

lhea

lthau

thor

ities

(p7

7)N

ˆ16

(p7

7)

Not

stat

edG

ener

alpe

rcep

tions

C

lose

stfo

cus

onch

ildhe

alth

prob

lem

ssu

chas

co-tr

imox

azol

em

ater

nal

mor

talit

yvi

tam

inA

b

reas

tfeed

ing

and

pol

io(p

p77

79)

UK

1998

27

The

imp

actt

hatt

heE

iexclect

ive

Hea

lthca

reB

ulle

tinha

sin

side

the

NH

Sor

gan

isat

ion

(p1

)

Tele

phon

ein

terv

iew

sst

ruct

ure

notm

entio

ned

(p1

)

Hea

lthau

thor

ities

(p

2)

Nˆ

34Tw

osp

ecifi

cbu

lletin

sJu

ne19

97an

dO

ctob

er19

97(p

1)

Hyp

othe

tical

deci

sion

s(e

g

itin

fluen

ced

the

dev

elop

men

tofo

urpo

licy

alth

oug

hth

isha

sbe

eng

oing

onfo

rsom

etim

ersquo)

(p1

0)

The

date

isno

tcle

arly

stat

edb

utit

is19

98or

1999

Tr

usts

and

GPs

are

incl

uded

inth

est

udy

buta

reno

trel

evan

tin

this

revi

ew

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



Tab

le1

Cha

ract

eris

tics

ofin

clud

edst

udie

s

Co

un

try

year

cond

uct

ed(

year

pu

blis

hed

)re

fere

nce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fev

iden

ce(f

rom

whi

chd

ecis

ion

sw

ere

mad

e)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

The

Net

herla

nds

1969

^197

17 Th

eco

nditi

ons

and

func

tions

influ

enci

ngut

ilisa

tion

ofso

cial

pol

icy

rese

arch

(p4

9)

Sem

i-stru

ctur

edsu

rvey

s(p

50)

Res

earc

hers

and

polic

y-m

aker

sth

atw

ere

resp

onsi

ble

for

inco

rpor

atin

gth

ere

sear

chre

sults

(p5

0)N

ˆno

tst

ated

120

rese

arch

proj

ects

on`c

lient

-orie

nted

rsquosoc

ial

rese

arch

with

inin

dust

rial

and

labo

urre

latio

ns

regi

onal

and

urb

anpl

anni

ngs

ocia

lwel

fare

and

publ

iche

alth

(p5

0)

Per

ceiv

ed(r

etro

spec

tive)

dec

isio

ns(p

50)

Rev

ised

vers

ion

ofpa

per

pres

ente

din

1979

(p4

9)

40ou

toft

he12

0pr

ojec

tsw

ere

onso

cial

wel

fare

and

publ

iche

alth

US

A19

738

Wha

twas

done

wha

the

lped

and

wha

thin

dere

dth

esu

cces

sfo

rten

rese

arch

proj

ects

unde

rthe

Nat

iona

lIns

titut

eof

Men

tal

Hea

lth

(p1

41)

Pers

onal

inte

rvie

ws

(p1

41)

Prin

cipa

linv

estig

ator

oth

erst

affm

emb

ers

adm

inis

trato

rsan

dpe

rson

sm

ostl

ikel

yto

be

utili

sers

(p1

41)

Nˆ

notg

iven

Five

proj

ects

rate

dlo

wan

dfiv

epr

ojec

tsra

ted

high

for

`suc

cess

rsquo(p

140

)

Per

ceiv

edus

eT

hein

terv

iew

sw

ere

don

ein

eith

er19

67or

1968

(pp

140^

141)

Su

cces

sde

fined

aspr

ojec

ttha

tw

asdi

rect

edto

the

orig

inal

obje

ctiv

ese

mbo

died

ina

clea

ran

dco

gent

rep

ort

cand

iden

ough

tobe

repl

icab

lea

ndth

atth

efin

din

gsw

ere

diss

emin

ated

adeq

uate

ly(p

140

)U

SA

1973

^197

49(p

vii)

and

USA

1973

^197

410So

cial

scie

nce

rese

arch

utili

satio

nan

dpo

licy

form

atio

n(p

vii)

Aw

aren

ess

ofre

sear

chk

now

ledg

eof

sour

ces

info

rmat

ion

retri

eval

pra

ctic

ein

tere

stin

soci

alin

dica

tors

eva

luat

ion

ofw

orth

and

obje

ctiv

ity

met

hods

and

mea

sure

men

tpr

oced

ures

deg

ree

ofco

nfid

ence

onfin

ding

sat

titud

esab

outu

sef

acto

rsth

atin

fluen

ceus

e(p

ix)

Pers

onal

inte

rvie

ws

open

-en

ded

and

stru

ctur

ed(p

pv

ix)

Upp

er-le

veld

ecis

ion-

mak

ers

inth

eex

ecut

ive

bran

chof

the

US

fede

ral

gove

rnm

ent(

ppv

viii)

31

depu

tyun

ders

ecre

tarie

s54

inst

itute

dire

ctor

s67

depu

tyas

sist

antd

irect

ors

52ag

ency

pers

onne

lN

ˆ20

4(p

v)

575

repo

rted

inst

ance

sof

use

whe

re38

5in

volv

edpr

imar

yre

sear

chso

urce

s(p

8)

94

wer

eei

ther

fund

edby

the

gove

rnm

ent

cond

ucte

dby

the

gove

rnm

ento

rbot

h(p

8)

Per

ceiv

ed(s

elf-r

epor

ted)

use

with

spec

ific

follo

w-u

pq

uest

ions

whe

nus

ew

asre

porte

d(p

1)

15

ofth

eus

eof

know

ledg

ew

ason

heal

thor

wel

fare

prob

lem

s(p

5)

`Mos

toft

heex

ampl

esre

spon

dent

sof

fere

dto

illust

rate

know

ledg

eap

plic

atio

nsre

ally

invo

lved

the

appl

icat

ion

ofse

cond

ary

sour

cein

form

atio

nrsquo(p

187

)

US

A19

77

11`T

oex

amin

eth

ena

ture

and

degr

eeof

utiliz

atio

nof

fede

rale

valu

atio

nre

sear

chrsquo

(p1

43)

Op

en-e

nded

inte

rvie

ws

with

diffe

rent

ques

tions

for

the

deci

sion

-mak

eran

dth

eev

alua

tor(

p1

43)

Thre

eke

yin

form

ants

from

each

ofth

e20

stud

ies

wer

ein

terv

iew

eda

proj

ect

offic

ert

hed

ecis

ion-

mak

er(g

over

nmen

tlev

el)a

ndth

eev

alua

tor(

p14

3)N

ˆ60

(p1

43)

20na

tiona

lhea

lthca

reev

alua

tions

(cas

est

udie

s)w

ithso

me

syst

emat

icd

ata

colle

ctio

nm

ade

betw

een

1971

and

1974

bas

edon

11fa

ctor

sid

entif

ied

byth

eau

thor

s(p

p14

2^14

3)

Per

ceiv

edan

dse

lf-d

efin

edus

e(p

144

)H

ealth

train

ing

lab

orat

ory

prof

icie

ncy

neig

hbou

rhoo

dhe

alth

cent

res

heal

thse

rvic

esd

eliv

ery

syst

ems

alco

holis

ml

oan

forg

iven

ess

and

train

ing

wor

ksho

p(p

p14

2^14

3)U

SA

1980

2 an

dU

SA19

8012

Polic

y-m

aker

srsquope

rcep

tions

ofth

eus

eful

ness

ofso

cial

scie

nces

rese

arch

(pp

165

302)

Pers

onal

inte

rvie

ws

both

open

-end

edan

dst

ruct

ured

(pp

168

171

303)

51fe

dera

l52

stat

e(fr

omte

nst

ates

)and

52lo

cal

offic

iald

ecis

ion-

mak

ers

inup

per-

leve

lpos

ition

s(p

p16

730

3)N

ˆ15

5

Each

dec

isio

n-m

aker

read

two

abst

ract

s(tw

o-pa

gers

)ou

tof5

0fo

ract

uals

ocia

lsc

ienc

ere

sear

chre

ports

(p

p1

683

03)

Hyp

othe

tical

deci

sion

s(i

e`re

leva

ntto

the

issu

esyo

urof

fice

deal

sw

ithrsquo)

onm

enta

lhe

alth

alc

ohol

and

drug

abus

e

(pp

166

303

)

Not

stat

edb

utpu

blis

hed

Ap

ril19

80R

eprin

ted

from

Am

eric

anP

sych

olog

ist 1

981

Bas

edon

the

sam

ere

sear

chp

roje

ctas

US

A19

8012

(but

par

tlyon

othe

rpar

tsof

the

data

)

The

deci

sion

-mak

ers

wer

eg

iven

diffe

rent

èm

piri

calrsquo

inte

rven

tions

(ie

two-

page

rs)a

ndar

eth

eref

ore

diffi

cult

toco

mpa

reon

the

effe

ctva

riabl

esco

ntin

ued



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1982

13

Toob

serv

eth

eim

pac

ttha

tde

cisi

onco

mp

lexi

tya

ndam

ount

ofin

form

atio

nav

aila

ble

had

onth

eq

ualit

yof

the

deci

sion

proc

ess

(p1

87)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edqu

estio

n-na

ires

com

bine

dw

itha

cont

rolle

dex

perim

ent

Con

seq

uenc

esof

com

plex

ityw

ere

exam

ined

byef

fect

sof

info

rmat

ion

load

and

num

bero

fde

cisi

onal

tern

ativ

es(p

p1

871

931

94)

Exp

erie

nced

pub

lican

dp

rivat

ese

ctor

man

ager

sfro

msi

xm

id-c

aree

rp

rog

ram

mes

atH

arva

rd

MIT

Sta

nfor

dan

dth

eW

ashi

ngto

nP

ublic

Affa

irsC

ente

r(pp

187

188

)N

ˆ13

2(p

188

)

Each

dec

isio

n-m

aker

rece

ived

alo

ngm

emor

and

umde

scrib

ing

apr

oble

mp

rese

ntin

gev

iden

ceus

eful

inan

alys

ing

the

prob

lem

and

pose

da

dec

isio

nto

bem

ade

base

don

real

peo

ple

and

even

ts(p

188

)

Two

hypo

thet

ical

dec

isio

ns

one

onth

est

atersquo

sD

epar

t-m

ento

fHea

lthan

dR

ehab

i-lit

ativ

ese

rvic

esan

don

eon

am

etha

done

mai

nten

ance

pro

gram

me

inN

ewYo

rkC

ity(p

p1

88^1

89)

Dec

isio

nha

dto

bem

ade

from

avai

labl

ein

form

atio

nw

ithin

1ho

ur45

min

utes

(p1

93)

Rep

rinte

dfro

mth

eJo

urna

lof

Pol

icy

Ana

lysi

san

dM

anag

emen

t2

66^8

7(p

185

)

The

exp

erim

entw

asb

ased

onsi

xdi

ffere

ntve

rsio

nsof

each

ofth

etw

oca

ses

The

six

diff

eren

tver

sion

sof

the

case

sva

ried

inin

form

atio

nco

mpl

exity

Eac

hde

cisi

on-m

aker

had

only

one

deci

sion

tom

ake

aton

lyon

ele

velo

fcom

plex

ity(p

p18

9^19

019

3)U

SA19

83

14To

dete

rmin

eso

me

ofth

efa

ctor

sth

atin

fluen

cead

min

istra

tors

rsquouse

ofev

alua

tive

data

inde

cisi

on-

mak

ing

and

toex

amin

eth

ety

pes

ofde

cisi

ons

(p7

9)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p7

9^80

)

Dire

ctor

sin

com

mun

itym

enta

lhea

lthce

ntre

sin

Flor

ida

and

Nor

thC

arol

ina

(p7

9)N

ˆ42

Con

duct

edst

udie

son

prog

ram

me

effe

ctiv

enes

sor

imp

actd

urin

gth

ela

st12

-m

onth

per

iod

(p8

1)

Per

ceiv

ed(i

ere

por

ted

)us

e(p

p80

^81)

Swed

en19

84^1

98515

(p1

52)

Diff

eren

tway

sof

usin

gso

cial

rese

arch

in

stru

men

tal

polit

ical

co

nflic

ten

light

enm

ent

inte

ract

ive

and

tact

ical

(pp

141

143^

144)

Inte

rvie

ws

(p1

52)

Soc

ialw

orke

rs

man

agem

entd

irect

ors

loca

land

natio

nalp

oliti

cian

sin

15Sw

edis

hci

ties

and

com

mun

es(p

152

)N

ˆ90

(p1

52)

Doc

umen

tsth

atth

ere

spon

dent

sre

ferr

edto

Th

ese

docu

men

tsw

ere

also

anal

ysed

by

the

auth

ors

(p1

52)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

52)

USA

1990

16To

dete

rmin

ew

hatt

ype

ofin

form

atio

nd

ecis

ion-

mak

ers

use

and

how

ideo

log

yan

din

tere

sts

influ

ence

the

use

ofin

form

atio

n(p

109

)

Pers

onal

sem

i-stru

ctur

edin

terv

iew

s(p

112

)Lo

calp

lann

ing

com

mitt

ees

and

gove

rnm

ente

mp

loye

esin

two

rura

lcom

mun

ities

inth

eP

acifi

cN

orth

wes

t(p

110

)N

ˆ27

(p1

10)

Sour

ces

ofin

form

atio

nus

edin

com

mun

ityhe

alth

care

plan

ning

and

the

asse

ssm

ento

fthe

stra

tegi

cpl

anni

ngp

roce

ss(p

112

)

Per

ceiv

edus

e(p

112

)

USA

1991

17

Wha

tfac

tors

influ

ence

the

use

ofev

alua

tion

rese

arch

(on

heal

than

dhu

man

serv

ices

)in

cong

ress

(p5

08)

Stru

ctur

edte

lep

hone

inte

rvie

ws

(p5

13)

C

ongr

ess

mem

bers

and

thei

rsta

ff(p

512

)N

ˆ10

0Ea

chde

cisi

on-m

aker

chos

eon

est

udy

they

wer

efa

mili

arw

ith(p

515

)

Per

ceiv

edus

e(p

515

)E

valu

atio

nst

udie

sfro

mth

eD

epar

tmen

tofH

ealth

and

Hum

anSe

rvic

esth

atd

eal

with

natio

nalh

ealth

pro

gram

mes

(pp

507

515

)

Use

isde

fined

here

asbo

thin

stru

men

tal(

dire

ctim

ple

men

tatio

n)c

once

ptua

l(it

chan

ged

the

way

ofth

inki

ng)a

ndpe

rsua

sive

(use

dto

conv

ince

othe

rsp

516

)

The

inte

rvie

ws

last

ed15

^40

min

utes

(p5

14)

UK

1991

18U

sean

dre

ason

sfo

ruse

not

use

ofth

eE

iexclect

ive

Hea

lthC

are

Bul

letin

(pp

51^

54)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p5

0^51

)

Dis

trict

gen

eral

man

ager

sd

irect

ors

ofpu

blic

heal

than

dd

irect

ors

ofpl

anni

ng

com

mis

sion

ing

inU

Kd

istri

cthe

alth

auth

oriti

esor

com

mis

sion

ing

cons

ortia

(p5

0)N

ˆ17

2(p

51)

Eiexcle

ctiv

eH

ealth

Car

eB

ulle

tinN

o1

1991

on

scre

enin

gfo

rost

eop

oros

isto

prev

entf

ract

ures

(p5

0)

Per

ceiv

edus

eof

info

rmat

ion

and

hypo

thet

ical

dec

isio

ns

chan

geof

opin

ion

and

conf

iden

ceto

acto

nin

form

atio

nin

the

bulle

tin(p

p52

^54)

Res

pon

dent

sw

ere

que

stio

ned

abou

tabu

lletin

that

had

been

prev

ious

lypo

sted

toth

em(p

p51

52)

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1993

^199

419Th

ero

leof

rese

arch

info

rmat

ion

inp

olic

yde

velo

pmen

t(p

29)

Que

stio

nnai

res

(p1

3)

open

-end

edte

leph

one

inte

rvie

ws

(p1

4)an

dse

mi-

stru

ctur

edte

leph

one

inte

rvie

ws

(p1

4)

Key

info

rman

ts

inst

ates

that

chan

ged

cost

-sha

ring

pol

icie

sfo

rpre

scrip

tion

reim

burs

emen

tlim

itsan

dco

-pay

men

ts(p

14)

N

în

adeq

uate

lyd

escr

ibed

Res

earc

hp

ublic

atio

nsan

dco

mm

ents

(p2

5)P

erce

ived

retro

spec

tive

dec

isio

ns(p

25)

Onl

y2^

3pa

ges

outo

f21

wer

ere

leva

nt(p

25f

f)I

nfor

man

tsin

clud

edM

edic

aid

drug

prog

ram

me

adm

inis

trato

rs

dire

ctor

sor

polic

yan

alys

ts

phar

mac

yco

nsul

tant

san

dle

gisl

ator

s(p

15)

UK

1994

^199

5202

1`T

oid

entif

yfa

ctor

sw

hich

faci

litat

eor

imp

ede

evid

ence

-bas

edp

olic

y-m

akin

grsquo(

p2

9)

In-d

epth

inte

rvie

ws

(usi

nga

topi

cgu

ide)

(p2

9an

dre

spon

seto

ale

tter)

Lead

polic

y-m

aker

sG

Psan

dre

sear

cher

sin

nine

diff

eren

tcas

est

udie

sat

alo

call

evel

inth

eN

HS

(pp

293

1)N

ˆ28

Soci

alre

sear

ch(p

42

inth

epr

ojec

trep

ort)

Per

ceiv

edde

cisi

ons

Polic

yon

the

inte

rface

bet

wee

nR

ampD

and

com

mis

sion

ing

atth

ehe

alth

auth

ority

leve

l(p

30)

Mex

ico

1994

^199

522`T

here

latio

nshi

pbe

twee

nhe

alth

rese

arch

(and

rese

arch

ers)

and

heal

thpo

licie

s(a

ndpo

licy-

mak

ers)

info

urve

rtica

lhe

alth

prog

ram

mes

inM

exic

orsquo(p

104

)

Ope

n-en

ded

(in-

dept

hrsquo)

inte

rvie

ws

with

aco

mm

onin

terv

iew

prot

ocol

(p1

06)

`Res

earc

hers

and

polic

y-m

aker

sfro

mdi

ffere

ntin

stitu

tions

and

leve

lsof

resp

onsi

bilit

yrsquo(p

103

)N

ˆ67

Use

ofre

sear

chpu

blic

atio

ns`w

here

we

knew

that

ther

ew

asat

leas

tso

me

inte

ract

ion

bet

wee

nre

sear

cher

san

dde

cisi

on-

mak

ersrsquo

(p1

05)

Per

ceiv

ed

use

ofev

iden

ceco

ncer

ning

AID

Sch

oler

afa

mily

pla

nnin

gan

dim

mun

isat

ion

ing

over

nmen

talp

rogr

amm

es(p

103

)

Per

sona

lcom

mun

icat

ion

gav

ead

ditio

nali

nfor

mat

ion

onth

isis

sue

It

isd

iffic

ultt

od

isce

rnw

heth

erth

ede

cisi

ons

wer

ep

erce

ived

orhy

pot

hetic

als

ince

ther

ear

eno

actu

alde

cisi

ons

men

tione

d

Aus

tralia

1995

23

Dec

isio

n-m

aker

srsquoaw

aren

ess

and

use

ofec

onom

icev

alua

tions

inhe

alth

care

(p1

03)

Inte

rvie

ws

open

-end

edan

dst

ruct

ured

(not

stat

edif

post

alt

elep

hone

orfa

ce-to

-fa

ce)(

p10

5)

Sen

iorm

anag

ers

inth

eFe

der

alD

epar

tmen

tof

Hea

lthH

ousi

ngan

dC

omm

unity

Serv

ices

New

Sou

thW

ales

Stat

eth

eH

ealth

Dep

artm

ent(

p10

4)

Nˆ

34(p

105

)

Use

ofec

onom

icev

alua

tions

forp

ricin

gon

drug

ssc

reen

ing

radi

olog

ytra

nspl

ants

and

scan

ning

(p1

05)

`Per

ceiv

edus

eof

econ

omic

eval

uatio

nsrsquo(

p1

04)

USA

1996

24

and

USA

1997

25

Tode

term

ine

whe

ther

the

proc

ess

of(s

ocia

lsci

ence

)in

form

atio

nut

ilisa

tion

varie

sac

ross

diff

eren

tare

as(i

ese

rvic

epr

ovis

ion

and

finan

cing

)(pp

16^

17)

Stru

ctur

edin

terv

iew

s(p

16)

Fe

der

als

tate

and

loca

lp

olic

y-m

aker

sat

diff

eren

tle

vels

deal

ing

with

finan

cean

dse

rvic

es(p

16)

N

ˆ47

9(p

16)

No

spec

ific

repo

rtb

utin

foty

persquor

efer

topr

ogra

mm

eev

alua

tion

dem

onst

ratio

np

roje

ct

stat

istic

ald

ata

and

polic

yan

alys

is(p

34)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

7)O

pen-

ende

din

terv

iew

sw

ere

used

toob

tain

basi

cb

ackg

roun

din

form

atio

nab

outt

hep

olic

ypr

oces

sTh

est

ruct

ured

inte

rvie

ws

built

onth

eop

en-e

nded

inte

rvie

ws

(p3

3)Pa

kist

an19

9826

The

role

that

rese

arch

pla

ysin

expe

rienc

esin

linki

ngre

sear

chw

ithp

olic

y(p

77)

Ope

n-en

ded

in-d

epth

inte

rvie

ws

(p7

7)D

ecis

ion-

mak

ers

from

gov

ernm

ent

the

cent

ral

min

istry

ofhe

alth

the

pla

nnin

gco

mm

issi

onan

dp

rovi

ncia

lhea

lthau

thor

ities

(p7

7)N

ˆ16

(p7

7)

Not

stat

edG

ener

alpe

rcep

tions

C

lose

stfo

cus

onch

ildhe

alth

prob

lem

ssu

chas

co-tr

imox

azol

em

ater

nal

mor

talit

yvi

tam

inA

b

reas

tfeed

ing

and

pol

io(p

p77

79)

UK

1998

27

The

imp

actt

hatt

heE

iexclect

ive

Hea

lthca

reB

ulle

tinha

sin

side

the

NH

Sor

gan

isat

ion

(p1

)

Tele

phon

ein

terv

iew

sst

ruct

ure

notm

entio

ned

(p1

)

Hea

lthau

thor

ities

(p

2)

Nˆ

34Tw

osp

ecifi

cbu

lletin

sJu

ne19

97an

dO

ctob

er19

97(p

1)

Hyp

othe

tical

deci

sion

s(e

g

itin

fluen

ced

the

dev

elop

men

tofo

urpo

licy

alth

oug

hth

isha

sbe

eng

oing

onfo

rsom

etim

ersquo)

(p1

0)

The

date

isno

tcle

arly

stat

edb

utit

is19

98or

1999

Tr

usts

and

GPs

are

incl

uded

inth

est

udy

buta

reno

trel

evan

tin

this

revi

ew

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1982

13

Toob

serv

eth

eim

pac

ttha

tde

cisi

onco

mp

lexi

tya

ndam

ount

ofin

form

atio

nav

aila

ble

had

onth

eq

ualit

yof

the

deci

sion

proc

ess

(p1

87)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edqu

estio

n-na

ires

com

bine

dw

itha

cont

rolle

dex

perim

ent

Con

seq

uenc

esof

com

plex

ityw

ere

exam

ined

byef

fect

sof

info

rmat

ion

load

and

num

bero

fde

cisi

onal

tern

ativ

es(p

p1

871

931

94)

Exp

erie

nced

pub

lican

dp

rivat

ese

ctor

man

ager

sfro

msi

xm

id-c

aree

rp

rog

ram

mes

atH

arva

rd

MIT

Sta

nfor

dan

dth

eW

ashi

ngto

nP

ublic

Affa

irsC

ente

r(pp

187

188

)N

ˆ13

2(p

188

)

Each

dec

isio

n-m

aker

rece

ived

alo

ngm

emor

and

umde

scrib

ing

apr

oble

mp

rese

ntin

gev

iden

ceus

eful

inan

alys

ing

the

prob

lem

and

pose

da

dec

isio

nto

bem

ade

base

don

real

peo

ple

and

even

ts(p

188

)

Two

hypo

thet

ical

dec

isio

ns

one

onth

est

atersquo

sD

epar

t-m

ento

fHea

lthan

dR

ehab

i-lit

ativ

ese

rvic

esan

don

eon

am

etha

done

mai

nten

ance

pro

gram

me

inN

ewYo

rkC

ity(p

p1

88^1

89)

Dec

isio

nha

dto

bem

ade

from

avai

labl

ein

form

atio

nw

ithin

1ho

ur45

min

utes

(p1

93)

Rep

rinte

dfro

mth

eJo

urna

lof

Pol

icy

Ana

lysi

san

dM

anag

emen

t2

66^8

7(p

185

)

The

exp

erim

entw

asb

ased

onsi

xdi

ffere

ntve

rsio

nsof

each

ofth

etw

oca

ses

The

six

diff

eren

tver

sion

sof

the

case

sva

ried

inin

form

atio

nco

mpl

exity

Eac

hde

cisi

on-m

aker

had

only

one

deci

sion

tom

ake

aton

lyon

ele

velo

fcom

plex

ity(p

p18

9^19

019

3)U

SA19

83

14To

dete

rmin

eso

me

ofth

efa

ctor

sth

atin

fluen

cead

min

istra

tors

rsquouse

ofev

alua

tive

data

inde

cisi

on-

mak

ing

and

toex

amin

eth

ety

pes

ofde

cisi

ons

(p7

9)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p7

9^80

)

Dire

ctor

sin

com

mun

itym

enta

lhea

lthce

ntre

sin

Flor

ida

and

Nor

thC

arol

ina

(p7

9)N

ˆ42

Con

duct

edst

udie

son

prog

ram

me

effe

ctiv

enes

sor

imp

actd

urin

gth

ela

st12

-m

onth

per

iod

(p8

1)

Per

ceiv

ed(i

ere

por

ted

)us

e(p

p80

^81)

Swed

en19

84^1

98515

(p1

52)

Diff

eren

tway

sof

usin

gso

cial

rese

arch

in

stru

men

tal

polit

ical

co

nflic

ten

light

enm

ent

inte

ract

ive

and

tact

ical

(pp

141

143^

144)

Inte

rvie

ws

(p1

52)

Soc

ialw

orke

rs

man

agem

entd

irect

ors

loca

land

natio

nalp

oliti

cian

sin

15Sw

edis

hci

ties

and

com

mun

es(p

152

)N

ˆ90

(p1

52)

Doc

umen

tsth

atth

ere

spon

dent

sre

ferr

edto

Th

ese

docu

men

tsw

ere

also

anal

ysed

by

the

auth

ors

(p1

52)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

52)

USA

1990

16To

dete

rmin

ew

hatt

ype

ofin

form

atio

nd

ecis

ion-

mak

ers

use

and

how

ideo

log

yan

din

tere

sts

influ

ence

the

use

ofin

form

atio

n(p

109

)

Pers

onal

sem

i-stru

ctur

edin

terv

iew

s(p

112

)Lo

calp

lann

ing

com

mitt

ees

and

gove

rnm

ente

mp

loye

esin

two

rura

lcom

mun

ities

inth

eP

acifi

cN

orth

wes

t(p

110

)N

ˆ27

(p1

10)

Sour

ces

ofin

form

atio

nus

edin

com

mun

ityhe

alth

care

plan

ning

and

the

asse

ssm

ento

fthe

stra

tegi

cpl

anni

ngp

roce

ss(p

112

)

Per

ceiv

edus

e(p

112

)

USA

1991

17

Wha

tfac

tors

influ

ence

the

use

ofev

alua

tion

rese

arch

(on

heal

than

dhu

man

serv

ices

)in

cong

ress

(p5

08)

Stru

ctur

edte

lep

hone

inte

rvie

ws

(p5

13)

C

ongr

ess

mem

bers

and

thei

rsta

ff(p

512

)N

ˆ10

0Ea

chde

cisi

on-m

aker

chos

eon

est

udy

they

wer

efa

mili

arw

ith(p

515

)

Per

ceiv

edus

e(p

515

)E

valu

atio

nst

udie

sfro

mth

eD

epar

tmen

tofH

ealth

and

Hum

anSe

rvic

esth

atd

eal

with

natio

nalh

ealth

pro

gram

mes

(pp

507

515

)

Use

isde

fined

here

asbo

thin

stru

men

tal(

dire

ctim

ple

men

tatio

n)c

once

ptua

l(it

chan

ged

the

way

ofth

inki

ng)a

ndpe

rsua

sive

(use

dto

conv

ince

othe

rsp

516

)

The

inte

rvie

ws

last

ed15

^40

min

utes

(p5

14)

UK

1991

18U

sean

dre

ason

sfo

ruse

not

use

ofth

eE

iexclect

ive

Hea

lthC

are

Bul

letin

(pp

51^

54)

Stru

ctur

edan

dop

en-e

nded

post

alq

uest

ionn

aire

s(p

p5

0^51

)

Dis

trict

gen

eral

man

ager

sd

irect

ors

ofpu

blic

heal

than

dd

irect

ors

ofpl

anni

ng

com

mis

sion

ing

inU

Kd

istri

cthe

alth

auth

oriti

esor

com

mis

sion

ing

cons

ortia

(p5

0)N

ˆ17

2(p

51)

Eiexcle

ctiv

eH

ealth

Car

eB

ulle

tinN

o1

1991

on

scre

enin

gfo

rost

eop

oros

isto

prev

entf

ract

ures

(p5

0)

Per

ceiv

edus

eof

info

rmat

ion

and

hypo

thet

ical

dec

isio

ns

chan

geof

opin

ion

and

conf

iden

ceto

acto

nin

form

atio

nin

the

bulle

tin(p

p52

^54)

Res

pon

dent

sw

ere

que

stio

ned

abou

tabu

lletin

that

had

been

prev

ious

lypo

sted

toth

em(p

p51

52)

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1993

^199

419Th

ero

leof

rese

arch

info

rmat

ion

inp

olic

yde

velo

pmen

t(p

29)

Que

stio

nnai

res

(p1

3)

open

-end

edte

leph

one

inte

rvie

ws

(p1

4)an

dse

mi-

stru

ctur

edte

leph

one

inte

rvie

ws

(p1

4)

Key

info

rman

ts

inst

ates

that

chan

ged

cost

-sha

ring

pol

icie

sfo

rpre

scrip

tion

reim

burs

emen

tlim

itsan

dco

-pay

men

ts(p

14)

N

în

adeq

uate

lyd

escr

ibed

Res

earc

hp

ublic

atio

nsan

dco

mm

ents

(p2

5)P

erce

ived

retro

spec

tive

dec

isio

ns(p

25)

Onl

y2^

3pa

ges

outo

f21

wer

ere

leva

nt(p

25f

f)I

nfor

man

tsin

clud

edM

edic

aid

drug

prog

ram

me

adm

inis

trato

rs

dire

ctor

sor

polic

yan

alys

ts

phar

mac

yco

nsul

tant

san

dle

gisl

ator

s(p

15)

UK

1994

^199

5202

1`T

oid

entif

yfa

ctor

sw

hich

faci

litat

eor

imp

ede

evid

ence

-bas

edp

olic

y-m

akin

grsquo(

p2

9)

In-d

epth

inte

rvie

ws

(usi

nga

topi

cgu

ide)

(p2

9an

dre

spon

seto

ale

tter)

Lead

polic

y-m

aker

sG

Psan

dre

sear

cher

sin

nine

diff

eren

tcas

est

udie

sat

alo

call

evel

inth

eN

HS

(pp

293

1)N

ˆ28

Soci

alre

sear

ch(p

42

inth

epr

ojec

trep

ort)

Per

ceiv

edde

cisi

ons

Polic

yon

the

inte

rface

bet

wee

nR

ampD

and

com

mis

sion

ing

atth

ehe

alth

auth

ority

leve

l(p

30)

Mex

ico

1994

^199

522`T

here

latio

nshi

pbe

twee

nhe

alth

rese

arch

(and

rese

arch

ers)

and

heal

thpo

licie

s(a

ndpo

licy-

mak

ers)

info

urve

rtica

lhe

alth

prog

ram

mes

inM

exic

orsquo(p

104

)

Ope

n-en

ded

(in-

dept

hrsquo)

inte

rvie

ws

with

aco

mm

onin

terv

iew

prot

ocol

(p1

06)

`Res

earc

hers

and

polic

y-m

aker

sfro

mdi

ffere

ntin

stitu

tions

and

leve

lsof

resp

onsi

bilit

yrsquo(p

103

)N

ˆ67

Use

ofre

sear

chpu

blic

atio

ns`w

here

we

knew

that

ther

ew

asat

leas

tso

me

inte

ract

ion

bet

wee

nre

sear

cher

san

dde

cisi

on-

mak

ersrsquo

(p1

05)

Per

ceiv

ed

use

ofev

iden

ceco

ncer

ning

AID

Sch

oler

afa

mily

pla

nnin

gan

dim

mun

isat

ion

ing

over

nmen

talp

rogr

amm

es(p

103

)

Per

sona

lcom

mun

icat

ion

gav

ead

ditio

nali

nfor

mat

ion

onth

isis

sue

It

isd

iffic

ultt

od

isce

rnw

heth

erth

ede

cisi

ons

wer

ep

erce

ived

orhy

pot

hetic

als

ince

ther

ear

eno

actu

alde

cisi

ons

men

tione

d

Aus

tralia

1995

23

Dec

isio

n-m

aker

srsquoaw

aren

ess

and

use

ofec

onom

icev

alua

tions

inhe

alth

care

(p1

03)

Inte

rvie

ws

open

-end

edan

dst

ruct

ured

(not

stat

edif

post

alt

elep

hone

orfa

ce-to

-fa

ce)(

p10

5)

Sen

iorm

anag

ers

inth

eFe

der

alD

epar

tmen

tof

Hea

lthH

ousi

ngan

dC

omm

unity

Serv

ices

New

Sou

thW

ales

Stat

eth

eH

ealth

Dep

artm

ent(

p10

4)

Nˆ

34(p

105

)

Use

ofec

onom

icev

alua

tions

forp

ricin

gon

drug

ssc

reen

ing

radi

olog

ytra

nspl

ants

and

scan

ning

(p1

05)

`Per

ceiv

edus

eof

econ

omic

eval

uatio

nsrsquo(

p1

04)

USA

1996

24

and

USA

1997

25

Tode

term

ine

whe

ther

the

proc

ess

of(s

ocia

lsci

ence

)in

form

atio

nut

ilisa

tion

varie

sac

ross

diff

eren

tare

as(i

ese

rvic

epr

ovis

ion

and

finan

cing

)(pp

16^

17)

Stru

ctur

edin

terv

iew

s(p

16)

Fe

der

als

tate

and

loca

lp

olic

y-m

aker

sat

diff

eren

tle

vels

deal

ing

with

finan

cean

dse

rvic

es(p

16)

N

ˆ47

9(p

16)

No

spec

ific

repo

rtb

utin

foty

persquor

efer

topr

ogra

mm

eev

alua

tion

dem

onst

ratio

np

roje

ct

stat

istic

ald

ata

and

polic

yan

alys

is(p

34)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

7)O

pen-

ende

din

terv

iew

sw

ere

used

toob

tain

basi

cb

ackg

roun

din

form

atio

nab

outt

hep

olic

ypr

oces

sTh

est

ruct

ured

inte

rvie

ws

built

onth

eop

en-e

nded

inte

rvie

ws

(p3

3)Pa

kist

an19

9826

The

role

that

rese

arch

pla

ysin

expe

rienc

esin

linki

ngre

sear

chw

ithp

olic

y(p

77)

Ope

n-en

ded

in-d

epth

inte

rvie

ws

(p7

7)D

ecis

ion-

mak

ers

from

gov

ernm

ent

the

cent

ral

min

istry

ofhe

alth

the

pla

nnin

gco

mm

issi

onan

dp

rovi

ncia

lhea

lthau

thor

ities

(p7

7)N

ˆ16

(p7

7)

Not

stat

edG

ener

alpe

rcep

tions

C

lose

stfo

cus

onch

ildhe

alth

prob

lem

ssu

chas

co-tr

imox

azol

em

ater

nal

mor

talit

yvi

tam

inA

b

reas

tfeed

ing

and

pol

io(p

p77

79)

UK

1998

27

The

imp

actt

hatt

heE

iexclect

ive

Hea

lthca

reB

ulle

tinha

sin

side

the

NH

Sor

gan

isat

ion

(p1

)

Tele

phon

ein

terv

iew

sst

ruct

ure

notm

entio

ned

(p1

)

Hea

lthau

thor

ities

(p

2)

Nˆ

34Tw

osp

ecifi

cbu

lletin

sJu

ne19

97an

dO

ctob

er19

97(p

1)

Hyp

othe

tical

deci

sion

s(e

g

itin

fluen

ced

the

dev

elop

men

tofo

urpo

licy

alth

oug

hth

isha

sbe

eng

oing

onfo

rsom

etim

ersquo)

(p1

0)

The

date

isno

tcle

arly

stat

edb

utit

is19

98or

1999

Tr

usts

and

GPs

are

incl

uded

inth

est

udy

buta

reno

trel

evan

tin

this

revi

ew

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

USA

1993

^199

419Th

ero

leof

rese

arch

info

rmat

ion

inp

olic

yde

velo

pmen

t(p

29)

Que

stio

nnai

res

(p1

3)

open

-end

edte

leph

one

inte

rvie

ws

(p1

4)an

dse

mi-

stru

ctur

edte

leph

one

inte

rvie

ws

(p1

4)

Key

info

rman

ts

inst

ates

that

chan

ged

cost

-sha

ring

pol

icie

sfo

rpre

scrip

tion

reim

burs

emen

tlim

itsan

dco

-pay

men

ts(p

14)

N

în

adeq

uate

lyd

escr

ibed

Res

earc

hp

ublic

atio

nsan

dco

mm

ents

(p2

5)P

erce

ived

retro

spec

tive

dec

isio

ns(p

25)

Onl

y2^

3pa

ges

outo

f21

wer

ere

leva

nt(p

25f

f)I

nfor

man

tsin

clud

edM

edic

aid

drug

prog

ram

me

adm

inis

trato

rs

dire

ctor

sor

polic

yan

alys

ts

phar

mac

yco

nsul

tant

san

dle

gisl

ator

s(p

15)

UK

1994

^199

5202

1`T

oid

entif

yfa

ctor

sw

hich

faci

litat

eor

imp

ede

evid

ence

-bas

edp

olic

y-m

akin

grsquo(

p2

9)

In-d

epth

inte

rvie

ws

(usi

nga

topi

cgu

ide)

(p2

9an

dre

spon

seto

ale

tter)

Lead

polic

y-m

aker

sG

Psan

dre

sear

cher

sin

nine

diff

eren

tcas

est

udie

sat

alo

call

evel

inth

eN

HS

(pp

293

1)N

ˆ28

Soci

alre

sear

ch(p

42

inth

epr

ojec

trep

ort)

Per

ceiv

edde

cisi

ons

Polic

yon

the

inte

rface

bet

wee

nR

ampD

and

com

mis

sion

ing

atth

ehe

alth

auth

ority

leve

l(p

30)

Mex

ico

1994

^199

522`T

here

latio

nshi

pbe

twee

nhe

alth

rese

arch

(and

rese

arch

ers)

and

heal

thpo

licie

s(a

ndpo

licy-

mak

ers)

info

urve

rtica

lhe

alth

prog

ram

mes

inM

exic

orsquo(p

104

)

Ope

n-en

ded

(in-

dept

hrsquo)

inte

rvie

ws

with

aco

mm

onin

terv

iew

prot

ocol

(p1

06)

`Res

earc

hers

and

polic

y-m

aker

sfro

mdi

ffere

ntin

stitu

tions

and

leve

lsof

resp

onsi

bilit

yrsquo(p

103

)N

ˆ67

Use

ofre

sear

chpu

blic

atio

ns`w

here

we

knew

that

ther

ew

asat

leas

tso

me

inte

ract

ion

bet

wee

nre

sear

cher

san

dde

cisi

on-

mak

ersrsquo

(p1

05)

Per

ceiv

ed

use

ofev

iden

ceco

ncer

ning

AID

Sch

oler

afa

mily

pla

nnin

gan

dim

mun

isat

ion

ing

over

nmen

talp

rogr

amm

es(p

103

)

Per

sona

lcom

mun

icat

ion

gav

ead

ditio

nali

nfor

mat

ion

onth

isis

sue

It

isd

iffic

ultt

od

isce

rnw

heth

erth

ede

cisi

ons

wer

ep

erce

ived

orhy

pot

hetic

als

ince

ther

ear

eno

actu

alde

cisi

ons

men

tione

d

Aus

tralia

1995

23

Dec

isio

n-m

aker

srsquoaw

aren

ess

and

use

ofec

onom

icev

alua

tions

inhe

alth

care

(p1

03)

Inte

rvie

ws

open

-end

edan

dst

ruct

ured

(not

stat

edif

post

alt

elep

hone

orfa

ce-to

-fa

ce)(

p10

5)

Sen

iorm

anag

ers

inth

eFe

der

alD

epar

tmen

tof

Hea

lthH

ousi

ngan

dC

omm

unity

Serv

ices

New

Sou

thW

ales

Stat

eth

eH

ealth

Dep

artm

ent(

p10

4)

Nˆ

34(p

105

)

Use

ofec

onom

icev

alua

tions

forp

ricin

gon

drug

ssc

reen

ing

radi

olog

ytra

nspl

ants

and

scan

ning

(p1

05)

`Per

ceiv

edus

eof

econ

omic

eval

uatio

nsrsquo(

p1

04)

USA

1996

24

and

USA

1997

25

Tode

term

ine

whe

ther

the

proc

ess

of(s

ocia

lsci

ence

)in

form

atio

nut

ilisa

tion

varie

sac

ross

diff

eren

tare

as(i

ese

rvic

epr

ovis

ion

and

finan

cing

)(pp

16^

17)

Stru

ctur

edin

terv

iew

s(p

16)

Fe

der

als

tate

and

loca

lp

olic

y-m

aker

sat

diff

eren

tle

vels

deal

ing

with

finan

cean

dse

rvic

es(p

16)

N

ˆ47

9(p

16)

No

spec

ific

repo

rtb

utin

foty

persquor

efer

topr

ogra

mm

eev

alua

tion

dem

onst

ratio

np

roje

ct

stat

istic

ald

ata

and

polic

yan

alys

is(p

34)

Per

ceiv

edre

trosp

ectiv

ed

ecis

ions

(p1

7)O

pen-

ende

din

terv

iew

sw

ere

used

toob

tain

basi

cb

ackg

roun

din

form

atio

nab

outt

hep

olic

ypr

oces

sTh

est

ruct

ured

inte

rvie

ws

built

onth

eop

en-e

nded

inte

rvie

ws

(p3

3)Pa

kist

an19

9826

The

role

that

rese

arch

pla

ysin

expe

rienc

esin

linki

ngre

sear

chw

ithp

olic

y(p

77)

Ope

n-en

ded

in-d

epth

inte

rvie

ws

(p7

7)D

ecis

ion-

mak

ers

from

gov

ernm

ent

the

cent

ral

min

istry

ofhe

alth

the

pla

nnin

gco

mm

issi

onan

dp

rovi

ncia

lhea

lthau

thor

ities

(p7

7)N

ˆ16

(p7

7)

Not

stat

edG

ener

alpe

rcep

tions

C

lose

stfo

cus

onch

ildhe

alth

prob

lem

ssu

chas

co-tr

imox

azol

em

ater

nal

mor

talit

yvi

tam

inA

b

reas

tfeed

ing

and

pol

io(p

p77

79)

UK

1998

27

The

imp

actt

hatt

heE

iexclect

ive

Hea

lthca

reB

ulle

tinha

sin

side

the

NH

Sor

gan

isat

ion

(p1

)

Tele

phon

ein

terv

iew

sst

ruct

ure

notm

entio

ned

(p1

)

Hea

lthau

thor

ities

(p

2)

Nˆ

34Tw

osp

ecifi

cbu

lletin

sJu

ne19

97an

dO

ctob

er19

97(p

1)

Hyp

othe

tical

deci

sion

s(e

g

itin

fluen

ced

the

dev

elop

men

tofo

urpo

licy

alth

oug

hth

isha

sbe

eng

oing

onfo

rsom

etim

ersquo)

(p1

0)

The

date

isno

tcle

arly

stat

edb

utit

is19

98or

1999

Tr

usts

and

GPs

are

incl

uded

inth

est

udy

buta

reno

trel

evan

tin

this

revi

ew

cont

inue

d



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



Tab

le1

cont

inue

d

Co

un

try

year

con

duc

ted

(ye

arp

ub

lish

ed)

refe

ren

ce

Ob

ject

ive

Stu

dy

des

ign

Par

tici

pan

tsl

evel

of

po

licy-

mak

ing

Typ

eso

fevi

den

ce(f

rom

wh

ich

dec

isio

ns

wer

em

ade)

Typ

eso

fdec

isio

ns

(hyp

oth

etic

alp

erce

ived

or

actu

al)

typ

eso

fpo

licy

Co

mm

ents

UK

and

USA

1998

28

H

owU

Kan

dU

Sp

urch

aser

sva

lue

and

use

rese

arch

asse

ssin

ghe

alth

tech

nolo

gy

and

clin

ical

effe

ctiv

enes

s(p

pv

1)

Sem

i-stru

ctur

edte

leph

one

inte

rvie

ws

(p4

)U

K1

4no

n-m

edic

alhe

alth

auth

ority

pur

chas

ers

seve

np

ublic

heal

thp

hysi

cian

sni

neG

PsU

SA1

3p

ublic

offic

ials

from

11st

ates

nin

ep

rivat

ese

ctor

purc

hase

rs

thre

eco

nsul

tant

sN

ˆ55

(p4

)

Acc

ess

toan

dus

eof

heal

thte

chno

logy

asse

ssm

ent

clin

ical

effe

ctiv

enes

sin

form

atio

n(p

p4

11)

Per

ceiv

edus

e(p

1)

Sta

ted

1999

(p3

)

Can

ada

1998

^199

929A

case

stud

yof

cros

s-se

ctor

alre

sour

ceal

loca

tion

inth

ehu

man

serv

ices

syst

emin

Prin

ceE

dwar

dIs

land

(p4

)

Sem

i-stru

ctur

edan

dop

en-

ende

din

terv

iew

san

dfo

cus

grou

ps(p

p13

^16

App

endi

x1)

Key

info

rman

tsho

ldin

gd

iffer

entp

ositi

ons

with

inth

efiv

ehe

alth

regi

ons

the

Dep

artm

ento

fSoc

ial

Ser

vice

san

dva

rious

sect

ors

(p4

)al

lreg

ions

all

sect

ors

and

man

agem

ent

leve

ls(p

13)

Nˆ

58(p

9)

Not

stat

edP

erce

ived

deci

sion

son

cros

s-se

ctor

alre

sour

ceal

loca

tions

(p5

)

Bur

kina

Faso

1999

30A

case

stud

yon

how

rese

arch

pla

yed

aro

lein

deci

sion

-mak

ing

abou

t`S

hare

dC

arersquo

fo

rch

ildho

odill

ness

(pp

19^

20)

Sem

i-stru

ctur

edin

terv

iew

s(p

20)

Min

istry

ofH

ealth

offic

ials

(p2

2)N

ˆno

tgiv

enW

orks

hops

and

rese

arch

pape

rsth

atth

eau

thor

sha

dpr

ovid

edM

inis

tryof

Hea

lthof

ficia

lsw

ithd

urin

gpr

evio

usye

ars

(p2

1)

Per

ceiv

edus

eof

the

pro

vide

dsc

ient

ific

info

rmat

ion

inde

cisi

on-

mak

ing

(p2

0)

`Sha

red

care

rsquois

abou

tim

prov

edco

ntac

tand

colla

bora

tion

bet

wee

nhe

alth

cent

rest

affa

ndm

othe

rs(p

20)

Sout

hA

frica

2000

31B

arrie

rsag

ains

tand

mec

hani

sms

fori

mpr

ovin

gth

evi

tam

inA

rese

arch

^po

licy^

actio

nco

nnec

tion

(p5

5)

Sem

i-stru

ctur

edin

terv

iew

s(p

55)

Key

info

rman

tsth

atw

ere

rele

vant

role

-pla

yers

(p5

5)

Nˆ

nots

tate

d

Not

stat

edP

erce

ived

use

(p5

5)

Can

ada

2000

32

Toas

sess

the

influ

ence

ofin

form

atio

nab

outt

hehe

alth

cons

eque

nces

ofun

empl

oym

enta

ndin

secu

rity

onp

olic

y-m

akin

g(p

5)

Stru

ctur

eds

emi-s

truct

ured

and

open

-end

edte

lep

hone

inte

rvie

ws

(pp

59

)

Pol

icy-

mak

ers

inth

ehe

alth

and

empl

oym

ents

ecto

rsof

allt

hree

leve

lsof

Can

adia

ng

over

nmen

tan

dex

ecut

ive

dire

ctor

sof

ten

Can

adia

nno

n-g

over

nmen

talo

rgan

-is

atio

ns(p

5)

Nˆ

48(p

5)

Not

stat

edP

erce

ived

retro

spec

tive

usef

ulne

ssof

past

rese

arch

info

rmat

ion

abou

tthe

heal

thco

nseq

uenc

esof

unem

ploy

men

tand

inse

curit

yon

polic

y-m

akin

g(p

5)

Sta

ted

date

forw

orki

ngpa

peri

sM

arch

2000

Can

ada

2000

33To

unde

rsta

ndw

heth

er

how

and

und

erw

hat

cond

ition

she

alth

serv

ices

rese

arch

pla

ysa

role

inpr

ovin

cial

polic

y-m

akin

g(p

i)

Bot

hpe

rson

als

emi-

stru

ctur

edin

terv

iew

san

dsu

rvey

s(p

p8

11)

Key

info

rman

tsw

ithin

ava

riety

ofke

yhe

alth

polic

ies

intw

oC

anad

ian

prov

ince

s(O

ntar

ioan

dS

aska

tche

wan

)(p

ii)

Nˆ

22(p

18)

Not

stat

ed

Act

uala

ndpe

rcei

ved

dec

isio

nsw

ithin

the

field

sof

heal

thse

rvic

esju

risdi

ctio

nfin

anci

alar

rang

emen

ts

pro

gram

me

deliv

ery

arra

ngem

ents

and

pro

gram

me

cont

ent

(pp

713

)

Iti

sno

tsta

ted

wha

tkin

dof

evid

ence

was

used

but

refe

renc

esto

dire

ctus

eof

evid

ence

inde

cisi

onre

ports

rep

ortin

gon

actu

alus

eof

rese

arch

im

plie

sth

atev

iden

cew

asus

ed(p

p13

18^

19)

GPs

gen

eral

pra

ctiti

oner

sM

ITM

assa

chus

etts

Inst

itute

ofTe

chno

log

yN

HS

Nat

iona

lHea

lthS

ervi

ceR

ampD

res

earc

han

dd

evel

opm

ent



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



Tab

le2

Stu

dym

etho

ds

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

The

Net

herla

nds

1969

^197

17SF

not

stat

edSS

not

stat

edN

otst

ated

Aqu

antit

ativ

e4-

poi

ntsc

ore

wei

ghte

dac

cord

ing

toth

ree

deci

sion

leve

lsw

ithsu

mm

edsc

ore

rang

ing

from

0to

18(p

51)

Two-

way

AN

OV

A(p

66)

Not

adeq

uate

lyde

scrib

ed

USA

1973

8SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

No

stat

istic

alte

chni

que

Not

adeq

uate

lyde

scrib

ed

USA

1973

^197

49

and

USA

1973

^19

7410

(pv

ii)

SFp

artia

l(p

viii)

SSn

otm

entio

ned

partl

ysn

owb

alle

d(p

viii

)

95

(p2

4)B

oth

quan

titat

ivel

yra

ted

and

wei

ght

ed(p

p25

28

333

536

)and

qua

litat

ivel

yde

scrib

ed(p

p1^

4)D

escr

iptiv

elis

ting

ofra

tings

(pp

28

333

536

)and

MC

A(m

ultiv

aria

tere

gre

ssio

n)an

alys

is(p

27)

Parti

alIt

isst

ated

that

`this

repo

rtre

pre

sent

only

afir

stsu

mm

ary

ofso

me

ofth

ehi

ghlig

hts

and

mor

eim

por

tant

findi

ngsrsquo

(pv

i)S

ubse

quen

trep

orts

not

foun

d

USA

1977

11

SFp

artia

l(p

143)

SSs

now

bal

led

partl

yra

ndom

(p1

43)

Not

stat

edD

escr

iptiv

eN

ost

atis

tical

tech

niqu

ePa

rtial

The

inte

rvie

ws

appe

arto

have

bee

nm

ade

betw

een

1973

and

1976

USA

1980

2an

dU

SA19

8012

SFp

artia

l(p

167)

SSn

otst

ated

90

(p1

67)

Bot

hqu

antit

ativ

ede

scrip

tions

rat

edan

dw

eig

hted

and

qual

itativ

e(p

p16

9ff

303)

Des

crip

tive

listin

gof

ratin

gs(p

p1

70^1

72

174)

and

25va

riabl

esan

alys

edus

ing

fact

oran

alys

isto

dev

elop

five

dim

ensi

ons

(p

304

)

Parti

alB

oth

a4-

poin

t(p

168

)and

a5-

poi

ntw

eigh

ted

ratin

gsc

ale

(pp

173

303)

The

qual

itativ

era

ting

com

esfro

mlis

ting

ofm

ostm

entio

ned

char

acte

ristic

sof

usef

ulne

ss

The

five

dim

ensi

ons

are

liste

din

Tabl

e3

und

erfa

cilit

ator

s

USA

1982

13SF

Par

tial

(pp

187

^188

)SS

ran

dom

sam

ple

(pp

187

^188

)

Volu

ntee

red

(p1

88)

Qua

ntita

tive

listin

gof

the

deci

sion

qua

lity

ofev

iden

ce^

use

onsi

xdi

ffere

ntle

vels

ofco

mpl

exity

(pp

195^

208)

List

ing

offre

quen

cies

(pp

196

^198

)Pa

rtial

The

132

dec

isio

n-m

aker

sw

ere

rand

omly

dist

ribut

edto

two

of50

case

stud

ies

and

six

diff

eren

tlev

els

(p1

94)

USA

1983

14SF

not

stat

ed(p

79)

SS

not

stat

ed(p

79)

65

(p7

9)Q

uant

itativ

elis

ting

and

ratin

gs(p

p8

2^86

)Fr

eque

ncie

san

dra

ted

impo

rtanc

e(a

sve

ry

mod

erat

ely

orno

tim

porta

nt)(

p8

2)

SF

and

SSco

mpl

ete

ifth

est

udyrsquo

sfo

cus

cons

ider

edon

lydi

rect

ors

inFl

orid

aan

dN

orth

Car

olin

aan

dif

ther

ew

ere

only

one

dire

ctor

inea

chC

omm

unity

Men

talH

ealth

Cen

ter

Swed

en19

84^

1985

15SF

not

stat

edSS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(p

p14

6^15

0)Li

stin

gof

frequ

enci

es(p

p1

451

49)

Not

adeq

uate

lyde

scrib

edTh

eau

thor

sm

ainl

yd

iscu

ssan

dde

mon

stra

ted

iffer

entw

ays

ofut

ilisi

ngre

sear

cha

sei

ther

ins

trum

enta

lpo

litic

alc

onfli

cte

nlig

hten

men

tin

tera

ctiv

eor

tact

ical

(p1

44)

`

usin

gre

sear

chìn

stru

men

tally

rsquorsquoor

for

ènl

ight

enm

entrsquorsquo

purp

oses

mea

ntdi

ffere

ntth

ings

indi

ffere

ntse

tting

san

ddi

ffere

ntor

gani

zatio

nalp

ositi

onsrsquo

(p1

45)

USA

1990

16SF

par

tial(

p11

0)SS

com

ple

te

(p1

10)

100

(p1

10)

Qua

litat

ive

and

desc

riptiv

e(p

112

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Com

ple

tere

fers

toth

e27

rep

rese

ntat

ives

inth

etw

opl

anni

ngco

mm

ittee

sin

the

two

com

mun

ities

No

furth

erge

nera

lisat

ion

can

be

mad

eon

the

basi

sof

this

sam

ple

USA

1991

17SF

com

plet

e(p

p5

13^5

14)

SSr

ando

msa

mp

le(p

513

)

60

(p5

14)

Qua

ntita

tivel

yde

scrip

tive

rate

dan

dw

eig

hted

(p

p51

9^52

2)D

escr

iptiv

elis

ting

ofw

eigh

ted

ratin

gs(p

p5

17^5

23)a

ndm

ultip

lere

gre

ssio

n(p

p5

185

23^5

27)

Com

ple

te

Wei

ghte

don

3-p

oint

and

5-p

oint

scal

es

UK

1991

18SF

com

plet

e(p

51)

SSr

ando

msa

mp

le(p

51)

73

(p5

1)Q

uant

itativ

era

ted

desc

riptio

ns(p

p51

^54)

Per

cent

ages

(pp

51^5

4)C

omp

lete

cont

inue

d



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



Tab

le2

cont

inue

d

Stu

dy

refe

ren

ceS

amp

lefr

ame

(SF

)aan

dsa

mp

lest

rate

gy

(SS

)b

Res

po

nse

rate

Mea

sure

men

tofu

seo

fev

iden

ceM

easu

rem

ento

fd

eter

min

ants

ofu

seS

um

mar

yo

fdes

crip

tio

no

fmet

ho

ds

(co

mp

lete

par

tial

or

no

tad

equa

tely

des

crib

ed)a

nd

com

men

ts

USA

1993

^199

419SF

par

tial(

p15

)SS

not

stat

edN

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

esof

pol

icy

chan

gea

nym

entio

nno

rank

ing)

(p1

6)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edO

nly

Med

icai

dp

rog

ram

mes

with

spec

ific

cost

-con

tain

men

tpol

icie

sin

the

last

seve

raly

ears

(p2

9)U

K19

94^1

99520

21

SFp

artia

l(pp

461

46

5)SS

sno

wb

alle

d(p

p4

614

65)

100

(all

thos

eap

proa

ched

fori

nter

view

agre

edto

the

inte

rvie

w)

Qua

litat

ive

and

desc

riptiv

e(p

31f

f)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Mex

ico

1994

^199

522SF

not

stat

ed

SSn

otst

ated

N

otst

ated

Qua

litat

ive

and

desc

riptiv

e(i

esu

bjec

tive

view

onke

yva

riabl

es)(

pp

104

106^

108)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edP

urpo

sive

sam

ple

(per

sona

lcom

mun

icat

ions

)A

ustra

lia19

9523

SFp

artia

lSS

not

stat

ed

Not

stat

edQ

uant

itativ

ean

dd

escr

iptiv

e(p

p1

05^1

08)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

edC

alle

d`p

urpo

sive

sam

plin

gfra

mersquo

(p1

04)

USA

1996

24an

dU

SA19

9725

SFp

artia

l(pp

16

30^3

1)SS

com

bin

edra

ndom

and

snow

-ba

lled

(pp

30^3

1)

Not

stat

edQ

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Q

uant

itativ

ere

gres

sion

and

apa

thm

odel

(p1

8)Pa

rtial

but

with

ago

odde

scrip

tion

ofSS

and

SF

Paki

stan

1998

26SF

par

tial(

p77

)SS

not

stat

edN

otst

ated

Des

crip

tive

(pp

788

082

)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

UK

1998

27SF

par

tial(

pp1

^2)

SSn

otst

ated

(pp

1^2

)

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(ie

subj

ectiv

evi

ewon

awar

enes

sge

nera

lper

cept

ions

and

impa

ct)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

UK

and

USA

1998

28SF

par

tial(

p4)

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(pp

12^2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Can

ada

1998

^199

929SF

par

tial(

p16

)SS

not

stat

ed

Not

stat

edQ

ualit

ativ

ede

scrip

tions

and

quan

titat

ive

listin

gs(p

p18

^30

32^3

4)Q

ualit

ativ

ePa

rtial

Aqu

alita

tive

valid

atio

nof

the

inte

rvie

ws

isd

escr

ibed

(pp

16^1

7)B

urki

naFa

so19

9930

SFn

otst

ated

SSn

otst

ated

Not

stat

edQ

ualit

ativ

ean

dde

scrip

tive

(p2

2)Q

ualit

ativ

eN

otad

equa

tely

desc

ribed

Sout

hA

frica

2000

31SF

not

stat

edSS

not

stat

edN

otst

ated

Des

crip

tive

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

Can

ada

2000

32SF

par

tial

SSn

otst

ated

42

(p

23)

Bot

hqu

antit

ativ

ean

dqu

alita

tive

des

crip

tions

ofba

rrie

rs(p

p11

^14

26)

Freq

uenc

ies

(p2

6)Pa

rtial

Cal

led

àpu

rpos

ive

sam

ple

ofC

anad

ian

polic

y-m

aker

srsquo(p

8)

Can

ada

2000

33SF

not

stat

edSS

sno

wb

alle

d(p

ii)

88

(22

25)

(p1

8)B

oth

quan

titat

ive

and

qual

itativ

ed

escr

iptio

nsof

faci

litat

ors

(pp

ii

i35

^37)

Qua

litat

ive

Not

adeq

uate

lyde

scrib

ed

a Sam

ple

fram

ec

omp

lete

ˆth

esa

mp

lew

asd

escr

ibed

inre

latio

nto

allt

hed

ecis

ion

-mak

ers

inth

eco

ntex

toft

hest

udy

par

tialˆ

inco

mp

lete

info

rmat

ion

not

stat

edˆ

nod

escr

iptio

nb S

amp

lest

rate

gy

com

ple

tesa

mp

ler

and

omsa

mp

les

now

bal

led

sam

ple

orn

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



Tab

le3

Mai

nre

sults

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

The

Net

herla

nds

1969

^197

17Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p5

4)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

60)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

54)

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

65)

USA

1973

8Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p1

41)

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

h(p

142

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

143

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

43)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p14

114

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(pp

142

145)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p1

43)

USA

1973

^197

49an

dU

SA

1973

^197

410

(pv

ii)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

333

6)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

33)

Goo

dqu

ality

rese

arch

(p3

3)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

8)

USA

1977

11Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

155

158)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p1

45)

Pow

eran

dbu

dget

stru

ggle

s(p

p14

915

2)P

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p15

9^16

0)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

155

158)

USA

1980

2an

dU

SA19

8012

Goo

dqu

ality

rese

arch

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

stTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sM

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

sP

ower

and

budg

etst

rugg

les

USA

1982

13N

one

ofth

efa

cilit

ator

sw

ere

men

tione

din

the

othe

rstu

dies

Non

eof

the

bar

riers

wer

em

entio

ned

inth

eot

hers

tud

ies

USA

1983

14C

omm

unity

pres

sure

orcl

ient

dem

and

forr

esea

rch

(pp

82

84)

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

82)

Not

stat

ed

Swed

en19

84^1

98515

Res

earc

hth

atco

nfirm

scu

rren

tpol

icy

oren

dors

esse

lf-in

tere

st(p

p14

614

9)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

148

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

149

)

Pow

eran

dbu

dget

stru

ggle

s(p

150

)

USA

1990

16R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

114

)R

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

(p1

18)

Pow

eran

dbu

dget

stru

ggle

s(p

p11

111

5)P

oorq

ualit

yof

rese

arch

(pp

119

^122

)U

SA19

9117

Goo

dqu

ality

rese

arch

(p5

19)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(pp

519

520)

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

19)

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

519

)U

K19

9118

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

(p5

5)N

otst

ated

USA

1993

^199

419G

ood

qual

ityre

sear

ch(p

27)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p2

7)Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

9)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

chA

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(pp

13^1

427

)

UK

1994

^199

5202

1Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pp

32^3

4)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

p32

^34)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

pp

31^3

2)P

oorq

ualit

yof

rese

arch

(p3

3)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p3

3)M

exic

o19

94^1

99522

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

sTi

mel

ines

san

dre

leva

nce

ofth

ere

sear

chR

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

nsR

esea

rch

that

conf

irms

curr

entp

olic

yor

endo

rses

self-

inte

rest

Com

mun

itypr

essu

reor

clie

ntde

man

dfo

rres

earc

hIn

clus

ion

ofef

fect

iven

ess

data

(pp

106^

108

109^

113)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oliti

cali

nsta

bilit

yor

high

turn

over

ofpo

licy-

mak

ing

staf

f(p

p10

7^11

1)

USA

1996

24Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p2

1)A

ustra

lia19

9523

Pers

onal

cont

actb

etw

een

rese

arch

ers

and

dec

isio

n-m

aker

s(p

107

)Ti

mel

ines

san

dre

leva

nce

ofth

ere

sear

ch(p

108

)G

ood

qual

ityre

sear

ch(p

108

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

108

)

Poo

rqua

lity

ofre

sear

ch(p

106

)La

ckof

timel

ines

sor

rele

vanc

eof

rese

arch

(p1

06)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

106

)

cont

inue

d



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated



Tab

le3

cont

inue

d

Stu

dy

refe

ren

ceFa

cilit

ato

rsto

use

ofr

esea

rch

iden

ticente

din

thre

eo

rm

ore

stu

die

sB

arri

ers

tou

seo

fre

sear

chid

enti

cented

inth

ree

or

mo

rest

ud

ies

Paki

stan

1998

26N

otst

ated

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

p78

82)

Pow

eran

dbu

dget

stru

ggle

s(p

80)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p80

82)

Poo

rqua

lity

ofre

sear

ch(p

80)

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

(p8

2)U

K19

9827

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p8

)R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

13)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p8

12)

UK

and

USA

1998

28Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

Goo

dqu

ality

rese

arch

Res

earc

hth

atin

clud

esa

sum

mar

yw

ithcl

earr

ecom

men

dat

ions

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

Incl

usio

nof

effe

ctiv

enes

sda

ta(p

p3

17^2

1)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

sP

oorq

ualit

yof

rese

arch

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p5

12^1

7)

Can

ada

1998

^199

929Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(p3

2)P

ower

and

budg

etst

rugg

les

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

p33

^34)

Bur

kina

Faso

1999

30N

otst

ated

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p2

1)A

bse

nce

ofpe

rson

alco

ntac

tbet

wee

npo

licy-

mak

ers

and

rese

arch

ers

(p2

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

23)

Sout

hA

frica

2000

31R

esea

rch

that

incl

udes

asu

mm

ary

with

clea

rrec

omm

end

atio

ns(p

62)

Tim

elin

ess

and

rele

vanc

eof

the

rese

arch

(p6

2)M

utua

lmis

trust

betw

een

polic

y-m

aker

san

dre

sear

cher

s(p

63)

Ab

senc

eof

pers

onal

cont

actb

etw

een

polic

y-m

aker

san

dre

sear

cher

s(p

61)

Lack

oftim

elin

ess

orre

leva

nce

ofre

sear

ch(p

61)

Pol

itica

lins

tab

ility

orhi

ghtu

rnov

erof

polic

y-m

akin

gst

aff(

p6

1)C

anad

a20

0032

Not

stat

edP

ower

and

budg

etst

rugg

les

Mut

ualm

istru

stbe

twee

npo

licy-

mak

ers

and

rese

arch

ers

Poo

rqua

lity

ofre

sear

ch(p

26)

Can

ada

2000

33Pe

rson

alco

ntac

tbet

wee

nre

sear

cher

san

dd

ecis

ion-

mak

ers

(pi

)N

otst

ated

Documents

Health policy-makers' perceptions of their use of evidence: a systematic review