Upload
elfrieda-lindsey
View
213
Download
0
Embed Size (px)
Citation preview
1
How Policy-Makers View Evidence: Lessons from the
RWJF Synthesis Project
Claudia WilliamsAZA Consulting
June 25, 2006
2
Why aren’t research results better used by policy-makers?
Policy-makers are in information overload
Research does not pass the “so what” test
Research results are not translated for policy decisions
• Too technical and indirect
• Isolated findings that have not been synthesized
Results are not fed into best channels
Opinion leaders, lobbyists and press
Use people not paper
Often need basic information
Won’t read long reports
Skim for “story line” and conclusions
Policy-maker Researcher
Journals and research conferences
Peer-reviewed journals
Information is technical and sophisticated
Reports often 50+ pages
Lead with methodology and background. Results last.
Information Channels
Informationhabits
Level ofInformation
Format
Mismatch between policy and research
4
Joseph Nye expressed it well
“It's a totally different world, government, from academics. In academic life, there's no premium on time, the premium is on getting it just right. In government, if you haven't got the right answer by four o'clock this afternoon when the president meets with the prime minister, that perfect paper you get in a little bit late is an "F.” The idea that I would have time to read a thirty page paper -- ! I used to write them, and I found that I couldn't read them (when I worked in government)…
So this problem of how do you take chaotic reality and try to shape the right questions, even before you get answers, is very different in the government setting than in the academic setting. Because in the academic setting there's a luxury, there's no time limit. You can sort your way through it, figure it out; if you don't have the answer you go back to the library and look up more data and so forth. In government you either solve the problem or get the right answer quickly or it doesn't happen at all. And it's quite a different set of skills. The premium we put on time makes a huge difference.”
Joseph Nye, Dean of the Kennedy School of Government1997 University of California at Berkeley Interview
Synthesis development process
6
Synthesis strategies Choose topics carefully
Create meaning not noise
Synthesize bodies of information
Pair research with policy expertise
Use active dissemination
7
Choose topics carefully
Select perennial “thorn in the side” issues
Solicit topic ideas from policy audience• Start with “what is on their plates”• Make sure topics pass “so what” test
Certain policy decisions—often big ideological questions—are not answered or addressed through better information
Work with advisory group to narrow and select best topics
8
9
Create meaning not noise
Help organize and manage information, addressing not aggravating information overload
Make synthesis visual and skimmable, leading with conclusions and telling a story
Policy questions drive the information, not the intricacies of the research
Use multiple layers so users can read at different levels
10
Synthesize bodies of information
Synthesize bodies of evidence instead of producing isolated findings
Organize, structure and make sense of information, putting it into understandable frameworks
Bring diverse findings onto the same footing • Compare and weigh findings• Reach conclusions based on best evidence
11
Pair research with policy expertise
This approach does not come easily to many researchers
Best known researcher often not best synthesizer
Requires a team approach
The process is iterative and intense
12
Use active dissemination
Goals of dissemination: share findings and foster discussion and dialogue
Researchers need a way to communicate more directly with policy-makers
Partner with knowledge broker and convening organizations
13
Methodological issues
Started with notion of a clear bar…but weaker literature sometimes needs to be discussed
Policy staff want to reach their own conclusions about research findings
Synthesis should not mask details on the approach, methodologies and bias of underlying evidence
• Compare results• Evaluate methods• Technical information separated out
Synthesis framework
TRANSLATE AND ORGANIZE• Build conceptual frameworks• Construct bridges• Organize around policy issues
RESEARCH FINDINGS ARE:• Too long and complicated• Fragmented • Unclear and confusing
BUILD INFRASTRUCTURE• Tracking function • Credible review• Advisory group
CREATE FORMAT AND DESIGN• Short and skimmable• Objective and balanced• Provocative• Links to other resources
The Problem
The Synthesis Solution
ENGAGE AND DISSEMINATE• Foster dialogue• Actively engage• Get press attention
Use Results to
Improve
Research
Topics?
Audience?
Partnerships?
15
Rules to live by Government “think tank” staff are ideal audience
Syntheses should lay out areas where the evidence points to a clear conclusion as well as areas where there is debate
Syntheses need policy as well as research input throughout
Syntheses should be candid, objective and credible
16
Contact information
Projectemail [email protected] site www.policysynthesis.org
David Colby, RWJF
Claudia Williams, AZA Consulting phone 571.641.3030email [email protected]