Research Communications

Caroline Cassidy

Click icon to add partner logo

How to communicate impact evaluations…

A short clip to watch

http://blip.tv/ictkm-program/how-science-can-not-only-predict-but-also-mitigate-the-effects-of-natural-disasters-4114862

Communication channels

Publications Media

Events Digital

Brainstorm: Communications channels

• Make a list of different formats and outputs you could use for communication of research within your assigned channel (publications, media, events, online)

• For each format or output, try to identify:– Which audience it best serves?– When it may be best to use it within a research

process?– Any costs or resourcing?

10 mins in your groups

Policy briefs:

A research communication tool

Evaluating policy briefs

Exercise 1

Evaluating policy briefs

• Each group has a Policy Brief. DON’T LOOK YET!• When we say ‘GO’, you will have THREE MINUTES to

read through the policy brief (you may want to take notes)

• And then hide the Policy Brief again…

Evaluating policy briefs

• In your groups, note down the answers to the following questions - you have 5 minutes and no peeping at the briefs!:1. What issue is the policy brief aiming to change,

validate or inform?2. Who do you think is the audience for this brief?3. What recommendations are made?4. Is there any specific piece of evidence that sticks

out in your mind?• Each group will have two minutes to present your

answers

Experience sharing

So what is a policy brief?

What is a policy brief?

A concise, standalone document that focuses on a particular issue requiring policy attention:• Explains and conveys the urgency of an issue• Presents policy recommendations or implications• Gives evidence to support these

recommendations• Points the reader to additional resources

What is a policy brief?

• Policy Briefs are the most popular form of publication in think tanks. Why? Because they can help bring together the research and policy communities

Source: CIPPEC and GDN’s Comparative study of policy research institutes in developing countries

Policy briefs as one part of your communications strategy

• Timing is important

• Where does your policy brief sit alongside other activities in your strategy?

Source: Jones and Walsh (2008), Policy briefs as a communication tool for development research.

A tool within communications activities

• Policy Briefs are one of many communications tools

Source: CIPPEC and GDN’s Comparative Analysis of think tanks in Asia, Africa and Latin America

Types of policy briefs

The type of policy brief depends on:• the objectives and messages of the research • the specific context within which your research takes place • the audience identified• the author(s)•other communications activities that are being planned alongside your policy brief

Different types of policy brief

– Policy Briefs as a memo

– Policy Briefs as a publication – print and/or online

– Policy Briefs as a marketing tool / hand out

– Policy Briefs as a targeted research summary

– Policy brief as part of a presentation

Types of policy briefs

The PB as memo The PB as publication

• Often internally focused• Demand-driven• Narrow audience• A general overview of the subject showing multiple opinions or view points• Might give multiple, and even competing, solutions

• Externally focused

• Supply-driven• Often broad audience• Targeted research summary that supports a main argument

• Gives strong, clear and coordinated policy recommendations or implications• Often used as a marketing tool

Planning your policy brief

19

Considering the objective(s)

Analyse an issue / put an issue on the public agenda

Make recommendations on an issue / advocacy

Describes a problem or situation, analyses causes and points out options to improve the situation.

Gives a clear opinion on a specific relevant topic in a particular moment and proposes a certain action

Presents alternatives to solution without suggesting the best one; offering costs and benefits from each alternative.

Offers a range of options and then proposes a specific solution using an evidence-based argument

Is oriented to a general audience (technical concepts explained)

Is oriented to an expert audience

Context-driven

Monitoring and Evaluation

Agenda Setting Decision

Making

Policy Implementation

Policy Formulation

Civil Society

DonorsCabinet

Parliament

Ministries

Private Sector

The shoes of your audience

• Who is your brief aimed at?• Does the audience know you? • How much does your target

audience know about the issue? • How do they perceive the issue?• What questions do they need answers

to?• How open are they to your

message(s)?

Audience-driven

Common audiences

Non-academic / non-specialistDecision-makers who may have varying degrees of expertise on a given issueIn certain cases may target practitionersNot usually general public

Policy-makers views:• Do present evidence-

informed opinions• Don’t shy away from

opinion and value judgements

• But signpost which content is subjective and which is objective

Audience-driven

• Contextualise your evidence within existing research

• Consider institutional factors

• What are our areas of credibility?– The writers– The organisation

Considering the authors

Types of author(s)

ResearchersPolicy-oriented research institutesThink tanksCivil society organisationsAdvocacy organisationsInternational NGOsMultilateral organisationsGovernment bodiesNetworks/ coalitions of any of the above

What does an effective policy brief look like?

• Potential sections of a policy brief:– Title– Executive Summary – Introduction (10-15%)– Background and methodology (10-15%)– Results or policy options (30%)– Implications or recommendations (30%)– References and useful resources (10%)

REMEMBER: Policy briefs are two, four or a maximum of eight pages in length (approx. 1,200, 2,500 or 4,000 words)

Structure and content

ODI • Design to make people pick

them up• Key points – three short

statements, key messages• Boxes to indicate related

but not integral content (case studies, methodology)

• Policy implications or recommendations

• References / useful resources

• Authors not given prominenceCI

PPEC

• Executive summary to transmit key arguments without reading whole document.

• Problem description / diagnosis of the situation

• Chart of policy options presenting different views to highlight and acknowledge key actors

• (Conclusions and results)• Policy recommendations• References and authors

Examples

Starting off…

29

1. Title

• You might like to come back to this at the end!

• Keep it punchy – NOT TOO LONG• Include all key words for online search

• And a standfirst

• Example:Rising food prices: A global crisisAction needed now to avert poverty and hunger

• Designed to give an overview of the content of the brief

• Should contain ALL standard elements of a policy brief

• Emphasis on capturing the attention of the reader

• Appears on the first page

• Usually written last

This policy brief outlines actions the Tunisian government can take to reduce the remaining barriers to entry, exit and competition in the Tunisian manufacturing industries, with the aim of stimulating a greater dynamism and flexibility than the market has yet to achieve. It concludes that policies should be considered to:•   Work with Statistics bureaus to develop and collect information on entry and exit and other indicators of firm dynamics.

• Reduce entry barriers by developing market access and increasing the access of small and medium enterprises (SMEs) to financial resources.

‘Entry and Exit Dynamics and Economic Performance in Tunisian Manufacturing Industries’, University of Tunis 2009

2. Executive summary

• Designed to capture the attention of the reader– Explains the importance/

urgency of the issue– Creates curiosity about

the rest of the brief

• Gives a brief overview of the conclusions or the direction of the rest of the brief

• Outlines the structure for the rest of the brief

The increased fear of a pandemic atthe current time is due to the fact that theH5N1 strain causing the outbreak is capable of mutating rapidly and acquiring genes from viruses infecting other animal species, including humans. If the new virus contained sufficient human influenza virus genes, direct transmission from one person to another (human-to-human) could occur, heralding a possible influenza pandemic. The current H5N1 strain causes high mortality among humans, with mortality rates exceeding 50 per cent in most countries.AN EFFECTIVE REGIONAL RESPONSE TO THE THREAT OF A PANDEMIC UN ESCAP Issue N.1 2005

3. Introduction

The substance….

33

• Designed as an overview of the findings/ facts

• Constructed around the line(s) of argument behind the policy recommendations

• Move between general and specific information

• Base conclusions on evidence, data and findings

• Make clear, balanced and defensible assertions

Do The Electric Fences Work?Overall, it was found that although the electric fencing does help…it is not capable of completely eliminating conflict. In each …area…technical as well as socioeconomic factors affect…success. Technical failures mainly affected the early fences…Other problems resulted from failure to take into account elephant behavior and distribution patterns.Elephants and Electric Fences: A Study From Sri Lanka. EEPSEA 2005-PB11

4. Results & conclusions

Designed to strengthen the credibility of the brief

• Might include:– Description of the issue and

context of the investigation– Description of the research and

analysis activities

o What methods were used to conduct the study?

o Who undertook the data collection and analysis?

TIPS and TRICKS

• This section is not always applicable – make sure it’s relevant before including it• Avoid overly technical language• Highlight unique methods or data collection• Focus on methodological issues related to the main messages/ recommendations

5. Methodology

Implications • What policy changes or

actions do the results point to?

• Supported by evidence

• Must be actionable

• Less direct than recommendations

• Useful when advice not requested or not welcome

Recommendations • What does the researcher

think should happen?

• Supported by evidence

• Must be actionable

• Describe clearly what should happen next

• State as precise steps

6. Implications / recommendations

• Especially when synthesising from multiple sources, always cite them appropriately and be sure to add a list of references

• Include seminal works in the area for the audience to easily find out more on the topic

• Good opportunity to direct readers to other resources produced by the same author/ organisation/ network

7. Reference and useful resources

Evaluating policy briefs

Exercise 2

Evaluating policy briefs

• In pairs – have a further look at the policy briefs and discuss the following questions:

• What content of the policy brief did you like?• What content of the policy brief did you not like?• What about the format of the policy brief did you

like?• What about the format of the policy brief did you

not like?

Evaluating policy briefs....again!

Think back to the policy brief that you evaluated at the start of the session:

• Has your opinion changed? Is it better or worse than you thought?

• What is good about the content and design?

• What could be done better?

• Format & design can:– Engage people– Highlight crucial information– Separate content and signpost

your narrative– Convey authority– Show information in different

ways

Structure and the role of design

Some design ideas…

• Photographs• Graphs and

charts• Text boxes• Pull quotes• Side bars

Outlining policy briefs:

where to start

43

Brainstorming for policy briefs

1• Identify the purpose and overarching message of the

policy brief

2• Determine three key policy recommendations/

implications

3• Construct a logical line of argument for making these

recommendations

4• Based on an understanding of the context around the

issue, identify one or two entry points for the message

Defining your key messages…

45

Presenting your argument

• What do you want to say, and to whom?– Summarise the main point, argument or line of

reasoning into one or two clear and accessible sentences

• Messaging is about prioritisation, not about ‘dumbing down’

• Messages should be both clear and consistent, and should tell a coherent story

Presenting your argument

The elevator pitch

ExerciseThe elevator pitch tool

• Write your 3 key messages (5 minutes)

• Split into pairs – one of you is a policy maker that you meet in the lift, going from the 1st to the 12th floor.

• You have 2 minutes to pitch your key messages. Switch. (4 minutes – 2 minutes each)

ExerciseThe elevator pitch tool

• Write your 3 key messages (5 minutes)

• Split into pairs – one of you is a policy maker that you meet in the lift, going from the 1st to the 12th floor. You have 2 minutes to pitch your key messages. Switch. (4 minutes – 2 minutes each)

An example to use in the online

toolkit…

51

Example: objective

The objective for this policy brief is to encourage national governments around the world to embed impact evaluations in government decision-making processes by setting up independent evaluation departments.

Example: Recommendations

• All government programmes should be evaluated using rigorous impact evaluations.

• Governments should institute capacity development programmes for bureaucrats to help them design evaluable programmes.

• These processes could be supported through the establishment of an independent evaluation department in each country.

Example: Line of argument

Governments should institute capacity development programmes for bureaucrats to help them design evaluable programmes.

-Many policy makers do not understand what an impact evaluation actually is-This means they have limited understanding of the benefits and constraints of IEs-Many programmes and policies are currently not evaluable

In South Africa, the gov’t announced a policy on subsidies for companies who employ young workers, but they don’t know if this is an effective way to reduce youth unemployment.

With the global financial crisis, many gov’t budgets have been cut and many programmes are losing funding. It is therefore important to make sure funding for programmes that work is not cut.

In the UK, there were budget cuts amounting to x% of the annual budget.

Example: Hooks (entry points)

• An ‘age of austerity’ is sweeping through governments as the medium term impacts of the global financial crisis start to be felt.

• Budgets are being cut, but usually on an ideological basis rather than because a programme is ineffective.

Example : Outline

Introduction

– In the 2010/11 budget announced in the UK, government spending will drop by 20%.

– But programmes are being cut without information as to whether they are effective.

– MAIN ARGUMENT: Governments must institutionalise impact evaluations of their programmes so that they know what works.

Example : Outline

Methodology

– (As this is secondary research instead of primary research, we do not think this is a useful section in this policy brief)

Example : Outline

What are impact evaluations?– Rigorous studies of the effectiveness of

programmes– Comprise different methodologies, but commonly

use Randomised Control Trials to establish statistical significance

– Examples of well known IEs:• Progresa/Oportunidades in Mexico on conditional

cash transfers• Deworming programmes and their effect on

school attendance

Example : Outline

What role do IEs have in policy making?

– IEs show whether or not a programme is effective at achieving its stated goals.

– Example:• Progresa/Opportunidades in Mexico: results from

the IE kept the programme alive even after a change in government

Example : Outline

Why are IEs not used currently?– Programmes are not currently designed in a way

that they can be evaluated using IEs• RCTs, for example, require a control group and a

treatment group– There is little understanding of what IEs are among

policy makers• One policy maker in South Africa suggested an IE

of his programme and asked for it in a month. But IEs often take a minimum of two years to complete

Example : Outline

Recommendations:– All government programmes should be evaluated

using rigorous impact evaluations.• A• B• C

– Governments should institute capacity development programmes for bureaucrats to help them design evaluable programmes.

– These processes could be supported through the establishment of an independent evaluation department in each country.

A tool within communications activities

• Policy Briefs are one of many communications tools

Source: CIPPEC and GDN’s Comparative Analysis of think tanks in Asia, Africa and Latin America

The views presented here are those of the speakers, and do not necessarily represent the views of ODI, or our partners.

Click icon to add partner logo Click icon to add partner logo Click icon to add partner logo

www.odi.org.uk

c.cassidy@odi.org.uk

Click icon to add partner logo