View
4
Download
0
Category
Preview:
Citation preview
www.3ieimpact.org Hugh Waddington
Farmer field schools: a
systematic review
International Initiative for Impact Evaluation
Hugh Waddington
www.3ieimpact.org Hugh Waddington
Co-authors
• Birte Snilstveit
• Jorge Hombrados
• Martina Vojtkova
• Daniel Phillips
• Howard White
www.3ieimpact.org Hugh Waddington
Agriculture starting to come back on the agenda
Source: Cabral and Howell 2012, ODI
www.3ieimpact.org Hugh Waddington
Agricultural extension
• But age old questions remain:
– how to raise farmer productivity?
– how to reach the poorest and marginalised (eg
women)?
• Ag extension has been part of tool box forever but got a
bad rap in last two decades – e.g. rise and fall of T&V
• Participatory extension like Farmer Field Schools are
what is new, the latest fad "that works"
www.3ieimpact.org Hugh Waddington
• Originally associated with
FAO and Integrated Pest
Management (IPM)
• Originated in response to
the overuse of pesticides in
irrigated rice systems in
Asia
• Belief that farmers need
confidence to reduce
dependence on pesticides,
through „discovery learning‟
• Now applied in 90+
countries, range of crops
and curricula
FFS history lesson
www.3ieimpact.org Hugh Waddington
• Group of 25 farmers, meeting once a week in a designated field during the growing season
• Exploratory: facilitator encourages farmers to ask questions, and to seek answers, rather than lecturing or giving recommendations.
• Experimentation: group manages two plots
• Participatory: emphasis on social learning with exercises to build group dynamics
• Field days and follow-up activities may be provided for diffusion of message to neighbours
A „best practice‟ FFS
www.3ieimpact.org Hugh Waddington
Farmer field school
stylised ToC... Input 1 Training
of trainers Input 2 Field
school
Adoption
(FFS
participants)
Capacity
building (FFS
participants)
Capacity
building (FFS
neighbours)
Adoption
(FFS
neighbours)
Measured impacts: Yield, input-output ratio,
income, empowerment,
environmental
outcomes, health
www.3ieimpact.org Hugh Waddington
Input 1 Training
of trainers Input 2 Field
school
Adoption
(FFS
participants)
Capacity
building (FFS
participants)
Capacity
building
(neighbours)
Adoption
(neighbours)
- Facilitators
adequately trained
- Farmers and
facilitators attend
sufficient meetings
- FFS synchronised
with planting
season
- Curriculum
relevant to
problems facing
farmers
-Farmer attitudes
changed
(convinced
message
appropriate)
- Relative
advantage over old
techniques
- Field
days/follow-up
- High degree of
social cohesion
- Geographical
proximity to other
farmers
(observation) or
market
(communication)
Measured impacts: Yield, input-output ratio,
income, empowerment,
environmental
outcomes. health
- New technology
appropriate
- Market access
- Favorable prices
- Environmental factors
including weather, soil
fertility
…with assumptions
www.3ieimpact.org Hugh Waddington
Polarised debate on FFS
• "Studies reported substantial and consistent reductions
in pesticide use attributable to the effect of training. In a
number of cases, there was also a convincing increase
in yield due to training.... Results demonstrated
remarkable, widespread and lasting developmental
impacts” (Van den Berg 2004, FAO)
• “The analysis, employing a modified „difference-in-
differences‟ model, indicates that the program did not
have significant impacts on the performance of
graduates and their neighbors” (Feder et al. 2004)
• But how good are they really - what does a systematic
look at the evidence say?
www.3ieimpact.org Hugh Waddington
Theory-based systematic review
• Review registered with Campbell Collaboration
• Uses theory of change to examine program mechanisms and outcomes along causal chain
• 3-part data review: – Quantitative review of effects (impact evaluations)
– Qualitative review of barriers and facilitators
– Global portfolio review of projects
• Integrated synthesis based around causal chain
www.3ieimpact.org Hugh Waddington
• Population is farm households in low and middle income
countries (data collected and analysed at household level)
• Intervention: programmes explicitly referred to as „farmer
field school‟
• Outcomes: effectiveness across the causal chain
– Knowledge (+ attitudes): what was learnt?
– Adoption: did farmers utilise new technologies (methods of planting,
approach to disease/pest control, other inputs)?
– Impact on yields, revenues, environment, health, empowerment etc.
• Study designs:
– Effects: experimental, quasi-experimental with controlled comparison
(no treatment, pipeline, other intervention)
– Barriers/facilitators: qualitative with reporting on data collection
(CASP)
Review inclusion criteria (PICOS)
www.3ieimpact.org Hugh Waddington
• Comprehensive search for published and
unpublished literature:
– General: SSCI, IBSS, EconLit,
– Subject specific: AgEcon, CAB Abstracts, Agricola,
US National Agricultural Library
– „Unpublished‟: JOLIS, BLDS, IDEAS, Google, Google
Scholar, Theses and Dissertations
• Hand search (journals, organisation websites)
• Literature snowballing (citation tracking)
• Contact key researchers and organisations
Search methods
www.3ieimpact.org Hugh Waddington
1,112 abstracts
screened
751 excluded
312 full text sought
49 no access
183 Extension impact papers: 134 FFS
49 non-FFS
257 excluded
1453 abstracts
screened
27,866 titles screened
369 full text obtained
126 no access
186 excluded: 128 on relevance 58 on design (no
comparison)
134 FFS impact papers
80 individual FFS studies
25 qualitative
papers
Causal Chain
Analysis Effectiveness
20 individual
FFS studies
30 IE and sister
papers
11 individual
FFS studies
Qualitative
Synthesis BB+ Synthesis
www.3ieimpact.org Hugh Waddington
• 93 FFS interventions: East Asia, South Asia, Latin America, Middle East, sub-Saharan Africa
– Cotton, rice, other cash crops, and food crops
– IPM, IPPM, IPNM, ICM, IWM
– Some with co-interventions (input support, marketing support)
• Design: No RCTs; quasi-experiments of varying quality (PSM, diff-in-diff, instrumental variables, Heckman, group means comparison)
• All effects measured relative to non-FFS farmers comparison; study arms include FFS-participants and ‘neighbouring’ farmers to measure spillovers (farmer-to-farmer diffusion)
• Small samples (approx. 200 farmers, often only a handful of villages) and short follow-up periods (most studies less than 2 years)
Characteristics of included impact evaluations
www.3ieimpact.org Hugh Waddington
Synthesis of outcomes across
the causal chain:
Knowledge adoption diffusion
agriculture yields net income (profits)
Environment
Health outcomes
Empowerment
www.3ieimpact.org Hugh Waddington
Effect sizes
• Intermediate and endpoint outcomes
synthesised
• Standardised mean differences for outcomes
without „natural‟ scale unit and zero points
(knowledge and adoption indexes)
• Response ratios for outcomes based on ratio
scale (pesticide use, yields, income,
environment) and probabilities (health,
empowerment)
www.3ieimpact.org Hugh Waddington
Positive
impacts on
knowledge
among
participants
NOTE: Weights are from random effects analysis
.
.
FFS participants
Huan et al., 1999 (Vietnam)
Endalew, 2009 (Ethiopia)
Price et al., 2001 (Philippines)
Rao et al., 2012 (India)
Reddy & Suryamani, 2005 (India)
Kelemework, 2005 (Ethiopia)
Mutandwa & Mpangwa, 2004 (Zimbabwe)
Dinpanah et al., 2010 (Iran)
Khan et al., 2007 (Pakistan)
Bunyatta et al., 2006 (Kenya)
Erbaugh, 2010 (Uganda)
Rebaudo & Dangles, 2011 (Ecuador)
Subtotal (I-squared = 93.3%, p = 0.000)
FFS neighbours
Khan et al., 2007 (Pakistan)
Reddy & Suryamani, 2005 (India)
Ricker-Gilbert et al, 2008 (Bangladesh)
Rebaudo & Dangles, 2011 (Ecuador)
Subtotal (I-squared = 0.0%, p = 0.610)
ID
Study
0.02 (-0.06, 0.10)
0.27 (-0.06, 0.60)
0.42 (-0.17, 1.01)
0.43 (-0.02, 0.87)
0.45 (-0.04, 0.94)
0.54 (-0.22, 1.29)
0.59 (0.25, 0.92)
0.67 (0.41, 0.92)
0.79 (0.29, 1.29)
1.03 (0.65, 1.41)
1.14 (0.93, 1.34)
1.79 (1.17, 2.41)
0.66 (0.33, 1.00)
-0.13 (-0.68, 0.42)
0.05 (-0.45, 0.56)
0.17 (-0.25, 0.59)
0.38 (-0.15, 0.91)
0.13 (-0.12, 0.37)
ES (95% CI)
0.02 (-0.06, 0.10)
0.27 (-0.06, 0.60)
0.42 (-0.17, 1.01)
0.43 (-0.02, 0.87)
0.45 (-0.04, 0.94)
0.54 (-0.22, 1.29)
0.59 (0.25, 0.92)
0.67 (0.41, 0.92)
0.79 (0.29, 1.29)
1.03 (0.65, 1.41)
1.14 (0.93, 1.34)
1.79 (1.17, 2.41)
0.66 (0.33, 1.00)
-0.13 (-0.68, 0.42)
0.05 (-0.45, 0.56)
0.17 (-0.25, 0.59)
0.38 (-0.15, 0.91)
0.13 (-0.12, 0.37)
ES (95% CI)
Favours intervention
0-.5 0 .5 1 3
www.3ieimpact.org Hugh Waddington
Reduced
pesticide
demand
among
participants
not
neighbours
NOTE: Weights are from random effects analysis
.
.
FFS neighboursPananurak, 2010 (India)Khan et al., 2007 (Pakistan)Yamazaki & Resosudarmo, 2007 (Indonesia)Wu Lifeng, 2010 (China)Pananurak, 2010 (Pakistan)Labarta, 2005 (Nicaragua)Pananurak, 2010 (China)Praneetvatakul & Waibel, 2006 (Thailand)Khan et al., 2007 (Pakistan)Feder et al, 2004 (Indonesia)Subtotal (I-squared = 84.6%, p = 0.000)
FFS participantsYamazaki & Resosudarmo, 2007 (Indonesia)Birthal et al., 2000 (India)Yang et al., 2005 (China)Yorobe & Rejesus, 2011 (Philippines)Yang et al., 2005 (China)Khan et al., 2007 (Pakistan)Khalid, n.d. (Sudan)Rejesus et al, 2010 (Vietnam)Pananurak, 2010 (India)Mutandwa & Mpangwa, 2004 (Zimbabwe)Pananurak, 2010 (Pakistan)Amera, 2008 (Kenya)Pananurak, 2010 (China)Mancini et al., 2008 (India)Wu Lifeng, 2010 (China)Huan et al., 1999 (Vietnam)Van den Berg et al., 2002 (Sri Lanka)Praneetvatakul & Waibel, 2006 (Thailand)Murphy et al., 2002 Vietnam)Cole et al., 2007 (Ecuador)Ali & Sharif, 2011 (Pakistan)Khan et al., 2007 (Pakistan)Labarta, 2005 (Nicaragua)Feder et al, 2004 (Indonesia)Cavatassi et al., 2011 (Ecuador)Friis-Hansen et al., 2004 (Uganda)Subtotal (I-squared = 93.2%, p = 0.000)
IDStudy
0.54 (0.25, 1.15)0.61 (0.51, 0.74)0.67 (0.12, 3.88)0.68 (0.62, 0.76)0.78 (0.40, 1.49)0.99 (0.42, 2.33)1.11 (0.69, 1.79)1.15 (0.92, 1.43)1.20 (0.40, 3.53)1.30 (1.09, 1.55)0.88 (0.68, 1.14)
0.20 (0.01, 3.23)0.21 (0.17, 0.26)0.32 (0.21, 0.48)0.37 (0.18, 0.78)0.41 (0.36, 0.46)0.46 (0.39, 0.54)0.48 (0.31, 0.75)0.52 (0.24, 1.12)0.52 (0.30, 0.92)0.57 (0.36, 0.89)0.59 (0.41, 0.87)0.61 (0.52, 0.71)0.65 (0.50, 0.84)0.67 (0.46, 0.97)0.71 (0.64, 0.80)0.72 (0.62, 0.84)0.82 (0.74, 0.90)0.82 (0.68, 0.98)0.83 (0.75, 0.93)0.88 (0.68, 1.13)0.90 (0.75, 1.09)0.91 (0.28, 2.94)0.95 (0.39, 2.34)1.30 (1.08, 1.57)1.34 (0.99, 1.80)1.42 (1.09, 1.86)0.66 (0.56, 0.78)
ES (95% CI)
0.54 (0.25, 1.15)0.61 (0.51, 0.74)0.67 (0.12, 3.88)0.68 (0.62, 0.76)0.78 (0.40, 1.49)0.99 (0.42, 2.33)1.11 (0.69, 1.79)1.15 (0.92, 1.43)1.20 (0.40, 3.53)1.30 (1.09, 1.55)0.88 (0.68, 1.14)
0.20 (0.01, 3.23)0.21 (0.17, 0.26)0.32 (0.21, 0.48)0.37 (0.18, 0.78)0.41 (0.36, 0.46)0.46 (0.39, 0.54)0.48 (0.31, 0.75)0.52 (0.24, 1.12)0.52 (0.30, 0.92)0.57 (0.36, 0.89)0.59 (0.41, 0.87)0.61 (0.52, 0.71)0.65 (0.50, 0.84)0.67 (0.46, 0.97)0.71 (0.64, 0.80)0.72 (0.62, 0.84)0.82 (0.74, 0.90)0.82 (0.68, 0.98)0.83 (0.75, 0.93)0.88 (0.68, 1.13)0.90 (0.75, 1.09)0.91 (0.28, 2.94)0.95 (0.39, 2.34)1.30 (1.08, 1.57)1.34 (0.99, 1.80)1.42 (1.09, 1.86)0.66 (0.56, 0.78)
ES (95% CI)
Favours intervention
1.1 .25 .5 1 2
www.3ieimpact.org Hugh Waddington
Positive
impacts on
other
adoption
measures
among
beneficiaries
NOTE: Weights are from random effects analysis
Overall (I-squared = 94.2%, p = 0.000)
Kelemework, 2005 (Ethiopia)
Dinpanah et al., 2010 (Iran)
Mauceri et al., 2007 (Ecuador)
Rao et al., 2012 (India)
Ricker-Gilbert et al, 2008 (Bangladesh)
Bunyatta et al., 2006 (Kenya)
Endalew, 2009 (Ethiopia)
ID
Zuger 2004 (Peru)
Study
0.73 (0.14, 1.32)
0.70 (-0.28, 1.67)
1.46 (1.18, 1.73)
1.41 (0.96, 1.86)
0.11 (-0.33, 0.55)
0.92 (0.29, 1.55)
1.45 (1.04, 1.85)
0.24 (-0.09, 0.58)
ES (95% CI)
-0.41 (-0.71, -0.11)
0.73 (0.14, 1.32)
0.70 (-0.28, 1.67)
1.46 (1.18, 1.73)
1.41 (0.96, 1.86)
0.11 (-0.33, 0.55)
0.92 (0.29, 1.55)
1.45 (1.04, 1.85)
0.24 (-0.09, 0.58)
ES (95% CI)
-0.41 (-0.71, -0.11)
Favours intervention
0-.5 0 .5 1 2 3
www.3ieimpact.org Hugh Waddington
Increased
yields among
FFS-
beneficiaries
not
neighbours
NOTE: Weights are from random effects analysis
.
.
FFS neighbours
Pananurak, 2010 (India)
Khan et al., 2007 (Pakistan)
Feder et al, 2004 (Indonesia)
Labarta, 2005 (Nicaragua)
Pananurak, 2010 (China)
Wu Lifeng, 2010 (China)
Pananurak, 2010 (Pakistan)
Yamazaki & Resosudarmo, 2007 (Indonesia)
Subtotal (I-squared = 49.5%, p = 0.054)
FFS participants
Pananurak, 2010 (India)
Van Rijn, 2010 (Peru)
Naik et al., 2008 (India)
Huan et al., 1999 (Vietnam)
Labarta, 2005 (Nicaragua)
Rejesus et al, 2010 (Vietnam)
Feder et al, 2004 (Indonesia)
Wu Lifeng, 2010 (China)
Ali & Sharif, 2011 (Pakistan)
Pananurak, 2010 (China)
Gockowski et al., 2010 (Ghana)
Yang et al., 2005 (China)
Hiller et al., 2009 (Kenya)
Khan et al., 2007 (Pakistan)
Cavatassi et al., 2011 (Ecuador)
Davis et al, 2012 (Tanzania)
Birthal et al., 2000 (India)
Pananurak, 2010 (Pakistan)
Dinpanah et al., 2010 (Iran)
Wandji et al., 2007 (Cameroon)
Mutandwa & Mpangwa, 2004 (Zimbabwe)
Palis, 1998 (Philippines)
Zuger 2004 (Peru)
Carlberg et al., 2012 (Ghana)
Yamazaki & Resosudarmo, 2007 (Indonesia)
Van den Berg et al., 2002 (Sri Lanka)
Davis et al, 2012 (Kenya)
Pande et al., 2009 (Nepal)
Dinpanah et al., 2010 (Iran)
Orozco Cirilo et al., 2008 b) (Mexico)
Todo & Takahashi, 2011 (Ethiopia)
Subtotal (I-squared = 93.0%, p = 0.000)
ID
Study
0.79 (0.63, 1.00)
0.97 (0.74, 1.26)
0.99 (0.97, 1.01)
1.00 (0.99, 1.01)
1.02 (0.98, 1.07)
1.03 (0.99, 1.08)
1.03 (0.86, 1.25)
1.43 (1.05, 1.96)
1.01 (0.98, 1.03)
0.80 (0.61, 1.05)
0.86 (0.63, 1.18)
0.89 (0.83, 0.96)
0.95 (0.92, 0.98)
0.97 (0.92, 1.02)
0.97 (0.72, 1.31)
0.98 (0.96, 1.01)
1.08 (1.03, 1.14)
1.09 (1.03, 1.15)
1.09 (1.04, 1.14)
1.14 (1.03, 1.25)
1.15 (0.94, 1.41)
1.17 (0.53, 2.56)
1.17 (0.97, 1.42)
1.22 (0.97, 1.53)
1.23 (1.00, 1.51)
1.24 (1.13, 1.36)
1.24 (1.01, 1.54)
1.32 (1.22, 1.42)
1.32 (1.07, 1.63)
1.36 (1.06, 1.73)
1.36 (0.97, 1.92)
1.44 (1.09, 1.92)
1.58 (1.19, 2.10)
1.67 (1.23, 2.26)
1.68 (1.30, 2.18)
1.81 (1.15, 2.84)
2.11 (1.25, 3.56)
2.52 (2.05, 3.11)
2.62 (2.23, 3.08)
2.71 (1.11, 6.60)
1.23 (1.16, 1.32)
ES (95% CI)
0.79 (0.63, 1.00)
0.97 (0.74, 1.26)
0.99 (0.97, 1.01)
1.00 (0.99, 1.01)
1.02 (0.98, 1.07)
1.03 (0.99, 1.08)
1.03 (0.86, 1.25)
1.43 (1.05, 1.96)
1.01 (0.98, 1.03)
0.80 (0.61, 1.05)
0.86 (0.63, 1.18)
0.89 (0.83, 0.96)
0.95 (0.92, 0.98)
0.97 (0.92, 1.02)
0.97 (0.72, 1.31)
0.98 (0.96, 1.01)
1.08 (1.03, 1.14)
1.09 (1.03, 1.15)
1.09 (1.04, 1.14)
1.14 (1.03, 1.25)
1.15 (0.94, 1.41)
1.17 (0.53, 2.56)
1.17 (0.97, 1.42)
1.22 (0.97, 1.53)
1.23 (1.00, 1.51)
1.24 (1.13, 1.36)
1.24 (1.01, 1.54)
1.32 (1.22, 1.42)
1.32 (1.07, 1.63)
1.36 (1.06, 1.73)
1.36 (0.97, 1.92)
1.44 (1.09, 1.92)
1.58 (1.19, 2.10)
1.67 (1.23, 2.26)
1.68 (1.30, 2.18)
1.81 (1.15, 2.84)
2.11 (1.25, 3.56)
2.52 (2.05, 3.11)
2.62 (2.23, 3.08)
2.71 (1.11, 6.60)
1.23 (1.16, 1.32)
ES (95% CI)
Favours intervention
1.5 1 2 3
www.3ieimpact.org Hugh Waddington
Increased
revenues
among
participants
of FFS and
FFS+
NOTE: Weights are from random effects analysis
.
.
.
FFS neighbours
Pananurak, 2010 (India)
Pananurak, 2010 (China)
Pananurak, 2010 (Pakistan)
Labarta, 2005 (Nicaragua)
Khan et al., 2007 (Pakistan)
Subtotal (I-squared = 0.0%, p = 0.706)
FFS participants
Labarta, 2005 (Nicaragua)
Pananurak, 2010 (India)
Waarts et al., 2012 (Kenya)
Pananurak, 2010 (China)
Pananurak, 2010 (Pakistan)
Naik et al., 2008 (India)
Van de Fliert 2000 (Indonesia)
Van den Berg et al., 2002 (Sri Lanka)
Yang et al., 2005 (China)
Khan et al., 2007 (Pakistan)
Subtotal (I-squared = 57.1%, p = 0.013)
FFS+ participants
Birthal et al., 2000 (India)
Van Rijn, 2010 (Peru)
Cavatassi et al., 2011 (Ecuador)
Palis, 1998 (Philippines)
Subtotal (I-squared = 96.2%, p = 0.000)
ID
Study
0.93 (0.66, 1.32)
1.07 (1.00, 1.14)
1.13 (1.01, 1.26)
1.39 (0.66, 2.92)
1.51 (0.51, 4.45)
1.08 (1.03, 1.15)
0.28 (0.02, 3.48)
1.06 (0.68, 1.66)
1.14 (0.92, 1.41)
1.17 (1.08, 1.27)
1.23 (1.09, 1.40)
1.25 (1.09, 1.42)
1.31 (1.11, 1.55)
1.41 (1.19, 1.67)
1.53 (1.10, 2.15)
3.40 (1.94, 5.97)
1.28 (1.17, 1.41)
1.43 (1.19, 1.72)
2.00 (1.02, 3.94)
3.34 (1.56, 7.15)
4.61 (3.83, 5.56)
2.57 (1.18, 5.58)
ES (95% CI)
0.93 (0.66, 1.32)
1.07 (1.00, 1.14)
1.13 (1.01, 1.26)
1.39 (0.66, 2.92)
1.51 (0.51, 4.45)
1.08 (1.03, 1.15)
0.28 (0.02, 3.48)
1.06 (0.68, 1.66)
1.14 (0.92, 1.41)
1.17 (1.08, 1.27)
1.23 (1.09, 1.40)
1.25 (1.09, 1.42)
1.31 (1.11, 1.55)
1.41 (1.19, 1.67)
1.53 (1.10, 2.15)
3.40 (1.94, 5.97)
1.28 (1.17, 1.41)
1.43 (1.19, 1.72)
2.00 (1.02, 3.94)
3.34 (1.56, 7.15)
4.61 (3.83, 5.56)
2.57 (1.18, 5.58)
ES (95% CI)
Favours intervention
1 .5 1 2 3
www.3ieimpact.org Hugh Waddington
Limited knowledge spillovers among neighbours
explains lack of adoption and impact
Num complex practices known
Num intermediate practices known
Num simple practices known
ID
Study
-0.09 (-0.49, 0.32)
0.11 (-0.31, 0.53)
0.49 (0.11, 0.88)
ES (95% CI)
-0.09 (-0.49, 0.32)
0.11 (-0.31, 0.53)
0.49 (0.11, 0.88)
ES (95% CI)
Favours intervention 0-.5 0 .5 1 2
www.3ieimpact.org Hugh Waddington
Reduced environmental risk factors
NOTE: Weights are from random effects analysis
.
.
FFS participants
Pananurak, 2010 (India)
Praneetvatakul & Waibel, 2006 (Thailand)
Pananurak, 2010 (Pakistan)
Cavatassi et al., 2011 (Ecuador)
Subtotal (I-squared = 8.0%, p = 0.353)
FFS neighbours
Pananurak, 2010 (India)
Pananurak, 2010 (Pakistan)
Cavatassi et al., 2011 (Ecuador)
Praneetvatakul & Waibel, 2006 (Thailand)
Subtotal (I-squared = 0.0%, p = 0.878)
ID
Study
0.52 (0.32, 0.85)
0.54 (0.39, 0.76)
0.55 (0.41, 0.75)
0.82 (0.55, 1.23)
0.59 (0.49, 0.71)
0.58 (0.24, 1.41)
0.64 (0.37, 1.10)
0.69 (0.43, 1.11)
1.04 (0.32, 3.40)
0.68 (0.49, 0.93)
ES (95% CI)
0.52 (0.32, 0.85)
0.54 (0.39, 0.76)
0.55 (0.41, 0.75)
0.82 (0.55, 1.23)
0.59 (0.49, 0.71)
0.58 (0.24, 1.41)
0.64 (0.37, 1.10)
0.69 (0.43, 1.11)
1.04 (0.32, 3.40)
0.68 (0.49, 0.93)
ES (95% CI)
Favours intervention
1 .1 .2 .5 1 2
www.3ieimpact.org Hugh Waddington
Farmers feel empowered, more confident
Favours intervention
Hiller et al., 2009 (Kenya)
Friis-Hansen & Duveskog, 2012 (Uganda)
Friis-Hansen & Duveskog, 2012 (Tanzania)
Friis-Hansen & Duveskog, 2012 (Kenya)
Van Rijn, 2010 (Peru)
Rusike et al., 2004 (Zimbabwe)
ID
Study
Favours intervention 1.2 .5 1 2 3
www.3ieimpact.org Hugh Waddington
Publication bias in evidence on yields
95% L ES 95%
U #
MA 1.10 1.18 1.25 23
Filled MA 1.00 1.07 1.14 30
Egger’s test: P=0.002
www.3ieimpact.org Hugh Waddington
Sensitivity
analysis:
Yields by
risk of bias
status
High risk of
bias studies
over-
estimate
impacts
NOTE: Weights are from random effects analysis
.
.
Overall (I-squared = 93.0%, p = 0.000)
Birthal et al., 2000 (India)
Subtotal (I-squared = 95.4%, p = 0.000)
Gockowski et al., 2010 (Ghana)
Todo & Takahashi, 2011 (Ethiopia)
Huan et al., 1999 (Vietnam)
Yamazaki & Resosudarmo, 2007 (Indonesia)Pananurak, 2010 (Pakistan)
Cavatassi et al., 2011 (Ecuador)
Dinpanah et al., 2010 (Iran)
Van den Berg et al., 2002 (Sri Lanka)
Wu Lifeng, 2010 (China)Ali & Sharif, 2011 (Pakistan)
Rejesus et al, 2010 (Vietnam)
ID
Pananurak, 2010 (India)
Carlberg et al., 2012 (Ghana)
Pananurak, 2010 (China)
Hiller et al., 2009 (Kenya)
High risk of biasNaik et al., 2008 (India)
Medium risk of bias
Davis et al, 2012 (Kenya)
Zuger 2004 (Peru)
Davis et al, 2012 (Tanzania)
Dinpanah et al., 2010 (Iran)Pande et al., 2009 (Nepal)
Labarta, 2005 (Nicaragua)
Feder et al, 2004 (Indonesia)
Yang et al., 2005 (China)
Khan et al., 2007 (Pakistan)
Subtotal (I-squared = 81.0%, p = 0.000)
Palis, 1998 (Philippines)
Orozco Cirilo et al., 2008 b) (Mexico)
Wandji et al., 2007 (Cameroon)Mutandwa & Mpangwa, 2004 (Zimbabwe)
Van Rijn, 2010 (Peru)
Study
1.23 (1.16, 1.32)
1.24 (1.13, 1.36)
1.35 (1.19, 1.52)
1.14 (1.03, 1.25)
2.71 (1.11, 6.60)
0.95 (0.92, 0.98)
1.67 (1.23, 2.26)1.24 (1.01, 1.54)
1.22 (0.97, 1.53)
1.32 (1.22, 1.42)
1.68 (1.30, 2.18)
1.08 (1.03, 1.14)1.09 (1.03, 1.15)
0.97 (0.72, 1.31)
ES (95% CI)
0.80 (0.61, 1.05)
1.58 (1.19, 2.10)
1.09 (1.04, 1.14)
1.17 (0.53, 2.56)
0.89 (0.83, 0.96)
1.81 (1.15, 2.84)
1.44 (1.09, 1.92)
1.23 (1.00, 1.51)
2.52 (2.05, 3.11)2.11 (1.25, 3.56)
0.97 (0.92, 1.02)
0.98 (0.96, 1.01)
1.15 (0.94, 1.41)
1.17 (0.97, 1.42)
1.10 (1.03, 1.17)
1.36 (0.97, 1.92)
2.62 (2.23, 3.08)
1.32 (1.07, 1.63)1.36 (1.06, 1.73)
0.86 (0.63, 1.18)
1.23 (1.16, 1.32)
1.24 (1.13, 1.36)
1.35 (1.19, 1.52)
1.14 (1.03, 1.25)
2.71 (1.11, 6.60)
0.95 (0.92, 0.98)
1.67 (1.23, 2.26)1.24 (1.01, 1.54)
1.22 (0.97, 1.53)
1.32 (1.22, 1.42)
1.68 (1.30, 2.18)
1.08 (1.03, 1.14)1.09 (1.03, 1.15)
0.97 (0.72, 1.31)
ES (95% CI)
0.80 (0.61, 1.05)
1.58 (1.19, 2.10)
1.09 (1.04, 1.14)
1.17 (0.53, 2.56)
0.89 (0.83, 0.96)
1.81 (1.15, 2.84)
1.44 (1.09, 1.92)
1.23 (1.00, 1.51)
2.52 (2.05, 3.11)2.11 (1.25, 3.56)
0.97 (0.92, 1.02)
0.98 (0.96, 1.01)
1.15 (0.94, 1.41)
1.17 (0.97, 1.42)
1.10 (1.03, 1.17)
1.36 (0.97, 1.92)
2.62 (2.23, 3.08)
1.32 (1.07, 1.63)1.36 (1.06, 1.73)
0.86 (0.63, 1.18)
Favours intervention
1.5 1 2 3
www.3ieimpact.org Hugh Waddington
Moderator analysis: high risk of bias excluded
Crop Effect size 95% confidence interval Num. estimates
I-squared
Rice 1.14 0.85 1.54 3 82.3% (p=0.004)
Cotton 1.09 1.06 1.12 4 0.0% (p=0.675)
Staples/veg 1.37 1.10 1.70 4 43.6% (p=0.150)
Cash crops (tea, coffee, cocoa)
0.86 0.63 1.18 1 n/a
Region
East Asia & Pacific
1.07 0.99 1.17 4 88.0% (p=0.000)
Latin America & Caribbean
1.04 0.74 1.46 2 67.6% (p=0.079)
South Asia 1.12 1.01 1.24 2 29.3% (p=0.234)
Sub-Saharan Africa
1.58 1.06 2.36 3 58.5% (p=0.090)
Length of follow-up Up to 2 years
1.14 1.06 1.23 7 51.3% (p=0.055)
More than 2 years
1.06 0.95 1.17 5 83.4% (p=0.000)
www.3ieimpact.org Hugh Waddington
Summary of quant findings
• FFS increase knowledge and improve adoption
of the FFS practices (e.g., reduction in pesticide-
use)
• Without negatively affecting the yields and
incomes, on average increasing one or both
• Suggestions of farmers feeling empowered
• Limited, if any, spillovers: Some simple
knowledge may diffuse to neighbours, but not
complex
• Neighbours do not adopt the practices
consistently
www.3ieimpact.org Hugh Waddington
But what were the barriers and facilitators of
these effects on outcomes?
•Full story of qualitative
synthesis: Birte
Snilstveit‟s
presentation Friday
9.30am
www.3ieimpact.org Hugh Waddington
In brief, qualitative evidence highlights key
process and implementation factors
• Targeting and participation (interest, group dynamics)
• Appropriateness of the curriculum for the farmers
• Delivery of the curriculum (participatory)
• Facilitators facilitation skills, perception by the
participants
• Language of instruction and the way the ideas are
introduced
• Policy context (eg subsidies and promotion of pesticides)
www.3ieimpact.org Hugh Waddington
• Protocol development crucial – piloting of searches, data collection tool and critical appraisal technique
• Early development of theory of change (with underlying assumptions)
• Balance between comprehensive study and budget: keep interventions narrow, careful consideration of inclusion criteria for causal chain analysis
• Iteration between review components required to produce integrated rather than parallel synthesis
• Integrated synthesis is resource intensive, particularly for large bodies of evidence such as FFS
Lessons for design of SRs
www.3ieimpact.org Hugh Waddington
Thank you
Review available shortly:
http://campbellcollaboration.org/lib
/project/203/
Please visit:
www.3ieimpact.org/systematicrevi
ews
Recommended