Upload
mohammed-mujib
View
218
Download
0
Embed Size (px)
Citation preview
8/7/2019 Impact Monitoring in Value Chain Kenya
1/54
Concepts and Application
Deutsche Gesellschaft frTechnische Zusammenarbeit (GTZ) GmbH
Impact Monitoringin Value Chain Promotion
Heike Hffler, GTZ Kenya
First sequence of thoughts, prepared for the Training Course
Promoting Promoting Value Chains for Agribusiness
Development in Africa
held in Nairobi, 4th - 7thApril 2005
8/7/2019 Impact Monitoring in Value Chain Kenya
2/54
2
Impact Monitoring: Conceptsand Application
Measuring Impacts in Value Chain Promotion1. Impact Hypotheses chain of changes
2. Scope of influence
3. Benchmark Data, Milestones and Indicators
4. Methods of Measuring Impact5. Using Monitoring Information
2
A Concept for Impact Monitoring
1. What to monitor? What are impacts?
2. GTZ Impact Model
3. Theory: Six steps to set up a monitoring system
1
Practice: PSDA Impact Monitoring System3
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
3/54
3
Impact Monitoring: Conceptsand Application
Measuring Impacts in Value Chain Promotion1. Impact Hypotheses chain of changes
2. Scope of influence
3. Benchmark Data, Milestones and Indicators
4. Methods of Measuring Impact5. Using Monitoring Information
2
A Concept for Impact Monitoring
1. What to monitor? What are impacts?
2. GTZ Impact Model
3. Theory: Six steps to set up a monitoring system
1
Practice: PSDA Impact Monitoring System3
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
4/54
4
Monitoring is an ongoing activity!
Projects and Programmes need to follow-up theirprogress
Successes and failures need to be looked atwhile projects and programmes are running
Monitoring is an internalised process of teamcommunication, continuously undertaken whileimplementing,
whereas
Evaluation is an act of stopping implementationto reflect past activities (but drawing frominformation from monitoring).
1
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
5/54
5
The World of Impact Monitoring
Impact Chain
Impact Model
Impact Indicators
Outputs
Outcome
Outreach
Benefits
Performance Monitoring
Input Monitoring
Activity Monitoring
Output Monitoring
Implementation Monitoring
Impact Monitoring
Impact Assessment
Process Monitoring
Result-based MonitoringEvaluation
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
6/54
6
The World of Impact Monitoring
Impact Chain
Impact Model
Impact Indicators
Outputs
Outcome
Outreach
Benefits
Performance Monitoring
Input Monitoring
Activity Monitoring
Output Monitoring
Implementation Monitoring
Impact Monitoring
Impact Assessment
Process Monitoring
Result-based MonitoringEvaluation
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
7/54
7
The World of Impact Monitoring
Performance Monitoring Impact Monitoring Impact Assessment
Input Monitoring
Activity Monitoring
Implementation Monitoring
Process Monitoring
Output Monitoring
Impact Chain
Impact Model
Impact Indicators
Result-based Monitoring
Outputs ImpactsBenefits
Outreach
Evaluation
Outcome
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
8/54
8
What to monitor in projects
Performance & Impact Monitoring1 Performance
Observing outputsagainst plannedactivities
Providinginformation forproject management
Day-to-day activity
To trigger short-termadjustments in
operation
Impact
Focussing on effectof outputs: impacts!
Observing the directbenefit of outputs
Strategic steering ofimplementation
To self-evaluatewhether activitiescontribute to
objectives
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
9/54
9
Impacts are ...
... changes that have a causal - or at least aplausible - link to a project/programme
... a change of circumstances as a consequence of
an intervention, it can be intended orunintended, positive or negative.
... there: from the first moment of intervention andthey continue to occur all the time.
... rather the result of social interaction than astraight-forward interventions
... the result of complex interactions and thus, acomplex matter to deal with!
1
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
10/54
10
Why concentrating on Impacts?
Broad international discussion late 90ies
Criticism about the efficiency of development
cooperation
Criticism about the Monitoring and Evaluationsystem of GTZ
1
BMZ is a contractor, GTZ is an agent
New structure of the project documents
Changes in political dialogue
Changes in Report obligations
AURA
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
11/54
11
1
The GTZ Impact Model (I)
Impact Orientation has become a principle of
GTZs corporate development.
quality at entry quality at exit
what has been done what has changed
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
12/54
ProjectUse of OutputDirect BenefitIndirect Benefit
Engel, P., The Social Organization of Innovation, 1997
Model of Interaction
1
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
13/54
Use ofOutputs
PovertyAlleviation
Direct
Benefit
(Goal)
Outputs
Indirect
Benefit
Activities
The GTZ Impact Model (II)
1
Partner
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
14/54
14
Illustration: which of the 100
Bricks did we donate?
8/7/2019 Impact Monitoring in Value Chain Kenya
15/54
Use ofOutputs
PovertyAlleviation
Direct
Benefit
(Goal)
Outputs
Indirect
Benefit
Activities
The GTZ Impact Model (II)
1
Partner
diff.ImpactLevels
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
16/54
16
Increase in rural
employment & income
Indirect Benefits
Direct Benefit (Goal)
Use of Output
Output
Activities
Inputs
Producers can access international markets
Market information is used to change production standards
Presentation of Research to stakeholders and publishing
Market research for a strategic agricultural product
Advisory Services in the Ministry of Agriculture
The GTZ Impact Model (III)
1
Attribution Gap
Pro-poor Rural Economomic Growth
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
17/54
17
Impact Models form the essential methodological part in
project design!
Project reports are reduced to the information relevant for
political intervention, i.e.
Changes in risk
Changes in assumptions
and: IMPACTS! i.e.
Which changes can be observed?
Which impacts can be plausibly attributed to the project?
Are there unintended impacts?
Consequences for GTZ Projects
1
Changes in Monitoring!
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
18/54
18
8/7/2019 Impact Monitoring in Value Chain Kenya
19/54
19
Theory: Six Steps (GTZ)
Step 1: Identify the System Boundaries
Step 2: Agree on Purpose and Procedures for
Results-based Monitoring
Step 3: Agree on Results Hypotheses
Step 4: Review Indicators and Define Milestones
Step 5: Conduct Data SurveyStep 6: Using Monitoring Results
1
Heike Hffler Kenya
GTZ 2004: Result-based Monitoring: Guidelines for Technical
Cooperation Projects and Programmes
8/7/2019 Impact Monitoring in Value Chain Kenya
20/54
20
Theory: Six Steps (Herweg/Steiner)
Step 1: Involvement of Stakeholders and
Information Management
Step 2: Review of Problem Analysis
Step 3: Formulation of Impact Hypotheses
Step 4: Selection of impact Indicators
Step 5: Development and Application of ImpactMonitoring Methods
Step 6: Impact Assessment & Follow-up
1
Heike Hffler Kenya
Herweg, K & Steiner, K. 2002: Impact Monitoring and Assessment,
Vol. I & II
8/7/2019 Impact Monitoring in Value Chain Kenya
21/54
21
Theory: Seven Steps (MAPP, GDI)
Step 1: Preparing a Life Line
Step 2: Preparing a Trend Analysis
Step 3: Cross-checking with other sources
Step 4: Compiling an Intervention List
Step 5: Developing the Influence Matrix
(connecting trends and interventions)Step 6: Developing the Impact Profile
Step 7: Attribution of Impacts to MDGs
1
Heike Hffler Kenya
DIE 2004: Briefing Paper: Impact Analysis of Development
Cooperation is Feasible
8/7/2019 Impact Monitoring in Value Chain Kenya
22/54
22
Impact Monitoring: Conceptsand Application
Measuring Impacts in Value Chain Promotion1. Impact Hypotheses chain of changes
2. Scope of influence
3. Benchmark Data, Milestones and Indicators
4. Methods of Measuring Impact5. Using Monitoring Information
2
A Concept for Impact Monitoring
1. What to monitor? What are impacts?
2. GTZ Impact Model
3. Theory: Six steps to set up a monitoring system
1
Practice: PSDA Impact Monitoring System3
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
23/54
23
Impact Models: A chain of changes
What do we expect to happen in value chain promotion?
2
SpecificInput
providers
Primaryproducers Traders
Final Con-
sumers
Logisticscentres,
Industry
Specific
Inputs
Trans-
formationTrade ConsumptionProduction
C H A N G E !
Intervention;
i.e. extension
Trans-
formationTrade Consumption
Specific
Inputs
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
24/54
24
Specific
Input
providers
Primary
producers TradersFinal Con-
sumers
Logistics
centres,
Industry
Specific
Inputs
Trans-
formationTrade2
Defining the Impact Hypothesis (I)
Consumption
Use of Output Farmer group gets EuroGap certified
Output
Activities Training on EuropGap Compliance and Certification
Inputs Advisory Services for Extension Providers
Production
Direct Benefit (Goal) Small-scale producers access international markets
Farmer Groups are setting up an Internal Control System
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
25/54
25
Specific
Input
providers
Primary
producers TradersFinal Con-
sumers
Logistics
centres,
Industry
Specific
Inputs
Trans-
formationTrade2
Defining the Impact Hypothesis (II)
Consumption
Use of Output
Output
Activities
Inputs
Production
Direct Benefit (Goal)
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
26/54
26
Specific
Input
providers
Primary
producers TradersFinal Con-
sumers
Logistics
centres,
Industry
Specific
Inputs
Trans-
formationTrade2
Defining the Impact Hypothesis (III)
Consumption
Use of Output
Output
Activities
Inputs
Production
Direct Benefit (Goal)
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
27/54
27
Specific
Input
providers
Primary
producers TradersFinal Con-
sumers
Logistics
centres,
Industry
Specific
Inputs
Trans-
formationTrade2
Defining the Impact Hypothesis (IV)
Consumption
Use of Output
Output
Activities
Inputs
Production
Direct Benefit (Goal)
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
28/54
8/7/2019 Impact Monitoring in Value Chain Kenya
29/54
29
Impact Hypotheses Food for Thought:
2
Pre-formulation of impact
chains can lead to mono-causal
conclusions.
Linear following of impact
chains can fade out the
complex interaction of
interventions and impacts.
The higher the impact level, themore a context oriented
approach is needed, taking into
account development trends.
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
30/54
30
Defining the Scope of Influence (I)
Specific single interventions
2
SpecificInput
providers
Primaryproducers Traders
Final Con-sumers
Logisticscentres,
Industry
Specific
Inputs
Trans-
formationTrade ConsumptionProduction
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
31/54
31
Defining the Scope of Influence (I)
Specific single interventions
2
SpecificInput
providers
Primaryproducers Traders
Final Con-sumers
Logisticscentres,
Industry
Specific
Inputs
Trans-
formationTrade ConsumptionProduction
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
32/54
32
Defining the Scope of Influence (II)
Chain promotion at all stages
2
SpecificInput
providers
Primaryproducers Traders
Final Con-sumers
Logisticscentres,
Industry
Specific
Inputs
Trans-
formationTrade ConsumptionProduction
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
33/54
33
Defining the Scope of Influence (III)
Full chain promotion, all stages, all levels
2
Specific
Inputproviders
Primary
producers Traders
Final Con-
sumers
Logistics
centres,Industry
Specific
Inputs
Trans-
formationTrade ConsumptionProduction
Subsector-specific Technical Agencies
Subsector-specific BDS providers
Associations
Local Government, Providers of Utilities / Infrastructure
National Government (Line Ministries) & Public Agencies
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
34/54
34
Impact Hypothesis - Conclusion
2
Specific
Inputproviders
Primary
producers Traders
Final Con-
sumers
Logistics
centres,Industry
Specific
Inputs
Trans-
formationTrade ConsumptionProduction
Subsector-specific Technical Agencies
Subsector-specific BDS providers
Associations
Local Government, Providers of Utilities / Infrastructure
National Government (Line Ministries) & Public Agencies
Heike Hffler Kenya
2
8/7/2019 Impact Monitoring in Value Chain Kenya
35/54
35
2
Benchmarks, Milestones & Indicators
Benchmark Data & Information
The state of the art at given point in time (i.e.
programme start)
Milestones
Desired output at a given point in time for (programme)
achievements
Indicators
Determinants for checking whether activities led to the
expected output (performance) or expected
implications (impact).
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
36/54
36
2
Benchmarks, Milestones & Indicators (I)
Indicators
Heike Hffler Kenya
0t
Benchmark
Data
Programme level
Value chain level
MilestonesProgress
Reports
Evaluation
8/7/2019 Impact Monitoring in Value Chain Kenya
37/54
37
Specific
Inputs
Trans-
formationTrade
2
Benchmarks in Value Chains
ConsumptionProduction
Usage of
InputsUsage of
services
Outreach:
How manyare
involved?
Employment
Gross
margins
Regionaldistribution
Number of
processorsDegree of
vertical
concentra-
tion and
integration
Market
Concentra-tion
Entry
Barriers
Price
Volatility
Seasonality
Trade
margins
Quality
Food Safety
Product
Diversity
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
38/54
38
Specific
Inputs
Trans-
formationTrade
From Benchmarks to Milestones
ConsumptionProduction
Output
Activities
Inputs
3 processing
firms
Low process
quality
High costs of
processing
In 18 Month:
2 more firms
established
At least 2 certified
under ISO xxx
Cost-effectiveness
of at least 2 firms
improved by x %
Milestones serve as process indicators.
They are directly derived from programme indicators
and the impact hypothesis.
2
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
39/54
39
Specific
Inputs
Trans-
formationTrade
Benchmarks & Milestones in Value Chains
ConsumptionProduction
Try to build these couples per stage in a chain that
you are targeting with your project.
Not all Benchmarks need to be quantitative data;
qualitative description also serves the purpose.
Market Analyses need to take the need for benchmarks
into account (into the TORs!)
We might not be the first to undertake this keep
looking for partners in gathering data!
2
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
40/54
40
2
From Milestones to Indicators
or was it the other way round?
Output
Activities
Inputs
Use of Output
Direct Benefit (Goal)
Production
Heike Hffler Kenya
Several Milestones in a chain
Single Milestone
Programme Indicator(s)
Value chain Indicator(s)
8/7/2019 Impact Monitoring in Value Chain Kenya
41/54
41
Specific
Inputs
Trans-
formationTrade
Benchmarks, Milestones & Indicators (II)
ConsumptionProduction
Output
Activities
Inputs
3 processing
firms
Low process
quality
High costs of
procesing
In 18 Months:
2 more firms
established
At least 2 certified
under ISO xxx
Cost-effectiveness
of at least 2 firms
improved by x %
...
2
Heike Hffler Kenya
Use of Output
Direct Benefit (Goal)
Impact Indicator,
Component 3,
Phase 2
The number of
profitably operating
private processors
doubles (2002: 6)
8/7/2019 Impact Monitoring in Value Chain Kenya
42/54
42
8/7/2019 Impact Monitoring in Value Chain Kenya
43/54
43
Example: Employment
Customers
Market 2
Customers
Market 1
(ResearchInstitute)
Retailers A
Retailers B
Industrial
SMEs
Importers
Large processing companies
3
2
1Primary
Producers 1
SM Primary
Producers 2
n = 400 n = 450n = 20
n = 6000
n = 40
n = 2400
n = 20
n = 200
3
Heike Hffler Kenya
The number of
primary producers
doubles.
8/7/2019 Impact Monitoring in Value Chain Kenya
44/54
44
100,00
Specific
InputsProduction Trans-
formationTrade Con-
sumption
Example: Value Added
Specific
Input
providers
Primary
producers TradersFinal Con-
sumers
Logistics
centres,
Industry
Sales Prices / unit 60,0025,00
Cost / unit
Income / unit
Distribution in %
- of value-added
- of income
23,00
2,00
27,00
8,00
30,00
10,00
25 %10 %
35 %40 %
40 %50 %
3
Heike Hffler Kenya
The valueadded at
producer
level
increases
from 25%
to 35%.
8/7/2019 Impact Monitoring in Value Chain Kenya
45/54
45
Example: Better Business Relations
Heike Hffler Kenya
Specific
InputsProduction Trans-
formationTrade Con-
sumption
Specific
Input
providers
Primary
producers TradersFinal Con-
sumers
Key informant interviewsshow that wholesale
traders are more satisfied
with the
quality/quanity/reliability
of produce delivered by
small scale farmers.
Interviews show
that small calefarmers have
improved
relationship to
traders and trust
them.
8/7/2019 Impact Monitoring in Value Chain Kenya
46/54
46
2
Indicators in Value Chains
Some examples:
Customer satisfaction
Repeated customers
Number of enterprises demanding a service
Satisfaction with last service purchased
Percentage of women-owned enterprises
Heike Hffler Kenya
There is increasing interest in, and use of, private
sector tools for performance and impact measurement;
i.e. consumer market research tools for measuring
changes in markets and private sector business tools
for measuring service provider performance.
8/7/2019 Impact Monitoring in Value Chain Kenya
47/54
47
Methods of Measuring Impacts
2
Heike Hffler Kenya
Market Studies
Key Informant Interviews
Point of Leverage
comparisons (before after) Participatory Monitoring
Time-series
etc...
Specific
InputsProduction Trans-
formationTrade Con-
sumption
Where to measure in
the chain?
What data qualitative
or quantitative?
Which degree of
participation?
8/7/2019 Impact Monitoring in Value Chain Kenya
48/54
48
Example: Spider Web Diagrammes
0
5
10
oduc ua
onsume a s ac on
Fa m ncome
o h n xpo s
o e
se o e ces
eg nng ng
a e 3 ea s
a e 10 ea s
2
Heike Hffler Kenya
8/7/2019 Impact Monitoring in Value Chain Kenya
49/54
49
E-Val
2
Heike Hffler Kenya
GTZ tool to catch different perceptions of projects and
programmes.
There are only subjective opinions to changes and
impacts
Triangulation of opinions:
GTZ,partners, and
target group
8/7/2019 Impact Monitoring in Value Chain Kenya
50/54
50
Using Impact Monitoring Systems
2
Heike Hffler Kenya
The objective of
Impact Monitoring
was ....
Focussing on effectof outputs: impacts!
Observing the directbenefit of outputs
Strategic steering ofimplementation
To self-evaluatewhether activitiescontribute toobjectives
8/7/2019 Impact Monitoring in Value Chain Kenya
51/54
51
Using Impact Monitoring Systems
2
Heike Hffler Kenya
SteeringObserve direct Benefit Self-evaluation
Steering
Committee
Reports
Annual
ProgressReports
Participation!
Discussion with
Stakeholders
Finetuning
Communication
Team
Meetings
Partner
Meetings
Other
donors
Private
Sector
8/7/2019 Impact Monitoring in Value Chain Kenya
52/54
52
Using Impact Monitoring Systems
2
Heike Hffler Kenya
More ideas lets be creative!
Replanning
Communication of
Results
Newsletter
Business Fora
Publications
Go- No-go- Decisions
Accountability
8/7/2019 Impact Monitoring in Value Chain Kenya
53/54
53
Its better to be approximatelyright
than precisely wrong!
( John Maynard Keynes )
Please email your comments & questions to [email protected]
Heike Hffler Kenya
H ik Hffl K
8/7/2019 Impact Monitoring in Value Chain Kenya
54/54
54
Impact Monitoring: Conceptsand Application
Measuring Impacts in Value Chain Promotion1. Impact Hypotheses chain of changes
2. Scope of influence
3. Benchmark Data, Milestones and Indicators
4. Methods of Measuring Impact5. Using Monitoring Information
2
A Concept for Impact Monitoring
1. What to monitor? What are impacts?
2. GTZ Impact Model
3. Theory: Six steps to set up a monitoring system
1
Practice: PSDA Impact Monitoring System3
Heike Hffler Kenya