From logframes to ToC?

PCM and LFA

2

The Logframe matrix

1. Fewer accidents in the production shop2. Employees get fewer complaints from clients

Employees feeladequately challenged at work

To contribute to an adequate retention of employees

1. 80% less serious accidents*2. 50% less complaints

1. Safety records2. Customer service records

Employee satisfaction score of 8,5

ES survey administered yearlyby consultancy global satisfaction score**

Annual % of leaversreduced to 3%

HR records

Employees are praised for doing wellEmployees understand the use of their tasks

Employees wages stay stable or increase

1. Project Description

2. Verifiable indicators

3.Sources of verification

4. Assumptions

Overall objective

Project purpose

Results

*serious = resulting in more than 5 days off work** instrument example attached

4

Theory of change-1• Theory of change:

• All preconditions of a key outcome clear in terms of timing and causality (blue boxes):• Those the intervention will

act on• Those that are assumed

(assumptions) incl. regarding what other stakeholders will do

• Rationale (green boxes) behind each arrow (why is a pre-condition important)

• Actions intervene throughout the chain (red boxes)

Example courtesy of ActKnowledge’s Helene Clark

Theory of change-2

6

Simplified version

Theory of change-3

7

© Benedict Wauters / Latitude C&T

Source: Chen

Theory of change-4

Change model

Intervention

Determinants

Outcome

Theory of change-5

• Let’s compare logframes and ToC

11

Logframe is oversimplification (lack-frame)

Output Purpose

Visits of teachers to parents

Better study-results

Betterstart

in labour market

Global LFATheory of change

Adapted from C. Weiss

ActionTeacher engages in scripted dialogue with parents

Results

Better homework

+More at school

Logframe obscures causality (logic-less)

Original example: World Bank, R.Rist, 2004 Action1 Action 2

Logframe obscures causality (logic-less)

Original example: World Bank, R.Rist, 2004 Action1 Action 2

Many logframes arejust retro fitting box

filling exercises wherethere is not necessarily

a lot of logic behind (“logicless frames”).

Logframe obscures causality (logic-less)

15

Logframe cannot show how elements at each level of the intervention logic interact with each other (e.g. how certain activities can only start as some outcomes already visible)

ACTIVITIESACTIVITIES

ACTIVITIES

Outcom

eO

utcome

Outcom

e

Outcom

eO

utcome

Outcom

e

Logframe obscures causality (logic-less)

16

Logframe cannot show how elements at each level of the intervention logic interact with each other (e.g. how certain activities can only start as some outcomes already visible)

Outcome

Outcome

Outcome

Outcome

Outcome

Outcome

Outcome

Outcome

Outcome

Outcome

Outcome

ACTIVITIES ACTIVITIES

ACTIVITIES

ACTIVITIES

In LFA terminology is adding to the confusion…

Ultimate Impact End Outcomes Intermediate Outcomes Outputs Interventions

Needs-based Higher Consequence Specific Problem Cause Solution Process Inputs

CARE terminology1 Program Impact Project Impact Effects Outputs Activities Inputs

CARE logframe Program Goal Project Final Goal Intermediate Objectives Outputs Activities Inputs

PC/LogFrame2 Goal Purpose Outputs Activities

USAID Results Framework3 Strategic Objective Intermediate Results Outputs Activities Inputs

USAID Logframe4 Final Goal Strategic Goal/ Objective Intermediate results Activities 202E

DANIDA + DfID5 Goal Purpose Outputs Activities

CIDA6 + GTZ7 Overall goal Project purpose Results/outputs Activities Inputs

European Union8 Overall Objective Project Purpose Results Activities

FAO9 + UNDP10 + NORAD11 Development Objective Immediate Objectives Outputs Activities Inputs

UNHCR12 Sector Objective Goal Project Objective Outputs Activities Input/Resources

World Bank Long-term Objectives Short-term Objectives Outputs Inputs

AusAID13 Scheme Goal Major Development Objectives Outputs Activities Inputs

1 CARE Impact Guidelines, October 1999. 2 PC/LogFrame (tm) 1988-1992 TEAM technologies, Inc. 3 Results Oriented Assistance Sourcebook, USAID, 1998. 4 The Logical Framework Approach to portfolio Design, Review and Evaluation in A.I.D.: Genesis, Impact, Problems and Opportunities. CDIE, 1987. 5 A Guide to Appraisal, Design, Monitoring , Management and Impact Assessment of Health & Population Projects, ODA [now DFID], October 1995 6 Guide for the use of the Logical Framework Approach in the Management and Evaluation of CIDA’s International Projects. Evaluation Div ision. 7 ZOPP in Steps. 1989. 8 Project Cycle Management: Integrated Approach and Logical Framework, Commission of the European Communities Evaluation Unit Methods and Instruments for Project Cycle Management, No. 1, February 1993 9 Project Appraisal and the Use of Project Document Formats for FAO Technical Cooperation Projects. Pre -Course Activity: Revision of Project Formulation and Assigned Reading. Staff Development Group, Personnel Division, August 1992 10 UNDP Policy and Program Manual 11 The Logical Framework Approach (LFA). Handbook for Objectives-oriented Project Planning. 12 Project Planning in UNHCR: A Practical Guide on the Use of Objectives, Outputs and Indicators for UNHCR Staff and Implementing Partners. Second Ver. March 2002. 13 AusAID NGO Package of Information, 1998

Advantages of ToC relative to Logframe

• Logic is more clear: Different strands of cause-effect linked to various actions Assumptions (also about other stakeholders) on equal footing to

actionable outcomes

• Not necessarily based on “problems”• Less issues with terminology as relies on visuals rather

than categories• Draws more heavily on research based theories (if done

properly)• If alternative theories allowed, then greater likelihood of

learning and improving• However, even harder to do than Logframe?

18

• However, some issues remain…

19

Issues with both LFA and ToC-1

• Both in LFA and ToC participation of relevant stakeholders is seen as key to create common understanding and ownership, but in practice, especially under time pressure, participation tends to be problematic target groups themselves are usually ignored and participants in

meetings/workshops usually are not the decision-makers differences of opinion among stakeholders are bound to exist and it may

not be feasible to resolve them without resorting to abuse of power on the other hand stakeholders can be all too “happy to agree” as long

as there will be funding coming in for them These processes are very demanding (e.g. formulating problems

“correctly” in LFA is quite difficult as is formulating an outcome chain) It discriminates against people who cannot read or have impaired eye-

sight as the method relies heavily on visualisation the assumption is also that it is not too hard to find a facilitator schooled

in “LFA/ToC” AND in group dynamics; in practice, these people are quite rare

Rogers and Funnel

Issues with both LFA and ToC-2

• although the logic of how the intervention should work is much more elaborate,…

• … backwards re-engineering, in someone’s office, AFTER a project has been designed is unavoidable: ToC/LFA establish a parallel process to what

is already going on (strategic planning processes, informal decision-making processes within existing power structures)

21

22

Issues with both LFA and ToC-3• the logic is linear:

if we do A, B will happen, and then C, and so on = mechanistic, engineering idea of cause and effect as if we can turn the key in the engine of development and the wheels start turning

it is assumes project actions set into motion a chain of events more or less automatically without feed-back loops or delaying effects

• “assumptions, risks, etc.” very difficult to identify in LFA the assumptions column usually is a formality (fill the box)

• limited by the imagination and experience • perception that too detailed a risk analysis might be seen negatively by funders

as it builds up a risky picture in ToC non-intervention pre-conditions have a bigger chance of being

identified due to seeing more of what happens in the outcome chain and by drawing on multiple, science based theories, but still there are many “chimney” ToC

in any case so many factors (systemic view) involved which lie beyond the scope of the planned initiative that will change the way things actually turn out, that it is unlikely you can identify them all

Training

Better skills

Better employability

Employment

Better income

Poverty reduced

Europe 2020

Eternal peace

Danger of „Chimney thinking“

Source: V. Kvaca

Adapted from C. Weiss

ActionTeacher engages in scripted dialogue with parents

?

?

?

?

?

?

?

?

• Ray Pawson (Realist evaluation) cautions: “We cannot contemplate, let alone observe and

control, every supposition that will find its way into a programme…. enlightenment describes rather well the working relationship between research and policy (slow dawning…)… I think the aim should be to produce a sort of ‘highway code’ to programme building, alerting policy makers and practitioners to the problems that they might expect to confront and some of the safest measures to deal with them. … remember A, beware of B, take care of C, D can result in both E and F, if you try G make sure that H is in place….”

R. Pawson, Nothing as practical as a good theory, Evaluation 2003, 9

26

27

Theories give us perspectives we can apply to interpret reality and how it may function…

they can help us to devise numerous strategies, depending on what perspective we adopt …

Theories are NOT comprehensive, air tight plans, nor should we confuse them.