Indicators for Program Quality

The Ethiopian Great RaceP-shift Launch WorkshopApril 30, 2008

Measuring Program Quality

Three areas of focus: Alignment to CI Programming

Principles in our work. Other priority strategies, such as

Advocacy, and Learning and Knowledge Management.

Adherence to technical and project management standards.

Key Definitions

Program Initiative A set of coherent activities with a final

goal and one or more programmatic objectives, and which has specific resources (human, financial, etc.) available to achieve the objectives and contribute to the attainment of the goal.

May be project-based or non-project based initiatives

Review of IndicatorsPromote Empowerment 1. # and % of program initiatives* that have deliberate strategies* to shift power

relations and to empower specific marginalized and excluded groups, in particular women.

Work with Partners 2. # of strategic partnerships (non-project; non-contract specific) in place.

Ensure Accountability and Promote Responsibility

3. # and % program initiatives that have a functioning system* in place to be held accountable by participants and civil society.

4. # and % program initiatives that have a deliberate strategy* for compelling those with responsibilities toward poor and marginalized people to fulfill their obligations.

Address Discrimination 5. # and % of program initiatives that have a deliberate strategy* for opposing discrimination and promoting equity, in particular gender equity.

Promote Non-Violent Conflict Resolution

6. # and % of program initiatives that have a deliberate strategy* to address potential or existing conflicts arising from shifts in power relations.

Seek Sustainable Results

7. # and % of program initiatives that have a coherent strategy* to address and measure impact on the underlying causes of poverty and social injustice.

Advocacy 8. # and % of program initiatives that include a deliberate strategy* for advocacy.

Learning and Knowledge Management

9. # and % of program initiatives that have a deliberate strategy* to identify/share lessons learned and make improvements in a timely manner (organizational learning).

10. # and % of program initiatives that have a deliberate strategy* to promote knowledge sharing within and between groups in society (social learning).

Technical and project management standards**

11. # and % of program initiatives that apply relevant technical and project management standards adopted by CARE

Emergency Readiness** 12. Level of risk versus level of preparedness

*Note that operational definitions exist for all underlined terms. These operational definitions are given in the next section.** Indicators 11 and 12 are considered to be indicators of program quality “basics,” while the other indicators are “beyond the basics.” Indicator 12 (emergency readiness) is listed in this table for information purposes only. This tool kit does not provide guidance for collecting data for this indicator because the Emergency and Humanitarian Assistance Unit (EHAU) in Atlanta already measures this indicator each quarter for every country office.

Proposed Process for Baseline

1. Develop an inventory of all program initiatives

2. Identify which program initiatives will be assessed

3. Train 2 to 3 facilitators to guide data gathering and reflection processes with program initiative teams

4. Facilitators apply data gathering tool with program initiative teams

5. Digitize and process data6. Data analysis and AOPs

Identify Program Initiatives to Assess

Generate an inventory of all program initiatives currently being implemented

Identify which program initiatives will be assessed (all or representative sample)

Identify program initiative managers and location of teams for each selected program initiative

Training

Identify and train 2 to 3 facilitators to guide data gathering and reflection process with program initiative teams

Initial testing of tools Learn by doing Troubleshooting support Resources to be perfected over time

Glossary of terms Tips for facilitating

Data gathering and reflection

To assess the extent to which a program initiative incorporates our principles and strategic priorities

To provide guidance to staff on how to better incorporate these principles and priorities into their program initiatives

Leads to immediate improvements to quality of assessed program initiatives

Processing of data

Digitization and processing of data Need to identify who will be digitizing

data First year, simple technology for

storing and processing data like Excel is sufficient

In future years, more robust solutions will need to be developed

OPM Unit and PIKL can provide support

Analyses and action planning

Baseline results reviewed by SLT to identify initial insights and implications for FY09 priorities

Creation of spaces for staff beyond SLT to analyze baseline results (determine where we are doing well and what the areas of improvement are)

Identify for FY09 AOP Setting targets for indicator for FY 08

Glimpse of the future

1. Learning to set targets2. Inserted into existing business process3. Collected yearly

Questions In the future, it is recommended that COs assess

program quality of all program initiatives under implementation. However, for the baseline, you may decide if you want to assess all program initiatives or just a representative sample. If you choose a representative sample, what criteria

will you use to identify the sample of program initiatives?

What changes would you make to the overall proposed process?

What timeline do you propose for the process if it must be completed by June 30? (establish a timeline with key steps, dates and leads)