M&E Frameworks How frameworks guide program monitoring and evaluation.

  • Published on
    31-Mar-2015

  • View
    217

  • Download
    0

Embed Size (px)

Transcript

<ul><li>Slide 1</li></ul> <p>M&amp;E Frameworks How frameworks guide program monitoring and evaluation Slide 2 Learning Objectives Describe function of Conceptual, Logical and Results frameworks Apply Proximate Determinants Framework to M&amp;E Planning Use logical frameworks to guide M&amp;E of HIV programs Apply achievements to results frameworks Slide 3 Session Overview Function of frameworks Conceptual Frameworks Proximate Determinants Logic Frameworks PMTCT program implementation, GFATM Results Frameworks PEPFAR GFATM M&amp;E Reporting Framework Activity Developing a logic framework for an HIV program Slide 4 Discussion Why do we use frameworks in M&amp;E? Slide 5 Functions of Frameworks for M&amp;E Roadmap to program planning, monitoring and evaluation Delineate clear pathways to program goals &amp; objectives Define relationships between program inputs, processes, outputs, and outcomes Describe how program factors interact with external context (environmental factors) Lead to sound implementation and good M&amp;E plans Slide 6 Types of Frameworks - Conceptual Definition Illustrate relationships among all factors (systemic, organizational, individual) that influence program operation &amp; successful achievement of goals Purpose Show how program components will operate to influence outcomes within wider context Guide identification of indicators Guide impact analysis (causal pathways) Slide 7 Conceptual Frameworks Makes explicit connections among relevant contextual (environmental) factors and your program Helps clarify the why and how questions of program operation and design: Assumptions that underlie chosen activities Factors activities are expected to affect Causal linkages leading to achievement of program objectives Slide 8 Conceptual Framework Examples: Proximate Determinants Premise of proximate determinants frameworks: Factors leading to health or demographic outcome operate through an ordered path of determinants Determinants grouped in proximity to outcome Underlying (socioeconomic) factors farthest away, biological closest Proximate determinants take intermediate position between underlying and biological Proximate determinants are affected by underlying factors &amp; affect the biological determinants Proximate determinants are what most programs seek to influence Slide 9 Proximate Determinants Example: HIV (STI) Incidence 1 Underlying determinants Socioeconomic and socio-cultural context (personal and community) Health programs environment: VCT, ARV, STD control, condom promotion, IEC Proximate determinants (interventions) Partner acquisition, mixing patterns, concurrency, abstinence Condom use, STI present, risky practices, STI treatment Treatment Biological Determinants Rate of contact between susceptible and infected Efficiency of transmission Duration of activity Outcomes: HIV incidence/mortality, STI incidence 1 Boerma and Weir, 2005 Slide 10 Proximate Determinants Framework for HIV/STI 1 1 Boerma and Weir, 2005 Underlying Determinants Proximate Determinants Biological Determinants Health Outcome Demog. Outcome Context Sociocultural Socioeconomic Demographic Interventions VCT STI control Condom prom. IEC Blood safety Safe injections Partner acquis. Mixing patterns Concurrency IDU Condom use Other STI Type of sex Needle safety ARV Treatment STI Treatment Exposure of susceptible to infected Efficacy of transmission per contact Duration of infectivity HIV infection Mortality Disease 1 Boerma and Weir, 2005 Slide 11 Ecological Framework: Links between violence against women and HIV 1 1 WHO 2010, Addressing violence against women and HIV AIDS: What works? More risky male partners : More controlling &amp; violent masculinities More sexual risk taking More likely HIV/STIs More risky male partners : More controlling &amp; violent masculinities More sexual risk taking More likely HIV/STIs Gender &amp; relationship inequality Rape, child sex abuse &amp; IPV Psychological Distress : Chronic Anxiety Depression PTSD Substance use Psychological Distress : Chronic Anxiety Depression PTSD Substance use Reduced protective powers : More acquiescent femininities More frequent sex Less condom use Reduced protective powers : More acquiescent femininities More frequent sex Less condom use More risky sex : More partners Concurrency Transact. Sex Sex while intox. More risky sex : More partners Concurrency Transact. Sex Sex while intox. HIV Rape Direct transmission Indirect transmission Slide 12 Types of Frameworks -Logic Definition Pictorial diagram that summarizes essential program information by describing its main elements (inputs, activities, outputs, outcomes) &amp; how they work together to affect a desired health outcome Purpose Make intended program outcomes explicit Show internal logic of program Reveal assumptions of how program leads to outcomes Guide indicator selection Promote communication among stakeholders Guide impact evaluation Source: CDC Evaluation Guidance Handbook at: http://www.cdc.gov/hiv/aboutdhap/perb/guidance/chapter4.htm Slide 13 Logic Frameworks - Components Problem statement The why of program implementation Statement what puts a population at risk for HIV Inputs Program resources like staff, curricula, money Activities What the program does like outreach, training, testing Outputs Deliverables of program like people trained or tested Outcomes Program results like change in knowledge, service use Impact Long term change in health status like decreased HIV incidence Source: CDC Evaluation Guidance Handbook at: http://www.cdc.gov/hiv/aboutdhap/perb/guidance/chapter4.htm Slide 14 Logic Framework Example: PMTCT Program 1 Problem Statement HIV rates rising in pregnant women &amp; infants in Thailand. Transmission risk from mom to child significant during pregnancy &amp; delivery, with additional risk pp via breastfeeding Inputs Staff, funding from CDC GAP &amp; govt, PMTCT counseling/protocol, HIV test kits, ARV drugs, breast milk substitutes Activities ANC services include counseling, testing &amp; referral &amp; referral, distribute ARV prophylaxis, provide infant feeding counseling &amp; support 1 CDC M&amp;E Guide, Version 1, Appendix 1 Slide 15 Logic Framework Example: PMTCT Program, contd. 1 Outputs (per quarter) 500 women participate in a 2 hr. ANC session 250 of these will get HIV test results All (of 500) who test positive will be referred for ARV prophylaxis, infant feeding C&amp;S Half of all positive women will receive prophylaxis and infant feeding counseling and breast milks substitutes Outcomes Increase in access to HIV testing Increase in knowledge of status Increase in knowledge about prophylaxis Increase in ARV prophylaxis use by pregnant HIV+ Increase in rapid infant weaning/use breast milk substitutes Impact: HIV incidence in infants, morbidity/mortality among people living with HIV/AIDS 1 CDC M&amp;E Guide, Version 1, Appendix 1 Slide 16 Developing a Logic Model 1 Input from all levels of stakeholders Program staff, program participants and others to analyze intended program outcomes A good problem statement leads to a good logic model Analyze the problem, program resources, desired outputs, outcomes and impact Design activities to achieve outputs and outcomes Usually expressed in diagram or table 1 CDC, ETA Evaluation Briefs, Logic models, No. 2, Dec. 2005 Slide 17 Types of Frameworks Results 1 USAID funded projects, like a logic framework Definition A tabular/graphic description of a programs strategy for achieving a specific strategic objective (SO) through intermediate results, with a narrative supplement. Purpose Like a logic framework, planning &amp; management tool for programs Specifies out exactly how, through a set of intermediate results, the program SO will be achieved Provides a basis of consensus for all program stakeholders Conveys the development hypothesisplausible linkages between the intermediate results and SO 1 USAID, Performance Monitoring and Evaluation TIPS: Building a Results Framework, PN-ACA-947 Slide 18 Results Frameworks - Components 1 Strategic Objective The ultimate end point of the Results Frameworkthe goal of the program Standard by which the operational unit (performance) is willing to be judged Appropriate level of ambition means the SO should be achievable, clear and measurable Intermediate results Achievable in the short term, lower level than SO An intermediate step leading to achieving the SO Lower level of impact than the SO; cumulative effect leads to the SO 1 USAID, Performance Monitoring and Evaluation TIPS: Building a Results Framework, PN-ACA-947 Slide 19 Results Frameworks - Components 1 Clarification of causal linkages Narrative that shows how intermediate results are linked to the SO Links can be between results which then leads to SO Critical Assumptions Set of conditions necessary under which the development hypothesis (strategy for achieving the SO) will hold true Not under project/USAID control, but will affect SO/results 1 USAID, Performance Monitoring and Evaluation TIPS: Building a Results Framework, PN-ACA-947 Slide 20 Results Framework Example PEPFAR funded ART Program SO: Utilization of ART services IR-1: Availability of quality services IR-2: Demand for services IR-1.1: Increase ART sites IR-1.2: Supply sites with ARVs IR-1.3: Training for providers IR-2.1: Increase knowledge of ART IR-2.2: Increase referral from VCT Slide 21 Developing a Results Framework Set an appropriate strategic objective Identify intermediate results Clarify causal linkages between results Identify critical assumptions Spend time on the narrative portion so that all stakeholders understand What is trying to be achieved How to report verifiable achievements Slide 22 GFATM M&amp;E Framework 1 Key Principles Country ownership, harmonization, consistency, balance between routine &amp; survey data, avoid duplication of efforts Small set of indicators Focus on multi-agency M&amp;E Toolkit (http://www.theglobalfund.org/en/performance/monitoring_evaluation/)http://www.theglobalfund.org/en/performance/monitoring_evaluation/ 12-18 indicators per grant Relies on UNAIDS MERG documents/indicators No Global Fund indicators (country ownership) Mix of short- &amp; long-term indicators Quarterly or 6-monthly for some 15-18months for others, important for next Phase 1 The Global Fund Monitoring and Evaluation KEY MESSAGES, http://www.theglobalfund.org/en/files/about/structures/lfa/backg round/LFATrainingMaterials/ME/ME_Key_Principles.pdf Slide 23 GFATM M&amp;E Framework, contd. 1 Avoid duplication Use existing in-country indicators, data systems, planned surveys Strengthen existing systems and budget for data gaps Indicator targets need to match grant proposals Existing tools for performance evaluation M&amp;E Toolkit, M&amp;E Systems Strengthening Tool, Data Quality Assessment Tool Tools developed for GFATM grants, but applicable to many contexts, as well see 1 The Global Fund Monitoring and Evaluation KEY MESSAGES, http://www.theglobalfund.org/en/files/about/structures/lfa/backgr ound/LFATrainingMaterials/ME/ME_Key_Principles.pdf Slide 24 New Donor Demands: Gender PEPFAR, UNAIDS, GFATM, WB require gender integration into programs Addressing gender in meaningful way = measureable All agencies have strategies to address gender equality Many resources out there (well examine some at the end of the week) Slide 25 Activity: Developing a Logic Framework Assemble workshop group-work members Use provided CDC-GAP activity sheet: Logic Model Worksheet for Planned Implementation and Outcomes Take about an hour to complete sheet Resulting logic models to be presented back (5 minutes for each group) by group representative This draft logic model will guide your group work and form the basis for indicator selection; you will have the opportunity to revise based on feedback Slide 26 MEASURE Evaluation is a MEASURE project funded by the U.S. Agency for International Development and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group International, ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. Government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide. </p>

Recommended

View more >