MODULE 4 CONSOLIDATING PERFORMANCE FRAMEWORKS AND M&E PLANS GLOBAL FUND GRANT CONSOLIDATION...

MODULE 4

CONSOLIDATING PERFORMANCE FRAMEWORKS

AND M&E PLANS

GLOBAL FUND GRANT CONSOLIDATION WORKSHOP

DATE

1

MODULE OVERVIEW

SESSION I: Steps in consolidating Performance Frameworks

SESSION II: Setting Periodic Review cycles

SESSION III: Areas to update in M&E plan

SESSION IV: Group Work

2

SESSION I: STEPS IN DEVELOPING A CONSOLIDATED PF (CPF)

1. Define duration of

PF and periodic review dates

2. Align reporting periods

3. Harmonize/ combine &

adjust goals,

objectives, SDA &

activities

4.Update baseline

s

5. Streamline impact & outcome

indicators and targets

6. Select and

consolidate key

programmatic

indicators (<20)

7. Complete CPF

template and update M&E plan

3

STEP 1: DEFINING PF DURATION & PERIODIC REVIEW DATES

Discuss options for alignment with national cycles• National fiscal cycle; reporting cycles; review cycles

Define start date

Define duration of the PFo 12-36 monthso Consider flexibilities in initial implementation period that may be

needed to align with national cycles or other factors.

Agree on Periodic Review date At least 6m before end of 3 year cycles; if shorter cycles, first

review might be 3 m or less before period end

4

STEP 2: ALIGNING REPORTING PERIODS BETWEEN EXISTING GRANTS

Align reporting periods between existing grants

Insert reporting periods

Provide any additional clarifications in the comments section

5

Source: Consolidated PF Template Instructions http://www.theglobalfund.org/documents/me

STEP 3: HARMONIZING GOALS, OBJECTIVES, SDA AND ACTIVITIES

Line up, combine and adjust goals and objectives Merge and rationalize activities

Ensure approved activities from EXISTING grants are KEPT

Identify gaps in activities and coverage Add and reorganize activities

STEP 4: UPDATING BASELINES

Choose updated / latest available baselines Sources of baselines include latest proposal, surveys, reviews etc

Indicate source

6

STEP 5: CONSOLIDATING IMPACT AND OUTCOME INDICATORS AND TARGETS

Line up indicators by objective and level of measurement (output, outcome or impact)

Streamline by determining relevance of the proposed indicator vis-à-vis proposal goals and objectives

Ensure consistency of targets with targets in grant/ NSP, latest results reported by PR and new epidemiological or other evidence

7

IMPACT AND OUTCOME INDICATORS

Round Impact indicator formulationLatest available baselines Year 1

Report due datevalue Year Source 2011

Round 4 RCC % of children U5 sleeping under an ITN

62%   2009MIS (Malaria

Indicator Survey)

 Round 7% of children U5 sleeping under an ITN the previous night

Round 9% of children U5 sleeping under an ITN the previous night

SSF (consolidated)

% of children U5 sleeping under an ITN the previous night 70%   Jan 2012

Section filled out automatically

Insert the calendar year

Align indicators Insert latest available baseline

data

Insert targets and reporting due dates

8

STEP 6: PRIORITIZE, SELECT AND CONSOLIDATE KEY PROGRAMMATIC INDICATORS

Prioritize, select & consolidate key programmatic indicators Decide whether and which to combine At least 15, no more than 20 Ensure important programmatic indicators are not dropped - process

indicators can be removed to reduce the number if necessary

Adjust duplication and consolidate targets Target setting and target accumulation

Ensure consistency of targets with targets in grant/NSP, latest results reported by PR and other new epidemiological evidence

9

PROGRAMMATIC INDICATORS: WHEN TO COMBINE?

As a general rule indicators that have a different numerator or denominator should not be combined in the template but listed separately.

Round SDA Indicator formulation CommentsRound 3 RCC

Insecticide-treated nets (ITNs)

Number of ITNs distributed

Round 8 Number of long lasting insecticide treated bed nets distributed to people

SSF (consolidated) Number of long lasting insecticide treated bed nets distributed to people

Example: indicators that should be combined:

Example: indicators that should be listed separately:Round SDA Indicator formulation Comments

Round 3 RCC BCC - community

outreach and schools

Number of people trained in HIV prevention and communication methods This indicator will not be part of the

SSF

Round 8 N/A

SSF (consolidated) N/A

Round SDA Indicator formulation CommentsRound 3 RCC BCC -

community outreach and

schools

N/ARound 8 Number of people reached by HIV prevention activities

SSF (consolidated) Number of people reached by HIV prevention activities 10

PROGRAMMATIC INDICATORS: ‘CONSOLIDATING CUMULATIVE ANNUALLY WITH NON-CUMULATIVE INDICATORS’

Round Indicator formulation Accumulation P1 P2 P3 P4

Round 3 RCCNumber of people trained in HIV prevention and communication methods

Y - cumulative annually

10 20 30 40

Round 8Number of people trained in HIV prevention and communication methods

N - not cumulative

10 10 10 10

SSF (consolidated)

Number of people trained in HIV prevention and communication methods

N - not cumulative

20 20 20 20

Ensure that indicators that are cumulated differently are consolidated correctly

11

PROGRAMMATIC INDICATORS: ‘CONSOLIDATING ANNUAL CUMULATIVE AND CUMULATIVE OVER PROGRAM TERM INDICATORS’

Round Indicator formulation Accumulation P1 P2 P3 P4 P5

Round 3 RCC # of people who received testing and counseling services for HIV and received their result

Y - over program term

500 600 700 800 900

Round 8 # of people who received testing and counseling services for HIV and received their result

Y - cumulative annually

100 200 300 400 100

SSF (consolidated)

# of people who received testing and counseling services for HIV and received their result

Y - cumulative annually

200 400 600 800 200

Cumulative annually

It is recommended to make use of “annual cumulative” and “not cumulative” indicators but not to include “cumulative over the program term” indicators

12

PROGRAMMATIC INDICATORS: ‘TIED TO’

Round Indicator formulation Tied to Accumulation P1 P2

Round 3 RCC # of adults and children with advanced HIV infection currently receiving antiretroviral therapy Current grant

N - not cumulative

1,000 1,100

Round 8 # of adults and children with advanced HIV infection currently receiving antiretroviral therapy Current grant

N - not cumulative

1,500 1,600

SSF (consolidated) # of adults and children with advanced HIV infection currently receiving antiretroviral therapy

National Program

N - not cumulative

10,500 10,700

Indicators can be tied to:• National Program• Current grant• GF when national target subset is funded by >1 GF grant• GF & Other donors when more than GF contribute

When reporting on national targets specify the contributions of the GF and other donors in the comments section of the framework.

Example: shifting to national targets

13

STEP 7: COMPLETE CPF TEMPLATE AND UPDATE M&E PLAN

Should be submitted by PR

If national M&E Plan level of detail not at par with GF requirements, PR to submit additional info as annexure to M&E plan

Where there is no national M&E plan: Develop one if process will be shorter than grant negotiation

phase, OR Submit M&E plan for consolidated grant as a provisional document

pending conclusion of the national plan

14

POTENTIAL CHALLENGES WITH THE CPF

Identifying a priority small set of indicators (<20)

Target accumulation and dealing with targets that cumulate differently

Overlapping targets e.g. targets in one grant are a sub-set of the other

Setting targets for uncommitted Phase 2 grants

Completing the ‘tied to’ section of the CPF

15

KEY MESSAGES - THE CONSOLIDATED PF

The PF should measure all key activities of the grant

Provide sufficient detail in the PF in the comments section

The PF should maintain approved targets from existing grants

Ensure M&E systems are in place to measure/report on the indicators

Ensure sufficient funds for M&E are available

Contact FPM for additional guidance if needed

16

SESSION II: SETTING PERIODIC REVIEW CYCLES

PERIODIC REVIEW: TIMING AND DURATION

… …

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 … …

IP 3…Implementation Period 1 Implementation Period 2

Year 7Year 1 Year 2 Year 3 Year 4 Year 5 Year 6

Cut off date (month 24)

CCM request(month 27)

Board decision (month 30)

SSF extension signing

Periodic Review Process # 1

Periodic Review Process # 2

Cut off date (month 60)

CCM request(month 63)

Board decision (month 66)

SSF extension signing

Review of 24 months of performance

Review of 36 months of performance

17

Maintains core aspects of current Phase 2 review Programmatic and financial evaluation at individual grant level Informs future funding commitments for each PR CCM request with assessment by the LFA

Additional aspects for new Periodic Review Reviews all PRs in a disease area at the same time Review dates correspond with in-country review processes More holistic assessment of program performance and impact

and comprehensive funding decisions

PERIODIC REVIEW

18

PERIODIC REVIEW ASSESSMENT FRAMEWORK: 5 DIMENSIONS

5- EXTERNAL CONTEXTUAL FACTORS (Legal, Political, ‘Force Majeure’, etc.)

4- EFFECTIVENESS (Equity, Value for Money, Aid Effectiveness)

3- MANAGEMENT • Monitoring and Evaluation• Pharmaceutical and Health products management• Institutional and Programmatic arrangements• Financial management and systems

2- PROPOSAL GOALS• Impact Results Vs. Targets (+ evaluation)•Coverage and Outcome of Key Interventions

1- SERVICE DELIVERY•Output Results Vs. Targets•Quality of Data •Quality of Services

NEW

ENHANCED

NEW

19

1 Consolidated request per disease vs. multiple requests

Longer commitment period up to 3 years

Review date decided with country in line with country cycles

CCM oversight role strengthened: can adjust implementation arrangements at time of review

Greater potential to adapt the grant in the next implementation period to updated priorities and scale up needs Possibility to accelerate strong performing grants; Stronger potential for reprogramming

PERIODIC REVIEW: EXPECTED BENEFITS

20

SESSION III: AREAS TO UPDATE IN M&E PLAN: ACTIVITY OBJECTIVES

By the end of the activity participants should be able to…

Identify the key areas of an M&E plan that require review following consolidation.

The M&E plan: key concepts

Essential document -provides the background information for the indicators included in the Performance Framework

Describes how the M&E system should be run Should be accompanied by an annual costed work plan

21

KEY CONTENT OF THE M&E PLAN

1. Indicator definitions and measurement

2. Data collection, analysis and reporting

3. Dealing with evaluations, reviews, surveys, surveillance and special studies

4. DQA and support supervision

5. M&E coordination

6. Capacity building

7. M&E budget and work plan

8. Annexure to include: M&E framework Indicator references

22

LINKAGES BETWEEN THE M&E PLAN, M&E SYSTEMS AND M&E USE

23

SUBMISSION OF M&E PLAN

By PR before grant signing.

Submit the National M&E Plan (disease specific; covering all diseases; or as part of national health M&E plan)

Should ideally cover the entire life of the grant - at least the first two years

Should clearly describe whether a quality M&E system is in place or will be established, to measure at a minimum all the PF indicators

No specific format- but certain key elements must be included…

24

M&E: TAKE HOME MESSAGES

1. National M&E plan should cover the interventions of the consolidated grants

2. M&E plans should be updated to include any new SR reporting formats, procedures and schedule

3. Focus is on medium and long-term results i.e. impact and outcome indicators.

4. Refer as much as possible to the list of top ten indicators and the partners (Global Fund, UNAIDS, WB, CDC M&E toolkit.)

5. FPM can provide guidance on key issues if needed

25

SESSION IV: GROUP EXERCISES

GE A: Choosing Indicators (Tanzania)

GE B: Consolidating Indicators and Targets (TGF exercise)

Completing the new CPF

Updating the M&E plan

26

USEFUL RESOURCES

1. GF: M&E Toolkit

2. GF: Completing consolidated performance frameworks

3. GF: Guidelines on M&E Plans

4. GF: Completing the Performance Framework

5. GF: Top 10 Indicators

Further information available from: The GLOBAL FUND webpage:

For communications, guidance materials & more www.theglobalfund.org/en/policies/

For further questions: FPMs Architecture inbox: ARCinbox@theglobalfund.org

27