Upload
lorraine-gallagher
View
221
Download
2
Tags:
Embed Size (px)
Citation preview
Monitoring & Evaluation for Routine Immunization: Data For Action
Using information to improve routine immunization services
Presentation 2 Objectives
• Be familiar with the role of information in routine immunization
• Be able to use, modify or develop tools for monitoring or improving data quality and use at health facility / district level
• Be aware of resources available for vaccination information in routine immunization
Overview of M&E framework for RI
Inputs Outcome Impact
Short-Term Long-Term
Vaccines, services delivered, advocacy, planning, training
Increase vaccination coverage
Reduce vaccine-preventable disease (VPD) burden
Reduce infant and child morbidity and mortality
In every country:at least 90% fully immunized, with at least 80% in every district*
-Eradicate polio-Reduce VPD mortality and morbidity
Reduce child mortality by 2/3 by 2015 (from 1990 levels)**
* WHO / UNICEF Global Immunization Vision and Strategy ** Millennium Development Goal 4
Goa
ls
How monitoring works for RI
• RI staff at all levels routinely monitor system performance and use this information to drive decision-making
• Monitoring indicators include– Doses administered, coverage, dropout
• Monitoring tools include– Monitoring charts, tables, monthly summary forms
• Internal monitoring activities include– Supervision visits– Rapid community assessments
How evaluation works for RI
• Internal evaluation activities include– Annual reporting of ~40 EPI indicators to
WHO/UNICEF including doses administered and immunization coverage
– Data quality self-assessments– Lot quality assurance surveys
• External evaluation activities include– External immunization coverage surveys– Data quality audits
How Data for Action works for EPI
• RI staff are expected to monitor their system AND use this information for action
• Use of RI information to improve coverage is often the weakest part of RI
• Supervisors (& you) should observe and ask staff if they know how to both calculate and use monitoring indicators, tools and activities
Common RI Indicators• Service performance
– Defined as number of children receiving one of the later vaccines– Coverage with DTP3 (DTP = DPT = DTC) – Coverage with measles– Percentage of children never reached
• Access – Defined as availability/start of services (number of children who
received one of the earliest vaccines)– Coverage with BCG or DTP1
• Utilization – Defined as continued use of services
– “Dropout rate” e.g. the number of children who received an early vaccination (DTP1) compared to the number of children who received a later vaccination (DTP3)
How to calculate coverage
– Coverage formula
________number of doses given_______ X 100estimated number of age-eligible children in catchment area
– Sources of error for coverage• Number of doses given• Estimated number of age-eligible children in the
catchment area
How to calculate dropoutDropout definition: Child begins but does not complete
immunization schedule
Drop-out rate formula
[ ( # Beginning – # Ending ) / # Beginning ] X 100
– Usually, series begins with DTP1 or BCGand ends with DTP3 or measles
= [( DTP1 – measles ) / DTP1] X 100 = [( DTP1 – DTP3 ) / DTP1] X 100 = [( BCG – measles ) / BCG] X 100 = [( BCG – DTP3 ) / BCG] X 100
Visualization of Never-reached versus Drop-outs
District A District B0
20
40
60
80
100
50
85
4258
DTP1DTP3Pe
rcen
t
Never-reached
Drop-outs
Never-reached
Drop-outs
Practice: Calculate Coverage and Dropout
Compile population and coverage
data from previous 12 monthsProblems analysis
Name of Health
Facilities
Target pop < 1 yr
Doses givenImmunization coverage
(%)Infants non immunised
Drop out rate (%)
Identification of problems
Category
DTP1 DTP3 Measles DTP1 DTP3 Measles DTP3 MeaslesDTP1 & DTP3
Access Utilization1, 2, 3, 4
A B C D E F G H I J K L M N
# # # #(C/
B)x100(D/
B)x100(E/
B)x100 B - D B - E(C-D)/C x
100 See flow chart "Interpretation"
Berma 430 502 472 445
Dokuy 781 836 353 538
Dara 490 355 342 524
Ira 916 651 573 572
TOTAL
Tool 1: Monitoring chart
• Used at multiple levels to visually track monthly performance• Charts include:
– Number of doses administered per month– Cumulative number of doses administered in year– Coverage rate– Difference in number of doses administered between 2 vaccines
(dropout)– Monthly target and cumulative target population
• Good charts are:– Up to date– Properly filled
Completing an Immunization Monitoring Chart
• Enter monthly objectives on the y-axis (target population ÷ 12)
• Diagonal line represents regular progress over a 12 month period ending with 100% coverage
• Every month – Enter the antigen total for the month– Calculate the cumulative total for the antigen– Mark the cumulative total on the chart with a dot– Calculate the cumulative number of drop outs– Calculate the cumulative drop out rate
Practice: How to fill in a monitoring chart
• Zorgho, 2003• Target population < 1 =
12,897
DTP1 DTP3
Jan 836 525
Feb 909 631
Mar 818 539
Apr 1027 729
May 909 728
Jun 660 529
Jul 1001 650
Aug 860 562
1075
7523
6449
5374
4299
3224
2150
8598
9673
10748
11822
12897
1075Zorgho 2003 12897
0
836
525
836
525
909 1745
631 1156
818
539
2563
1695
1075
7523
6449
5374
4299
3224
2150
8598
9673
10748
11822
12897
1075Zorgho 2003 12897
0
836
525
836
525
909 1745
631 1156
818
539
2563
1695
311
37
589
34
868
34
1075
7523
6449
5374
4299
3224
2150
8598
9673
10748
11822
12897
DTP1
DTP30
Zorgho 2003
Tool: Immunization summary form
• Facility summary form• Summary of doses administered at facility• Data summarized from session tally sheets
• District summary form• Summary of doses administered across all facilities in
district• Data summarized from facility summary sheets
• Sent monthly by each level• Used nationally as source of information on
number of doses administered
Illustrative flow of immunization summary information
Health CenterHealth postsOutreach sites
Tally sheetsPatient registersCase reports
Health Centerconsolidates
Monthly reportsCase reportsWeekly reports
DistrictConsolidated monthlySurveillance reportWeekly reports
Consolidated monthly reports, vaccinationCase reports
National Department of Epidemiology
EPI program
Liberia Immunization Summary Form
Tool: Defaulter Tracking System
• Method to identify children who did not return for vaccination (drop outs)
• Multiple tools used as “tracking system”– Ledger of children’s names who dropped out– Tickler box or blanket
• Copy of vaccination card is kept with the month when child is expected to return
• At end of month, remaining cards used to identify and then find children who did not return
Defaulter ledger example
Source: Increasing immunization coverage at the health facility level, WHO 2001
What proportion of the children have ACCESS to immunization services? (what is DTP1 coverage?)
Tool: Prioritize your locations by severity of problems (I)
What proportion of children COMPLETE the immunization schedule (what are the DTP1 - DTP3 drop-out rates?)
Categorize the problem
HIGH coverage with DTP1 (>80%) Low coverage with DTP1 (<80%)
Drop-out rate <10%
Drop-out rate >10%
Drop-out rate <10%
Drop-out rate >10%
• Dropout rates are low = good utilization
• Coverage is high = good access
• Category 1 (no problem)
• Dropout rates are high= poor utilization
• Coverage is high = good access
• Problem Category 2
• Dropout rates are low = good utilization
• Coverage is low= good access
• Problem Category 3
• Dropout rates are high = poor utilization
• Coverage is low= poor access
• Problem Category 4
Compile population and coverage
data from previous 12 months
Problems analysis
Name of Health
Facilities
Target pop < 1 yr
Doses givenImmunization coverage
(%)Infants non immunised
Drop out rate
(%)
Identification of problems
Category
Priority
DTP1 DTP3 Measles DTP1 DTP3 Measles DTP3 MeaslesDTP1 & DTP3
Access Utilization1, 2, 3, 4
1, 2, 3, 4
A B C D E F G H I J K L M N O
# # # # (C/B)x100 (D/B)x100 (E/B)x100 B - D B - E (C-D)/C x 100 See flow chart "Interpretation"
Berma 430 502 472 445 117% 110% 103% 30 -15 6% 1 3
Dokuy 781 836 353 538 107% 45% 69% 483 243 58% X 2 1
Dara 490 355 342 524 72% 70% 107% 13 -34 3.7% X 3 2
Ira 916 651 573 572 71% 62% 62% 78 344 12% X X 4 4
Tool: Prioritize your locations by severity of problems (II)
Tool: Rapid community assessment• Monitoring tool used by supervisors to rapidly understand whether
vaccination services are reaching the community and identify barriers to vaccination
• Not a survey and does not generate coverage rates
• Useful to use in places where administrative vaccination coverage is reportedly high– A rapid assessment should not find many unvaccinated children
• General method– Select ~20 households and assess vaccination status and reasons for no vaccination– Should be conducted in parts of a catchment area where low coverage is expected
Thanks
Questions?