Upload
dana-osborne
View
267
Download
0
Tags:
Embed Size (px)
Citation preview
Health Facility Surveys and Quantified Supervisory
Checklists
Health System Innovations Workshop Abuja, Jan. 25-29, 2010
Health Facility Surveys – What are they?
• Assessments of different types of health facilities using a standardized questionnaire
• Usually done through a simple random sample of all health facilities
• Surveyors are usually trained health workers (often doctors)
• Usually look at many different aspects of service delivery including technical quality of care
2
Health Facility Surveys
Advantages• Can assess quality
of care• Can be
independent of service providers
• Can be done more frequently than HHS
Disadvantages• Complex to design • Lots of data, can
overwhelm • Cannot provide
information on coverage, equity
3
An example from Afghanistan:
• 600+ facilities surveyed every year 2004 to 2008 by a team led by JHU
• Contents developed through consultative process
• Very careful quality assurance• Each facility rated on a score of 0-100, can be
aggregated at county, state, national level• Present results through “balanced scorecard”
4
What the BSC Looks At:
• Presence of staff• Knowledge of staff• Quality of patient-provider interaction• Availability of drugs and supplies (also quality
on sample basis)• Patient satisfaction (different from HH results)• Waste management • Use of facilities, use by women, and the poor • etc. 5
Can Look at Provincial Progress – Color Coded
6
32% Improvement in Total Scores in Contracted Facilities (from health facility survey)
40
45
50
55
60
65
70
75
80
2004 2005 2006 2007
MOPH Alone
PPA Median
Looking at Provincial Progress on Total Score
Balanced Score Card Results from 2007 Compared to Previous Years
Province 2004 2005 2006 2007 Change from 2004
to 2007
Badghis 48.7 59.3 49.8 80.2 31.5
Balkh 55 71 71.6 78.6 23.6
PPA Median 53.4 60 62.8 75.8 22.4
National Median 53.2 59 65.4 70.2 17
8
Can Look At Areas Needing Attention
Index 2004 2005 2006 2007 Change 2007-2004
Patient Counseling 29.6 35.1 36.6 48.7 19.1 Equipment Functionality 65.7 67 78.7 83.8 18.1 Family Planning Availability 61.4 70 82.9 93.7 32.3 Patient History & Exam 70.6 73.5 82.2 83.1 12.5 Proper sharps disposal 62.2 52 77.5 84.4 22.2 Obstetrical care 25.4 22.3 42.3 59.5 34.1 BHC's with >750 patients 22.2 32.3 55 57.4 35.2 HMIS Implementation 67.7 65.8 74.9 91.5 23.8 Provider Knowledge 53.5 69 68.7 68.7 15.2 Drug Availability 71.1 83.7 85.7 81 9.9
9
Health Facility Assessment in Nigeria under Malaria + Program covering 327 facilities
Illnesses: Fever/Malaria; Pneumonia; Dysentery and Diarrhea
13
Poor awareness of PMVs regarding new Malaria treatment policy
What are the challenges with Health Facility Surveys?
• Deceptively difficult to do• Requires talented technical staff
experienced in survey design• Need to do it every year or so to look at
changes• Costs about $300,000 per year (more during
development)
17
What are the challenges with Health Facility Surveys?
• Generates a lot of data (400+ questions on each facility)
• Tough to explain to managers – need means, like BSC, to summarize data
• Quality assurance is a real challenge• Easy to do badly – consumers won’t
know
18
Quantitative Supervisory Checklist – What is it?
• A reduced version of a health facility assessment
• Objectively assesses a variety of indicators to come up with total score.
• Takes about 2-3 hours to complete• A copy of results left in the health facility, easy
to track progress• QSC is both a management intervention and
tool for M&E19
Example of a Quantitative Supervisory Checklist
Date of Visit 5/12 7/19 8/11 10/21
Availability of Drugs (0-10) 3 5 4 6Presence of staff (0-5) 2 1 2 2HMIS implementation (0-10) 3 3 5 5TB Case Detection Rate (0-5) 0 1 1 2DPT3 coverage rate (0-10) 2 3 3 4Consultations per capita (0-10) 2 4 2 5Deliveries in facility (0-10) 0 1 1 3TOTAL SCORE (out of 60) 12 18 18 27Supervisor’s signatureHF in-charge signature 20
Development of QSC in the Philippines
• New HMIS forms developed which were supposed to facilitate supervision
• “Checklist Safari” in 7 provinces found:– 25 different checklists– 95 items, average 4.5 pages long– Rarely used, never found in health facilities– Designed in such a way to make follow up difficult
• Supervision was sporadic, not systematic, mostly dreaded by health workers
21
Development of QSC in the Philippines
• Discussions with key program managers led to definition of 20 indicators.
• Indicators scored from 0-3 with specific definitions and means of calculation
• Copy of QSC could be left in HF so future supervisors & staff could track progress
• Copy with supervisor so s/he could track which indicators were lagging
• Before & after assessments in 4 experimental provinces and 6 control provinces
22
Example of a Quantitative Supervisory Checklist
Date of Visit 5/12 7/19 8/11 10/21
Availability of Drugs (0-10) 3 5 4 6Presence of staff (0-5) 2 1 2 2HMIS implementation (0-10) 3 3 5 5TB Case Detection Rate (0-5) 0 1 1 2DPT3 coverage rate (0-10) 2 3 3 4Consultations per capita (0-10) 2 4 2 5Deliveries in facility (0-10) 0 1 1 3TOTAL SCORE (out of 60) 12 18 18 27Supervisor’s signatureHF in-charge signature 23
Evidence for the Effectiveness of QSC
24
% Change in Scores from Baseline
Other Findings from QSC• Health workers liked it because it made it
clear what was expected. Supervisors not angry
• Supervisors liked it because made interaction with HWs more focused on key results
• HWs tracked performance and became adept at tracking their own performance
• Was launched nation-wide but fell into dis-use after devolution
25
Quantified Supervisory Checklists
Advantages• Can assess QOC.• Can be independent
of service providers • Can be done often• Inexpensive• Clarifies what is
expected of HWs• Can be adapted to
conditions as they change
Disadvantages• Challenging to design • Cannot provide
information on coverage, equity
• Ensuring continued use is difficult
26