Upload
alexis-blake
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
may 9, 2008 1
Implementation and use of operations research
Dr. Lesly Michaud
World Vision, Haiti
may 9, 2008 2
Presentation Outline
• Description of implementation
• Objectives & methods of operations research (OR)
• Selected OR results
• Conclusions and implications of OR for overall evaluation
may 9, 2008 3
Timeline
2002 2003 2004 2005Qualitative/ formative research
Development of BCC and 2 program models
Implementation of full program models
Operations Evaluation (Round 1)
Consultation with WV; program improvements
Operations Evaluation (Round 2)
BaselineSurvey
Final Survey
Development Implementation
Impact evaluation
Program implementation
-Service delivery points
-Staffing
may 9, 2008 5
Program service delivery points
Rally PostIdentification of beneficiaries
Mandatory attendance at
- Mothers’ Clubs*(mother of beneficiary child)- Home visits (severely malnourished
- Prenatal/ postnatal clinics- Mothers’ Clubs(pregnant/lactating women)- Home visits (post-delivery)
Eligible to receive food atFood distribution points
*Organized differently in preventive and recuperative
PREG/LACTATING WOMENCHILD BENEFICIARIES
may 9, 2008 6
Organizational structure
Headquarters (Port-au-Prince)Director of MCHN
Director of Commodities
Regional Office (Central Plateau)Regional MCH Coordinator; Assistants to MCH
Coordinator
MCH Supervisors(Nurses)
Health Promoters
Assistant Health Promoters
Food Monitors
Regional Office (Central Plateau)
Regional Commodities Officer
may 9, 2008 7
Services at Rally Posts (RP)
Identical services in both program models
Services include:• Growth monitoring and promotion• Determination of eligibility for food assistance*• Vaccination• Distribution of vitamin A capsules, ORS, deworming
medications• Brief group education sessions
*Based on program model: age-based eligibility in preventive and nutritional status-based eligibility in recuperative
may 9, 2008 8
Services at Mothers’ Clubs (MC)
• Preventive BCC- Pregnancy: 6 sessions- Lactation: 6 sessions- When child is 6-23 mo:
18 sessions, of which 7 focus on complementary feeding and 11 on other topics; some reinforcement
• Recuperative BCC- Pregnancy: 6 sessions- Lactation: 6 sessions- Malnourished child is 6-59
mo: 9 sessions of which 7 sessions focus on recuperation, nutrition; 2 on other topics
Small group education sessions, using interactive methods, meeting monthly and located close to homes of beneficiary mothers
may 9, 2008 9
Services at pre- and post-natal consultations
• Identical in both program models
• Services include– Physical examination– Iron-folate supplements– Tetanus toxoid immunization– Post-partum vitamin A
may 9, 2008 10
Services at Home Visits
Identical in both program models
• Post-delivery
• Newborn visit
• Follow-up of severely malnourished children
may 9, 2008 11
Food Distribution Points
• Monthly ration is the same in both groups (except for longer duration for children in preventive program)
• Ration for children is different from ration for pregnant/lactating women.
• Indirect household ration is included, but only one indirect ration even if multiple beneficiaries in a single household
may 9, 2008 12
Commodity Children 6-24 mo or malnourished <5 years
Pregnant/lactating women
Direct ration (kg)
Indirect ration (kg)
Direct ration (kg)
Indirect ration (kg)
WSB 8
SFB 10 5 5
Lentils 2.5 2 2
Oil 2 1.5 1.5
WSB=Wheat-soy blend; SFB=Soy-fortified bulgur; oil is fortified with vitamins A & E
Rations distributed at FDPs
Operations Research
may 9, 2008 14
Main objectives
OR Round 1
• Evaluate program implementation relative to plan
• Identify and prioritize solutions to problems
OR Round 2
• Evaluate implementation of solutions
• Look for differences in implementation and/or program use between preventive and recuperative models
– To rule out implementation differences as a reason for differences in impact at the final survey
may 9, 2008 15
Methods
• Structured observations Rally Posts, Mothers’ Clubs, Food Distribution Points
• Exit interviews with beneficiaries and staff
• Group interviews with beneficiaries & staff
may 9, 2008 16
Examples of issues covered
• Rally Posts (RP) – OR-1 & OR-2Staffing and participationOrganization of activities, bottlenecksAvailability of suppliesLarge group education: staffing, qualityGrowth monitoring: accuracy, quality, counseling
• Mothers’ Clubs (MC) – OR-1 & OR-2Organization (age groupings) relative to planStaff facilitation and teaching skills
• Food Distribution Points (OR Round 1 only)OrganizationQuality of foodAmounts distributed (compliance with intended amount)
may 9, 2008 17
Other issues covered
• Staff-level factorsMotivation, job satisfaction
Perceptions of supervision/management
Technical knowledge
Time allocation/workload
• ParticipantsKnowledge of BCC messages
Trial and adoption of key practices
Factors influencing trial and adoption (constraints, facilitating factors)
may 9, 2008 18
Key results from Round 1 Rally Posts
• High participant/staff ratios, crowded
• Sequence of activities often did not follow plan
• Problems with availability of supplies Vaccines, ORS, vitamin A, deworming
• Group education Only one session, crowded, poor ambience, little use of visual materials, did not follow monthly plan for topics
• Growth monitoring Some errors in weighing and plotting, communication with mothers somewhat limited and vague
may 9, 2008 19
Key results from Round 1Mothers’ Clubs
• Overall quality very goodFacilitation, teaching, technical content
• In the preventive model, difficulties to organize MCs by age groups
• Quality not consistent across all staff/program areas
may 9, 2008 20
Key results from Round 1 Food Distribution Points
• Minor inconsistency in measurement of commodities but beneficiaries largely got intended amount
• Staff and food transport problems and a very high beneficiary/staff ratio lead to high time burden for beneficiaries and staff
• Other issues– Food handling/hygiene concerns– Communication issues (health & commodities)
may 9, 2008 21
Consultative workshop with staff Port-au-Prince February 2004
• Presentation of OR 1 results, dialogue, use of decision tool to:• Prioritize problems• Identify feasible solutions• Make an action plan
• Program management played an active role in process
• Field staff were also involved
may 9, 2008 22
OR 2 Rally Post ResultsOrganization & supplies 2003 vs 2004
• At 10 RPs observed both years:
Slight decrease in average participant/staff ratio; range decreased substantially
2003: 21 (7-59) 2004: 17 (6-33)
• Sequence of activities followed plan designed following workshop
• Numbering system used to ensure first-come first-served
• Availability of supplies still a problem: some better, some worse than 2003
may 9, 2008 23
OR 2 Rally Post ResultsGroup education 2003 vs 2004
• Many positive changes
• No differences between program areas
# of sessions observed
2003n = 24
2004n = 62
Mean number of sessions (range)
1.3
(1 - 3)
3.1
(2 – 6)
Mean number of participants (range)
30
(4 - 76)
15
(5 – 30)
Mean duration of session (range)
10
(1 – 23)
22
(6 – 41)
may 9, 2008 24
OR 2 Mothers’ Clubs Results
• Large improvement in organizing mothers into groups by child age in preventive MCs
• Session topics appropriate to groups
• Quality of facilitation and teaching remains very high (e.g. technical content 84% correct; same as OR1)
• No differences between program areas
may 9, 2008 25
Summary of OR 2 results Staff interviews
• Staff-level factors
Motivation, job satisfaction – very positive
Perceptions of supervision – positive, but some variability
Technical knowledge – generally very good, some gaps
• No differences between program areas
may 9, 2008 26
Conclusions 1:Implementation of solutions
• Achieved success in implementing majority of solutions identified at workshop
• Demonstrated commitment to program improvement
• Operations research helped build staff interest and commitment in improving program
• Areas for continuing improvement were identified
may 9, 2008 27
Conclusions 2: Comparing preventive and recuperative program areas
• No major differences in quality of implementation
• Some suggestion that communication during growth monitoring is better in preventive areas
• No major differences in staff-level factors or in participant trial/adoption of practices
may 9, 2008 28
Conclusions 3: Implications for evaluation of effectiveness
and cost-effectiveness
• Any differences in effectiveness can reasonably be attributed to differences in program models, but NOT program implementation