International Journal of Gynecology and Obstetrics 127 (2014) 108–112

Contents lists available at ScienceDirect

International Journal of Gynecology and Obstetrics

j ourna l homepage: www.e lsev ie r .com/ locate / i jgo

EVIDENCE FOR ACTION

Using scorecards to achieve facility improvements for maternal andnewborn health

Mohamed Yilla a,⁎, Sara L. Nam b, Austine Adeyemo a, Samuel A. Kargbo c

a Evidence for Action, Freetown, Sierra Leoneb Evidence for Action - Options Consultancy Services, London, UKc Reproductive and Child Health Directorate, Ministry of Health and Sanitation, Freetown, Sierra Leone

⁎ Corresponding author at: 12A King Street, Off theConsultants, Freetown, Sierra Leone. Tel.: +232 7919 846

E-mail address: m.yilla@evidence4action.net (M. Yilla

http://dx.doi.org/10.1016/j.ijgo.2014.07.0110020-7292/© 2014 Published by Elsevier Ireland Ltd. on b

a b s t r a c t

a r t i c l e i n f o

Keywords:

AccountabilityEmergency obstetric and neonatal careEvidence for Action (E4A)Maternal healthNeonatal healthQuality of careScorecardsSierra Leone

The Government of Sierra Leone launched the Free Health Care Initiative in 2010, which contributed to increaseduse of facility basedmaternity services. However, emergency obstetric and neonatal care (EmONC) facilitieswerefew andwere inadequately equipped tomeet the increased demand. To ensure provision of EmONC in some pri-ority facilities, the Ministry of Health and Sanitation undertook regular facility assessments. With the use of as-sessment tools and scorecards it is possible to make improvements to the services provided in the period afterassessment. The exercise shows that evidence that is shared with providers in visually engaging formats canhelp decision-making for facility based improvements.

© 2014 Published by Elsevier Ireland Ltd. on behalf of International Federation of Gynecology and Obstetrics.

1. Background

How can we save the 2400 mothers and 10 000 newborns that dieeach year in Sierra Leone [1], and how can we ensure the country ison track to achieve Millennium Development Goals (MDGs) 4 and 5?

Despite a decade of conflict that left the healthcare infrastructure indisarray, there is gradual progress. In April 2010, the Government ofSierra Leone launched the Free Health Care Initiative (FHCI), which con-tributed to increased use ofmaternity services. The proportion of facilitydeliveries has doubled in the period between the two Demographic andHealth Survey (DHS) periods, from25% in the 2008DHS [2] to 54% in the2013 DHS [3]. However, significant gaps in quality of care pose a threatto the sustainability of the FHCI [4].

A needs assessment conducted in 2008 [5] found that there were nobasic emergency obstetric and neonatal care (BEmONC) facilities in thecountry and the comprehensive emergency obstetric and neonatal care(CEmONC)—mostly nongovernment-owned facilities—were concen-trated in the western and northern regions, leaving six districts withno CEmONC facility. The framework of the UN Process Indicators focus-ing on signal functions [6] recommends that five emergency obstetricand neonatal care (EmONC) facilities are in place per 500 000 popula-tion, of which at least one should be a CEmONC facility. This underpinstheMinistry of Health and Sanitation’s (MoHS) long-term goal to equip

Maze - DALAN Development4.).

ehalf of International Federation of G

all hospitals and community health centers (CHCs)with themeans nec-essary to achieve quality outcomes in EmONC.

In 2010, 65 CHCs and 13 government hospitals were prioritized toprovide EmONC services plus five BEmONC facilities per 500 000 popu-lation [7]. Selection of facilities was guided by administrative bound-aries at district and chiefdom level. In each of the 13 districts, onegovernment hospital and five community health centers per districtwere assessed quarterly for their readiness to provide comprehensiveand basic EmONC, respectively.

2. Using scorecards: Facility Improvement Team (FIT)assessment process

To ensure provision of EmONC in the priority facilities, MoHSassessed the health facilities quarterly, from October 2010 to July2013. It established an assessment team—the Facility ImprovementTeam (FIT)—that included health personnel from different MoHS pro-grams and was led by a coordinator. Assessment was done using achecklist that resulted in a scorecard and provided a mechanism toidentify what was required to raise the standard of the facilities’ readi-ness to provide EmONCby focusing on seven enablers (Fig. 1). These en-ablers contribute to the enabling environment inwhich signal functionsare performed. Signal functions as per the UN framework are used todefine an EmONC and include: (1) administering parenteral antibiotics;(2) administering uterortonic drugs; (3) administering parenteral anti-convulsants for pre- eclampsia and eclampsia (i.e. magnesium sulfate);(4) manually removing the placenta; (5) removing retained products(e.g. manual vacuum extraction, dilation and curettage); (6) performassisted vaginal delivery; (7) perform basic neonatal resuscitation;

ynecology and Obstetrics.

Fig. 1. The seven enablers assessed through the Facility Assessment Team (FIT) process.

109M. Yilla et al. / International Journal of Gynecology and Obstetrics 127 (2014) 108–112

(8) perform surgery (e.g. cesarean delivery); and (9) perform bloodtransfusion. A BEmONC facility is one in which all functions one toseven are performed, and a CEmONC facility is one in which all func-tions one to nine are performed [6].

Each quarter the FIT working group reviewed previous findings,adapted approaches, and conducted field visits. The field visits involvedinitial meetings with district level stakeholders such as members of dis-trict health management teams, local councils, ministry of local

Fig. 2. Facility Assessment Team (FIT) scoring system for each enabler.

110 M. Yilla et al. / International Journal of Gynecology and Obstetrics 127 (2014) 108–112

government, and local government finance departments. The FIT teamreviewed the findings and recommendations of previous assessmentsand gained permission to conduct another assessment.

FIT assessments included a member of the district health manage-ment team (for example, the District Health Sister or Monitoring andEvaluation Officer) and upon completion of assessments, preliminaryfindings and key recommendations were presented to the districtlevel stakeholders for them to address any gaps. The MoHS FIT coordi-nator received the completed data and, after analysis, a report was

Fig. 3. Screenshot of a Facility Assessment Team (FIT) scorecard showing the progress of selectedstatus (at January 2013). Abbreviations: Gvt, Government.

disseminated at national level and scorecards summarizing the findingswere disseminated at district level.

3. FIT assessment tool and FIT scorecards

The FIT questionnaire tool uses checklists and closed questions to as-sess whether facilities meet minimum standards of infrastructure forsignal functions under each enabler required to achieve EmONC certifi-cation. The enablers are: water and sanitation; electricity; staffing;referrals; drugs; blood storage and laboratory services; and specialequipment. The number of components met is graded on a traffic lightgrading system as shown in Fig. 2, and the details of theminimum stan-dards required for each of the enablers to achieve a green status basedon whether they are BEmONC or CEmONC facilities, are provided inFig. 1. When all seven enablers are graded as green, the facility is certi-fied as EmONC compliant (green). A facility that demonstrates achiev-ing four to six enablers is graded yellow, one to three is graded amber,and none is graded red. This traffic light system is then summarized inscorecards for hospitals and CHCs (Fig. 3).

4. Achievements

Since implementation of the FIT assessments, more than 50% ofthe government hospitals and nearly 20% of the CHCs assessed raisedtheir standards to green for all seven enablers over the period from

government hospitals in achieving comprehensive emergency obstetric andneonatal care

Fig. 4.Histogramwith linear trend line demonstrating overall progress of community health centers assessed by the Facility Assessment Team process fromNovember 2010 to July 2013.

111M. Yilla et al. / International Journal of Gynecology and Obstetrics 127 (2014) 108–112

November 2010 to July 2013, thus attaining comprehensive and basicEmONC certification, respectively [8]. Overall progress since November2010 has been achieved, based on the total mean scores of all enablersfor both CHCs andhospitals and expressed as a percentage (Figs. 4 and 5).

The FIT assessment scorecards are a visual situation analysis thatmade it easier to communicate findings to councils and other planners.This enabled more accessible dissemination of the findings, and use ofthe tools to advocate for allocation of resources. The scorecards havealso strengthened coordination among partners and created healthycompetition among facilities, communities, anddistrict level authorities.

5. The future for FIT

Dedicated donor funding for FIT assessments has now ended, al-though Evidence for Action (E4A) and UNICEF have responded to a re-quest for a FIT assessment to take place in July 2014 co-funded by theMoHS. For the assessments to continue in a more sustainable manner,advocacy efforts for dedicated funding have to be aimed at governmentand donor partners. More cost-effective methods of feedback on thefindings are required to ensure that the standards for all enablers arenot only met but are also sustained. For example, facilities can fallshort of achieving full green status owing to drug and equipment stock

Fig. 5. Histogram with linear trend line demonstrating overall progress of government hospit

outs, which requires coordination amongdifferent agencies and govern-ment departments to achieve continued safe EmONC compliant clinics.

In the absence of a donor-funded FIT coordinator, MoHS directorshave been visiting districts to discuss the most recent FIT assessmentfindings and emphasize the need to improve quality of facilities. How-ever, identifying a focal person to act as liaison between the health facil-ities and the ministries of local government and health will ensureeffective communication of findings and coordination of interventions.

We envisage that the FIT scorecardswill be adopted as a tool by com-munities to hold planners and decision-makers to account by strength-ening partnerships between community level health developmentcommittees, civil society, and nongovernmental organizations. As evi-dence for the use of community involvement in such accountabilitytools grows [9], this is an opportunity to ensure the continuation ofup-to-date scorecards to improve and maintain quality of care and lifesaving EmONC.

Acknowledgments

This work was funded by the UK Department for International De-velopment (DFID). The most recent and current FIT assessments inJuly 2013 and 2014 were made possible through collaborative fundingeffort by MoHS, UNICEF, and DFID.

als assessed by the Facility Assessment Team process from November 2010 to July 2013.

112 M. Yilla et al. / International Journal of Gynecology and Obstetrics 127 (2014) 108–112

Conflict of interest

Mohamed Yilla, Austine Adeyemo, and Sara Nam are supported bythe DFID-funded Evidence for Action (E4A) program. Samuel Kargbo,

Director of the Reproductive and Child Health Department, is employedby the Ministry of Health and Sanitation of Sierra Leone.

References

[1] World Health Organization, UNICEF, UNFPA, the World Bank, United NationsPopulation Division. Trends in maternal mortality: 1990 to 2013. Estimates by WHO,UNICEF, UNFPA, theWorld Bank and the United Nations Population Division. Geneva:WHO; 2014. http://apps.who.int/iris/bitstream/10665/112682/2/9789241507226_eng.pdf?ua=1. Accessed June 1, 2014.

[2] Statistics Sierra Leone, Measure DHS. Sierra Leone Demographic and Health Sur-vey 2013. Preliminary report without results of HIV prevalence. Freetown andRockville: SSL, MEASURE DHS; 2014. http://dhsprogram.com/pubs/pdf/PR42/PR42.pdf.

[3] Statistics Sierra Leone, ICF Macro. Sierra Leone Demographic and Health Survey2008. Calverton, Maryland, USA: Statistics Sierra Leone and ICF Macro; 2009.http://dhsprogram.com/pubs/pdf/FR225/FR225.pdf.

[4] Maxmen A. Sierra Leone's free health-care initiative: work in progress. Lancet 2013;381(9862):191–2.

[5] Oyerinde K, Harding Y, Amara P, Kanu R, Shoo R, Daoh K. The status of maternal andnewborn care services in Sierra Leone 8 years after ceasefire. Int J Gynecol Obstet2011;114(2):168–73.

[6] World Health Organization, UNFPA, UNICEF, Averting Maternal Death andDisability. Monitoring emergency obstetric care: a handbook. Geneva: WHO;2009. http://whqlibdoc.who.int/publications/2009/9789241547734_eng.pdf?ua=1. Accessed July 18, 2014.

[7] Ministry of Health and Sanitation Reproductive and Child Health Programme.Report of Q1 2012 Facility Improvement Team (FIT) Assessment Exercise, May2012. http://www.mamaye.org.sl/sites/default/files/evidence/MOHS_2012_Q1%20FIT%20Report.pdf. Accessed July 18, 2014.

[8] Government of Sierra LeoneMinistry of Health and Sanitation Reproductive and ChildHealth Programme. Report of Q1 2013 Facility Improvement Team (FIT) AssessmentExercise, January 2013. http://www.mamaye.org.sl/sites/default/files/evidence/MOHS_2013_Q1%20FIT%20report.pdf. Accessed July 18, 2014.

[9] Joshi A. Do theywork? Assessing the impact of transparency and accountability initia-tives in service delivery. Dev Policy Rev 2013;31(S1):S29–48.