Rapid Assessment of the Health Worker In-Service Training ... · A. Types of Education and Training Provided ... In-service training program provider and training program questionnaire

Rapid Assessment of the Health Worker In-Service Training Situation in Ethiopia: Survey of Training Program Provider Practices and Key Informant Interviews

MARCH 2014

This assessment report was prepared by University Research Co., LLC (URC) for review by the United States Agency for International Development (USAID) and authored by Fikreab Kebede, Keneni Gutema, Abyot Asres, Tana Wuliji, and Emily Lanford of URC. The work described was conducted under the USAID Health Care Improvement Project and the report completed under the USAID Applying Science to Strengthen and Improve Systems (ASSIST) Project, both made possible by the generous support of the American people through USAID and its Office of Health Systems. The Ethiopia in-service training assessment was supported by the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR).

R ESE A RCH A ND E VA LUAT ION R EPORT

RESEARCH AND EVALUATION REPORT

Rapid Assessment of the Health Worker In-Service Training Situation in Ethiopia: Survey of Training Program Provider Practices and Key Informant Interviews

MARCH 2014

Fikreab Kebede, University Research Co., LLC Keneni Gutema, University Research Co., LLC Abyot Asres, University Research Co., LLC Tana Wuliji, University Research Co., LLC Emily Lanford, University Research Co., LLC

DISCLAIMER

The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

Acknowledgements

The authors would like to thank the Federal Ministry of Health for their advice and leadership in facilitating stakeholder consultations in the design and development of this assessment, identifying in-service training program providers to participate in the assessment and convening key stakeholders to review and discuss the assessment findings – especially the Honorable State Minister Dr. Amir Aman, Dr. Fitsum Girma and Mrs. Rahima Shikur. The Federal Ministry of Health Technical Working Group on health worker in-service training contributed actively throughout the assessment process and played a key role in shaping the design of the assessment and guiding the identification of respondents. Many thanks to the Jhpiego led consortium in the USAID Ethiopia Strengthening Human Resources for Health Program for their collaboration in hosting the stakeholder workshop where the assessment findings were reviewed, discussed and used to develop the national strategic framework for in-service training. Special thanks to Dr Samuel Hailemariam and Mr. Eshete Yilma of the USAID Ethiopia Mission staff for their guidance and support. The authors would like to appreciate Ms. Diana Frymus, USAID Washington, for her ongoing support and advocacy for health worker training improvement. Finally, the authors have appreciated the many hours and efforts taken by the assessment respondents to participate in this assessment, the information and insights they have shared have been critical towards our enhanced understanding of the in-service training situation in Ethiopia and priorities for improvement.

This report was prepared by University Research Co., LLC (URC) under the USAID Applying Science to Strengthen and Improve Systems (ASSIST) Project, which is funded by the American people through USAID’s Bureau for Global Health, Office of Health Systems. The project is managed by URC under the terms of Cooperative Agreement Number AID-OAA-A-12-00101. URC's global partners for USAID ASSIST include: EnCompass LLC; FHI 360; Harvard University School of Public Health; HEALTHQUAL International; Institute for Healthcare Improvement; Initiatives Inc.; Johns Hopkins University Center for Communication Programs; and Women Influencing Health Education and Rule of Law, LLC.

The assessment was designed and implemented under the USAID Health Care Improvement Project with the assessment publication finalized with the support of ASSIST. The assessment was funded by the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR).

For more information on the work of the USAID ASSIST Project, please visit www.usaidassist.org or write [email protected].

Recommended citation:

Kebede F, Gutema, K, Asres A, Wuliji T, Lanford E. 2014. Rapid assessment of the health worker in-service training situation in Ethiopia: Survey of training program provider practices and key informant interviews. Research and Evaluation Report. Published by the USAID Health Care Improvement Project. Bethesda, MD: University Research Co., LLC (URC).

Rapid assessment of the health worker in‐service training situation in Ethiopia i

TABLE OF CONTENTS

List of Tables and Figures ............................................................................................................... i

Acronyms ....................................................................................................................................... ii

EXECUTIVE SUMMARY ...................................................................................................................... iii

I. INTRODUCTION .......................................................................................................................... 1

A. Rationale .............................................................................................................................. 1

B. Study Context ....................................................................................................................... 1

C. Assessment Objectives ......................................................................................................... 2

II. METHODOLOGY ......................................................................................................................... 2

A. Study Design ........................................................................................................................ 2

B. Sampling ............................................................................................................................... 2

C. Description of the Study Sample ........................................................................................... 3

D. Data Collection ..................................................................................................................... 3

E. Survey Validation Process and Findings ............................................................................... 4

F. Analysis ................................................................................................................................ 6

III. RESULTS ..................................................................................................................................... 6

A. Types of Education and Training Provided ........................................................................... 6

B. IST Program Provider Staffing .............................................................................................. 6

C. IST Resources and Infrastructure ......................................................................................... 7

D. Training Content ................................................................................................................... 8

E. Trainees ................................................................................................................................ 9

F. Length, Frequency, Modalities, and Locations of IST ........................................................... 9

G. IST Program Provider Practices, Linkages, and Interaction ................................................ 12

H. Stakeholders’ Recommendations on Priority IST Issues ..................................................... 29

IV. DISCUSSION ............................................................................................................................. 29

A. Relation to Other Evidence ................................................................................................. 29

B. Limitations .......................................................................................................................... 30

V. REFERENCES ........................................................................................................................... 30

VI. ANNEXES .................................................................................................................................. 31

A. Annex I: In-service training program provider and training program questionnaire ............. 31

B. Annex 2: Key informant interview guide .............................................................................. 41

List of Tables and Figures

Table 1: In-service Training Program Providers, Ethiopia, July 2012 .................................................... 3 Table 2: Survey Response Validation Findings: Available Infrastructure and Educational Resources .. 4 Table 3: Survey Response Validation Findings: Availability of Training/Learning Materials and Methods of Training Delivery ................................................................................................................ 5 Table 4: Survey Response Validation Findings: Number of Training Programs Provided ..................... 5 Table 5: Survey Response Validation Findings: Training Facilities and Additional Infrastructure ......... 5 Table 6: IST Provider Staffing Levels with Training Evaluation Competencies ..................................... 7 Table 7: Percentage of Surveyed IST Program Providers with Infrastructure for Training ................... 8 Table 8: Percentage of Surveyed IST Programs Offering Various Content Areas ................................ 8 Table 9: Training Programs by Cadre Trained ...................................................................................... 9 Table 10: Frequency of Trainings Jan 2011-Dec 2011 ....................................................................... 10 Table 11: Training Sites, by Region: Urban vs. Rural and Type of Site .............................................. 11 Table 12: IST Program Provider Practices ......................................................................................... 12 Table 13: Formal Recognition of IST Program Providers and IST Programs ...................................... 20

Figure 1: Types of Training offered by IST Providers (% of IST providers), 2011 ................................. 7 Figure 2: Resources Available to IST ProviderTeaching Staff and Trainees......................................... 7

Rapid assessment of the health worker in‐service training situation in Ethiopia ii

Figure 3: Word Cloud of main training topics ........................................................................................ 8 Figure 4: Length of Surveyed Training Programs ............................................................................... 10 Figure 5: Average Percent of Training Programs Delivered Through Each Modality .......................... 11 Figure 6: IST Program Provider Practices for Developing Learning Materials and Curricula .............. 18 Figure 7: Training evaluation methods used ....................................................................................... 18 Figure 8: Types of Partners IST Programs Cited Working With .......................................................... 26

Acronyms

AIDS Auto-Immune Deficiency Syndrome

ASSIST USAID Applying Science to Strengthen and Improve Systems Project

CBO Community-based organization

CDC U.S. Centers for Disease Control and Prevention

EHNRI Ethiopian Health and Nutrition Research Institute

FBO Faith-based organization

FMOH Federal Ministry of Health

FTE Full-time equivalent

HCI USAID Health Care Improvement Project

HIV Human immunodeficiency virus

HMIS Health management information system

HRIS Health resource information system

IMCI Integrated management of childhood illness

IST In-service training

I-TECH University of Washington International Training and Education Center for Health

MEPI-AAU Medical Education Partnership Initiative-Addis Ababa University

NGO Non-governmental organization

PEPFAR U.S. President’s Emergency Plan for AIDS Relief

PMTCT Prevention of mother-to-child transmission of HIV

RHBs Regional Health Bureaus

SNNPR Southern Nations, Nationalities and Peoples Region

TWG Technical working group

URC University Research Co., LLC

USAID United States Agency for International Development

USG United States Government

Rapid assessment of the health worker in‐service training situation in Ethiopia iii

EXECUTIVE SUMMARY This report presents the results of an assessment of in-service training (IST) provided to health care workers in Ethiopia. Undertaken in 2012, the assessment used a questionnaire to elicit information from bodies providing IST (referred to in this report as IST providers) and an interview guide to elicit the views of key stakeholders, such as representatives of the Federal Ministry of Health (FMOH), professional associations, donors, and IST program providers.

With the ultimate goal of providing IST programs that would be effective, efficient, and sustainable, the Ministry, the U.S. Agency for International Development (USAID) Mission in Ethiopia, and the USAID Health Care Improvement Project (HCI) with the support from PEPFAR had collaborated in 2011 to define practices that would improve IST. They identified a short list of “practices” that would help the Ministry achieve its goal. The practices involved training design; delivery; follow-up/ monitoring; evaluation/capacity building; standardization and institutionalization; the tracking of training (e.g., who has been trained in what); and linkages among health care workers and community health workers, para-social workers, program managers, and policy makers. An example of such practices is “Our organization provides technical assistance to build the capacity of other IST program providers.”

The assessment sought to analyze the IST situation in Ethiopia and provide evidence to inform policy, planning, and implementation to 1) improve IST effectiveness, efficiency, and sustainability and 2) standardize and institutionalize the practices that caused such improvement. The findings would identify the strengths, weaknesses, and best practices in IST and inform the development of 1) a database to track IST training and 2) a strategy to standardize and institutionalize IST.

Methods

Mixed methods were applied in this assessment including a survey of IST program providers and a survey for each IST program they offered in the calendar year 2011, as well as semi-structured key informant interviews with opinion leaders. All 63 IST program providers identified by the FMOH were invited to participate in this study and 20 purposively selected opinion leaders were invited to participate in the key informant interviews. The survey of IST providers and IST programs were applied using the Internet and received responses from 34 (55%) civil society organizations, development partners, United Nations agencies, donors and funding agencies, Ministry technical units, regional health bureaus, and academic and training institutions. The 20 interviewed stakeholders represented the Ministry, regional health bureaus, professional organizations, donors, the private sector, and IST program providers. The qualitative analysis used a thematic approach, resulting in “themes” that are named, described, and illustrated in tables in the report.

Results: Ethiopia’s IST situation

The survey asked IST providers what kinds of training they provided between January and December 2011: 94% of trainings had been IST and 65% had been training-of-trainers, while pre-service and post-graduate trainings numbered far fewer, with each below 30%.

IST providers’ staffs ranged from nine to 750 full-time equivalents; 53% of IST program providers had one or more FTE staff competent in designing curricula, and 53% had two or more FTE staff with training competencies. Key informants mostly agreed with the need to partner with local IST program providers to build their capacity for sustainability; however, they also had concerns on feasibility and how this would be coordinated.

With regard to resources and infrastructure, most respondent IST providers provided access to computers and Internet to their staff, but only about half extended these facilities to trainees. Journal access was even less available, and having rooms for training even less so (rooms were accessed at hotels and other venues).

From the 34 IST providers responding to the survey, details of a total of 72 IST programs offered in 2011 were submitted. HIV and AIDS programs were provided most frequently (28%), followed by communication skills (22%), and family planning (17%). No IST program covered human resources management, injuries and violence, or social work and care.

Recipients of training were most frequently nurses, then health officers, and then physicians and rarely information system or lab personnel, health extension workers, and midwives.

Rapid assessment of the health worker in‐service training situation in Ethiopia iv

The assessment looked into 16 best IST provider practices (e.g., “Our organization provides technical assistance to build the capacity of other IST program providers”), asking survey respondents about the frequency with which each engaged in each practice on a five-point scale (“rarely” to “always”). The best practices that were reported most frequently to be always practiced related to describing to trainees the purpose of training and its objectives (79%, Table 12, line 9), ensuring compliance with national efforts (71%, Table 12, line 4), and having programs be formally recognized (59%, Table 12, line 2).

The report also presents scenarios of ideal provider practices that had been selected from the IST Improvement Framework and that the stakeholders were asked to comment on. The framework was developed by the Federal Ministry of Health IST Technical Working Group, and an example of a scenario is, “All IST providers are formally authorized or accredited by local/national authorities to provide training programs.” For example, many providers (responding to the survey) claimed to have been authorized/accredited, but (interviewed) stakeholders called for better authorizing mechanisms. Similarly, 65% of IST providers said they submit IST information to a training-tracking mechanism all or most of the time, but stakeholders suggested that such mechanism should be strengthened.

Recommendations

In reviewing the findings of this assessment, stakeholders prioritized IST actions as follows: 1) designing/implementing an IST strategy, 2) standardizing IST, 3) accrediting and licensing IST, 4) setting IST monitoring and evaluation guidelines, 5) establishing an IST database (tracking mechanism), 6) setting IST policy direction, 7) institutionalizing IST, 8) ensuring the accountability and commitment of trainees to share their knowledge, 9) linking IST to pre-service trainings, and 10) getting key partners to work together. They also recommended a focus on program impact, cost, approaches to curriculum standardization, alternatives to off-the-job training, potential assistance from higher education institutions, and the tracking of IST data.

Limitations

The involvement in the technical working group (TWG) in the design of the survey instruments, and qualitative interviews might have influenced their response in the survey. Thus the findings presented in this report may not adequately represent all IST program providers and programs offered in Ethiopia. Nevertheless, the study gained insight from the relatively high number of IST programs reported by those IST providers who did take part and serves as the most comprehensive IST assessment to date.

Rapid assessment of the health worker in‐service training situation in Ethiopia 1

I. INTRODUCTION This report describes the methodology and results of a rapid assessment of in-service training (IST) provided to health care workers in Ethiopia between January and December 2011. Undertaken in 2012, the assessment used a questionnaire to elicit information from IST providers and an interview guide to elicit the views of key stakeholders, such as representatives of the Ministry of Health and donors, as well as IST program providers.

The assessment sought to analyze the IST situation in Ethiopia and provide evidence to inform policy, planning, and implementation to 1) improve IST effectiveness, efficiency, and sustainability and 2) standardize and institutionalize the practices that caused such improvement. The findings would identify the strengths, weaknesses, and best practices in IST and inform the development of 1) a database to track IST training and 2) a strategy to standardize and institutionalize IST. With these goals in mind, the Federal Ministry of Health and USAID Health Care Improvement Project (HCI), with input from the IST Technical Working Group (TWG), designed and implemented a rapid assessment of the national IST situation; IST program provider practices; and key stakeholder opinions on IST priorities, issues, and strategic development.

The findings were presented in a workshop in 2012 and used to inform the development of the Federal Ministry of Health’s (FMOH’s) Strategic Framework for IST. This framework calls for the standardization and institutionalization of IST programs to facilitate better, more harmonized, and locally owned training that is more sustainable (details of the workshop and strategic framework are published in Kebede et al. [1]).

A. Rationale

Scaling up health care services is not possible without significant investments in IST to build the capability of health workers to competently, safely, and efficiently provide quality services. While IST has been widely seen as an effective way to enhance health workers’ knowledge and skills [2], many IST programs in Ethiopia had been found in a previous rapid assessment to be weak with poor planning, coordination, and quality. The percentage of health care providers offered IST was observed to be low, and even when IST programs were in place, they were often not needs based [3].

The effectiveness, efficiency, and sustainability of IST programs could be further improved if health workers, training program providers, Ministries of Health, donors, and partners worked together toward an evidence-informed vision for such improvement.

The assessment design process revealed the following key challenges in Ethiopia that potentially impede progress toward IST standardization and institutionalization:

1. Limited and/or inadequate training evaluations;

2. Training duplication, where health care workers repeatedly receive the same content;

3. Poor coordination among IST providers,

4. Inadequate training quality;

5. Fragmented training systems; and

6. Fragmented and incomplete guidance for IST, where the guidance is provided by multiple sources.

The TWG and development partners agreed to objectively inform the direction of the IST strategic development process. The FMOH and HCI, with input from the TWG, designed and implemented a rapid assessment of the national IST situation; IST program provider practices; and key stakeholder opinions on IST priorities, issues, and strategic development.

B. Study Context

Ethiopia policies, plans, and initiatives toward IST improvement

The Ministry’s Human Resources Directorate drafted an IST implementation guideline with the aim of standardizing and harmonizing the delivery of IST. As part of this process, the Ministry established and worked closely with the IST TWG.

Efforts have been undertaken to introduce training programs for Ethiopian health care providers that will place greater emphasis on improving the quality of service delivery [4]. The U.S. government (USG) also launched the five-year Strengthening Human Resources for Health project in 2012. One of its major components is strengthening health worker competencies through IST throughout the


country [5]. The FMOH led the development of the IST guide, as well as a concept note outlining strategies for the institutionalization and standardization of training programs to facilitate better, more harmonized, and locally owned training that is more sustainable.

Global Health Worker IST Improvement Framework

The development of the rapid assessment and the strategic framework was informed by a global activity in 2011 in which FMOH and USAID Ethiopia collaborated to define practices that would improve IST effectiveness, efficiency, and sustainability. Between June and December 2011, HCI facilitated a process that engaged training providers, professional and regulatory bodies, Ministries of Health, development partners, donors, and experts to develop and reach consensus on an improvement framework for IST. The development process involved 89 participants representing 26 countries and comprising stakeholder groups and experts (including the Ethiopian FMOH and USAID Mission). They engaged in five rounds of content development and review (a modified Delphi approach); the last round was an external validation of recommendations developed during the first four rounds. The resulting improvement framework recommends practices agreed upon by the stakeholders and experts to improve IST. It covers the following themes [6]:

1. Strengthening IST systems,

2. Coordination of training,

3. Continuum of learning from pre-service to in-service training,

4. Design and delivery of training,

5. Support for learning, and

6. Evaluation and improvement of training.

In this connection, the FMOH and TWG identified a shortlist of practices from the framework relevant to Ethiopia’s priority interests and needs. The FMOH and TWG then agreed to undertake a rapid assessment of the prevalence of these practices and the stakeholder opinion on their desirability and feasibility and strategies for their implementation.

The FMOH and TWG also decided to undertake a structured group process where the TWG and IST experts and key stakeholders would collaborate to design and reach a consensus on a national IST strategic framework that would focus on improving IST effectiveness, efficiency, and sustainability.

C. Assessment Objectives

The assessment’s objectives were to analyze the current IST situation in Ethiopia and provide evidence to inform policy, planning, and implementation to 1) improve IST effectiveness, efficiency, and sustainability and 2) standardize and institutionalize the practices that resulted in such improvement. The assessment aimed to:

Analyze Ethiopia’s IST situation;

Analyze IST provider practices in training design; delivery; follow-up/monitoring; evaluation; capacity building; standardization and institutionalization; tracking the trainings of health care workers by training recipient; and linkages among health care workers, community health workers, para-social workers, program managers, and policy makers;

Identify the strengths, weaknesses, and best practices in IST;

Inform the development of a database for health sector in-service trainings; and

Inform the development of a strategy for the standardization and institutionalization of health sector in-service trainings.

II. METHODOLOGY

A. Study Design

This assessment utilized a cross-sectional, mixed-methods study design and included a questionnaire for self-completion by IST program providers and structured interviews of key informants.

B. Sampling

Seventy-six IST providers were initially identified by the FMOH, IST TWG, and entities that implement and/or support IST. All providers were included in the study sample. Similarly, the FMOH and TWG


identified 20 key IST stakeholders to participate in the interviews. The study group believed that 20 would ensure a sufficient yet manageable sample.

C. Description of the Study Sample

Of the 76 IST providers identified by the FMOH and TWG that had been asked to complete the survey, eight did not provide training, contact details were incorrect for four, and one was a duplicate. These 13 were thus excluded leaving a total of 63 IST providers that were invited to participate in the study. Thirty-four IST program providers (54.8% response) completed the IST program provider survey for 72 IST programs and 28 did not respond. The FMOH and TWG identified the 20 key IST stakeholders to enable the development of a representative sample of informants to share their experiences, expertise, and guidance on strategies to improve IST. In identifying these stakeholders, the FMOH and TWG sought to ensure an institutional and geographic mix. Included were: three representatives from the FMOH, two from regional health bureaus (RHBs), eight from IST program providers, two from professional associations, four from donors, and one from the private sector. One RHB representative declined the interview due to an inability to attend a face-to-face meeting. This representative was replaced by another RHB representative, maintaining the desired balance of representation.

Respondent IST program providers: Most IST providers were NGOs (local and international), including development partners. Of the 34 respondents representing IST program providers, 32% were from local NGOs; 30% were from international NGOs; 21% were from public sector entities; and 15% represented such institutions as professional associations, multilateral organizations, and academia. Details of ownership and organization type are shown in Table 1.

Table 1: In-service Training Program Providers (n=34), Ethiopia, July 2012

Ownership type % (n)

Government 21 (7)

Private for profit: local 3 (1)

Private non-profit: local NGO 32 (11)

Private non-profit: international NGO 30 (10)

Other 15 (5)

Organization type

Training institution 15 (5)

Health facility 3 (1)

NGO, FBO, CBO 59 (20)

Multilateral agency 3 (1)

Company/business 3 (1)

Donor, foundation 3 (1)

Note: FBO: faith-based organization; CBO: community-based organization.

D. Data Collection

The rapid assessment used both quantitative (online survey) and qualitative (structured key informant interviews) to collect information that would achieve the study objectives.

Quantitative data were collected online using a survey administered through SurveyMonkey software (Annex 1). A pre-test of the online questionnaire was administered in institutions that were not included in the actual assessment and resulted in minor changes to the instrument. This effort was designed and managed by the headquarters office of University Research Co., LLC, which manages HCI, in Bethesda, Maryland, USA. A local consultant followed up the data collection process, mainly communicating with the surveyed institutions so that their responses would be provided by the desired date. He also helped resolve respondents’ difficulties in responding to the online survey. Survey data were stored electronically and transferred to SPSS for analysis.


Interviews were conducted from June 27th to July 27th, 2012. The interview instrument proposed “scenarios,” brief statements describing an alternative situation to one known to exist in Ethiopia (Annex 2). For example, one scenario was “All in-service training program providers are formally authorized or accredited by local/national authorities to provide training programs.” All in-service training informants were asked to share their opinions of the scenarios as well as answer questions. The study team also developed a guide providing instructions to data collectors.

The study team then shared a draft of the instrument with the FMOH and TWG and then refined it. Two experienced consultants were hired to manage the qualitative data collection and analysis. Before administering the instrument, they practiced using it to identify and manage problems that might have presented during data collection. Interviews were digitally audio-recorded and transcribed for analysis.

E. Survey Validation Process and Findings

Of the 34 program providers who completed the survey, four were chosen at random and asked to supply supporting documentation or evidence to verify their answers to certain questions. This exercise sought to confirm the accuracy of the providers’ responses. Survey answers were validated either through site visits to see the claimed infrastructure and training materials or by gathering and reviewing literature from providers to confirm training materials and the number of available training programs. Validation results are presented in the balance of this section, first through text and then in tables.

Infrastructure and educational resources available to teaching staff

Four of these providers each stated in the survey that both computers and Internet access were available to teaching staff, but the site visits could confirm the existence of such access at only three (75%). Furthermore, all four also claimed to have access to up-to-date references, but the documentation they provided was sufficient to confirm only three of these claims (75%). Lastly, while two of the four indicated in their survey responses that teaching staff had access to journals, neither of them provided sufficient evidence to support this claim.

Infrastructure and educational resources available to students/trainees

Three of the randomly selected providers reported that computers were available to students/trainees, and one also claimed that Internet access was available to them. These responses were confirmed at 67% and 100%, respectively. Additionally, three indicated that up-to-date references were available to students/trainees, and the documentation each provided was sufficient (100%). Finally, one provider stated in its survey responses that teaching staff had access to journals, which could not be confirmed (0%). See Table 2.

Table 2: Survey Response Validation Findings: Available Infrastructure and Educational Resources (n=4)

Availability to teaching staff Availability to students

Resource Number indi- Availability Confirmation Number indi- Availability Confirmation cating resource confirmed (%) cating confirmed (%) was available resource was

available

Computers 4 3 75% 3 2 67%

Internet 4 3 75% 1 1 100%

Up-to-date 4 3 75% 3 3 100% references

Journals 2 0 0% 1 0 0%

Availability of training/learning materials and methods of training delivery

All four selected providers indicated in the survey that learning materials were used in training programs, and all four submitted sufficient evidence to confirm their reports (100%). Furthermore, all four also stated that they had one of the following training offerings: full-time, classroom-based


training; part-time, classroom-based training; workplace-based training; distance learning; and E-learning (Table 3).

Table 3: Survey Response Validation Findings: Availability of Training/Learning Materials and Methods of Training Delivery (n=4)

Number of positive Number Confirmation responses confirmed (%)

Availability of learning materials used in 4 4 100%training programs

Training options offered (e.g., full-time, 4 2 50%classroom-based training; part-time classroom-based training; workplace-based training; distance learning; E-learning)

Curriculum for training programs 4 3 75%

Number of training programs provided

In order to confirm the number of training programs offered, the selected providers were asked for documentation to prove the existence of each they had reported on the survey. Such reports could be confirmed by only one of the three (33%) organizations that responded positively to this survey question (Table 4).

Table 4: Survey Response Validation Findings: Number of Training Programs Provided (n=4)

Number indicated Number confirmed (Y/N)

Organization 1 10 N

Organization 2 No response N/A

Organization 3 4 N

Organization 4 29 Y

Training facilities and additional infrastructure

Site visits to these program providers were done to validate responses related to training facilities and infrastructure (Table 5). Only one stated in the survey that it possessed halls with a 50+ capacity and classrooms for <50 persons, both of which were confirmed (100%). Similarly, two providers claimed to have access to projectors, and site visits confirmed the existence at both (100%). None of the four claimed to have speakers, so no confirmation was sought.

Table 5: Survey Response Validation Findings: Training Facilities and Additional Infrastructure (n=4)

Number reported Number confirmed Confirmation rate (%)

Halls (capacity for 50+) 1 1 100%

Classrooms (<50) 1 1 100%

Projector(s) 2 2 100%

Speaker(s) 0 N/A N/A

Validation conclusions

Most survey responses were confirmed as accurate either through site visits (pertaining to infrastructure) or documentation review (training materials and programs). We interpret this to mean that most responses were true to the best of respondents’ knowledge. Also, survey responders were advised before completing the survey that they could be asked to verify their responses, likely discouraging them from providing misleading information.


Ethical Considerations

The FMOH TWG members and Human Resources Development reviewed and approved the instruments and guidelines. The FMOH sent each IST provider a letter explaining the study’s purpose and requesting cooperation. The study objectives and purpose appeared on the questionnaire’s cover page to prompt a high response rate. Participants’ identities were not revealed to the study team, and data were presented to the study team in a way that concealed participants’ identities.

Informed (verbal) consent was obtained from each interviewed stakeholder after a brief explanation of the study’s objectives and contents. Audio-recorded data were deleted after transcription and coding.

F. Analysis

Quantitative analysis

Data were descriptively analyzed in SPSS. Categorical data were cross-tabulated and analyzed for significant differences between groups using chi-square tests.

Qualitative analysis

Audio-taped interviews were transcribed and then checked against the audiotape for accuracy. Then, a study team member read and re-read the transcripts to develop familiarity with the data.

The assessment’s analysis used a thematic approach, as proposed by Braun and Clarke [7], for its comparative advantages of flexibility, relative ease and speed of application, facility in organizing key data features, and capacity to support the development of insights that might not be developed with other approaches. The transcribed data were thoroughly examined so that key themes could be identified and categorized.

Next, as preliminary coding, a segment of data that fit into a pre-set specific aspect of each research question/scenario was identified and gathered in a systematic way, covering the entire dataset. Third, the extracts of data from the second step were categorized and sorted into themes and sub-themes that could contribute to the study’s objectives. Fourth, the themes were reviewed by checking their relationships with the original data to validate each. In this step, some themes were merged, others were divided into more than one theme, and those lacking sufficient supporting data were removed from consideration. Finally, each theme was described, named, and illustrated with examples. The theme names and examples are presented in the tables below that present the key qualitative findings.

III. RESULTS

A. Types of Education and Training Provided

The assessment explored the types of education and training the surveyed providers offered. Most program providers (34/36: 94%) indicated that they offered IST, and a substantial proportion (22/36: 65%) that they offered training of trainers. Fewer offered pre-service and post-graduate training (Figure 1).

B. IST Program Provider Staffing

Responses from IST program providers on staffing levels were low, with only half of the IST program providers submitting this information. Of the IST program providers submitting information, the total number of staffs ranged from five to 750 full-time equivalent (FTE) staff members, with 16 organizations (47%) having 50 or fewer FTEs. The questionnaire asked each provider to indicate the number of FTE employed with his/her organization who had competencies in designing curricula and providing and evaluating training. Further analysis showed that the proportion of staff in each IST provider with curriculum design competencies was low, ranging from 50–56% of total FTE staff.

Fifteen IST providers (44%) reported that they had one or more FTE staff with curriculum development competencies (ranging up to 105 FTEs), and 16 (47%) had fewer than three FTE staff with curriculum development competencies. 1 organizations (53%) indicated that they had two or more FTEs with training competencies (ranging up to 105 FTEs). Sixteen (47%) had 12 or fewer FTEs with competencies in providing training. Eighteen (53%) indicated that they had two or more FTEs with training evaluation competencies (ranging up to 105 FTEs), and 17 (50%) had seven or fewer FTEs. Some key informants echoed inadequate competencies among IST trainers in the


qualitative interviews. One respondent said “Trainers for different trainings are not adequately knowledgeable.”

Figure 1: Types of Training offered by IST Providers (% of IST providers), 2011 (n=34)

Table 6: IST Provider Staffing Levels with Training Evaluation Competencies

Mean Range

Full time equivalent staff (26 IST providers responded) 85 5 – 750

Full time equivalent staff with (17 IST providers responded)

competencies in

designing curricula 9 0 -105

Full time equivalent staff with IST providers responded)

competencies in providing training (23 17 0 – 105

Full time equivalent staff with (22 IST providers responded)

competencies in

evaluating training 14 2 – 105

C. IST Resources and Infrastructure

Among the surveyed IST program providers, they mentioned they had computers and internet services for their training staff (82% and 80%, respectively), while only 53% and 41%, respectively, and made these available to trainees. Over half of program providers (56%) mentioned that journals were available to training staff, while only one-third (32%) said they were available for trainees (Figure 2). Only 21% of the surveyed IST providers reported that they had halls and classrooms for training (Table 7). No significant differences were found between types of training providers and accessibility of these resources for staff and students and availability of halls and classrooms for training.

Figure 2: Resources Available to IST ProviderTeaching Staff and Trainees (n=34)

0

10

20

30

40

50

60

70

80

90

Computers Internet References Journals

Teaching staff %

Students %


Table 7: Percentage of Surveyed IST Program Providers with Infrastructure for Training (n=34)

Infrastructure % (n)

Halls (50+ participants) 21 (7)

Classrooms (<50 participants) 21 (7)

Projector 36 (11)

Audio speakers 18 (6)

D. Training Content

Respondent’s IST programs were asked what content areas were covered in their training programs and were offered the choice of the topics described in Table 8. Respondents could also indicate additional topics not covered in this listing and could select all the relevant topics covered in the training.

The most common training topics are illustrated in the word cloud in Figure 3.

Figure 3: Word Cloud of main training topics

The survey found that the most frequently covered topic was HIV and AIDS (28% ), followed by communication skills (22%) and family planning (17%). No program reported offering training on human resources management injuries and violence, or social work and care (Table 8).

Table 8: Percentage of Surveyed IST Programs Offering Various Content Areas (n=72)

Topic % (n) Topic % (n)

HIV and AIDS 28 (20) Emergency medicine 7 (5)

Communication skills 22 (16) Environmental health 7 (5)

Family planning 17 (12) Pharmaceuticals (rational use of medicines) 7 (5)

Monitoring and evaluation 15 (11) Research methods 7 (5)

Maternal health 13 (9) Water and sanitation 7 (5)

Nutrition 13 (9) Computers and technology 6 (4)

Infectious diseases (including TB) 13 (9) Teaching skills 6 (4)

Quality improvement 13 (9) Malaria 6 (4)


Topic % (n) Topic % (n)

Pharmaceutical management 11 (8) Immunization 4 (3)

Health leadership and management 11 (8) Sexually transmitted infections 4 (3)

Reproductive health 11 (8) Health economics and finance 3 (2)

Disease surveillance 10 (7) Life-long learning skills 3 (2)

Health information systems 10 (7) Childhood illnesses 3 (2)

Laboratory 10 (7) Mental health 1 (1)

Chronic disease 8 (6) Radiology 1 (1)

Logistics and supply chain management 8 (6) Human resources and management 0 (0)

Health policy and planning 8 (6) Injuries and violence 0 (0)

Health promotion and communication 8 (6) Social work and care 0 (0)

E. Trainees

IST providers were asked to indicate the number of trainees that completed training programs during the previous two calendar years (calendar years 2010 and 2011). For 59 (82%) of the 72 programs, respondents could provide the total number of trainees trained. Table 9 shows that 36 (50%) training programs trained nurses, 33 (46%) trained health officers, and 30 (42%) trained physicians. Fewer training programs trained personnel in health management information system (HMIS) (two, 3%), laboratory personnel (eight, 11%), health extension workers (15, 21%), and midwives (16, 22%).

Table 9: Training Programs by Cadre Trained (n=59)

Cadre % (n) Cadre % (n)

Nurses 50 (36) Laboratory personnel 11 (8)

Health officers 46 (33) Others 4 (3)

Physicians 42 (30) District and woreda health personnel 3 (2)

Facility managers 22 (16) HMIS personnel 3 (2)

Midwives 22 (16) Project/program managers 3 (2)

Pharmacy personnel 22 (16) Teachers 3 (2)

Health extension workers 21 (15) Health assistants 1 (1)

Environmental health personnel 11 (8) Trainers/supervisors/preceptors 1 (1)

The survey asked providers whether they conducted a training needs assessment before trainings. Only 29% always did so (Table 12). In the qualitative interview, a key informant, representing a development partner, indicated the need for training needs assessments, saying, “Training should be need based and in response to an identified, felt need.” - [Development partner representative/001]

A representative from a professional association further noted, “Some of the trainings are even provided without any demonstrated need for the country.” - [Professional association representative/002]

F. Length, Frequency, Modalities, and Locations of IST

The length of trainings was reported for 53 programs and ranged from three hours to three years, with a median length of 40 hours. Training programs were most frequently reported to be around 40 hours, followed by 24 hours. Eight (15%) training programs were reported to lasted 80 hours (Figure 4).


Figure 4: Length of Surveyed Training Programs (n=53)

Regarding the frequency of IST provision offered in the previous calendar year (January-December 2011), 20% were offered twice a year and 17% once a year. Some providers IST training programs hundreds of times (Table 10).

Table 10: Frequency of Trainings Jan 2011-Dec 2011 (n=54)

Number of times training program was provided

% (n) of providers offering programs at each frequency

0 4 (2)

1 17 (9)

2 20 (11)

3 6 (3)

4 11 (6)

5 9 (5)

6 2 (1)

7 2 (1)

8 2 (1)

9 4 (2)

10 13 (7)

11 2 (1)

48 2 (1)

108 2 (1)

130 2 (1)

200 2 (1)

332 2 (1)

With regard to training modalities, of the reported 72 training programs, four included a distance education element for approximately 10% of the total hours of training. Six programs had a self-study component, ranging from 5%–40% of the total hours of training.

Almost 40% of IST programs were conducted entirely outside the work place, only two programs (3%) were completely taught at the work place. However, most training programs were found to use a mix of training modalities. Figure 5 indicates the breakdown of modalities (as a percentage of the total program length) used in the average training program. The average training program had 38% of its curriculum delivered through workplace-based training. Table 11 also offers data on onsite training.


Figure 5: Average Percent of Training Programs Delivered Through Each Modality

With regard to training locations by Ethiopian region, training programs were most frequently reported to have been conducted in Southern Nations, Nationalities and Peoples Region (SNNPR: 47%), Addis Ababa (44%), Amhara (43%), and Oromia (43%), with only 10% of the trainings provided in Afar and Somali. Not all providers indicated whether a program was conducted in a rural or urban area, but clearly urban offerings predominated (Table 11). Training programs were mostly provided at hotels, although providers apparently used a mix of training sites.

Table 11: Training Sites, by Region: Urban vs. Rural and Type of Site (% [n])

Region

IST programs provided in region

Area Training site (more than one response permitted*)

Urban Rural Trainee's workplace

Local public training institution

Local private for-profit training institution Hotel Other

Addis Ababa 44 (32) 9 (30) NA 60 (19) 22 (7) 3 (1)

91 (29) 6 (2)

Afar 10 (7) 43 (3) 14 (1) 14 (1) 29 (2) -- 29 (2) --

Amhara 43 (31) 55 (17) 35 (11) 26 (8) 16 (5) --

58 (18) 19 (6)

Benishangul-Gumuz 15 (11) 45 (5) 9 (1) 27 (3) 27 (3) --

54 (6) --

Dire Dawa 18 (13) 77 (10) NA 23 (3) 15 (2) --

54 (7) 7 (1)

Gambela 15 (11) 55 (6) 9 (1) 27 (3) 27 (3) -- 45 (5) --

Harari 13 (10) 50 (5) 10 (1) 20 (2) 20 (2) -- 30 (3) --

Oromia 43 (31) 65 (20) 19 (6) 19 (6) 13 (4) 3 (1)

71 (22) 16 (5)

Somali 10 (7) NA 14 (1) 14 (1) 14 (1) -- 14 (1) --

SNNP 47 (34) 62 (21) 29 (10) 26 (9) 15 (5) --

59 (20) 15 (5)

Tigray 29 (21) 62 (13) 14 (3) 14 (3) 10 (2) --

62 (13) 10 (2)


Notes: Not all providers indicated whether a program was conducted in a rural or urban area. “NA” means data were not available; “--“means no such program was found to be offered there.

*Training programs may be conducted in more than one type of site – for example both in the workplace and in a hotel.

G. IST Program Provider Practices, Linkages, and Interaction

Table 12 presents data on the extent to which IST providersproviders engaged in 16 best in-service training practices, such as providing technical assistance to build the capacity of other IST providers. Many respondents mentioned that their training programs had undertaken best practices most of the time or always. For a number of the best practices providers mentioned that tese were conducted rarely or only some of the time. The study revealed no significant difference between organization type providing IST and the existence of learning objectives, partnership with organizations, sponsorship of trainees, and the application of pre-tests or post-tests.

The findings from the IST program provider survey are presented alongside the findings from the key informant interviews. Scenarios discussed by key informants are numbered by the order in which they appeared in the key informant interview guide (Annex 2).

Table 12: IST Program Provider Practices (n=34)

Practice

Providers engaging in practice % (n)

Rarely (<10%)

Some of the time (10–40%)

Half the time (40–60%)

Most of the time (60–90%)

Always (>90%)

1 Our organization provides technical assistance to build the capacity of other IST program providers. 12 (4) 44 (15) 9 (3) 18 (6) 15 (5)

2 The training programs that our organization provides are formally recognized. 12 (4) 9 (3) 21 (7) 59 (20)

3 Our organization submits information about our training programs and trainees completing training to a training-tracking mechanism. 18 (6) 9 (3) 3 (1) 18 (6) 47 (16)

4 Our organization has processes in place to make sure the training provided is in compliance with national policies, strategies, laws and regulations regarding the scope of practice for health workers. 3 (1) 0 6 (2) 18 (6) 71 (24)

5 Our organization shares training materials with curriculum developers and trainers outside of our organization for the training programs we provide. 21 (7) 21 (7) 3 (1) 21 (7) 27 (9)

6 When providing training, our organization uses and strengthens local infrastructure and resources, such as local (national, regional, zonal, woreda) trainers and training institutions. 3 (1) 12 (4) 3 (1) 29 (10) 50 (17)

7 Our organization invests in continuing professional development of our staff to improve their training, management and relevant technical competencies. 12 (4) 15 (5) 9 (3) 21 (7) 35 (12)

8 Our organization coordinates the training programs we provide with other training program providers that offer similar training

6 (2) 21 (7) 6 (2) 21 (7) 35 (12)


Practice

Providers engaging in practice % (n)

Rarely (<10%)



Most of the time (60–90%)

Always (>90%)

programs.

9 The purpose of the training, learning objectives, learning methods and expectations of the training program are described to trainees. 0 0 0 18 (6) 79 (27)

10 Training programs are evaluated with the input of trainees, and feedback is used to continuously improve training programs. 9 (3) 3 (1) 6 (2) 18 (6) 56 (19)

11 Training programs are scheduled with the supervisors of trainees so that health services are not disrupted over the training period. 3 (1) 6 (2) 27 (9) 53 (18)

12 Before training programs are conducted, all possible barriers to performance are identified and assessed to determine training needs. 6 (2) 15 (5) 12 (4) 32 (11) 29 (10)

13 This organization collaborates with supervisors to apply trainee selection strategies, including criteria to identify those that need training before each training program is provided. 3 (1) 6 (2) 9 (3) 35 (12) 35 (12)

14 Training programs are designed, planned and delivered in a way that prioritizes time efficiency to prevent or minimize trainee absenteeism from work. 3 (1) 6 (2) 15 (5) 35 (12) 38 (13)

15 Trainees are routinely followed up with after training to assess whether performance has improved and to identify further specific training needs. 12 (4) 15 (5) 15 (5) 27 (9) 27 (9)

16 Pre-determined eligibility criteria are used to select trainees. 6 (2) 3 (1) 9 (3) 29 (10) 50 (17)

Alignment of IST to needs

Most IST providers (79%) had used pre-determined eligibility criteria to select trainees most of the time/always (Table 12, row16), but stakeholders indicated concerns in this regard: A development partner representative said, “I encountered one young medical doctor trainee a few months ago who came for a third time for similar training. Similarly, an individual may take similar trainings several times due to the lack of designed monitoring strategy.” - [Development partner representative/001] A representative of academia said, “In one of the in-service trainings I was involved in, I found a person coming three times to take similar training, and I said to him that ‘The first was your bachelor’s, the second was your master’s, and the now your PhD for the same course.’” - [Academia representative/011]

The key informant interviews complement the survey findings with regard to issue of the alignment of IST with national and/or regional training plans and assessed training (Scenario 1). While some reservations toward this scenario were expressed by some key informants, an academic


representative stated: “Yes, it has to be aligned to a health sector development plan to capacitate human resource for the achievement of the plan.” - [Academia representative/010]

Someone representing the donor community said, “It is possible to develop trainings in response to the training needs of specific regions or woreda or whatever health care priorities, so, I agree with the scenario and it should be in place.” - [Donor representative/016]

Yet another, an academic, said, “I wouldn’t say all trainings [must be aligned with a master plan], because there are courses provided by interested individuals that may not need alignment.” - [Academia representative/011]

Someone from a training and research institution addressed the feasibility of alignment, saying, “Some regions have already started to identify their needs and share [them with] their partners.” - [Training and research institution representative/007]

Another donor representative simply noted that alignment “is feasible.” - [Donor representative/014]

A representative of a civil society organization expressed doubt as to the feasibility of Scenario 1: “I have doubts of its feasibility because [alignment] needs basic structural establishment and intensive need assessment.” - [Civil society representative/003]

Scenario 1

Scenario 1: All training programs are in alignment with national and/or regional training plans and are designed to address assessed training needs.

Response to the scenario

Key themes

Need based Not all training has to be aligned

Training has to be aligned

All IST has to be need based.

Not all IST needs to be aligned as exceptions may arise.

IST can support achievement of health sector plans if they are aligned.

Illustrations

“This is great ...training should be need based and in response to an identified felt need.” (001)

“It is possible to develop trainings in response to training needs of specific regions or woreda or whatever health care priorities. So, I agree with the scenario and it should be in place.” (016)

“I wouldn’t say all trainings, because there are courses provided by interested individuals that may not need alignments.” (011)

“Sometimes trainings might be an emergency solution and conducted out of the plan. Hence, it is difficult to have all trainings be planned.” (005)

“Yes it has to be aligned to health sector development plan to capacitate human resource for the achievement of the plan.” (010)

“I think so. Yes, every training should be in line with national and regional plans.” (015)

Feasibility

Key themes

Feasible Doubtful

It is feasible. Have concern/difficult to say so

Illustrations

“Some regions have already started to identify their needs and share them with their partners”(007)

“It is feasible.” (014)

“It can be feasible. It only needs evaluating needs at the ground and identify weakness and strength of the trainings conducted so far. And design the current trainings in response to the need and weakness of the previous trainings”.

“I have doubt on its feasibility because it needs basic structural establishment and [an] intensive needs assessment.” (003)


(017)

Strategy/ mechanism for realizing the scenario

Key themes

Policy Collaboration and coordination

Responsible office

Policy/ strategy to guide IST

Collaborate and coordinate with stakeholders

Designate responsible office for IST

Illustrations

“There has to be strategy or policy to guide how to operate the in-service training.”(004)

“It takes commitment of different stakeholders, both donors and Ministry of Health. It needs also collaboration and coordination among different stakeholders.” (002)

“At the level of the Ministry . . . my suggestion is there should be an office or unit whatever responsible for this purpose.” (011)

Best practice

UNICEF conducted a needs assessment before it funded a project to build basic competencies in emergency obstetric and newborn care (EmONC)

FMOH has established a TWG of stakeholders to plan and follow-up trainings, including IST.

Some USG partners usually conduct an annual needs-based assessment and communicate priority needs to the respective RHBs and/or FMOH for a decision.

FMOH provides training for health extension workers after identifying skill and knowledge gaps; it has designed an integrated refresher training to respond to the gaps and prevailing conditions. This training is held at the workers’ health centers to avoid long interruptions of routine activities.

Coordinating IST

Eighty percent (80%) of the survey respondents said that their trainings were scheduled so as to not disrupt health services in most cases or always (Table 12, row 11), but some key informants aired concern the interruption of health care delivery services with IST schedules. A development partner representative stated that “all the trainings are off-the-job, compromising duties of the [health care] organization.” - [Development partner representative/001] Twenty-seven percent of the program provider respondents noted that their organization had coordinated training programs they provide with other organizations that offer similar training programs rarely or sometimes (Table 12, line 8). Stakeholders emphasized the importance of such coordination (Scenario 8); a representative of a civil society organization said, “[Training] has to be coordinated under the leadership of the Ministry of Health.” - [Civil society organization representative]

Scenario 8

Scenario 8: In-service training program providers participate in a national and regional in-service training coordination mechanism that includes all IST program providers and key stakeholders to improve planning and coordination of training.

Response to scenario

Key theme Agreement

It is good and innovative idea.

Illustrations

“Is a very good and innovative idea and what the Ministry of Health is saying.” (003)

“Yes, it has to be coordinated under the leadership of the Ministry of Health.” (010)

“Yes, definitely. They should participate.” (009)

“This is also good and important; it creates sharing of duties among stakeholders.” (008)


Scenario 8: In-service training program providers participate in a national and regional in-service training coordination mechanism that includes all IST program providers and key stakeholders to improve planning and coordination of training.

Feasibility

Key themes

Feasible Doubtful

It is feasible scenario. Feasibility is questionable and problematic.

Illustrations

“The scenario is feasible since there are initiatives started at the Ministry of Health and different partners. So, what is needed is coordination and integration at the regional and national levels.” (001)

“It is feasible and needs collaborative work towards similar goal among the stakeholders.” (008)

“The feasibility is questionable because bringing people together is not an easy task.” (009)

“Feasibility is problematic.” (020)

Strategy/ Mechanism for realizing the scenario

Key themes

Form a forum FMOH leadership

Formation of a stakeholders forum FMOH has to take the lead

Illustrations

“It can be achieved through the formation of a nationally representative forum composed of the key partners. And the scenario needs establishment of a taskforce or forum composed of different stakeholders who should be mandated to assess need, plan accordingly and share priority needs with all the stakeholders.” (005)

“This should be in place through involving stakeholders in all aspects of the program, that is, during planning, developing standards, implementation and evaluation.” (008)

“The Ministry has to take the lead to coordinate joint planning, assessing needs and communicating to partners for implementation through the establishment of coordinating unit.” (004)

“As for me, this should be under the human resource development and management directorates [which] at all levels should take the responsibility.” (017)

Best practices

The practice of duplication currently seen has been tackled at the specific partner level. For instance, the US Centers for Disease Control and Prevention (CDC) has started to regionalize HIV- and AIDS-related intervention tasks to its implementing agencies (I-TEC) in Amhara and Tigray, International Center for AIDS Care and Treatment Programmes (ICAP) of Columbia University Mailman School of Public Health in Oromiya, Johns Hopkins University in Addis Ababa, SNNPR, etc.).

The FMOH has established a working group composed of different stakeholders and veterans that is chaired by the Human Resources Directorate for in-service training-related issues.

A unit to coordinate joint IST planning among partners is established in some higher learning institutions among the key partners to share the costs required for the implementation, e.g., at Gondar University, I-TEC.

Different HIV and AIDS intervention program stakeholders usually meet to exchange information on the success and weakness of program implementation (including training).

IST tracking and information systems

Table 12 shows that 65% of respondents stated that they submit information about their training programs and trainees completing training to a training-tracking mechanism all or most of the time (row 3). Interviewed stakeholders disagreed, suggesting that the tracking issue is something that they hope will be strengthened (Scenario 9). A development partner representative stated: “It is exactly


what we are looking forward [to]. It is difficult to plan, design, and implement training without a timely database that tracks the trainers and trainees.” - [Development partner representative/001] Others were concerned about putting a tracking mechanism in place: Someone representing the government said, “[It is] difficult to keep [an] individual [level] database at [the] central level.” - [Government organization representative/012]

Scenario 9

Scenario 9: All in-service training programs and trainees are tracked through a national and regional mechanism (e.g.,, database).


Key themes

Very important Difficult

It is very important and has to be in place. Difficult at a national level

Illustrations

“This is exactly what we are looking forward [to], and I fully agree with scenario. It is difficult to plan, design, and implement training without an up-to-date database that tracks the trainers and trainees.” (001)

“This is [a] true scenario because many health workers have been taking similar trainings repeatedly, and the right people for the training have not been trained so far. If the database is in place, we can have records of who came for what training and from where.” (017)

“Difficult to keep individual database at central level. Rather it can be kept at regions, and the center should ensure existence at regions.” (012)

Feasibility

Key themes

Feasible Difficult

It is feasible. Tracking is difficult.

Illustrations

“It is not [a] difficult task and is feasible.” (002)

“In the era of computers and the Internet, this has to be very feasible.” (019)

“It is feasible because there is already an established HMIS through which we can adapt and capture the training database.” (009)

“Tracking could be very difficult, but to have the data is important.” (011)

Strategy/ mechanism for realizing scenario

Key theme Strengthen HMIS Resources

Strengthening the current HMIS Avail technical, financial and information technology resources.

Illustrations “Since there is an existing HMIS, it only needs strengthening through providing training for personnel working with it.” (010)

“The database should not be limited to [the] national [level] but should be at the level of every facility where the facilities should have their own training database.” (004)

“It needs [the] design of national database that can be accessed by providers and implementing agencies.” (006)

“This requires furnishing with different electronic materials, like computer, Internet service, and other accessory materials.” (005)

“It requires personnel, financial and technical inputs.” (008)

Best

The FMOH established the Human Resource Information System (HRIS) in collaboration with Tulane University. The software is powerful for monitoring and forecasting training


Scenario 9: All in-service training programs and trainees are tracked through a national and regional mechanism (e.g.,, database).

practices needs and is used in some regions.

The HMIS that might be adapted to capture IST information.

PEFPAR (President’s Emergency Plan for AIDS Relief) partners formerly tracked some IST programs using computer software called “Training Information Management System.”

USG partners such as the University of Washington International Training and Education Center for Health (I-TECH) designed software for monitoring and forecasting training needs.

Development of IST programs

With regard to methods of developing IST material, adaptation was the most frequently cited method: 85% of surveyed IST program providers reported that they had adapted their training/learning materials from existing resources, while 76% adapted their curriculum from existing curricula (Figure 6). Additionally, key informants expressed a concern for standardizing learning materials and curricula. An academic noted, “They have to be standardized in order to achieve the intended goal.” - [Academia representative/010]

Figure 6: IST Program Provider Practices for Developing Learning Materials and Curricula

Evaluating IST

IST program providers were asked about their practices relating to evaluating training. Seventy-four percent (74%) of the respondents said that their IST programs were evaluated most of the time or always (Table 12, line 10). Of those that evaluated their training the great majority were through knowledge pre-tests and post tests, followed by obtaining feedback from trainees through post-training feedback forms (Figure 7). Evaluating performance and learning outcomes and obtaining feedback from the trainee’s supervisor were less common.

Figure 7: Training evaluation methods used (%)


Stakeholders commented on the practice of IST evaluation (Scenario 10). A representative of a regional health bureau pointed out that “This [evaluation] is inescapable and mandatory,” indicating the importance the representative assigned to evaluation. - [Regional health bureau representative]

Scenario 10

Scenario 10: All in-service training program providers evaluate the effectiveness of their training programs.


Key theme Agreement

It should be in place.

Illustrations

“It [this scenario] is a must and should be implemented.” (005)

“This is inescapable and mandatory.” (020)

“There is no question about this, and it should be in place. It is a great scenario. It takes … coordination among the providers and employers sending trainees.” (001)

“The impacts of all the above scenarios should be evaluated [after implementation], hence the scenario is crucial in realization.” (008)

Feasibility

Key themes

Feasible Not feasible

It is feasible. Not feasible for now

Illustrations


“It will be feasible if we follow the desired accreditation process for training providers in the above scenarios.” (007)

“If partners collaborate and do these activities, it would be feasible.” (014)

“It is not feasible at the moment. It is difficult to follow large number of trainees.” (004)

“The follow-up depends on the number of trainees, which might dictate the feasibility of follow-up.” (006)


Key themes

Integrate evaluation into the IST system

Collaboration Resources

Should be part and parcel of the IST system

It needs collaborative work.

Avail technical and financial resources.

Illustrations

“It should be part of the whole system of the in-service training program.” (016)

“Monitoring and evaluation should be placed in the document of the program that has to be reported along with the other reports of the program.” (003)

“If partners collaborate and do these activities, it would be feasible.” (014)

“This is in the hands of different government structure. The trainer and facility recruiting the trainee should communicate about the trainee in order to identify problems related to practice of the training received.” (020)

“The scenario needs expertise, financing and a guiding strategy.” (005)

“Needs planning and training of expertise on techniques to conduct anevaluation.” (001)

Best practices

I-TECH follows up trainees through mentoring and certifies them in demonstrated competency.

MEPI-AAU evaluates training on the Integrated Management of Childhood Illness (IMCI, a protocol developed by the World Health Organization and addressing health care for children under five) through discussion and observation of the performance of


IMCI-trained health workers in selected health facilities.

Ipas does assessments of IST with periodic follow-up of trained health workers.

Formal authorization and recognition of IST program providers and programs

Seventy-one percent of IST program providers reported that their training programs were formally authorized by the relevant authorities to provide training. Stakeholders uniformly suggested that formal authorization or accreditation by local/national authorities to provide training programs is important. One from the private sector said, “It [having authorization/accreditation] is good and one of the key intervention areas.” - [Private sector representative/006]

Table 13: Formal Recognition of IST Program Providers (n=34) and IST Programs (n=72)

Practice Yes

% (n)

No

% (n)

Unsure

% (n)

IST program provider formally authorized by relevant authority to provide training (n=34 IST providers)

71 (24) 12 (4) 12 (4)

Practice Always (>90%)

Most the time (60–90%)



Rarely (<10%)

IST training program and materials are formally recognized (n=72 IST programs)

69 (50) 10 (7) 4 (3) 0 (0) 12.5 (9)

But even if the providers are authorized, there was concern about the quality of the training. In response to Scenario 6, one respondent representing the government said, “I think they are authorized to train. . . . But the main thing is, are they really practicing what is expected or not is the key? [A] mechanism for accreditation of trainers has to be created and implemented to [determine] whether the trainers are practicing in the right way or not.” - [Government institution representative/012]

Scenario 6

Scenario 6: All in-service training program providers are formally authorized or accredited by local/ national authorities to provide training programs.


Key themes

Agreement

Agreement that it has to be in place

Illustrations

“It is good and one of the key intervention areas.” (006)

“It is appropriate and we have started to some extent.” (007)

“It is good and actually it should be put in place. It avoids resource duplication and even the quality of the training by itself.” (013)

“Is also good and true scenario because it has been claimed that trainers for different trainings are not adequately knowledgeable about the topic they are providing.” (017)

Feasibility

Key themes

Feasible Implementation challenges

It is a feasible scenario. Difficult to implement

Illustrations

“It is feasible scenario and some criteria have been developed by Higher Education Relevance and Quality Assurance at the Ministry of Education.” (001)

“It will not work at this time.” (002)

“Implementation is not an easy task.” (006)

“It is difficult to accredit all training


Scenario 6: All in-service training program providers are formally authorized or accredited by local/ national authorities to provide training programs.

“It is feasible as long as we built capacity of the authorizing bodies through diversifying their activities.” (009)

“It is very feasible.” (019)

providers at this time. The Ministry has also documented limitation of such issues to be applied to all training situations.” (003)


Key themes

Policy framework and standards

Monitoring and evaluation

Regional capacity

Accrediting body

Develop policy framework and standards for accreditation.

Develop strong monitoring and evaluation mechanism.

Build regional capacity.

Establish an accrediting body.

Illustrations

“Developing policy framework for accrediting organization.” (003)

“Legal framework and criteria for accreditation has to be in place and system of governance should be there.” (001)

“There has to be [a] strong monitoring and evaluation mechanism, for even authorized providers.” (004)

“Build the regional capacity because they [the regions] should be able to have capacity to see the contents, organizational or technical capacities before they [select] these training providers.” (018)

“Establishing, equipping, and financing and developing policy framework for accrediting organization.” (003)

Best practices

The Ethiopian National Accrediting Office could accredit IST providers by integrating the task into its current work.

FMOH is authorized to prepare guidelines with the support of universities and NGOs. Guideline development has begun (most of the documents have been revised) and should continue.

Interviewed stakeholders recommended that all IST curricula and materials meet minimum standards and be accredited by the relevant national authorities (Scenario 7). A development partner representative noted, “We believe so and [it is] important.” - [Development partner representative/004] A representative from academia said, “Yes, [accreditation] should be [implemented] for the key areas like IMCI [Integrated Management of Childhood Illness, a protocol for treating children under five years who are taken to a health facility], ART [anti-retroviral treatment], and TB.” - [Academia representative/011] Stakeholders had differing views on the feasibility of accreditation. An academic said it would be feasible: “It is feasible because developing curricula in the era of [the] Internet is an easy task.” - [Academia representative/010] Someone from the government differed: “It will not be feasible to develop a curriculum for each training issue; it would be tiresome.” - [Government representative/012]

Strengthening in-country IST program providers and systems

Only 33% of IST providers said that they were giving technical assistance to build the capacity of other IST program providers most of the time or always (Table 12, row 1). On the other hand, some stakeholders suggested the importance of such practice (Scenario 4); one donor representative said, “Support the Ministry and the lower levels, including the regional health bureau, to empower them to develop their own plans and strategies so that they can build their capacity to take over in-service training.” - [Donor representative/014]


Scenario 7

Scenario 7: All in-service training curricula and materials meet minimum standards and are accredited by national authorities.


Key themes

Agreement Not for all

Yes, agreements should be in place.

Accreditation should not be for all but for specific areas.

Illustra-tions

“We believe so and important.” (004)

“Yes, the scenario should be in place.” (007, 019) “I fully agree with [the] scenario, and they [curricula and materials] have to be standardized in order to achieve the goal intended to be attained through training.” (010)

“Yes it should be for the key areas like IMCI, ART and TB.” (011)

“If you say that every training … should be accredited, I am afraid this might drag small things. So, we have to define which trainings will [require] some [amount] of accreditation and licensing.” (018)

“My concern here is the regions do have mandates to have their own training curricula or other materials. So, it is difficult to have nationally accredited materials for all regions.” (015)

Feasibility

Key themes

Feasible Not feasible

It is feasible Is not feasible

Illustra-tions


“It is feasible because developing curricula in the era of the Internet is an easy task.” (010)

‘It will not be feasible to develop curriculum for each training issue, and it would be tiresome.” (002)

“Difficult to implement in the near future (five years).” (003)

“I [doubt] its feasibility.” (020)


Key themes Collaboration

Establish and strengthen appropriate body

Develop guidelines

Avoid conflict of interest

Develop curriculum

Collaboration of different stakeholders

Establishing and strengthening appropriate body to standardize and accredit curricula

Developing standardization guidelines

Avoid conflict of interest among the key actors

Develop curriculum for key areas

Illustra-tions

“The realization of the scenario requires collaboration of different stakeholders.” (003)

“Takes involvement of relevant stakeholders of the Ministry.”

“Establish-ment of capable institutions to review and accredit curricula.” (001)

“A regulatory body like a taskforce composed of NGOs and governmental

“Develop-ment of guidelines, strategic direction, [scope of work], or the guide in order to standardize training and develop minimum criteria which

“Care should be taken to avoid conflict of interest among the existing agencies providing a license that is FMHACA, and professional associations. Therefore, a separate independent

“This can be achieved by designing curriculums and get endorsed at a national workshop. The Ministry of Health should design in collaboration with its key stakeholders.


Scenario 7: All in-service training curricula and materials meet minimum standards and are accredited by national authorities.

(006) organizations should be established. And I think specific assignment should be given for the Directorate of Human Resource Development in the Ministry of Health.” (013)

include specific standard for each training.” (014)

body needs to be established for accrediting the course curricula.” (001)

” (010)

Best practices

The Higher Education Relevance and Quality Agency is accrediting private institutions for pre-service training.

The FMOH has started relevant initiatives, such as the development of guidelines for refresher trainings.

Standardized training materials for HIV and AIDS (prevention of mother-to-child transmission of HIV or PMTCT), ART, and voluntary counseling and testing) have been developed in collaboration with universities and NGOs.

Note: FMHACA is the Food, Medicine and Health Care Administration and Control Authority.

Scenario 4

Scenario 4: All international NGOs and development partners providing in-service training partner with local in-service training program providers to build their capacity so that they can take on the responsibility of providing that training in the future.


Key Themes Important Difficult

Important to build capacity of the sector

Different NGOs have different goals.

Illustra-tions

“The scenario is important for the country.” (019)

“I think this is great and what is expected. NGOs bring skilled expertise and innovative approach[es]. Hence, it is desirable scenario.” (001)

“It is[a] very difficult scenario because different NGOs have different goals. We don’t have real power to refuse trainings designed by different NGOs.” (002)

Feasibility

Key Themes

Feasible Feasible with some considerations

Difficult

Is feasible Coordination and collaboration needed

Not feasible

Illustra-tions

“It is [a] workable scenario.” (013)

“It is not a difficult task when there is strong coordination.” (015)

“Collaboration is

“Will not be simply feasible. Rather, a lot has to be done in building capacity of local providers.” (016)

“Some trainings are even provided without any demonstrated need for the


Scenario 4: All international NGOs and development partners providing in-service training partner with local in-service training program providers to build their capacity so that they can take on the responsibility of providing that training in the future.

mandatory to implement.” (019)

country. So, the scenario will not be feasible given the current situation.” (002)


Key Themes

Collaborations and coordination

Regulatory mechanism

Leadership Outsourcing

Collaborations, joint planning, ongoing consultations and coordination among the different stakeholders

There has to be a regulatory mechanism.

Donors and government take the leadership.

Building local capacity through outsourcing

Illustra-tions

“It needs collaboration and coordination among different stakeholders, including professional associations.” (003)

“If transfer of skill, knowledge and institutional memory is desired, there has to be collaborative work.” (012)

“It has to be under the Ministry and, if given to private {sector}, has to be well controlled.” (011)

“Donors should insist on the requirement of building local capacity and the local government should create enabling environment.” (001)

“It takes outsourcing of some part of the work by different partners to local entities (consultants). Among the works to be outsourced might be conduct of training and post-training follow up.” (006)

Best practices

The Higher Education Relevance and Quality Agency isaccrediting private institutions for pre-service training.

FMOH has launched relevant initiatives, such as the development of guidelines for refresher trainings.

Standardized training materials for HIV and AIDS (voluntary counseling and testing, anti-retroviral therapy, and prevention of mother-to-child transmission of HIV) have been developed in collaboration with US universities and other NGOs.

Most (79%) surveyed IST providers responded that their organization used and strengthened local infrastructure and resources mostly or always (Table 12, row 6). Stakeholders endorsed this practice (Scenario 5). A representative of a government organization said, “This ensures sustainability through building local capacity for [the] provision of training.” - [Government organization representative/017]

Scenario 5

Scenario 5: In-service training program providers always use and strengthen local infrastructure and trainers (e.g.,, local training institutions, centers and sites).


Key themes

Agreement Not always

This is a good scenario. This scenario will not work for all. trainings.

Illustra-tions

“It is good and should be put in place.” (005)

“This is also true scenario since it ensures sustainability through

“We agree for most not for all since some new emerging training needs might require expertise and facilities beyond the capacity of local (institutions).“ (004)


Scenario 5: In-service training program providers always use and strengthen local infrastructure and trainers (e.g.,, local training institutions, centers and sites).

building local capacity for provision of training.” (017)

“Not necessarily always. It should be situation based.” (009)

Feasibility

Key themes

Feasible Not practical

It is a feasible scenario. The scenario may not be practical.

Illustra-tions

“It is feasible and we have been working with 20 health science colleges in Ethiopia in upgrading health extension workers.” (013)

“It is feasible but takes time, since establishing and equipping local capacity needs resources and time.” (017)

“Since we could not have the expertise (the way they teach, their experience) and facilities sometimes, the scenario will not work currently.” (009)

“Feasibility is also questionable because most of the trainings venues are a hotel, which is due to the lack of suitable training institutions outside of hotels.” (003)


Key themes

Establish training centers Coordination and collaboration

Accreditation

Establishing the centers Central coordinating body and collaboration

Accrediting providers and trainees

Illustra-tions

“Training centers have to be established that meet the needs of the adult learning environment.” (001)

“The government needs to build big regional or national trainingcenters - where everybody aspires to reach to” (004)

“There has to be also [a] central coordinating body to regulate the implementation of the intended scenario.” (004)

“There has to be collaborative work among different organizations.” (005)

“To implement the scenario, the government needs to strengthen local providers through accrediting (IST) providers and trainees.” (007)

Best practices

Almost all NGOs are using local infrastructure and trainers through training-of-trainers programs.

Some development partners, such as Jhpiego, are working with UNICEF in establishing training centers (at least one in every region). The center would have lodgings, classrooms, audiovisuals, and different models for clinical practices.

Abortion care training, which requires building the capacity of the training institutions and health facilities, has started.

The Ethiopian Management Institute serves as an inspiring training center.

Oromiya Health Bureau had experience in providing trainings at local facilities (health centers, hospitals, and the zonal health department) and minimized costs by doing so.

The Ethiopian Health and Nutrition Research Institute (EHNRI) is providing most of its trainings by itself, using its own expertise and facilities and is preparing to build its own training facility.

The Medical Education Partnership Initiative-Addis Ababa University (MEPI–AAU) in collaboration with Jhpiego has constructed a medical education unit with a clinical lab. The unit is now sustainable.

Mekele University and the Tigray Regional Health Bureau built the university’s IST capacity so that it has its own program coordination and management.


Partnerships with local and national organizations

Sixty-four (90%) of IST programs indicated that they partnered with other organizations in planning, delivering, and evaluating their training programs. Of them, approximately two-thirds ( partnered with national or sub-national authorities (64%) and the trainee’s supervisor/manager(60%). Only three training programs reported having partnered with professional associations (Figure 8).

Figure 8: Types of Partners IST Programs Cited Working With (n=64)

In addition, stakeholders expressed a belief in the necessity and possibilities of such collaborations (Scenario 3). An academic informant affirmed programs’ reports of such necessity by stating, “Collaboration is . . . mandatory.” - [Academia representative/019]

A donor representative said, “It is not a difficult task when there is strong coordination.” - [Donor representative/015]

Scenario 3

Scenario 3: In-service training program providers collaborate with pre-service training providers to ensure consistency in learning approaches between pre- and in-service education and training.


Key themes

Agreement Agreed with some exception

Agree that this scenario would be good to put in place

Recommend that it be put in place with some amendment

Illustrations

“The scenario is very good because in-service training should fill the gaps of the pre-service training and some other emerging issues.” (002)

“It is good and needs to be done.” (008)

“Generally, I agree with collaboration and consistency but not with the content. In-service is specific while the pre-service [is] general.” (001)

Feasibility

Key themes

Feasible It depends Difficult

Illustra-tions

It is feasible Feasibility is conditional

Difficult to implement

“It is feasible.” (005, 008)

“Feasible through time.” (007)

“I think it is very feasible and most of the training materials need to be adapted.” (019)

“The feasibility depends on the two ministries (MOH and MOE) complementary works.” (004)

“It is difficult to integrate the two [pre- and in-service training] because they are [provided by] separate institutions.” (002)


Scenario 3: In-service training program providers collaborate with pre-service training providers to ensure consistency in learning approaches between pre- and in-service education and training.


Key themes

Collaboration Institutionalization of IST

Integrate IST and pre-service training

Collaborative work and coordination among pre-service and in-service training providers/stakeholders (FMOH, Ministry of Education, universities, etc.)

Higher education centers can be used to enable institutionalization

Pre-service curriculum should have all the areas of in-service training except for refresher courses given on site.

Illustra-tions

“Ministries of Health and Education, stakeholders for in-service and pre-service trainings, have to work together and feed information to each other to supplement their programs.” (004)

“If a collaborative environment would be established, it will be easy for implementation.” (014)

“Using universities as in-service training providers will lead to institutionalizing trainings from the current practice of hotel based trainings.” (007)

‘As a strategy, the pre-service curriculum should have all the areas of in-service training and long in-service training has to be abandoned and only refresher courses should be given from time to time and preferably at the work site.” (019)

Best practices

Many USG-funded organizations are supporting both pre- and in-service training programs to enable collaboration among them.

Curriculum development for the new medical education initiative was developed in collaboration with all stakeholders providing both in- and pre-service trainings.

Jhpiego has been conducting annual discussions with both pre- and in-service training providers.

Oromiya Regional Health Bureau has experience in midwifery training at pre-service training institutions.

EHNRI had a practice of introducing its IST to universities by funding them to undertake the training, thus aiming to integrate the same material in the pre-service training curriculum.

Some international NGOs are upgrading the knowledge and skills of health extension workers with Open University and the FMOH.

To reduce the need for IST, USG partners supported Gondar, Jimma, and Addis Ababa Universities to revise the pre-service curriculum so that HIV and AIDS are adequately addressed.

Mandatory continuing professional development

Twenty-seven percent of the survey respondents stated that their organization invests in the continuing professional development of their staff rarely or sometimes (Table 12, row 7). However, key informants suggested that continuing professional education is very important. One NGO representative said, “It is very important and mandatory” (Scenario 2) - [NGO representative/013]

Scenario 2

Scenario 2: Continuing professional development is mandatory for all health workers and is linked to periodic renewal of professional practice licenses.

Response to the scenario

Key themes Agreement Do not fully agree

It is important and good. Currently it will not work

“It is very important and “I may not agree fully.” (014)


Illustrations mandatory.” (013)

“It [the scenario statement] is what should be and [is] an ideal statement.” (003)

“Is really very good and it’s what I look forward to see in the future.” (006)

“It will not work in our country given the current situation, because we don’t have [a] set-up to implement [it].” (002)

Feasibility

Key themes Feasible Doubtful

It is feasible Feasible with some uncertainties

Illustrations

“If our professional associations and skilled groups are strong enough to do it, the scenario will be feasible.” (012)

“If higher education and professional associations engaged actively in the process [the scenario would] be feasible.” (014)

“[It] is not easy to implement.” (017)

“It is very difficult to implement because it needs [the]strong tracing of health workers in the country, which is nonexistent. It is also very difficult now because of the shortage of a means of acquiring continuing professional development and also administering it to the whole professional group.” (019)

”The challenge is who is going to cover the tuition fee? It is ridiculous to think that the health workers will cover the fees and take trainings because their salary is very low. I question the feasibility.” (020)


Key themes Policy/legal framework Decentralization Governing body

Put in place a policy framework.

Decentralize roles and responsibilities

Establish broad-based governing council.

Illustrations

“There has to be policy framework to guide the implementation.” (003)

“A strategy and guideline has to be in place to implement the scenario.” (004)

“As strategy the Ministry of Health and any other licensing body should develop rules and regulations towards this.” (019)

“Decentralization to regional level and the Ministry in collaboration with professional associations.” (011)

“Establish[ing a] professional council that can be broader [in its] representation of the public; there has to be a regulatory mechanism.” (001)

“Higher education and professional associations should engage actively in the process.” (014)

“It takes collaboration and commitment of professional associations and donors.” (002

Best practices

The FMOH-established TWG for developing IST implementation guidelines comprises key stakeholders with different perspectives.

In South Africa professional associations are mandated to provide trainings and accredit professionals who are their members.

Ghana requires health care providers to have some training credits in order to receive a professional license.


H. Stakeholders’ Recommendations on Priority IST Issues

Asked to share their opinions and views on what they consider to be the priority IST issues, key informants identified the following:

Designing and implementing an IST strategy (covering the various levels)

Standardizing IST

Accrediting and licensing IST

Setting IST monitoring and evaluation guidelines

Establishing IST databases (central and regional)

Setting IST policy direction

Institutionalizing IST

Ensuring the accountability and commitment of trainees (i.e., those who take IST)

Linking IST to pre-service trainings

Getting key partners to work together.

Furthermore, stakeholders recommended the following six focus areas for future IST research:

Impact of IST programs

Cost-effectiveness of IST programs

Approaches to curriculum standardization

Alternative approaches to off-the-job training,

Potential of higher institutions to integrate and manage IST programs

Best approaches to an IST database system

IV. DISCUSSION This study presents the findings from the analysis of data from 34 IST providers in Ethiopia who represent the main providers of training and from 20 key stakeholders, such as the FMOH and donors.

Our findings show that local and international NGOs were the main IST providers and that about 29 different types of IST programs were provided in 2011. Nearly half the IST program providers did not have competent FTE staff for training, and four in five providers lacked training halls/classrooms.

About a third of IST providers said that they conducted training needs assessments of all their IST programs. HIV and AIDS was the most frequently covered topic, while the most commonly trained health cadres were nurses, health officers, and doctors.

Off-site facilities were commonly used for IST, usually hotels. The most common program duration was 40 hours. Distance learning and self-study were uncommon training modalities. Adaptation was the most frequently cited method of developing learning materials and curricula.

Seventy-nine percent of IST providers said that they used pre-determined eligibility criteria to select trainees most of the time (29%) or always (50%), but stakeholders indicated doubt that these claims were accurate.

Only one in three IST providers consistently extended technical assistance to local capacity building “most of the time” or “always”. In addition, partnership with professional societies was very low in delivering IST. Furthermore, only one in five IST programs undertook post-training evaluations, and independent evaluations were uncommon.

Most IST scenarios received favorable reaction along with strategies for the way forward, with some concerns relative to the feasibility of some scenarios.

A. Relation to Other Evidence

Findings from this rapid assessment identify concerns similar to other reports such as Elzinga et al., [2] where many IST programs in Ethiopia had been found in a previous rapid assessment to be weak with poor planning, coordination, and quality. Another assessment found that the percentage of health care providers offered IST was low, and even when IST programs were in place, they were often not needs based [3]. However this study did not survey health workers, although we also found that needs assessments were not routinely done.


B. Limitations

Some members of the TWG took part both in the quantitative and qualitative studies. The involvement in the TWG in the design of the survey instruments, and qualitative interviews might have influenced their response in the survey. Many IST providers were identified and repeatedly asked by email, telephone, and text messaging to participate in the survey but did not. Had they responded, further insight might have been gained to show a more complete picture of IST in the country. Nevertheless, the study gained insight from the relatively high number of IST programs reported by those IST providers who did take part and represents the most comprehensive assessment of IST programs in Ethiopia to date.

V. REFERENCES 1. Kebede F et al. Towards a National In-service Training Strategy: Workshop Report. 2012. Addis

Ababa, Ethiopia and Bethesda, MD: Federal Democratic Republic of Ethiopia Ministry of Health and USAID Health Care Improvement Project.

2. Elzinga G.et al. Human Resources for Health: Implications for Scaling Up for Universal Access to HIV/AIDS Prevention, Treatment, and Care: Ethiopia Rapid Situational Analysis. 2010. Global Health Workforce Alliance Technical Working Group.

3. Feysia B et al. The Health Workforce in Ethiopia: Addressing the Remaining Challenges. 2012. Washington DC, World Bank.

4. Africa Health Workforce Observator. Human Resources for Health: Country Profile: Ethiopia. 2010.

5. Embassy of the United States, Ethiopia. US Government Launches Health System Human Resources Expansion Program. Bahir Dar, Ethiopia. 2012.

6. USAID Applying Science to Strengthen and Improve Systems Project (ASSIST). A Global Improvement Framework for Health Worker In-service Training: Guidance for Improved Effectiveness, Efficiency and Sustainability. Bethesda, MD: University Research Co., LLC. 2013.

7. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology 2006, 3: 77-101.


VI. ANNEXES

A. Annex I: In-service training program provider and training program questionnaire

Rapid assessment of Health Workers in‐service training in Ethiopia: In‐service training program provider and training program questionnaire

Instructions This assessment has three parts. The first part asks some basic information about your organization, the second part asks about the way in which your organization coordinates designs, plans, conducts, monitors and evaluates training, and the third part asks about each in‐service training program your organization provides.

Background for respondents: All health worker in‐service training program providers in Ethiopia including public, private for‐profit and non‐profit organizations, training institutions, NGOs and others are invited by the Federal Ministry of Health (FMOH) and the USAID HCI project to participate in this rapid assessment of in‐service training in Ethiopia. The main objectives of this rapid assessment are to:

1. Analyze the in‐service training situation in Ethiopia 2. Analyze in‐service training program provider practices in training design, delivery, follow‐

up/monitoring, evaluation, capacity building, standardization and institutionalization, tracking training and linkages with health service providers, community health workers, para social workers, program managers and policy makers

3. Identify strengths, weaknesses and best practices 4. Inform the development of a data‐base for health sector in‐service trainings in Ethiopia 5. Inform the development of a strategy for standardization and institutionalization of health sector in‐

service trainings It is hoped that this assessment will provide greater information about the type of health worker in‐service training provided in Ethiopia and generate additional evidence to inform the development of a national in‐service training strategy. This rapid assessment includes a questionnaire for in‐service training program providers about the type of training they provide and their practices as well as interviews with up to 20 representatives of different stakeholders (public, private, NGOs, CSOs, FBOs and Bilateral Agencies) on priority health worker in‐service training issues and strategies. All information gathered will be kept anonymous. Findings will be presented in a way that protects the anonymity of participants and their organizations. All information collected will only be presented publicly in a way that does not disclose the identity of the organization or respondent. It is optional to respond to this survey and your response indicates your consent to use the collected information for the purposes explained. We greatly appreciate your time and efforts taken to provide complete information in response to this questionnaire. We look forward to sharing the results of this assessment with you. Please complete your response by the 27 June 2012. For more information or if you have any questions, please contact: Dr. Fikreab Kebede; [email protected]; mobile: +251‐910 075 355 USAID Health Care Improvement Project Consultant


Part 1 Basic information (Please note that this section begins on page 3 of the online survey) This part gathers basic information about your organization and its structure.

1. Respondent contact details:

Respondent name

Respondent position

Respondent phone

Respondent email

2. Contact details of the in‐service training program provider:

Organization name

Address (physical location of main office)

Organization phone

Institution e‐mail addresses 1. 2.

3. Organization type:

Year organization was established/ opened office in Ethiopia

Ownership type: select one Government Private for profit – local Private for profit – foreign Private not for profit – Local NGO Private not for profit – International NGO Other (describe):

Organization type: select one Training institution (e.g. University, College) Health facility (e.g. Teaching Hospital) NGO (includes FBOs, CBOs) Multilateral agency (e.g. UN agencies) Company Donor or foundation Other (describe):

Is your organization formally authorized by the relevant authority to provide training?

Yes, please state the authority: No

Type of training offered: select all that apply

Pre‐service/undergraduate programs Post‐graduate programs In‐service training programs Training of trainers Other (describe):

4. Does your organization have more than one centre/campus where training is provided?

Yes No a. If YES, state the number of other training centres/campuses:

b. If YES, provide contact details below:

Training centre/campus name Physical address Phone Email

1

2


3

4

5

6 Form designed so additional centers can be added

5. Staffing (actual evidence of work)

a. Total number of staff employed: full time equivalent staff (FTE) b. Total number of staff with competencies in designing curriculum: FTEs c. Total number of staff with competencies in providing training: FTEs d. Total number of staff with competencies in evaluating training: FTEs

6. Training resources and materials

Educational resources available to teaching staff/trainers: select all that apply

Interviewer to tick if respondent able to show evidence

Computers Internet Access to up to date reference texts Access to recent scientific and professional journals

Educational resources available to students/trainees: select all that apply


Computers Internet Access to up to date reference texts Access to recent scientific and professional journals

Learning materials used in training programs are usually: select all that apply


Designed by our organization Adapted by our organization from existing materials developed externally (e.g. – FMOH, WHO, etc.) Using existing materials developed externally Other, describe:

7. Training programs provided

Delivery of training options offered: select all that apply


Full‐time classroom based training Part‐time classroom based training Work‐place based training Distance learning programs E‐learning

Curriculum for training programs are usually: select all that apply


Designed by our organization Adapted by our organization from existing curriculum

developed externally (e.g. FMOH, WHO, etc.) Using existing curriculum developed externally Other, describe:

Number of different types of training programs provided over the past 2 calendar years (January 2010 – December 2011). Do not count repeats of the same program.



8. Training facilities

Training facilities and infrastructure operated by your organization/institution: select all that apply


Halls (capacity for 50+) Classrooms (<50) Projector(s) Speaker(s)

Or

Not applicable, our institution does not have its own training facility

Part 2 In‐service training program provider practices (Please note that this section begins on page 19 of the online survey) This part gathers information about ways in which your organization coordinates, designs, plans, conducts and evaluates training. Select the most suitable response to each statement. A random sampling of survey respondents may be contacted by the rapid assessment consultants as part of the validation process to provide additional information regarding their responses. We thank you in advance for your support. Please feel free to contact us if you have any questions regarding this step. Select one response to each statement only.

Rarely (<10%)

Some of the time (10‐40%)

Half the time

(40‐60%)

Most of the time (60‐90%)

Always (>90%)

1. Our organization provides technical assistance to build the capacity of other in‐service training program providers.

a. If done, provide specific examples as evidence:

b. If rarely done, explain reasons why:

2. The training programs that our organization provides are all formally recognized



3. Our organization submits information about our training programs and trainees completing training to a training tracking mechanism



4. Our organization has processes in place to make sure that the training provided is in compliance with national policies, strategies, laws and regulations regarding the scope of practice for health workers a. If done, provide specific examples as evidence:


5. Our organization shares training materials with curriculum developers and trainers outside of our


Select one response to each statement only.Rarely (<10%)


Half the time

(40‐60%)


Always (>90%)

organization for the training programs we provide



6. When providing training, our organization uses and strengthens local infrastructure and resources such as local (national/regional/zone/woreda) trainers and training institutions.



7. Our organization invests in the continuing professional development of our trainers to improve their training, management and relevant technical competencies



8. Our organization coordinates the training programs we provide with other training program providers that offer similar training programs



9. The purpose of the training, learning objectives, learning methods and expectations of the training program are described to trainees



10. Training programs are evaluated with the input of trainees and feedback is used to continuously improve the training program a. If done, provide specific examples as evidence:


11. Training programs are scheduled with the supervisors of trainees so that health services are not disrupted over the training period a. If done, provide specific examples as evidence:


12. Before training programs are conducted, all possible barriers to performance are identified and assessed to determine training needs (e.g. competencies, availability of materials).



Select one response to each statement only.Rarely (<10%)


Half the time

(40‐60%)


Always (>90%)


13. Predetermined eligibility criteria are used to select trainees.



14. This organization collaborates with supervisors to apply trainee selection strategies including criteria to identify those that need training before each training program is provided



15. Training programs are designed, planned and delivered in a way that prioritizes time efficiency to prevent or minimize trainee absenteeism from work



16. Trainees are routinely followed up after training to assess whether performance has improved and to identify further specific training needs



Part 3 In‐service training programs (Please note that this section begins on page 36 of the online survey) This part gathers information about the type of training your organization currently provides. Please provide information for each training program currently offered by your organization (eg – one set of information per training program).

1. Name of training program

2. Does this training program have learning objectives? Yes No

a. If YES, list learning objectives of training program: 1. 2. 3. 4. 5. 6.


3. Which of the following does this training program perform: select all that apply Pre‐test of learner’s knowledge or skills Post‐test of learner’s knowledge or skills

4. Tick relevant content areas covered: select all that apply to this training program

Chronic diseases Communication skills Computers and technology Disease surveillance Emergency medicine Environmental health Family Planning Health economics and financing Health information systems Health leadership and management Health policy and planning including supervisory skills Health promotion and communication HIV and AIDS Human resources management Immunization Infectious diseases Injuries and violence

Laboratory Life‐long learning skills Logistics and supply chain management Maternal health Mental health Monitoring and evaluation Nutrition Pharmaceutical management Pharmaceuticals Quality improvement Radiology Reproductive health Research methods Social work and care Sexually transmitted infection management Teaching skills Water and sanitation Other (list):

5. What teaching and learning methods have been applied in this training program – describe

and give specific examples

6. Cadres trained: select all that are trained in this program Environmental health personnel Facility managers Health Extension Workers Health officers Laboratory personnel Midwives Nurses Pharmacy personnel Physicians/MDs Others (list):

7. Total number of trainees that completed this training program over the past calendar year

(January ‐ December 2011):

8. Does your organization partner with other organizations in planning, designing, delivering, monitoring/follow‐up or evaluating this training program?

Yes No a. If YES, which stakeholders does your organization partner with? Describe specific

examples of the role of partners.


Stakeholder Role of stakeholder in partnership – describe specific examples

Trainee’s supervisor/manager

Local public training institution

Local private for‐profit training institution

National or sub‐national authorities

Professional associations

Other (describe):

Other (describe):

9. Length of training program: fill in the total length of the training program

hours and days

10. Delivery of training program: fill in the approximate % breakdown of the training program % taught face to face out of the work place % taught face to face in the work place % taught by distance % self‐study

11. How many times was this training program provided over the past calendar year (January 2011 – December 2011): fill in frequency times

12. Where is this training program provided? Fill in the table below

Region – select regions where training is provided

Rural/urban – select where training is provided

Place of training – select all that apply

Addis Ababa Rural areas Urban areas

At the trainee’s workplace Local public training institution Local private for‐profit training institution Hotel Other (describe):

Afar Rural areas Urban areas


Amhara Rural areas Urban areas


Benishangul‐Gumuz Rural areas Urban areas


Dire Dawa Rural areas Urban areas



Region – select regions where training is provided

Rural/urban –select where training is provided

Place of training – select all that apply

Gambela Rural areas Urban areas


Harari Rural areas Urban areas


Oromia Rural areas Urban areas


Somali Rural areas Urban areas


SNNP Rural areas Urban areas


Tigray Rural areas Urban areas


13. Have the training materials for this program or is this training program formally recognized

by relevant authorities (e.g. approvals from professional bodies, regulatory bodies or government authorities)

Yes No

a. If YES, describe the authority: b. If YES, describe how formal recognition was obtained:

14. Has this training program been evaluated?

Yes No

a. If YES, who did the evaluation? Staff from our organization Independent consultants hired by our organization An external organization/institution Other, describe:

b. If YES, what methods were used to evaluate the training?


c. If NO, are there plans for an evaluation to be conducted? Yes No

15. Funding for trainees: a. Are participants fully (e.g. perdiem, lodging, transportation, learning materials, tuition fee)

sponsored to participate in this training program? Yes No

i. If YES, who funds their participation?

b. Are participants partially (any one or more of the above) sponsored to participate in this training program?

Yes No i. If YES, who funds their participation?

c. Do participants fully fund their own participation in this training program? Yes No

Many thanks for your participation in this survey. Please read and sign the statement below: The information provided in this survey is to the best of my/our knowledge is accurate and representative. Signed (respondent): ________________________________________ Respondent name: ________________________________________ Date: ___/_____/_______ Please provide any comments/feedback about the overall survey or the process.


B. Annex 2: Key informant interview guide

Rapid assessment of Health Workers in‐service training in Ethiopia: Key informant interview guide

Instructions for interviewer: Please read the following background to the respondent. Ensure that he/she understands that participation is voluntary and give an opportunity for the respondent to answer any questions. If he/she consents to participate, please have him/her sign before proceeding to the questions.

Explain to the respondent that this interview will ask them about their experiences and seek their expertise and guidance on strategies to improve in‐service training and that it may take approximately one hour. First, hand this to the respondent so that they can fill in some basic information about themselves.

Respondent information Respondent name Title:

First name(s): Surname:

Respondent position

Background for respondents: All health worker in‐service training program providers in Ethiopia including public, private and non‐profit organizations, training institutions, NGOs and others are invited by the Federal Ministry of Health (FMOH) to participate in this rapid assessment of in‐service training in Ethiopia. The main objectives of this rapid assessment are to:

1. Analyze the in‐service training situation in Ethiopia 2. Analyze in‐service training program provider practices in training design, delivery, follow‐

up/monitoring, evaluation, capacity building, standardization and institutionalization, tracking training and linkages with health service providers, community health workers, para social workers, program managers and policy makers

3. Identify strengths, weaknesses and best practices 4. Inform the development of a data‐base for health sector in‐service trainings in Ethiopia 5. Inform the development of a strategy for standardization and institutionalization of health sector in‐

service trainings It is hoped that this assessment will provide greater information about the type of in‐service training provided in Ethiopia and generate additional evidence to inform the development of a national in‐service training strategy. This rapid assessment includes a questionnaire for in‐service training program providers about the type of training they provide and their practices as well as interviews with up to 20 representatives of different stakeholders (public, private, NGOs, CSOs, FBOs and bilateral and multilateral agencies) on priority health worker in‐service training issues and strategies. All information gathered will be kept anonymous; interviews will be audio‐recorded and transcribed for analysis. Findings will be presented in a way that protects the anonymity of participants and their organizations. All information collected will only be presented publicly in a way that does not disclose the identity of the organization or respondent. It is optional to participate in this interview and your participation indicates your consent to use the collected information for the purposes explained. We greatly appreciate your time and efforts taken to participate in this key informant interview. We look forward to sharing the results of this assessment with you. For more information or if you have any questions,


Respondent’s organization

Respondent phone

Respondent email

Respondent gender Female Male

Respondent age <30 years 30‐50 years >50 years

Background related to health worker in‐service training – select all that apply

Developing and designing training programs Providing training Coordinating and/or managing training programs Training trainers Monitoring and evaluating training programs Developing training policies and/or plans Accrediting/formal recognition of training programs Other (please describe): ________________

Questions 1. Tell me about your work relating to health‐workers in‐service training: 2. How do you see current in‐service training addressing performance gaps in Ethiopia? 3. What do you think are the key issues or challenges that affect the effectiveness, efficiency

and sustainability of health workers in‐service training in Ethiopia at the national, in‐service training program provider, and learner/workplace levels?

a. Which issues do you think are the highest priority to address within the next 5 years at each level?

i. Why? 4. What do you think future research or assessments on health workers in‐service training

should focus on? a. Why?

5. I’m going to read you 10 scenarios, after listening to each scenario I’d like to get your responses to a few questions on how you feel about the scenario, how feasible the scenario would be and what strategies or steps are needed to achieve it.

Scenario 1: All training programs are in alignment with national and/or regional training plans and are designed to address assessed training needs

How do you feel about this scenario?

How feasible is this scenario?

What would it take to achieve this scenario?

What best practices do you know of that should be scaled up to help achieve this scenario? Scenario 2: Continuing professional development is mandatory for all health workers and is

linked to periodic renewal of professional practice licenses.




What best practices do you know of that should be scaled up to help achieve this scenario? Scenario 3: In‐service training program providers collaborate with pre‐service training

providers to ensure consistency in learning approaches and content between pre‐service and in‐service education and training.




What best practices do you know of that should be scaled up to help achieve this scenario?


Scenario 4: All international NGOs and development partners providing in‐service training partner with local in‐service training program providers to build their capacity so that they can take on the responsibility of providing that training in the future.




What best practices do you know of that should be scaled up to help achieve this scenario? Scenario 5: In‐service training program providers always use and strengthen local

infrastructure and trainers (eg – local training institutions, centers and sites).




What best practices do you know of that should be scaled up to help achieve this scenario? Scenario 6: All in‐service training program providers are formally authorized or accredited by

local/national authorities to provide training programs.




What best practices do you know of that should be scaled up to help achieve this scenario? Scenario 7: All in‐service training curricula and materials meet minimum standards and are

accredited by national authorities.




What best practices do you know of that should be scaled up to help achieve this scenario? Scenario 8: In‐service training program providers participate in a national and regional in‐

service training coordination mechanism that includes all in‐service training program providers and key stakeholders to improve planning and coordination of training.




What best practices do you know of that should be scaled up to help achieve this scenario? Scenario 9: All in‐service training programs and trainees are tracked through a national and

regional mechanism (eg – database).




What best practices do you know of that should be scaled up to help achieve this scenario? Scenario 10: All in‐service training program providers evaluate the effectiveness of their

training programs




What best practices do you know of that should be scaled up to help achieve this scenario? Ask if the respondent has any other comments or recommendations that s/he would like to share. At the end of the interview, thank the respondent for her/his time.

USAID APPLYING SCIENCE TO STRENGTHEN AND IMPROVE SYSTEMS PROJECT

University Research Co., LLC

7200 Wisconsin Avenue, Suite 600

Bethesda, MD 20814

Tel: (301) 654-8338

Fax: (301) 941-8427

www.usaidassist.org

Documents

Rapid Assessment of the Health Worker In-Service Training ... · A. Types of Education and Training Provided ... In-service training program provider and training program questionnaire