Self-Assessment and Peer Review: Improving Indonesian Service

Young Mi Kim is senior research and evaluation officer,and Adrienne Kols is consultant, Johns Hopkins Uni-versity Center for Communication Programs (JHU/CCP),Baltimore, MD, USA; Fitri Putjuk is country representa-tive, and Endang Basuki is consultant, JHU/CCP, In-donesia. This study was made possible by financial sup-port from the U.S. Agency for International Development(USAID) under cooperative agreements CCP-A-00-96-90001-00 and HRN-C-00-96-900013-03. The authors thankLeslie B. Curtin, Lana Dakan and Bambang Samekto(USAID/Indonesia); Pudjo Rahardjo, Sugiri Syarief andMamam Sundjana (National Family Planning Coordi-nating Board [BKKBN]); Erlaini, Emiria Sirman and Nur-fina Bachitar (JHU/CCP consultants); Anne Palmer, J.Douglas Storey and Gary Lewis (JHU/CCP); EugenitaGarot and Endang Iradati (JHU/CCP, Indonesia); DebraRoter and Susan Larson (JHU); Paul Richardson and HanyAbdallah (Quality Assessment Project); and Izhar M. Fihir(AVSC International).

4 International Family Planning Perspectives

While Indonesia’s family planningprogram is one of the most suc-cessful among developing coun-

tries, contraceptive discontinuation ratesare high: One-fourth of couples who adopta method stop using it within a year.1 In re-sponse, the National Family Planning Co-ordinating Board (BKKBN) has launcheda national initiative to improve the quali-ty of family planning and other repro-ductive health care. One major componentof this initiative is extensive refresher train-ing for family planning service providers,to improve their interpersonal communi-cation and counseling skills.

In a marked departure from past prac-tice in Indonesia, the training curriculumpromotes a client-centered approach tocare, drawing on the client-centered ther-apeutic relationship described in coun-seling and psychotherapy, as well as newmodels of medical care that have emergedas alternatives to the conventional physi-

Self-Assessment and Peer Review: Improving IndonesianService Providers’ Communication with Clients By Young Mi Kim, Fitri Putjuk, Endang Basuki and Adrienne Kols

cian-centered approach.2 Client-centeredcare calls on providers to respect clients’point of view, encourage clients to expresstheir needs and preferences, tailor infor-mation to clients’ situation and help clientsmake informed decisions, rather than tellthem what to do. Thus, the training cur-riculum demands substantial changes inproviders’ attitudes and behavior.

Health care workers face many obsta-cles in applying newly learned skills on thejob, and improvements in job performanceafter training are not necessarily sustainedover time.3 To realize the potential of train-ing, providers need reinforcement and asupportive environment when they returnto their jobs after training.4 Because fundsavailable for conventional forms of rein-forcement (such as refresher training orclose supervision) are limited, BKKBNsought to design and test a low-cost in-tervention to help providers consolidateand apply newly acquired skills.

Context: Training alone may not be sufficient to prompt complex and lasting changes in the per-formance of family planning providers. Affordable and effective reinforcement mechanisms areneeded to ensure that providers apply new skills on the job.

Methods: In December 1997 and January 1998, 201 providers working at 170 clinics in Indonesiaattended a training course on client-centered counseling. They were divided into three subgroupsfor follow-up. One group (controls) received no reinforcement, one conducted weekly self-assessments and the third attended peer-review meetings in addition to conducting self-assessments. Data were collected before training, immediately afterward and after four monthsof reinforcement to measure changes in provider and client behavior.

Results: In the month after training, counseling sessions were about twice as long as before,and providers offered twice as much information and counseling on medical and family planningissues. The frequency of providers’ facilitative communication (which fosters rapport and clientparticipation) doubled from 15 to 30 instances per session, and the number of clients’ questionsincreased from 1.6 to 3.3. After reinforcement, providers’ facilitative communication, clients’ ac-tive communication and clients’ ratings of self-expression and satisfaction increased in the self-assessment group, but did not change significantly in the control group. Both providers’ facilita-tive communication and clients’ active communication improved further in the peer-review group,but this intervention did not affect clients’ perspectives on the counseling experience.

Conclusions: Self-assessment and peer review help maintain providers’ performance aftertraining and prompt continuous quality improvement.

International Family Planning Perspectives, 2000, 26(1):4–12

ARTICLES

Prior research suggested that self-as-sessment and peer review would be effec-tive, low-cost strategies.5 Self-assessmentasks individual providers to judge theirown job performance against a set of out-side standards. In peer review, groups ofproviders give one another feedback andshare experiences and ideas. Providers’ ef-forts to learn and improve by themselvesis a critical element of both activities. Afteridentifying their weaknesses, providers setpersonal goals for behavior change, try outnew behaviors and assess the outcomes oftheir efforts; this is a continuous process,in which providers repeat the cycle untilthey are completely satisfied with their per-formance. The success of both self-assess-ment and peer review depends on pro-viders’ motivation, ability and diligencein completing the tasks required, since lit-tle or no supervision is involved.

While research on whether self-assess-ment and peer review improve health work-ers’ performance is limited and has proveninconclusive, results suggest that these ap-proaches may improve providers’ commu-nication skills. Studies in developed coun-tries that have examined the reliability ofself-assessment as a substitute for or com-plement to supervisors’ evaluations gener-ally suggest that it is useful in training health

5Volume 26, Number 1, March 2000

percent were midwives,* while the re-mainder were nurses. The providers wereresponsible for a wide range of health ser-vices in addition to family planning; 45%spent less than half their time on familyplanning. Four-fifths had more than fiveyears of experience offering family plan-ning services, and half saw more than 10family planning clients per week.

Clients were chosen randomly on theday of data collection; as a result, they mir-rored the general family planning popu-lation. All but two were women, 99% weremarried, 70% had at least two living chil-dren and 55% were aged 25–34. More thanhalf (57%) had at least some secondary ed-ucation; this proportion is slightly higherthan national levels among women 25–44years old (21–44%), reflecting that familyplanning users in Indonesia are better ed-ucated than nonusers.9 Three-fourths(76%) of participants were continuingclients, about half of whom were return-ing for routine checkups and resupply; theremainder came to the clinics because theyhad a problem with their method. Themost commonly used contraceptives werethe injectable (used by 28% of participants),IUD (27%), implant (17%) and pill (16%).

Interventions Providers attended five-day training work-shops, held in December 1997 and Janu-ary 1998, that emphasized client-centeredcounseling and covered skills such as es-tablishing rapport with clients, encourag-ing dialogue and helping clients make de-cisions. To assess the impact of the trainingand reinforcement, we randomly assignedeach district to one of three groups; everyclinic and every provider within a districtbelonged to the same group. Providers inone group, who served as controls, at-tended the training workshop but did notreceive any subsequent reinforcement.Those in a second group participated inself-assessment after the training, whileproviders in the third group both partici-pated in self-assessment and attendedpeer-review meetings. Workshops held inthe self-assessment and peer-review dis-tricts included a half-day of instruction onthese interventions. The reinforcement in-terventions began one month after theworkshops ended and lasted 16 weeks,from February to June 1998.

For the self-assessment intervention,providers spent about 15–20 minutes eachweek evaluating a single counseling ses-sion, using a two-page form. Each week’sform covered a single skill area fromamong these eight: listening, being re-sponsive to clients, expressing positive

care providers. However, they have ques-tioned the reliability of self-assessment be-cause its quality varies widely.6

An evaluation of an Indonesian programthat sent trained midwives to observe andassess their colleagues and to give them di-rect feedback found that this interventionenhanced midwives’ interpersonal andclinical skills.7 Some key elements of thepeer-review approach also have been test-ed as part of the COPE (client-oriented,provider-efficient) intervention, in whichclinic staff work as a group to assess andsolve problems. Results suggest thatproviders are intrinsically motivated tooffer better services, consider themselvesresponsible for self-improvement and cancontinue to learn through group self-assessment and support.8

The purpose of the study described inthis article was to assess whether training,self-assessment or peer review encour-aged client-centered counseling in the In-donesian program—that is, whether, afterthe intervention, providers worked morewith clients as partners, clients partici-pated more actively and clients felt moresatisfied with the counseling process.

MethodsStudy Sample The study was conducted in two provincesof Indonesia: East Java and Lampung.Within each province, investigators se-lected three districts (or kabupaten) thatwere matched in terms of size, proximityto the provincial capital and clients’ so-cioeconomic characteristics. From the listof all public service clinics (or puskesmas)in the six districts, 191 clinics were ran-domly chosen for the study. One or twoproviders at each clinic were asked to par-ticipate, for a total of 233. Each providerasked six family planning clients to par-ticipate: two before training, two immedi-ately after and two at a later date. Over thecourse of the study, 21 clinics, 32 providersand 55 clients were dropped becauseproviders did not attend the training work-shop or information was incomplete. Dataon 170 clinics, 201 service providers and1,210 clients are analyzed here.

The clinics, 77% of which were locatedin rural areas, varied widely in size, staffingand client flow. In June 1998, for example,the top 20% of clinics with respect to fam-ily planning caseload registered 90 or morefamily planning visits apiece, while the bot-tom 20% each recorded 18 or fewer. A fewof the busiest clinics recorded more than200 family planning visits that month.

All of the providers were women; mostwere married with children. Ninety-three

emotions, eliciting information, verifyingclients’ decisions, communicating duringclinical procedures, giving informationand encouraging clients’ participation.First, providers reported on their own andthe client’s behavior, using a checklist withsuch items as “Gave full attention toclient’s fears and anxieties” and “Clientasked for clarification.” Then, two open-ended questions prompted the providersto reflect more deeply about the interac-tion, asking, for example, what other ques-tions or concerns the client might havehad and what else the provider could havesaid. The providers finished the form bylisting two specific behaviors, related tothat week’s skill area, that they wanted tochange during the following week. Spacealso was provided to record the outcomesof their efforts to improve.

Providers in the peer-review group at-tended a weekly 30–60-minute meetingwith three or four other participants. Nomoderator or facilitator was assigned, butproviders were given a brief discussionguide that echoed the topic of the week-ly self-assessment form. Participants wereexpected to discuss issues that emergedfrom the self-assessment exercises but notto identify a specific case or to share theirself-assessment forms.

Data and Variables Research assistants collected data fromfour sources: interviews with providersabout their training and service experience;audiotapes of family planning counselingsessions; exit interviews with clients abouttheir perceptions of the session; and clin-ic records of the number of clients servedand the availability of contraceptive meth-ods. Prior to data collection, providers andclients signed a confidentiality and vol-untary participation consent form; theirnames were not known to the analysts.Data were collected at three points: beforethe training workshop (baseline), duringthe one-month gap between the end of thetraining and the start of the reinforce-ment interventions (posttraining) and 1–2months following the 16 weeks of rein-forcement (follow-up).

Two key variables, for which informa-tion was drawn from the audiotaped ses-sions, examined provider-client inter-actions. One of these, which we callproviders’ facilitative communication,refers to communication that promotes aninteractive relationship between client and

*These clinic-based midwives should not be confused

with village midwives, who are trained in fewer areas

of health care and have more limited job responsibilities.

sively in both developed and developingcountries.10 The adaptation was based onan earlier client-provider interaction studywe conducted in Indonesia, in which wereviewed 38 videotaped counseling ses-sions.11 Coders for the current study wereIndonesians who understood the local lan-guage and received special training inRIAS; they assigned one of 47 mutuallyexclusive codes to each utterance, usinga computerized data entry screen whilelistening to the audiotapes. The codeswere combined into the categories listedin Table 1.

The analysis of changes in providers’ fa-cilitative and clients’ active communicationpresented a methodological challenge, be-cause the average length of the counselingsessions varied between clients and fluc-tuated significantly from one round to thenext. Data on the proportion of utterancescontributed by the provider and the clientallow sessions of different lengths to becompared. However, an increase in one pro-portion means that the other must decrease,since the total is always 100%. Yet if sessionslast longer, the number of contributions ofboth may increase. Frequency data providea more accurate assessment of these kindsof changes than do proportions. Frequen-cy data also capture changes in importanttypes of communication that occur too sel-dom to achieve significance in percentagetables. To present a full picture of thechanges in communication patterns, we re-port both percentages and frequencies.

Three other variables,derived from interviewswith clients, assessedtheir perspectives on thecounseling experience.These were self-efficacy,self-expression and satis-faction. Self-efficacy is aprecondition to behaviorchange. In general, it denotes the extent towhich a person believesthat she or he is able toact;12 here, it refers toclients’ belief that theycan say what they wantto the provider. Self-expression is clients’ as-sessment of how muchthey spoke and whatthey said during the con-sultation. Two aspects ofclients’ satisfaction withthe quality of care wereassessed: the personal at-tention they were shownand whether they re-

provider by fostering dialogue, rapportand client participation. The other, termedclients’ active communication, refers tocommunication that allows the client toparticipate in the consultation and helpshape its direction or that indicates thatshe feels comfortable and is speakingopenly with the provider. Increasing bothtypes of communication was the prima-ry goal of the interventions.

To measure the communication vari-ables, we used an adaptation of the RoterInteraction Analysis System (RIAS), whichassigns a code to each utterance (i.e., com-plete thought, usually a phrase or sen-tence) made by a client or provider so thatthese may be grouped into categories foranalysis. RIAS has been employed exten-

ceived the help that they came for. To measure these variables, research as-

sistants read a series of statements toclients, who had a choice of four respons-es: strongly agree, agree, disagree or strong-ly disagree. For the purposes of analysis, aneutral option was added to the scale; thus,these responses were tabulated on a five-point scale, with five indicating strongagreement and one indicating strong dis-agreement. Multiple interview items weregrouped to create the indicators shown inTable 2. Statistical testing found that relia-bility for these indicators was high. Cron-bach reliability coefficients were .79 for self-efficacy, .84 for self-expression, .85 forsatisfaction with attention shown and .69for satisfaction with needs met.

ResultsBaseline Communication Patterns Baseline counseling sessions show howproviders and clients in Indonesia usual-ly interact. On average, sessions lasted sixminutes, but they varied widely in length(from one minute to 72 minutes). Like-wise, the total number of utterances spo-ken by both provider and client rangedwidely (from 11 to 512), averaging 95.Total utterances in sessions with newclients exceeded those in sessions withcontinuing clients (119 vs. 87; p<.0001).Providers dominated the interchange:They spoke 64% of all utterances, and theirutterances generally contained morewords than clients’.


Self-Assessment and Peer Review in Indonesia

Table 1. Provider and client communicationcategories

PROVIDERFacilitative communicationAsks lifestyle and psychosocial questions

Gives information and counsels on lifestyle and psychosocial issues

Builds partnership with clients (self-disclosure, checks for understanding, asks for opinion, states opinion)

Expresses positive emotion (approval,empathy,concern, reassurance)

Shows agreement or understanding

Makes personal or social remarks

OtherGives information and counsels onmedical and family planning issues

Asks medical, family planning androutine questions

Gives instructions

Expresses negative emotion (disapproval, criticism)

Miscellaneous (transition words,mechanical repetition, unintelligible)

CLIENTActive communicationAsks questions of all kinds

Seeks clarification

Shows concern or worry; seeks reassurance

Expresses opinion, approval, disapproval; requests service

Makes personal or social remarks

OtherGives medical, family planning and routine informa-tion

Gives lifestyle and psychosocial information

Shows agreement or understanding

Laughs (nervous or happy)

Miscellaneous (transition words, unintelligible,

Table 2. Concepts reflecting clients’ perspectives on their coun-seling experience, as measured in exit interviews

Self-efficacyWhen I come to the clinic, I feel confident that I can talk about whatever is onmy mind.

When I come to the clinic, I feel confident that I can ask for clarification when Ido not understand something.

When I am asked a question by the provider, I feel confident that I can givemore than brief answers.

Self-expressionI feel that I spoke as much as I wanted today.

I feel that I had the chance to say, in my own words, what I wanted to saytoday.

I feel that I asked all the questions I wanted to ask today.

SatisfactionAttention and care

The provider took time to find out what I was concerned about today.

The provider answered my questions.

The provider listened carefully to everything I had to say.

The provider made me feel that she cared about me. (I felt attended by theprovider.)

The provider treated me well today.

Needs metI feel that I received the information and services I wanted today.

I feel that I got appropriate assistance for my particular needs.


self-expression, 4.0 for self-efficacy and 4.1for satisfaction. These results are not sur-prising, given the strong social pressuresin Indonesia against expressing dis-agreement. Thus, in this setting, even rel-atively small shifts in client assessmentsmay be meaningful.

Posttraining Communication PatternsThe most dramatic impact of the trainingworkshop observed in the month after-ward was that the average length of coun-seling sessions almost doubled, to 11 min-utes. Similarly, the total number ofutterances by both provider and clientjumped 86%, to 177. (Sessions grew short-er over the months that followed, but werestill 40% longer at follow-up than they hadbeen at baseline.)

Providers used most of the extra timeto give clients additional information andcounseling on medical and family plan-ning issues. The number of utterances inthis category rose from 27 to 58, and theproportion climbed from 39% to 48%(Table 3); both differences were highly sta-tistically significant (p<.0001). Providersalso asked more medical and family plan-ning questions after training than before(18 vs. 16; p<.005), even though thesequestions accounted for a smaller pro-portion of their utterances. Since Indone-sian providers generally give family plan-ning clients limited information,13 thedramatic increase in information deliveredrepresents an improvement in one aspectof counseling. However, the quality of in-formation provided is also important, andthe RIAS coding scheme does not permitus to assess the clarity, accuracy and rele-vance of information.

Training also had a positive impact onproviders’ facilitative communication. Be-tween baseline and the posttraining peri-od, the average number of utterances inthis category doubled from 15 to 30; as aproportion of providers’ contributions,they increased from 25% to 28% (p<.0001for both increases). In percentage terms,providers increasingly acknowledgedwhat clients said and offered informationon lifestyle and psychosocial issues.

After training, clients’ active commu-nication remained steady in percentageterms (Table 3) but more than doubled infrequency, from 3.3 to 7.0 utterances persession (p<.0001). Most of the increase wasin acknowledging what the provider hadsaid, as a consequence of the sharp rise inproviders’ information-giving. However,the average number of questions per ses-sion also doubled (from 1.6 to 3.3;p<.0001), and that number held steady

The bulk of provider communication, asexpected, was a straightforward deliveryof information on family planning andmedical matters. Such information andquestions about health and family planningaccounted for about two-thirds of pro-viders’ utterances (Table 3). One-fourth ofproviders’ communication was facilitative.

More than three-quarters of clients’ ut-terances came in direct response to theprovider. Clients predominantly either an-swered providers’ questions about med-ical, family planning and routine matters(56%) or signaled agreement with or un-derstanding of providers’ remarks (24%).Only about 10% of clients’ utterances wereactive; half of these were questions. Thistranslates into clients’ asking an averageof 1.6 questions per session.

Clients’ baseline ratings of their coun-seling experiences were skewed to theupper end of the five-point scale: 3.9 for

during the four-month follow-up period,even as consultations grew shorter.

All client ratings showed small increas-es after training. The self-expression rat-ing rose to 4.1, self-efficacy to 4.1 and sat-isfaction to 4.2 (p<.0001 for each increase).

Impact of Self-Assessment As Figure 1 shows, self-assessment alonehad a significant impact on providers’ fa-cilitative communication. The proportionof providers’ utterances that fell into thiscategory increased from 28% to 33% in thegroup that conducted self-assessmentonly, while remaining essentially un-changed in the control group. A closer ex-amination of the data (Table 4, page 8)shows two patterns at work. Some typesof communication (partnership-buildingand asking questions) increased in the self-assessment group but stayed at the samelevel in the control group. Others (posi-tive emotion and acknowledging clients’remarks) declined significantly in the con-trol group but not in the self-assessmentgroup. Only the proportion of utterancesdevoted to personal or social conversationincreased in both groups.

The level of clients’ active communica-tion increased significantly in the self-assessment group but not in the controlgroup (Table 5, page 9). The change wasdue to the substantial increase in clients’social conversation in the self-assessmentgroup and the decrease in their expres-sions of concern in the control group. Self-assessment also had a positive impact onother types of client communication (theprovision of lifestyle, psychosocial, med-

Table 3. Percentage distribution of providers’and of clients’ utterances during counselingsessions, by category, before training inter-vention (baseline) and in the month afterward(posttraining)

Communication Baseline Posttrainingcategory (N=397) (N=406)

PROVIDERFacilitative 24.8 28.3***Lifestyle/psychosocial

questions 2.1 1.2***Lifestyle/psychosocial

information 3.9 4.8**Partnership-building 9.4 8.9***Positive emotion 4.3 4.5Acknowledgment 3.9 7.6***Personal/social

conversation 1.2 1.3

Other 75.2 71.7***Medical/family planning

information and counseling 38.7 47.6***

Medical/family planning questions 26.8 15.8***

Instructions 4.9 2.6***Negative emotion 0.1 0.1Miscellaneous 4.8 5.5*

CLIENTActive 10.3 11.3Questions 4.8 4.7Clarification 0.9 1.2Concern 1.0 0.9Opinion 2.4 2.6Personal/social

conversation 1.3 1.9***

Other 89.7 88.7Laughter 3.9 2.7***Lifestyle/psychosocial

information 5.1 4.1**Medical/family planning

information 56.3 44.4***Agreement 24.0 36.2***Other 0.5 0.3**

Total 100.0 100.0

*Difference between columns is statistically significant at p<.05.**Difference between columns is statistically significant at p<.01.***Difference between columns is statistically significant at p<.001.

ControlSelf-assessmentSelf-assessment and re-

Figure 1. Percentage of providers’ utterancesrepresented by facilitative communication, byreinforcement intervention, according to datacollection period

40

35

30

25

20

15

10

5

0B P F

Data collection period

%

Note: Data collection periods are as follows: B=baseline (beforethe training workshop); P=posttraining (during the one-month gapbetween the end of the training and the start of the reinforcementinterventions); F=follow-up (after 16 weeks of reinforcement).

ical and family planning information),which is important for client-oriented care.

Client ratings of self-expression in-creased significantly (p<.0001) in the self-assessment group (from 4.0 to 4.2) but heldsteady in the control group. Similarly, inthe self-assessment group, but not in thecontrol group, significant increases wereobserved in client satisfaction withprovider attentiveness (from 4.3 to 4.4)and with needs met (4.1 to 4.3). Self-efficacy did not change in either group.

Additional Impact of Peer Review Peer review made an impact beyond thatof self-assessment on facilitative commu-nication (Figure 1). The total level of fa-cilitative communication rose from 28%to 37% in the peer-review group (Table 4).Peer review had a greater effect than self-assessment alone on social conversationand lifestyle or psychosocial information;it had less of an influence on partnership-building and a similar impact on lifestyleor psychosocial questions. The number offacilitative utterances declined amongproviders in the self-assessment group(from 29 to 25, p<.05) but remained stable(30–32) in the peer-review group.

Clients’ active communication also in-creased more sharply in the peer-reviewgroup (from 12% to 17%) than in the self-assessment group (from 12% to 15%),mainly because of differences in levels ofsocial conversation (Table 5). During thefinal round of data collection, clients in thepeer-review group made an average of 8.5active utterances, including 4.9 questions;by comparison, those in the self-assess-ment group made 5.8 active utterances, in-cluding 3.3 questions.

Client ratings show a different pattern.Two of the three variables changed in theself-assessment group: Satisfaction in-creased from 4.2 to 4.4, and self-expressionrose from 4.0 to 4.2 (p<.0001 for both com-parisons). However, none of the client rat-ings changed significantly in the peer-review group.

Differential Impacts on SubgroupsWe conducted a series of multiple regres-sion analyses and Wald tests to assess theimpact of the reinforcement interventionsin various subgroups. The subgroupanalysis examined six variables: new ver-sus continuing clients, clients’ educationand age, providers’ experience, provinceand clinic size.*

Self-assessment and peer review had astronger impact on providers’ facilitativecommunication in sessions with continu-ing clients than in those with new clients,

perhaps because sessions with continuingclients tend to be shorter and focused moreon asking clients questions than on givinginformation. In contrast, reinforcementhad a stronger impact on clients’ activecommunication and satisfaction in sessionswith new clients; this result may reflect thatnew clients have fewer preconceptionsabout appropriate behavior during fami-ly planning consultations.

Clients’ characteristics also proved sig-nificant in determining the effect of the in-terventions. Self-assessment and peer re-view had more impact on both providers’facilitative and clients’ active communi-cation among clients with at least a sec-ondary education than among those withless schooling. Better-educated clients en-gaged in active communication more oftenthan their less-educated peers at baseline,and this gap widened over the study pe-riod. Education may give clients the con-fidence to take advantage of any oppor-tunities to speak that providers offer, whileproviders may be more open and respon-sive to better-educated clients, with whomthey may identify. Reinforcement inter-ventions also had a greater impact onproviders’ facilitative communication insessions with clients aged 35 or older.

The interventions were associated witha greater improvement in both providers’facilitative and clients’ active communi-cation among providers with more than10 years of experience offering familyplanning services than among their less-experienced counterparts. This findingimplies that experienced providers, farfrom resisting change, were better able tounderstand, carry out and apply lessonslearned from reinforcement activities thanwere those with less experience.

Self-assessment and peer review had astronger effect on levels of providers’ fa-cilitative communication in clinics with50 or fewer client visits per week than inthose with smaller caseloads. In contrast,client satisfaction was more affected inlarger clinics. Client satisfaction differedsubstantially in the two provinces stud-ied at baseline, and it was more affectedby reinforcement activities in Lampungthan in East Java.



Table 4. Percentage distribution of providers’utterances during counseling sessions, bycommunication category and training inter-vention, in the posttraining and follow-upperiods

Communication category Posttraining Follow-upand training intervention

FACILITATIVE COMMUNICATIONAllControl 28.8 26.9Self-assessment only 28.2 32.9***Peer review 28.1 36.5***

Lifestyle/psychosocial questionsControl 1.6 1.5Self-assessment only 1.1 1.9***Peer review 1.0 1.8***

Lifestyle/psychosocial informationControl 5.0 4.6Self-assessment only 4.8 5.7Peer review 4.7 7.5***

Partnership-buildingControl 9.2 9.5Self-assessment only 8.1 10.6***Peer review 9.4 10.6*

Positive emotionControl 4.0 2.7**Self-assessment only 4.9 4.1Peer review 4.6 4.5

AcknowledgmentControl 7.7 5.5**Self-assessment only 7.9 6.6Peer review 7.1 7.1

Personal/social conversationControl 1.2 3.1***Self-assessment only 1.4 4.0***Peer review 1.4 4.9***

OTHER COMMUNICATIONAllControl 71.2 73.1Self-assessment only 71.8 62.6***Peer review 71.8 63.5***

Medical/family planning information and counselingControl 48.1 45.4Self-assessment only 48.4 43.6***Peer review 46.4 41.3**

Medical/family planning questionsControl 16.4 21.8***Self-assessment only 14.7 12.2*Peer review 16.4 15.4

InstructionsControl 1.9 1.8Self-assessment only 2.1 3.3**Peer review 3.7 3.4

Negative emotionControl 0.1 0.1Self-assessment only 0.1 0.3*Peer review 0.1 0.2*

MiscellaneousControl 4.7 3.9Self-assessment only 6.5 3.2***Peer review 5.2 3.2***

Total 100.0 100.0

*Difference between columns is statistically significant at p<.05.**Difference between columns is statistically significant at p<.01.***Difference between columns is statistically significant at p<.001.Notes: Ns for the posttraining and follow-up periods were as fol-lows: for the control group, 121 and 119, respectively; for the groupreceiving self-assessment only, 142 in both periods; and for thegroup receiving peer review, 143 and 142, respectively. The post-training period was the month after the completion of the training

*The analysis compared follow-up data from the self-as-sessment and peer-review groups with posttraining datafrom all three groups, as well as follow-up data from thecontrol group. The Wald test was used to check the sig-nificance of differences between subgroups’ coefficients.For example, if the regression coefficient for provider fa-cilitative communication was significantly higher for con-tinuing than for new clients, it was interpreted to meanthat reinforcement had a stronger impact on provider fa-cilitative communication among continuing clients.


taled approximately U.S.$70 per providerfor training, $2 for self-assessment and $9for peer review.* The need to transportproviders between the scattered rural clin-ics that participated in the study boostedpeer-review costs. Including opportunitycosts (the cost of employee time divertedfrom regular duties to program activities)and the cost of a single supervisory visit, thetotal cost of training and reinforcement was$90 per provider for the control group, $106for the self-assessment group and $122 forthe peer-review group. (To put these figuresin context, the providers’ average monthlysalary at the time was $76.) Developmentcosts for the training curriculum, self-assessment forms and peer-review guideswere not included because they were one-time expenditures.

To calculate cost-effectiveness, we usedthe proportion of providers’ utterancesthat were classified as facilitative com-munication as the outcome measure. Thecost of raising facilitative communicationone percentage point in the control group(i.e., through training alone) was $500. Incomparison, the cost was $12 for the self-assessment group and $10 for the peer-review group. Thus, adding peer reviewto self-assessment proved cost-effectivedespite its higher cost.

DiscussionBenefits for Providers While the training workshop raised thequality of providers’ communication inthe short term, without reinforcement,their newly learned skills tended to erodeover time. This confirms the observationsof health care trainers in both developingand developed nations that the transfer ofskills from formal training to everydaypractice is extremely limited without con-tinuing support and reinforcement.14

Indeed, providers in the control groupfound little support for their new counsel-ing skills when they returned to their clin-ics after training. Routine supervision vis-its were irregular, and supervisors—whowere not included in the training—rarelyaddressed client-provider communication.Untrained colleagues expressed skepticismwhen providers tried out new counselingbehaviors—for example, by asking “Whyare you acting so funny now?” Continuingclients also were bewildered and uncom-fortable when providers acted friendlier oroffered lengthier explanations than theyhad in the past. Providers found it easierto try out new communication behaviorswith clients whom they had not seen be-fore. Faced with these obstacles to behav-ior change, providers in the control group

Costs and Cost-Effectiveness Direct costs of training and reinforcement(including materials, trainers’ honoraria,providers’ per diems and transportation) to-

may have stopped trying to apply theirnew skills before they had a chance to con-solidate them.

Self-assessment supplied the rein-forcement and continuing learning thatproviders needed to change their behav-ior on the job. Not only did it help main-tain providers’ posttraining performance,it also served as a mechanism of continu-ous quality improvement. Some key communication skills, such as partner-ship-building, continued to improvethroughout the study period. More expe-rienced providers were better able totranslate self-assessment and peer reviewinto behavioral change.

Prior studies suggest that by clarifyingstandards, self-assessment forms reduceproviders’ anxiety and confusion, increasemotivation, focus attention and facilitatelearning.15 The list of weaknesses thatproviders commonly identified during theirself-assessment exercises matched outsideassessments of their deficiencies. This sug-gests that self-assessment helped providersdiscriminate between good and poor skillsand allowed them to evaluate how wellthey were performing. The process of be-havior change built into the weekly self-assessment exercises also served as a con-stant reminder to providers to apply newskills. During their final interviews,providers agreed that the self-assessmentexercises had increased their confidence andtheir ability to apply the communicationskills they learned during training.

Adding peer review to self-assessmentheightened the impact on some aspects ofprovider communication. The effective-ness of peer groups for learning dependsupon the quality of the group process andthe richness of the discussion. However,the contents of the peer-review discus-sions were not systematically studied. Ac-cording to anecdotal reports, the providersgot to know each other well over the 16-week period, enthusiastically shared theirexperiences and gained self-confidencefrom their participation. If this is so, thenone important benefit of the peer-reviewmeetings may have been the psychologi-cal and emotional support that partici-pants derived from one another.

Impact on ClientsMost of the gains in clients’ active com-munication were in personal and socialconversation, suggesting that the inter-

Table 5. Percentage distribution of clients’ utterances during counseling sessions, bycommunication category and training inter-vention, in the posttraining and follow-upperiods

Communication Posttraining Follow-upcategory and training intervention

ACTIVE COMMUNICATIONAllControl 10.2 11.9Self-assessment only 11.8 14.7**Peer review 11.6 16.6***

QuestionsControl 3.7 6.0**Self-assessment only 4.7 6.2*Peer review 5.6 6.8

ClarificationControl 1.3 1.2Self-assessment only 1.1 0.9Peer review 1.3 1.2

ConcernControl 0.9 0.3**Self-assessment only 1.1 1.2Peer review 0.6 1.1*

OpinionControl 2.7 0.4***Self-assessment only 2.8 0.4***Peer review 2.1 0.2***

Personal/social conversationControl 1.6 4.0***Self-assessment only 2.1 5.9***Peer review 2.0 7.3***

OTHER COMMUNICATIONAllControl 89.8 88.1Self-assessment only 88.2 85.3**Peer review 88.4 83.4***

LaughterControl 3.2 3.5Self-assessment only 2.4 2.1Peer review 2.5 3.7*

Lifestyle/psychosocial informationControl 5.3 4.5Self-assessment only 3.6 6.6***Peer review 3.6 6.6***

Medical/family planning informationControl 46.4 50.1Self-assessment only 41.3 46.3**Peer review 44.9 41.8

AgreementControl 32.8 28.7Self-assessment only 39.1 29.9***Peer review 36.4 30.9**

OtherControl 1.1 0.2***Self-assessment only 1.8 0.4***Peer review 1.0 0.4**

Total 100.0 100.0

*Difference between columns is statistically significant at p<.05.**Difference between columns is statistically significant at p<.01.***Difference between columns is statistically significant at p<.001.Notes: See notes to Table 4.

*The cost-effectiveness analysis employed the exchange

rate that prevailed when each expense was incurred. This

generally ranged from 5,000 to 7,500 rupiahs to the dol-

lar, but spiked as high as 13,500 rupiahs.

Egypt has investigated the link betweenclient-centered counseling and contra-ceptive continuation.19 Further researchis needed to determine whether and howfacilitative counseling and increased clientparticipation affect contraceptive contin-uation in developing countries.

Ingredients for SuccessPast studies have reported problems inimplementing self-assessment because ofparticipants’ unfamiliarity with their rolein the intervention, resistance to self-eval-uation or distrust of the assessment.20 Bycontrast, although participants in thisstudy conducted the self-assessment ex-ercises in isolation from one other andwithout support from supervisors orcoworkers, nearly all of them completedthe 16-week series. Similarly, absenteeismfor the peer-review meetings was ex-tremely low, even though two-thirds ofproviders had to travel to other clinics forthe meetings. Why were these providersso diligent in carrying out their self-assessments and peer review?

The brief, uncomplicated design of theself-assessment exercises and form madethem easy to complete, even though theform took longer than planned to fill out.Providers initially had some difficultieswith the cycle of defining concrete be-havior goals, trying out new behaviorsand assessing the outcomes, but a singleproblem-solving meeting proved suffi-cient to overcome them.

Furthermore, the contents of the cur-riculum, self-assessment forms and peer-review guides were based on formative re-search findings and were completelyconsistent with one another. The trainingworkshop clearly defined the behaviorsto be evaluated and their performance cri-teria, so providers understood what eachbehavior meant and how to evaluate itsquality. Equally important, the workshoptaught providers how to complete the self-assessment forms. Without this kind oftraining, self-assessment might not work.

Providers’ high level of motivation alsowas essential. Training instilled providerswith the desire to offer better service, as ev-idenced by the increased length of sessionsimmediately afterward. The short (one-month) time lag between the trainingworkshop and the start of the reinforce-ment interventions allowed self-assess-ment and peer review to capitalize on thisstrong motivation. Further motivationmay have come from Indonesianproviders’ identification with the servicedelivery system and their tendency tocomply with their institutional obligations.

vention was more successful in promot-ing rapport between providers and clientsthan in boosting client input. This infer-ence is borne out by the relatively smallchange in client self-expression ratings.However, clients were asking twice asmany questions by the end of the study asthey had been at baseline—a meaningfulchange, given the passive behavior ofmost Indonesian clients.

Although these data suggest that train-ing and reinforcement are related to im-provement in providers’ and clients’ be-havior, they do not support the theory thatraising providers’ facilitative communi-cation increases clients’ confidence in theirability to express themselves, since self-efficacy grew only slightly over the studyperiod. Rather, providers’ and clients’ be-havior seem to be more directly linked,with providers offering clients more op-portunities to speak and prompting themto talk. Better-educated clients were ableto take greater advantage of these oppor-tunities than their less-educated peers.

Theoretically, increases in providers’ fa-cilitative and clients’ active communica-tion should make clients feel more satis-fied with their consultations. Althoughthere was a significant rise in client satis-faction, it was quite small. Also, contraryto expectations, satisfaction increased mostin the self-assessment group, although thepeer-review group displayed the greatestgains in provider and client behavior.Given the many methodological difficul-ties in defining and assessing client satis-faction,16 it is not clear whether these re-sults reflect measurement problems or realdifferences in client satisfaction. Substan-tial differences in client satisfaction be-tween Lampung and East Java also com-plicate interpretation of the results.

Ultimately, the success of this inter-vention should be measured by its impacton clients’ correct and continuing use ofa contraceptive method, since high dis-continuation rates were BKKBN’s moti-vation for launching the project. Howev-er, no data were collected on clients’contraceptive practices after they left theclinic. In theory, increased client partici-pation and more active communicationshould help clients make more realistic,better-informed contraceptive choices andshould increase their confidence in andcommitment to their decision—all ofwhich would raise their likelihood of con-tinuing to use the method.17 Studies inmedical settings in developed countrieshave found that health outcomes improvewhen clients take a more active role in con-sultations,18 but only a single study in

Study Strengths and Limitations By collecting data at three points in time(including a baseline) and creating a con-trol group, we were able to study severalissues at once: the impact of training onjob performance, retention of skills overtime, the impact of self-assessment aloneand the impact of self-assessment com-bined with peer review. The large samplesize provided the opportunity to compareimpacts in several ways and allowed thequantitative findings to be interpretedwith a high degree of confidence.

The use of multiple data sources and in-dicators to measure the impact of the in-terventions enriched the interpretation ofthe findings. Audiotaping the sessions hadseveral advantages over conventional ob-servation and interview methods: It pro-vided in-depth qualitative and quantita-tive data, allowed investigators to listenand recode as needed, and covered clientand provider behavior equally. Interviewsprobing clients’ perspectives on the coun-seling sessions enhanced our under-standing of client-provider interactionsand the impact of interventions. Con-flicting findings from different datasources provided important insights onmeasurement issues.

A critical limitation of this study is thatwe cannot relate quality improvementswith outcome behaviors, such as continu-ing use of a contraceptive method. We havedata showing that the number of familyplanning visits to study clinics droppedduring the four-month follow-up period;however, this statistic was so greatly af-fected by economic and social disruptionsin Indonesia that it may not reflect changesin quality. 21 Riots, contraceptive shortagesand new charges for contraceptives dur-ing this period all discouraged clients fromattending public clinics.

Refining the InterventionsBoth training and reinforcement over-looked a critical issue in raising the qual-ity of care: ensuring that providers giveequally good care to all clients, regardlessof their education, socioeconomic statusor ethnicity.22 Although the training cur-riculum and reinforcement materials en-couraged respect for clients, they did notask providers to reflect on how cultural bi-ases affected their interactions with pooror uneducated clients. Revised materialsshould directly address these issues andencourage providers to treat all clientswith equal respect and empathy.

Other changes in the strategies testedhere could help give providers the sup-portive work environment that experience




tural values may have made providers re-luctant to give specific feedback, for fearof being impolite. While giving construc-tive criticism is always risky behavior, itis especially so in Indonesia.

Meeting the Need for Reinforcement Self-assessment and peer review can maketraining more effective by helpingproviders apply new skills in the work-place, giving them an opportunity to con-tinue learning and motivating them to im-prove their performance long aftertraining is over. The learning cycle builtinto these reinforcement mechanisms alsocan support the kind of ongoing qualityimprovement sought by many primaryhealth care programs in the developingworld. Given the high cost of training,spending a small additional amount onself-assessment or peer review makesgood sense.

References1. Indonesia Central Bureau of Statistics et al., IndonesiaDemographic and Health Survey 1997, Calverton, MD, USA:Indonesia Central Bureau of Statistics and Macro Inter-national, 1998.

2. Rogers C, Client-Centered Therapy, Boston: HoughtonMifflin, 1951; Henbest RJ and Stewart MA, Patient-cen-tredness in the consultation. 1: a method for measure-ment, Family Practice, 1989, 6(4):249–253; and Smith RCand Hoppe RB, The patient’s story: integrating the pa-tient- and physician-centered approaches to interview-ing, Annals of Internal Medicine, 1991, 115(6):470–477.

3. Bradley J et al., Whole-Site Training: A New Approach tothe Organization of Training, AVSC Working Paper, NewYork: AVSC International, 1998, No. 11; and Sullivan RSand Smith T, On-the-Job Training for Family Planning Ser-vice Providers, Strategy Paper, Baltimore, MD, USA: JHPIEGO, 1996, No. 3.

4. Best K, Training involves many factors, Network, 1998,19(1):19–23 & 26; and Kols AJ and Sherman JE, Familyplanning programs: improving quality, Population Reports,1998, Series J, No. 47.

5. MacDonald P, The Peer Review Program of the Indone-sian Midwives Association, Jakarta: University ResearchCorporation and Indonesian Midwives Association, 1995;and Lynam P, McNeil Rabinovitz L and Shobowale M,Using self-assessment to improve the quality of familyplanning clinic services, Studies in Family Planning, 1993,24(4):252–260.

6. Calhoun J, Ten Haken J and Wooliscroft J, Medical stu-dents’ development of self and peer assessment skills,Teaching and Learning in Medicine, 1990, 2:25–29; FincherR and Lewis L, Learning, experience, and self-assessmentof competence of third-year medical students in per-forming bedside procedures, Academic Medicine, 1994,69(4):291–295; Kaiser S and Bauer J, Checklist self-evaluation in a standardized patient exercise, AmericanJournal of Surgery, 1995, 169(4):418–420; Falchikov N andBoud D, Student self-assessment in higher education: ameta-analysis, Review of Educational Research, 1989,59(4):395–430; and Gordon M, Self-assessment programsand their implications for health professions training, Aca-demic Medicine, 1992, 67(10):672–679.

7. MacDonald P, 1995, op. cit. (see reference 5).

8. Lynam P, McNeil Rabinovitz L and Shobawale M,

in both developing and developed coun-tries has found to be so necessary aftertraining.23 Training all of the providers ina clinic—rather than just one—and ori-enting the rest of the staff to the curricu-lum would help ensure that coworkersunderstood and encouraged providers’ ef-forts to change their approach to coun-seling. It would also substantially lowerthe cost of peer review by eliminating theneed for providers to travel between clin-ics. If training were extended to managersand supervisors, then they too could helpoversee and advise providers on inter-personal communication and counseling.(AVSC International has long advocatedthis kind of whole-site training.24)

Clients also contribute to a supportiveenvironment, as demonstrated by pro-viders’ difficulty in changing their style ofcounseling with continuing clients. Thisfinding suggests the importance of a strat-egy that raises clients’ expectations and im-proves providers’ communication andcounseling skills. Family planning pro-grams in Nepal and Egypt employingradio and television to model client-cen-tered interactions have changed culturalnorms about what is appropriate behav-ior during family planning consultations.25

While providers had little difficultycompleting the self-assessment forms, ittook them longer than planners had esti-mated, and some did not have enoughtime to fill out the form between clients.But some providers had trouble remem-bering the details of the consultation ifthey waited until later in the day to com-plete the form. One solution is for pro-viders to fill out the first section of the form(which asks specific questions about theirperformance) immediately after seeing theclient, but to save the rest for later.

Peer review proved to have a weakerdesign than self-assessment. Giving andreceiving feedback is the critical learningcomponent in group work, but providerslacked the expertise to offer the kind of ac-curate and concrete feedback that can bestinfluence knowledge, attitudes and be-havior. Adding a more skilled person tothe group, such as a supervisor or trainedpeer mentor, might have improved thequality of the feedback. Feedback wouldhave been more concrete and powerful ifthe group listened to an audiotaped seg-ment from one of their own sessions orparticipated in a role-play, rather than bas-ing their discussion on providers’ ac-counts of their experiences. Audiovisualmaterials and a more detailed discussionguide might have focused and enrichedthe peer review. Finally, Indonesian cul-

1993, op. cit. (see reference 5).

9. Indonesia Central Bureau of Statistics et al., 1998, op.cit. (see reference 1).

10. Roter D, The Roter Interaction Analysis System (RIAS)Coding Manual, Baltimore, MD, USA: Johns Hopkins Uni-versity School of Hygiene and Public Health, 1997; RoterD et al., The effects of a continuing medical education pro-gramme in interpersonal communication skills on doc-tor practice and patient satisfaction in Trinidad and Tobago, Medical Education, 1998, 32(2):181–189; Abdel-Tawab N, Provider-client communication in family plan-ning clinics in Egypt: styles predictors, and associationswith client outcomes, dissertation, Johns Hopkins Uni-versity School of Hygiene and Public Health, Baltimore,MD, USA, 1995; and Kim YM et al., Client participationand provider communication in family planning coun-seling: transcript analysis in Kenya, Health Communica-tion, 1999, 11(1):1–19.

11. Kim YM et al., Client-Provider Interaction Study: Pro-gram Support Research for the Quality Improvement Initia-tive in Indonesia, Baltimore, MD, USA: BKKBN and JohnsHopkins University Population Communication Services,1997.

12. Bandura A, Exercise of personal and collective effi-cacy in changing societies, in: Bandura A, ed., Self-Efficacy in Changing Societies, Cambridge, UK: CambridgeUniversity Press, 1995, pp. 1–45.

13. Kim YM et al. 1997, op. cit. (see reference 11).

14. Bradley J et al., 1998, op. cit. (see reference 3); and Sul-livan RS and Smith T, 1996, op. cit. (see reference 3).

15. Abrams R and Kelley M, Student self-evaluation ina pediatric-operative technique course, Journal of DentalEducation, 1974, 38(7):385–391; Stackhouse J and Furn-ham A, A student-centred approach to the evaluation ofclinical skills, British Journal of Communication Disorders,1983, 18(3):171–179; and Stuart M, Goldstein H and SnopeF, Self-evaluation by residents in family medicine, Jour-nal of Family Practice, 1980, 10(4):639–642.

16. Tavrow P, The relationship between quality of ser-vice and client satisfaction, paper presented at the an-nual meeting of the Population Association of America,Washington, DC, Mar. 27–29, 1997.

17. Kim YM et al., 1999, op. cit. (see reference 10).

18. Kaplan SH, Greenfield S and Ware JE Jr., Assessingthe effects of physician-patient interactions on the out-comes of chronic disease, Medical Care, 1989, 27(Supple-ment 3):S110–S127; and Henbest R and Fehrsen S, Patient-centredness: is it applicable outside the West? itsmeasurement and effect on outcomes, Family Practice,1992, 9(3):311–317.

19. Abdel-Tawab N, 1995, op. cit. (see reference 10).

20. Geissler P, Student self-assessment in dental tech-nology, Journal of Dental Education, 1973, 37(9):19–21; andJelly E and Friedman C, An evaluation system for resi-dency training, Journal of Family Practice, 1980, 10(1):73–80.

21. Kols AJ and Sherman JE, 1998, op. cit. (see reference4).

22. Hotchkiss DR and Jacobalis S, Indonesian health careand the economic crisis: is managed care the needed re-form? Health Policy, 1999, 46(3):195–216.

23. Bradley J et al., 1998, op. cit. (see reference 3); Sulli-van RS and Smith T, 1996 op. cit. (see reference 3); andKols AJ and Sherman JE, 1998, op. cit. (see reference 4).

24. Bradley J et al., 1998, op. cit. (see reference 3).

25. Storey D et al., Impact of the Integrated Radio Com-munication Project in Nepal, 1994–1997, Journal of HealthCommunication, 1999, 4(4):271–294; and Kols AJ and Sher-man JE, 1998, op. cit. (see reference 4).

zado el refuerzo, aumentaron significativa-mente en el grupo de auto-evaluación—perono en el grupo de control—la comunicación defacilitación de los proveedores, la comunicaciónactiva de los clientes, y la percepción de auto-expresión y satisfacción entre los clientes. Tantola comunicación de facilitación de los provee-dores como la comunicación activa de los clien-tes mejoraron aún más en el grupo de revisiónde pares, aunque esta intervención no afectólas perspectivas de los clientes con respecto asu experiencia con el asesoramiento.Conclusiones: La auto-evaluación y la eva-luación realizada por los pares ayudan a man-tener el rendimiento del trabajo de los prove-edores, y provocan una mejora continua en lacalidad de servicios.

RésuméContexte: La formation seule ne suffit pas né-cessairement à l’amélioration complexe et du-rable des performances des prestataires de plan-ning familial. Des mécanismes de renforcementfinancièrement abordables et efficaces doiventêtre mis en place pour assurer l’application surle terrain des nouvelles compétences acquises.Méthodes: En décembre 1997 et janvier 1998,201 prestataires employés dans 170 cliniquesd’Indonésie ont suivi un cours de formation àla consultation orientée sur les besoins de laclientèle. Ils ont ensuite été répartis en troissous-groupes de suivi: le premier (de contrôle)ne recevant aucune forme de renforcement, ledeuxième étant appelé à effectuer une auto-éva-luation hebdomadaire, et le troisième à parti-

ResumenContexto: Solamente la capacitación puedeno ser suficiente para realizar cambios opor-tunos, complejos y definitivos para mejorar elrendimiento del trabajo de los proveedores delos servicios de planificación familiar. Se re-quieren mecanismos de refuerzo viables y efi-caces para asegurar que los proveedores apli-quen sus nuevas técnicas al realizar sus tareas.Métodos: En diciembre de 1997 y en enero de1998, 201 proveedores que trabajaban en 170clínicas de Indonesia asistieron a cursos de ca-pacitación especializados en asesoramiento alos clientes. Se los dividió en tres subgrupospara realizar el seguimiento. Un grupo (con-troles) no recibió refuerzo, uno realizó auto-eva-luaciones semanales y el tercer grupo asistió areuniones en que los pares hacían una revisióndel desempeño, además de realizar su propiaauto-evaluación. Se recopilaron datos antes delcurso de capacitación, inmediatamente despuésy luego de un período de cuatro meses de re-fuerzo para avalar los cambios producidos enla conducta del proveedor y del cliente.Resultados: Un mes después de realizado elcurso, las sesiones de asesoramiento se pro-longaron al doble de tiempo, y los proveedoresofrecieron el doble de información y asesora-miento sobre temas médicos y de planificaciónfamiliar. Entre los proveedores, la frecuenciade su comunicación de facilitación (lo cual fo-menta la relación y la participación con el clien-te) se duplicó de 15 a 30 eventos por sesión, yel número de preguntas que formularon losclientes aumentó de 1,3 a 3,3. Luego de reali-

ciper, en plus de la procédure d’auto-évalua-tion, à des réunions d’évaluation par leurspairs. Les données ont été recueillies avant laformation, immédiatement après et après quatremois de renforcement afin de mesurer les chan-gements survenus dans les comportements desprestataires et de la clientèle.Résultats: Durant le premier mois ayant suivila formation, les consultations se sont révéléesenviron deux fois plus longues qu’auparavant,et les prestataires offraient deux fois plus d’in-formations et de conseils sur les questions mé-dicales et de planning familial. La fréquence dela communication d’invite des prestataires (pro-pice aux échanges et à la participation de lacliente) a doublé, passant de 15 à 30 instancespar session, tandis que le nombre de questionsposées par les clientes augmentait aussi, de 1,3à 3,3. Après le renforcement, la communica-tion d’invite des prestataires, la communica-tion active des clientes et l’évaluation d’auto-expression et de satisfaction des clientess’étaient améliorées dans le groupe soumis àl’auto-évaluation, sans toutefois présenter dechangement significatif dans le groupe decontrôle. La communication d’invite des pres-tataires et la communication active des clientesavaient augmenté plus encore dans le groupesoumis à l’évaluation des pairs, bien que cetteintervention soit restée sans effet sur les pers-pectives des clientes quant à l’expérience vécue.Conclusions: L’auto-évaluation et l’évalua-tion des pairs contribuent au maintien des per-formances des prestataires après la formation etfavorisent une amélioration de qualité continue.



CorrectionIn “The Incidence of Abortion World-wide,” by Stanley K. Henshaw,Susheela Singh and Taylor Haas [1999,25(Supplement):S30–S38], Australia’stotal abortion rate, shown in Table 2(page S34), should be 0.67, not 0.57.

Documents

Self-Assessment and Peer Review: Improving Indonesian Service