Upload
phungngoc
View
228
Download
6
Embed Size (px)
Citation preview
The World Bank Research Observer, vol. 13, no. 1 (February 1998), pp. 79–101© 1998 The International Bank for Reconstruction and Development / THE WORLD BANK 79
Evaluating Retraining Programsin OECD Countries: Lessons Learned
Amit Dar • Indermit S. Gill
Are retraining programs for the unemployed more effective than job search assistance?Governments of the member countries of the Organisation for Economic Co-operationand Development have considerable experience with retraining programs in a variety ofindustrial settings. Evaluations of these programs show that the results are disappointing,however. This article discusses the factors associated with retraining programs for twotypes of workers: those laid off en masse and the long-term unemployed. Evaluations indi-cate poor results for both groups: retraining programs are generally no more effective thanjob search assistance in increasing either reemployment probabilities or postinterventionearnings, and they are between two and four times more expensive than job searchassistance.
Industrial countries spend sizable amounts on labor programs for the unemployed.In 1992 member countries of the Organisation for Economic Co-operation andDevelopment (OECD) spent between 0.1 percent (Japan) and 2.6 percent (Sweden)of gross domestic product on labor programs. In several countries this training is“the largest category of active programs, and is often perceived as the principal alter-native to regular unemployment benefits” (OECD 1994a). Countries in EasternEurope—where the role of active labor programs is a topic of current debate—alsospent between 0.2 and 3 percent of gross domestic product on these programs (OECD
1994a). Yet possibly because such assistance is viewed almost as a fundamental rightin Western Europe, these programs are rarely evaluated outside the United States.This article surveys evaluations of retraining programs in the OECD countries, high-lighting the shortcomings of such schemes and illustrating the payoff to investing inrigorous evaluations.
We examine only the evaluations of retraining programs, so our focus is largely onadults with previous work experience rather than on unemployed school-leavers. Wedistinguish here between retraining schemes for workers displaced by plant closures
The World Bank Research Observer, vol. 13, no. 1 (February 1998)80
or restructuring and programs aimed at the long-term unemployed. These groupsdiffer in important ways. First, the long-term unemployed are a relatively hetero-genous group of individuals compared with those laid off en masse from a singleplant or firm. Second, although some programs are targeted at specific regions, thelong-term unemployed are generally more dispersed geographically. Third, the du-ration of unemployment is, almost by definition, greater for the long-term unem-ployed. Finally, retraining programs for the long-term unemployed are generally amix of classroom and in-plant training, while programs for displaced workers areusually confined to the classroom.
Evaluation Techniques
Techniques for evaluating the effectiveness of retraining programs can be broadlyclassified into two categories: scientific and nonscientific. Scientific evaluations canbe further divided into experimental and quasi-experimental. Experimental, or clas-sically designed, evaluations require selecting “treatment” and “control” groups be-fore the intervention: the treatment group receives the assistance, and the controlgroup does not. If large numbers of individuals are randomly assigned to each group,the average characteristics of the two groups should not differ significantly, so anydifference in outcomes can be attributed to program participation. In quasi-experimental studies treatment and control groups are selected after the interven-tion. To compute the program’s effects, statistical techniques are used to correct fordifferences in characteristics between the two groups. Nonscientific techniques donot use control groups to evaluate the effect of interventions but instead rely onstatistics compiled by a program’s administrators. These evaluations are of little use;without a control group, it is difficult to attribute the success or failure of the partici-pants to the intervention, because the changes in individuals’ behavior might haveresulted from other factors, such as worker-specific attributes or economywide changes(Grossman 1994).
Classically Designed (Randomized) Experiments
This technique, which was originally developed to test drug effectiveness, identifiesand randomly assigns individuals to either a treatment or a control group before theintervention. Its main appeal lies in the simplicity of interpreting results—the effec-tiveness of the program is computed as the simple difference in the outcome betweenthe participants in the treatment group and the nonparticipants in the control group(Newman, Rawlings, and Gertler 1994). The main pitfalls of this method are a failureto select individuals through random assignment, changes in behavior as a result oftheir assignment to either group (for instance, enrolling in private programs or intensi-
Amit Dar and Indermit S. Gill 81
fying their job search), high costs because of the number of participants in the sample,and ethical questions about excluding a group of people from the intervention.
Although randomization is thought to eliminate selection bias in determiningwho will participate, proponents of this methodology make an important assump-tion: that random assignment does not alter the behavior that is being studied. Thismay not be the case and, in fact, the bias may be quite strong (Heckman 1992). Forexample, ambitious individuals who would have taken a training course in any casewill not apply to the program if they stand a chance of being in the nontreatmentgroup. Such individuals, who might have enrolled in a nonrandomized regime, maymake plans anticipating enrollment in a training program. With randomization theymay alter their decision to apply or undertake activities complementary to training.Thus risk-averse persons will tend to be eliminated from the program.
Quasi-Experimental Techniques
In quasi-experimental methods the treatment and control groups are selected afterthe intervention. Econometric techniques are used to correct for the differences incharacteristics between the two groups. The main appeal of this method lies in itsrelatively low cost and the ability to undertake the evaluation at any time. The maindrawback is that these techniques—if done properly—are statistically complex. Thetechniques used to adjust for differences in observable attributes (for example, sex,age, region of residence, education) are relatively straightforward but subject to speci-fication errors; correcting for unobservable characteristics (for example, motivation,family connections) requires a convoluted procedure that can yield wildly differentresults. Quasi-experimental evaluations fall into three types: regression adjusted forobservable characteristics; regression adjusted for observable and unobservable char-acteristics (selectivity corrected); and matched pairs.
REGRESSION ADJUSTED FOR OBSERVED CHARACTERISTICS. This technique is used toassess the impact of participation in a program when the observable characteristics ofthe participants and the comparison group are different. This method is appropriatefor estimating the effect of a program when the difference between participants andnonparticipants can be explained by these observable characteristics. For example, ifbetter-educated workers are more successful in finding work regardless of whether ornot they had special training, then controlling for the effect of education (usingregression techniques) will provide more reliable estimates than would a simple com-parison of the reemployment probabilities of the control and treatment groups.
REGRESSION ADJUSTED FOR OBSERVED AND UNOBSERVED VARIABLES (SELECTIVITY
CORRECTED). When selection into programs is not random, and participation in aprogram is due to both observable and unobservable characteristics, the above tech-
The World Bank Research Observer, vol. 13, no. 1 (February 1998)82
nique, which corrects for observed characteristics, is likely to be biased. Even if par-ticipants and nonparticipants have similar observable characteristics, unobservablecharacteristics (such as innate ability) would lead to incorrect inferences about non-participants. This technique uses a method developed by Heckman (1979) called“sample selectivity correction” to try to control for these unobservables.
MATCHED PAIRS. Because the observed characteristics of individuals in the controland treatment groups are bound to be different to some degree, these groups arelikely to have different success rates in finding employment even in the absence ofactive labor market programs. To control for these differences, synthetic controlgroups are constructed using a matched pairs approach. The synthetic control group,which is a subset of the entire control group, is composed of individuals whose ob-servable characteristics most closely match those of the treatment group.
Relative Strengths of Techniques
Estimating the effect of an employment program on the earnings of trainees, usingrandomized and quasi-experimental techniques, LaLonde (1986) found that ran-domized experiments yielded results significantly different from those that relied onquasi-experimental techniques. Policymakers should be aware that availablenonexperimental evaluations of training programs may contain large biases. Whilerandomized experimentation is theoretically the best technique to estimate the ef-fects of interventions, quasi-experimental techniques may be superior in practice.
The main weakness of randomized experiments is their inability to ensure thatindividuals in the control group do not alter their behavior in a way that contami-nates the experiment. For example, individuals denied public job training mightenroll in private programs, which would bias the results of any evaluation of publicprograms.1 It may also be difficult to ensure that assignment is truly random. Forexample, applicants may be selected into the program because of nepotism, or pro-gram administrators may deliberately exclude high-risk applicants to achieve resultsthat reflect well on the program. A third problem concerns ethical questions abouttreating individuals as subjects in an experiment. Finally, experimental evaluationsare possible only for future programs, because the control and treatment groups haveto be selected before the program is initiated.
Using the dual criteria of rigor and feasibility then, randomized experiments arenot necessarily superior to quasi-experimental techniques. Because the decision toevaluate labor market programs often occurs after the programs are in place andbecause the costs of setting up the experiments are high, randomized evaluationsshould perhaps be the last alternative. Within quasi-experimental techniques, selec-tivity correction may not add much, especially when information is available for aconsiderable number of observable individual and labor-market characteristics (such
Amit Dar and Indermit S. Gill 83
as education, age, sex, household wealth, and region of residence). In addition tobeing cumbersome and somewhat counterintuitive, this method often gives arbi-trary results depending on the selectivity-correction specification that is used.
This leaves the matched pairs and regression-adjusted techniques. Of the two, thematched pairs technique is preferred because the procedure is less arbitrary. Becausethe observed differences between the treatment and comparison group are mini-mized, the exact specification of the model becomes less important. And because theprogram measures the simple difference in the variables that policymakers want an-swered (reemployment probabilities and wages) between the control and treatmentgroups, the results are easier for nonstatisticians to interpret.
One weakness shared by both the scientific and nonscientific evaluations is thatthey do not take into account the displacement that may result from the retrainingprogram. For example, in countries where demand for labor is constrained, retrain-ees may “bump,” or displace, employed workers, so aggregate unemployment maynot change despite the size of the program. In general, displacement implies that thesocial benefits from reemployment attributable to the retraining program are lowerthan indicated by the evaluation, however well done.
The Importance of Costs
For the purposes of informing policy decisions, an evaluation is not complete untilthe costs of both the retraining program and its alternatives are considered. For ex-ample, if retraining is twice as costly as job search assistance but no more effective infinding people jobs and increasing their wages, job search assistance is twice as cost-effective. At least at the margin, such a finding would constitute a case for reallocat-ing resources from retraining to job search programs. Unfortunately, costs appear tobe the least analyzed aspect of these programs in OECD countries.
Even the most careful evaluations of retraining programs cannot be used for socialcost-benefit analysis because of the displacement effects discussed above. But whendone correctly, evaluations are good guides for cost-benefit analysis of private train-ing programs, which policymakers can use to institute cost recovery in public pro-grams and to promote private provision. Evaluations may also help in deciding whetherretraining programs reduce budgetary expenditures by moving people off unem-ployment benefits into productive employment or whether the programs are a drainon the budget despite being effective.
Evaluating Retraining in OECD Countries
Retraining programs in OECD countries have been designed primarily to assist threecategories of workers: those laid off en masse; those who have lost their jobs because
The World Bank Research Observer, vol. 13, no. 1 (February 1998)84
of plant closures; and those who have been unemployed over the long term. Thisreview of the evaluations of eleven retraining programs classifies the specific situa-tion that the program was designed to address, describes the policy intervention thatwas selected and the type of evaluation used, and reports the main results. Becausethe success of retraining programs depends on aggregate or regional labor marketconditions, such as unemployment rates and the state of the leading industry, theseindicators are reported as well.
We examine the type of retraining provided, whether in classrooms or on-the-job,and whether it was accompanied by—or in lieu of—other measures, such as jobsearch assistance. The evaluations are classified as experimental, quasi-experimental,and nonscientific, and we do not consider inferences drawn from the nonscientificevaluations to be reliable. We then look at the effects of the program on reemploy-ment and wages, both for subgroups of retrainees and type of intervention. The costsof the program are included when they are reported.
Training Programs Instituted as a Result of Mass Layoffsand Plant Closures
The results of eleven programs (three in the United States, four in Sweden, andone each in Australia, Canada, Denmark, and France) examining the effectivenessof retraining programs for workers displaced through mass layoffs and plantclosures have been reviewed (Table A.1). Five of the evaluations were nonscientificand five were quasi-experimental. One study relied on more than one techniqueto evaluate the impact of the program; no study used experimental evaluationtechniques.
The retraining programs were undertaken to assist workers in the steel, pulp, min-ing, shipbuilding, and automotive industries. The number of workers who lost theirjobs varied from about 500 to 3,000 per plant. The rationale underlying the pro-grams was apparently to assist the affected workers in any way possible. Generally,these programs were instituted during periods of high or rising aggregate unemploy-ment or during a contraction in certain manufacturing industries. For example, theevaluations in the United States and Canada covered primarily the auto and steelindustries, which were battered by competition from Japan. Between 1978 and 1980,auto production in the two countries declined 25 percent, precipitating layoffs andplant closures in the early 1980s. In Europe and Australia the retraining programsseem to have been instituted during periods of high or rising rates of unemployment.Most of the retraining programs were classroom based, and accompanied by jobsearch assistance. With only one exception, on-the-job training was not provided orfacilitated. In France retraining was accompanied by financial incentives to regionalfirms that hired trainees, so the full costs of the program were likely high (OECD
1993a).
Amit Dar and Indermit S. Gill 85
Quasi-experimental and nonscientific techniques were used in evaluating all ofthese retraining programs. None of the studies was longitudinal, so they provide atbest a snapshot of the labor market benefits of the program. The longer-term ben-efits of retraining were not evaluated even by the scientific evaluations. Althoughnonscientific evaluations indicate that these programs are very effective in placinghigh numbers of male workers in wage-earning jobs or in self-employment and womenin business start-ups, more reliable quasi-experimental evaluations indicate that re-training programs are generally no more effective than job search assistance in in-creasing either reemployment probabilities or postintervention earnings. Some re-training programs resulted in modest gains in reemployment probabilities, but thewage changes were sometimes negative. Interestingly, evaluations of three retrainingprograms for U.S. auto workers showed the contrast between scientific and non-scientific techniques: in San José, California, a nonscientific evaluation indicatedhigh placement rates, while in Buffalo, New York, and Michigan—during the sameperiod—scientific evaluations showed that these programs were ineffective (OECD
1993a; Corson, Long, and Maynard 1985; Leigh 1992).The costs, when they are known, varied between $3,500 and $25,000 a person.
Evaluations seldom report the full costs of retraining or job search programs, how-ever, so determining cost-effectiveness is difficult. Retraining programs appear to bebetween two and four times more expensive than job search programs: for example,in Buffalo, job search services cost $850 a participant, while retraining cost $3,300(Corson, Long, and Maynard 1985). If, as the findings indicate, both programs haveroughly the same success, job search assistance may be more cost-effective than re-training in assisting displaced workers get jobs.
Training Programs for the Long-Term Unemployed
There is no reason to assume that the impact of retraining on the long-term unem-ployed is the same as it is on workers laid off en masse. The results of studies exam-ining the effectiveness of retraining programs for the long-term unemployed are shownin Table A.2. Of the eleven evaluations (four in the United States, three in Germany,two in the Netherlands, and one each in Canada and Britain), four were nonscien-tific, four were quasi-experimental, and three were experimental.
The clientele of retraining programs for the long-term unemployed is relativelyheterogeneous. Because these individuals are displaced from various sectors and somehave never worked, they are likely to be more varied in age, skills, and educationthan laid-off workers. New Jersey’s retraining program in 1986–87 included work-ers whose previous jobs were in manufacturing, trade, and services (Anderson, Corson,and Decker 1991); many were more than 55 years old. In contrast, half the workersin Germany’s retraining program for the long-term unemployed were less than 35years old (Johanson 1994).
The World Bank Research Observer, vol. 13, no. 1 (February 1998)86
These programs are generally instituted during improving conditions in industryor in aggregate employment. Interventions at this stage, if appropriately designed,are thought to enable the long-term unemployed to obtain some of the jobs that arebeing created. The programs are more comprehensive than are those for people laidoff en masse and generally provide a mix of classroom or workshop training, on-the-job training, and job search assistance.
Some experimental evaluations were conducted in the United States, and quasi-experimental techniques were used in both the United States and Europe. Some ofthese studies were longitudinal, providing an indication of the medium-term labormarket benefits of the retraining program (see Johanson 1994 and OECD 1993b forEurope; and Leigh 1992, and Anderson, Corson, and Decker 1991 for the UnitedStates). But many of the evaluations in Europe were nonscientific.
Again, the results of nonscientific evaluations, which were encouraging, were notconfirmed by evaluations based on scientific techniques. The effectiveness of pro-grams for the long-term unemployed, while not high, was better than that of pro-grams for those laid off en masse. A few programs did result, or were thought to haveresulted, in gains in either reemployment probabilities or wages; some evaluationsalso indicated that these programs were more effective in helping women. But whereparticipants did record gains in employment, longitudinal studies generally foundthat the effects of retraining dissipated within a couple of years. In this group, too,retraining programs were generally no more effective than job search assistance inincreasing either reemployment probabilities or postintervention earnings. For ex-ample, evaluations of the Texas Worker Adjustment Demonstration program indi-cated that participants were likely to be employed more rapidly than nonpartici-pants, but that over time, the employment opportunities of male participants wereno better than those of nonparticipants or of those who only had job search assis-tance (Bloom 1990). The costs, when known, varied between $900 and $12,000 aperson, about twice as much as job search services, but the lack of data makes itdifficult to determine the absolute cost-effectiveness of these programs. Still, jobsearch assistance programs appear to be somewhat more cost-effective than retrain-ing programs in finding jobs for the long-term unemployed.
Evaluation Results for Hungary
Rising unemployment and falling real wages are a vivid and costly aspect of countriesin transition from controlled to market economies. Long-run unemployment is par-ticularly pernicious, and many countries have mounted active labor programs (forinstance, public retraining programs and public service employment) to deal withthis problem. With the exception of Hungary, where the World Bank has sponsoredrigorous evaluations and collections of data on costs, little reliable information exists
Amit Dar and Indermit S. Gill 87
on their effectiveness. As an early reformer, Hungary provides valuable lessons forother transition economies. Since 1989 sharp declines in the country’s gross domes-tic product have been accompanied by rising unemployment and falling real wages.The unemployment rate appears to have stabilized since 1993, but employmentcontinues to decline, reflecting a continuing withdrawal of workers from the formallabor market. Since 1990 the government has offered workers retraining programsfinanced by a national employment fund. Trainees, who are either currently unem-ployed or are working but expect to become unemployed, include participants inpublic works programs. Instruction is largely classroom based.
Evaluations using different quasi-experimental techniques—matched pairs, regres-sion adjusted, and selectivity corrected—yielded different results (O’Leary 1995).Before adjusting for differences in the characteristics of program participants andcontrol groups, estimates of effectiveness indicated that retraining significantly raisedthe probability of reemployment. But when the observable characteristics of treat-ment and control groups were taken into account, retraining (and training, becauseabout 40 percent of trainees had not worked previously) was only marginally suc-cessful at best, increasing the probability of finding employment by 6 percent. Fur-ther controls for unobservable attributes led to ambiguous results. Retraining wasnot at all successful in raising earnings.
Preliminary analysis indicates that retraining was a substitute for attributes thatlead to higher reemployment probabilities in the absence of any intervention, such asbeing younger, better educated, and from more dynamic regions. That is, theprogram’s value-added (in terms of improving labor market outcomes) is greater forrelatively disadvantaged job-seekers. This finding may be country specific, and othercountries or regions should determine whether it applies to them before implement-ing similar large-scale programs. For the programs evaluated in Hungary, focusingmore on job-seekers who lack these attributes would appear to serve both equity andefficiency objectives better than simply ensuring support for programs whose re-trainees have high probabilities of reemployment and gains in earnings. The analysisimplies that public retraining programs should target older men from backwardregions.2
This finding also highlights the usefulness of rigorous impact evaluations, whichnet out the effects of such attributes in determining whether a program is effective.And it underscores the need to agree upon a reliable, feasible, and easily interpretedtechnique to evaluate the efficiency and equity effects of labor programs. Because ofits analytical rigor and feasibility, we argue that the preferred evaluation technique ismatched pairs analysis, where trainees are compared with a subset of the controlgroup whose characteristics most resemble their own.
Private cost-benefit analysis of the costs of retraining (an average of approximately$900 per trainee in 1994, according to Pulay 1995) and the level of gains in reem-ployment reveal that based on reasonable assumptions about the durability of
The World Bank Research Observer, vol. 13, no. 1 (February 1998)88
program effects and the amount and duration of unemployment benefits, it wouldtake more than 30 years to recover costs of the program (Gill and Dar 1995). Itseems difficult to justify retraining programs based on economic considerations alone,even in a country such as Hungary where the government provides benefits for un-employed workers.
Summary and Conclusion
The paucity of rigorous evidence on the costs and effectiveness of retraining pro-grams does not permit a definitive conclusion on whether such interventions can bejustified economically (Table 1). The scattered evidence does not appear to justifythe indiscriminate expansion of retraining programs to cover more of the unem-ployed. These conclusions are consistent with the findings in the OECD EmploymentOutlook (1993b), which concluded that “for the broadly targeted sub-group of pro-grams, the overall impression is most troubling. Available evidence does not permitstrong conclusions, but it gives remarkably meager support of a hypothesis that suchprograms are effective.”
From this discussion it appears that transition and other restructuring economiescan draw the following lessons from experience in the OECD countries:
Table 1. Summary of OECD Retraining Programs and EvaluationsDisplaced Long-term
Item workers unemployed
Previous sector of employment Manufacturing Heterogeneous
Labor market conditions Generally deteriorating Generally improving
Training venue Mainly classroom Classroom and OJT
Other measures provided JSA JSA
Type of evaluationNonscientific 6 4Quasi-experimental 5 4Experimental 0 3 (all U.S.)
EffectivenessNonscientific techniques Positive Generally positiveScientific techniques Negative Generally negative;
some groups benefitRelative to JSA No more effective No more effective
CostsRetraining Data are not available Data are not availableRelative to JSA At least twice as costly At least twice as costly
Note: JSA, job search assistance; OJT, on-the-job training.
Amit Dar and Indermit S. Gill 89
First, sound techniques should be used to evaluate retraining programs (and otherpublic interventions). Although the nonscientific evaluations of retraining programspresent a rosy picture based on placement rates and other informal evidence, scien-tific evaluations are quite discouraging. Relying on nonscientific evaluations maylead to incorrect policy conclusions.
Second, rigorous evaluations, although not necessarily allowing a complete socialcost-benefit analysis, can be useful for policymakers in allocating public expenditureon labor programs. Reviews of evaluations find, for example, that job search assis-tance measures—which cost less than retraining but appear equally effective—maybe a more cost-effective device in assisting displaced workers.
Third, OECD experience of retraining programs for workers displaced en massemay be useful in designing assistance programs in transition countries such as theformer Soviet Union and liberalizing economies that expect labor shedding in themanufacturing and mining sectors. Principally, these economies should recognizethat retraining should not be the main form of assistance.
Fourth, transition and developing economies that are beginning to experiencelong-term unemployment can learn from OECD experience, which indicates that re-training programs for the long-term unemployed are more beneficial for some groupsthan for others within this relatively heterogeneous group of job-seekers. Whichgroup will benefit most from retraining is difficult to predict, however. Principally,these results call for using modest untargeted pilot programs, evaluating them rigor-ously, and then tightly targeting public retraining programs to those for whom theyare found most cost-effective.
Notes
Amit Dar is in the Social Protection Group of the World Bank’s Human Development Network,and Indermit S. Gill is in the Latin America and Caribbean Country Department I. The authors aregrateful to Jane Armitage, Kathie Krumm, Ana Revenga, Michal Rutkowski, and anonymous refer-ees for helpful suggestions.
1. Heckman (1992) documents other limitations of this technique when applied to social experi-ments; these limitations derive from selectivity biases.
2. These findings contrast with those of Revenga, Riboud, and Tan (1994) for retraining pro-grams in Mexico. Using quasi-experimental techniques, they find that effectiveness can be improvedif programs better target relatively educated job-seekers of both sexes with previous work experience.They also find generally more encouraging results than those for OECD countries and Hungary.
The World Bank Research Observer, vol. 13, no. 1 (February 1998)90
Tab
le A
.1.
Ove
rvie
w o
f Stu
dies
Eva
luat
ing
Tra
inin
g Pr
ogra
ms f
or W
orke
rs D
ispla
ced
by P
lant
Clo
sure
s and
Mas
s Lay
offs
Labo
r m
arke
t pro
blem
Rel
evan
t ind
icat
ors
Inte
rven
tion
des
ign
Typ
e of
eva
luat
ion
Res
ult
Com
men
ts
Ford
Mot
or C
o. p
lant
clos
ure
in S
an J
osé,
Cal
if. (1
982)
. Som
e2,
400
wor
kers
affe
cted
(OE
CD
199
3a).
Aut
omot
ive
plan
tcl
osur
es in
Mic
higa
n(1
980–
83).
Mor
e th
an3,
000
wor
kers
laid
off,
mos
tly e
xper
ienc
ed,
high
ly p
aid
blue
-col
lar
mal
e em
ploy
ees (
Leig
h19
92; 1
994)
.
Aut
o an
d st
eel p
lant
clos
ures
in B
uffa
lo,
N.Y
. (19
82–8
3). M
ass
layo
ffs o
f hig
hly
paid
,ex
peri
ence
d bl
ue-c
olla
rm
ale
wor
kers
(Cor
son,
A 2
5 pe
rcen
t dec
line
inau
to p
rodu
ctio
n be
-tw
een
1978
and
80.
Une
mpl
oym
ent r
ates
rose
from
7.5
per
cent
in 1
981
to 9
.5 p
erce
ntin
198
2 an
d 19
83, a
ndm
anuf
actu
ring
em
ploy
-m
ent d
eclin
ed b
y 5
perc
ent d
urin
g 19
81–
83.
A 2
5 pe
rcen
t dec
line
inau
to p
rodu
ctio
n be
-tw
een
1978
and
198
0.H
igh
unem
ploy
men
tra
tes i
n 19
81–8
2 na
-tio
nally
(9.5
per
cent
),w
hich
fell
to 7
.5 p
er-
cent
by
1984
. Man
u-fa
ctur
ing
empl
oym
ent
rose
by
5 pe
rcen
t be-
twee
n 19
83 a
nd 1
984.
Hig
h un
empl
oym
ent
rate
s in
1981
–82
na-
tiona
lly (9
.5 p
erce
nt),
whi
ch fe
ll to
7.5
per
-ce
nt b
y 19
84.
Man
ufac
turi
ng
Bas
ic sk
ills t
rain
ing
asw
ell a
s tar
gete
d vo
ca-
tiona
l tra
inin
g in
mar
-ke
tabl
e sk
ills.
Bot
h JS
A a
nd c
lass
room
trai
ning
wer
e pr
ovid
edpr
ompt
ly a
fter
pla
ntcl
osur
es. R
etra
inin
g w
asof
fere
d in
occ
upat
ions
in w
hich
ther
e w
asgr
owin
g de
man
d,in
clud
ing
blue
-col
lar
trad
es.
Dis
plac
ed w
orke
rsw
ere
prov
ided
with
both
JSA a
nd e
ither
clas
sroo
m tr
aini
ng o
rO
JT. P
rogr
am s
ervi
ces
wer
e pr
ovid
ed a
fter
a
Hig
h ra
te o
f suc
cess
inpl
acin
g w
orke
rs in
jobs
.
Cla
ssro
om tr
aini
ng d
idno
t sig
nific
antly
im-
prov
e th
e pa
rtic
ipan
ts’
reem
ploy
men
t rat
e.T
rain
ees
did
no b
ette
rth
an th
ose
rece
ivin
gJS
A.
JSA s
ervi
ces
had
a fa
irly
larg
e im
pact
on
earn
-in
gs fo
r th
e fir
st s
ixpo
st-p
rogr
am m
onth
s.H
owev
er, t
here
is n
oev
iden
ce th
at e
ither
Perc
eive
d su
cces
s w
asdu
e to
ade
quat
e re
-so
urce
bas
e ($
6,00
0gr
ant
per
wor
ker)
and
a hi
gh d
egre
eof
coo
rdin
atio
npr
ovid
ed b
y Fo
rd a
ndby
gov
ernm
ent.
Ear
ning
s es
timat
es v
ary
(ran
ging
from
neg
ativ
eto
sig
nific
antly
pos
i-tiv
e). T
rain
ing
does
not s
eem
to h
ave
been
very
eff
ectiv
e, e
spe-
cial
ly in
ligh
t of t
hefa
ct th
at tr
aini
ng c
ost
twic
e as
muc
h as
JSA.
Cla
ssro
om tr
aini
ng a
ndO
JT w
ere
inef
fect
ive.
The
se p
rogr
ams c
ost
abou
t fou
r tim
es a
sm
uch
as JS
A, i
mpl
ying
that
JSA is
pot
entia
lly
Non
scie
ntifi
c
Qua
si-
expe
rim
enta
l
Qua
si-
expe
rim
enta
l
Amit Dar and Indermit S. Gill 91
Long
, and
May
nard
1985
).
Plan
t clo
sure
s at 1
3st
eel f
acto
ries
and
min
es in
Can
ada
in th
e19
80s (
Leig
h 19
92).
Stee
l and
coa
l pla
ntcl
osur
es in
Cre
usot
-Lo
ire,
Fra
nce
(198
4).
1,23
0 pe
ople
aff
ecte
d( O
EC
D 1
993a
).
empl
oym
ent r
ose
by 5
perc
ent b
etw
een
1983
and
1984
.
Une
mpl
oym
ent r
ates
rose
bet
wee
n 19
80 a
nd83
. Em
ploy
men
t in
nona
gric
ultu
ral a
ctiv
i-tie
s fel
l slig
htly
dur
ing
the
peri
od.
Stee
l sec
tor
cont
ract
ion
resu
lted
in th
e lo
ss o
f6,
000
jobs
dur
ing
1984
. Une
mpl
oym
ent
rate
s ro
se fr
om 8
.1pe
rcen
t in
1982
to10
.2 p
erce
nt b
y 19
85.
In m
inin
g em
ploy
men
tfe
ll by
alm
ost 4
0 pe
r-ce
nt b
etw
een
1980
and
1990
. Man
ufac
turi
ngem
ploy
men
t fel
l abo
ut6
perc
ent f
rom
198
3 to
1985
.
fair
ly le
ngth
y pe
riod
of
unem
ploy
men
t.
JSA a
nd tr
aini
ng p
ro-
gram
s wer
e pr
ovid
ed.
Wor
kers
rece
ived
70
perc
ent o
f the
ir fo
rmer
sala
ries f
or 1
0 m
onth
sw
hile
enr
olle
d in
re-
trai
ning
and
job-
sear
chac
tiviti
es. T
rain
ees (
inen
gine
erin
g, p
last
icm
oldi
ng, r
efrig
erat
ion,
etc.
) wer
e pr
omise
dre
empl
oym
ent i
n th
ere
gion
. Car
eer c
ouns
el-
ing
and
shor
t cou
rses
inw
ork
skill
s, pr
oduc
tion
met
hods
for s
mal
l firm
s,
clas
sroo
m tr
aini
ng o
rO
JT h
ad a
ny in
crem
en-
tal e
ffec
t.
Part
icip
ants
had
a 7
perc
ent h
ighe
r rat
e of
empl
oym
ent t
han
com
para
bly
disp
lace
dw
orke
rs w
ho d
id n
otpa
rtic
ipat
e in
the
pro-
gram
. Thi
s im
pact
was
attr
ibut
ed to
trai
ning
inw
hich
28
perc
ent o
fw
orke
rs p
artic
ipat
ed.
But
at t
wo
min
ing
sites
,th
e pr
ogra
m h
ad n
oim
pact
.
Hig
h su
cces
s ra
te in
findi
ng jo
bs fo
r pa
rtic
i-pa
nts.
the
only
cos
t-ef
fect
ive
prog
ram
. No
evid
ence
was
pro
vide
d on
the
part
icip
ants
’ em
ploy
-ab
ility
.
Tra
inin
g se
ems t
o ha
veha
d a
grea
ter i
mpa
ctth
an JS
A. N
o in
form
a-tio
n w
as a
vaila
ble
onth
e co
st o
f tra
inin
g. J
obco
unse
ling
has l
ittle
impa
ct in
Can
ada,
whe
re su
ch se
rvic
es a
rero
utin
ely
prov
ided
toth
e un
empl
oyed
by
the
Publ
ic E
mpl
oym
ent
Serv
ice.
No
evid
ence
that
re-
trai
ning
pro
gram
spr
oduc
ed lo
ng-t
erm
empl
oym
ent b
enef
its.
Qua
si-
expe
rim
enta
l
Non
scie
ntifi
c
(Tab
le c
ontin
ues o
n th
e fo
llow
ing
page
s.)
The World Bank Research Observer, vol. 13, no. 1 (February 1998)92
and
job
sear
ch te
ch-
niqu
es w
ere
also
pro
-vi
ded.
Firm
s wer
e gi
ven
finan
cial
ince
ntiv
es to
hire
thes
e w
orke
rs.
Part
icip
ants
wer
e pr
o-vi
ded
with
cla
ssro
omtr
aini
ng.
A si
gnifi
cant
num
ber o
fw
orke
rs jo
ined
retr
ain-
ing
prog
ram
s sev
eral
mon
ths b
efor
e th
eyw
ere
laid
off.
Cou
rses
of v
aryi
ng d
urat
ion
wer
e of
fere
d in
wel
d-in
g, e
ngin
eeri
ng, a
ndco
ntro
l eng
inee
ring
.T
hese
retr
aini
ng p
ro-
gram
s wer
e pr
ovid
ed b
yth
e st
ate-
owne
d tr
ain-
ing
boar
d, m
unic
ipal
educ
atio
n in
stitu
tions
,an
d ot
her a
dult
educ
a-tio
n se
rvic
es.
Qua
si-
expe
rim
enta
l
Non
scie
ntifi
c
Part
icip
ants
rec
eive
dlo
wer
wee
kly
wag
esth
an th
ose
who
did
not
rece
ive
trai
ning
. Dro
pin
ear
ning
s w
as e
spe-
cial
ly s
igni
fican
t in
the
first
yea
r; th
ere
wer
eno
app
reci
able
long
-te
rm g
ains
.
By
Nov
embe
r 19
87,
mor
e th
an 9
0 pe
rcen
tof
the
wor
kers
who
had
com
plet
ed tr
aini
ngha
d fo
und
jobs
or
beco
me
self-
empl
oyed
—m
ost o
f the
m in
occu
patio
ns fo
r w
hich
they
had
trai
ned.
Pulp
pla
nt c
losu
re in
Kra
mfo
rs, S
wed
en(1
977)
(OE
CD
199
1).
Abo
ut 2
,000
wor
kers
laid
off
at t
he U
dde-
valla
shi
pyar
d in
Sw
e-de
n (1
985)
(A
lftha
nan
d Ja
nzon
199
4).
Une
mpl
oym
ent r
ates
in S
wed
en r
ose
from
1.6
perc
ent i
n 19
76 to
2.2
perc
ent b
y 19
78.
Em
ploy
men
t in
man
u-fa
ctur
ing
decl
ined
by
8pe
rcen
t bet
wee
n 19
77an
d 19
79.
Vol
vo C
o. s
et u
p a
man
ufac
turi
ng p
lant
inU
ddev
alla
. Une
mpl
oy-
men
t rat
es w
ere
decl
in-
ing
in th
is p
erio
d w
hile
man
ufac
turi
ng e
m-
ploy
men
t was
fair
lyst
eady
.
No
info
rmat
ion
pro-
vide
d on
ree
mpl
oy-
men
t rat
es o
r co
sts.
Ben
efits
from
ret
rain
-in
g pr
ogra
m w
ere
insi
gnifi
cant
.
Tw
o fa
ctor
s ac
coun
ted
for
the
succ
ess
of th
etr
aini
ng p
rogr
am:
econ
omic
and
labo
rm
arke
t con
ditio
nsw
ere
buoy
ant i
n th
ere
gion
, and
man
age-
men
t, em
ploy
men
tof
fices
, and
trai
ning
agen
cies
wor
ked
incl
ose
coop
erat
ion.
Tab
le A
.1.
(con
tinu
ed)
Labo
r m
arke
t pro
blem
Rel
evan
t ind
icat
ors
Inte
rven
tion
des
ign
Typ
e of
eva
luat
ion
Res
ult
Com
men
ts
Amit Dar and Indermit S. Gill 93
A re
trai
ning
pro
gram
was
des
igne
d to
hel
p th
eex
istin
g w
ork
forc
em
anag
e th
e ch
ange
with
out j
ob lo
sses
. The
com
pany
acc
epte
d th
epr
opos
al u
nder
the
stip
ulat
ion
that
the
cost
be sh
ared
by
the
gove
rn-
men
t. T
he p
rogr
am w
asa
broa
d co
mpe
tenc
y-ra
ising
exe
rcise
, with
spec
ific
trai
ning
topr
epar
e th
e pa
rtic
ipan
tsfo
r the
pro
duct
ion
ofth
e ne
w a
utom
obile
s.
Thi
s pr
ogra
m (
1986
–89
) tr
aine
d w
omen
entr
epre
neur
s to
hel
pth
em s
tart
thei
r ow
nbu
sine
sses
. A to
tal o
f20
0 ho
urs
of s
peci
ficbu
sine
ss-o
rien
ted
trai
ning
was
pro
vide
d.
Non
scie
ntifi
c
Non
scie
ntifi
c
Prog
ram
has
not
yet
been
eva
luat
ed.
Fift
y-on
e bu
sine
sses
wer
e se
t up
in 1
989.
Less
than
a th
ird
of th
epa
rtic
ipan
ts o
pene
d a
full-
time
busi
ness
, and
few
hir
ed a
ny e
mpl
oy-
ees,
thus
the
tota
lad
ditio
nal e
mpl
oym
ent
gene
rate
d w
as lo
w.
Vol
vo C
o. p
lant
at
Göt
ebor
g, S
wed
en,
anno
unce
d th
at 1
,000
wor
kers
wou
ld b
e la
idof
f in
1992
as
it ph
ased
out a
n ol
d m
odel
and
tool
ed u
p fo
r a
new
one
(Alft
han
and
Janz
on 1
994)
.
Ship
yard
clo
sure
inSt
orst
rom
cou
nty,
Den
mar
k (1
986)
.2,
000
peop
le lo
st th
eir
jobs
(O
EC
D 1
993a
).
Vol
vo p
lann
ed to
recr
uit 8
00 w
orke
rs to
man
ufac
ture
the
new
line
of c
ars.
Une
m-
ploy
men
t rat
es w
ere
risi
ng s
harp
ly—
from
3.2
perc
ent i
n 19
91 to
5.9
perc
ent i
n 19
92.
Man
ufac
turi
ng e
m-
ploy
men
t dro
pped
by
9 pe
rcen
t in
1992
.T
otal
em
ploy
men
t fel
lby
4 p
erce
nt.
Hig
h un
empl
oym
ent
rate
s in
reg
ion,
esp
e-ci
ally
am
ong
wom
en.
Nat
iona
l une
mpl
oy-
men
t rat
es fo
r m
en in
1986
and
198
7 w
ere
6.1
and
6.4
perc
ent,
resp
ectiv
ely,
whi
le th
eco
rres
pond
ing
num
-be
rs fo
r w
omen
wer
e10
and
9.6
per
cent
.E
mpl
oym
ent i
n m
anu-
fact
urin
g w
as s
tagn
ant
from
the
mid
-198
0s.
Prog
ram
cos
ts a
reex
pect
ed to
be
abou
t$2
5 m
illio
n ($
25,0
00pe
r pe
rson
), c
lose
toha
lf of
whi
ch w
ill b
epa
id fo
r by
the
gove
rn-
men
t. In
judg
ing
the
cost
-eff
ectiv
enes
s of
this
pro
gram
, it s
houl
dbe
not
ed th
at th
e un
-em
ploy
men
t ben
efits
the
gove
rnm
ent w
ould
have
had
to p
ay
wou
ld a
mou
nt to
$6.5
mill
ion.
The
pro
gram
did
not
seem
to b
e su
cces
sful
and
was
app
aren
tlyqu
ite c
ostly
(re
liabl
eco
st e
stim
ates
are
not
avai
labl
e).
(Tab
le c
ontin
ues o
n th
e fo
llow
ing
page
.)
The World Bank Research Observer, vol. 13, no. 1 (February 1998)94
Swed
en, 1
980s
and
1990
s: g
ener
al e
valu
a-tio
n of
pub
lic r
etra
in-
ing
prog
ram
s ( O
EC
D
1991
).
Aut
omob
ile p
lant
clos
ure
in A
ustr
alia
in19
84. 4
45 w
orke
rs la
idof
f (Le
igh
1992
).
Une
mpl
oym
ent r
ose
stea
dily
ove
r th
epe
riod
.
Une
mpl
oym
ent r
ates
rose
ste
adily
in A
ustr
a-lia
, rea
chin
g a
high
of
9 pe
rcen
t in
1984
befo
re fa
lling
to 8
perc
ent f
or th
e ne
xtfe
w y
ears
. Bet
wee
n19
80 a
nd 1
984
em-
ploy
men
t in
man
u-fa
ctur
ing
fell
by 4
perc
ent.
Var
ious
type
s
Labo
r A
djus
tmen
tT
rain
ing
Arr
ange
men
tof
fere
d cl
assr
oom
trai
ning
(av
erag
ele
ngth
19
wee
ks).
Mai
n di
stin
ctio
n be
-tw
een
cour
ses
was
whe
ther
they
pro
vide
ddr
iver
trai
ning
or
not.
Var
ious
type
s
Qua
si-
expe
rim
enta
l
Ret
rain
ing
prog
ram
sha
ve b
ecom
e le
ss e
ffec
-tiv
e ov
er ti
me—
espe
-ci
ally
sin
ce th
eec
onom
y ha
s be
gun
dete
rior
atin
g. P
artic
i-pa
nts
have
had
mor
etr
oubl
e fin
ding
jobs
than
oth
er u
nem
-pl
oyed
wor
kers
.
Ove
r a
nine
-mon
thpe
riod
, dri
ver
trai
ning
incr
ease
d th
e pr
obab
il-ity
of r
eem
ploy
men
t.H
owev
er, f
ewer
trai
n-ee
s of
oth
er tr
aini
ngco
urse
s fou
nd jo
bs.
The
cos
t-ef
fect
iven
ess
of th
ese
prog
ram
sde
clin
ed b
oth
beca
use
they
wer
e le
ss s
ucce
ss-
ful a
nd b
ecau
se th
eco
sts
incr
ease
d.
The
re w
ere
no d
ata
show
ing
the
cost
-ef
fect
iven
ess
of d
iffer
-en
t typ
es o
f tra
inin
gco
urse
s. S
elf-
sele
ctio
nw
as a
pro
blem
bec
ause
indi
vidu
als
who
cho
seno
t to
part
icip
ate
wer
ein
the
cont
rol g
roup
.T
he im
pact
of l
onge
rtr
aini
ng w
as n
egat
ive.
Not
e: JS
A, j
ob s
earc
h as
sist
ance
; OJT
, on-
the-
job
trai
ning
.
Tab
le A
.1.
(con
tinu
ed)
Labo
r m
arke
t pro
blem
Rel
evan
t ind
icat
ors
Inte
rven
tion
des
ign
Typ
e of
eva
luat
ion
Res
ult
Com
men
ts
Amit Dar and Indermit S. Gill 95
Tab
le A
.2.
Ove
rvie
w o
f Stu
dies
Eva
luat
ing
Tra
inin
g Pr
ogra
ms f
or th
e Lo
ng-T
erm
Une
mpl
oyed
Labo
r m
arke
t pro
blem
Rel
evan
t ind
icat
ors
Inte
rven
tion
des
ign
Typ
e of
eva
luat
ion
Res
ult
Com
men
ts
Tra
inin
g fo
r wor
kers
elig
ible
for J
PTA T
itle
III
prog
ram
(disp
lace
dw
orke
r) in
Hou
ston
,T
exas
(198
3–85
).E
ligib
le in
divi
dual
sw
ere
unem
ploy
ed a
ndha
d a
low
pro
babi
lity
ofre
turn
ing
to th
eir p
revi
-ou
s occ
upat
ion
orin
dust
ry (B
loom
199
0).
Tra
inin
g fo
r w
orke
rsel
igib
le fo
r JP
TA T
itle
III
prog
ram
(di
spla
ced
wor
ker)
in E
l Pas
o,T
exas
(19
83–8
5).
(Cri
teri
a sa
me
asab
ove.
) W
orke
rs la
idof
f fro
m li
ght m
anu-
fact
urin
g pl
ants
(Blo
om 1
990)
.
A d
eclin
e in
pet
ro-
chem
ical
indu
stry
led
to la
yoff
. U.S
. une
m-
ploy
men
t rat
es d
e-cl
ined
from
9.5
per
-ce
nt in
198
3 to
7.4
perc
ent i
n 19
85 (
and
mai
ntai
ned
this
tren
dun
til th
e la
te 1
980s
).E
mpl
oym
ent i
n th
isin
dust
ry in
crea
sed
slig
htly
bet
wee
n 19
83an
d 19
85.
U.S
. une
mpl
oym
ent
rate
s de
clin
ed fr
om 9
.5pe
rcen
t in
1983
to 7
.4pe
rcen
t in
1985
(an
dm
aint
aine
d th
is tr
end
until
the
late
198
0s).
Em
ploy
men
t in
man
u-fa
ctur
ing
fell
by c
lose
to 2
per
cent
bet
wee
n19
82 a
nd 1
983
but
rose
to a
bout
5 p
erce
ntin
198
5.
Tex
as W
AD
pro
ject
.D
ispla
ced
wor
kers
wer
epr
ovid
ed w
ith JS
A o
r am
ix o
f JSA
with
cla
ss-
room
trai
ning
.
Tex
as W
AD
pro
ject
.D
ispla
ced
wor
kers
prov
ided
with
JSA a
lone
or c
ombi
ned
with
clas
sroo
m tr
aini
ng.
Exp
erim
enta
l
Exp
erim
enta
l
One
yea
r af
ter
the
prog
ram
’s co
mpl
etio
n,pa
rtic
ipan
ts r
egis
tere
dno
add
ition
al e
arni
ngga
in w
hen
com
pare
dw
ith th
e sa
mpl
e of
wor
kers
who
had
bee
ngi
ven
only
JSA.
One
yea
r af
ter
the
prog
ram
’s co
mpl
etio
n,th
is p
rogr
am r
ecor
ded
no e
ffec
t on
mal
eem
ploy
ees’
earn
ings
but d
id in
crea
se th
eea
rnin
gs o
f wom
en.
Des
pite
the
high
cos
tsof
cla
ssro
om tr
aini
ng(t
wic
e as
muc
h as
JSA),
no a
dditi
onal
gai
nsac
crue
d fr
om th
is ty
peof
trai
ning
.
The
incr
ease
in e
arn-
ings
of f
emal
e pa
rtic
i-pa
nts
was
slig
htly
mor
eth
an th
e pr
ogra
m’s
cost
s. M
ale
empl
oyee
sre
port
ed n
o be
nefic
ial
effe
cts.
(Tab
le c
ontin
ues o
n th
e fo
llow
ing
page
s.)
The World Bank Research Observer, vol. 13, no. 1 (February 1998)96
Ass
ista
nce
aim
ed a
t the
long
-ter
m u
nem
ploy
edin
New
Jer
sey
(198
6–87
). I
n ge
nera
l, w
ork-
ers
wer
e la
id o
ff fr
omm
anuf
actu
ring
, tra
de,
and
serv
ices
(C
orso
nan
d ot
hers
198
9).
Long
-ter
m u
nem
-pl
oyed
wor
kers
in th
eU
.S. (
1988
), p
rim
arily
resu
lting
from
pla
ntcl
osur
es (
Leig
h 19
92).
Une
mpl
oym
ent r
ates
in th
e U
.S. f
ell f
rom
7.1
perc
ent i
n 19
85 to
6.1
perc
ent b
y 19
87.
Whi
le e
mpl
oym
ent i
nm
anuf
actu
ring
re-
mai
ned
fair
ly s
tead
ybe
twee
n 19
85 a
nd19
88, t
rade
em
ploy
-m
ent i
ncre
ased
7 p
er-
cent
and
ser
vice
-sec
tor
empl
oym
ent r
ose
abou
t 5 p
erce
nt.
Une
mpl
oym
ent r
ates
in th
e U
.S. f
ell f
rom
7.1
perc
ent i
n 19
85 to
6.1
perc
ent b
y 19
87.
Whi
le e
mpl
oym
ent i
nm
anuf
actu
ring
re-
mai
ned
fair
ly s
tead
ybe
twee
n 19
85 a
nd19
88, t
rade
em
ploy
-m
ent i
ncre
ased
7 p
er-
cent
and
that
in s
er-
vice
s ab
out 5
per
cent
.
New
Jer
sey
UI
Ree
m-
ploy
men
t Dem
onst
ra-
tion
proj
ect.
Thi
spr
ogra
m p
rovi
ded
disp
lace
d w
orke
rs w
ithJS
A a
lone
, or
follo
wed
by O
JT, c
lass
room
trai
ning
(C
T),
or
are
empl
oym
ent b
onus
.
The
Tra
de A
djus
tmen
tA
ssis
tanc
e Pr
ogra
mw
as in
tend
ed to
de-
velo
p w
orke
rs’ s
kills
inne
w o
ccup
atio
ns. M
ost
of th
ese
skill
s w
ere
supp
lied
by a
voc
a-tio
nal c
olle
ge o
r lo
cal
com
mun
ity c
olle
ge in
cour
ses
that
wer
e m
ore
than
a y
ear
long
.
Exp
erim
enta
l
Qua
si-
expe
rim
enta
l
Mor
e th
an 1
0 qu
arte
rsaf
ter
the
prog
ram
ende
d, b
oth
CT
and
OJT
sig
nific
antly
in-
crea
sed
earn
ings
ove
rth
ose
who
rec
eive
d JS
A
only
. The
se in
divi
dual
sw
ere
also
em
ploy
ed fo
rlo
nger
per
iods
of t
ime
in e
ach
quar
ter
than
the
JSA-o
nly
grou
p.
Indi
vidu
als
who
re-
ceiv
ed tr
aini
ng b
egan
earn
ing
sign
ifica
ntly
mor
e th
an th
ose
who
rece
ived
ext
ende
din
com
e-m
aint
enan
cebe
nefit
s by
the
6th
quar
ter
and
cont
inue
dto
ear
n m
ore
until
the
12th
(la
st)
quar
ter,
reac
hing
a le
vel o
f$5
00 a
qua
rter
.
Whi
le th
e re
sults
of
trai
ning
wer
e po
sitiv
e,th
e fo
llow
ing
cave
ats
appl
y: th
e tr
aine
es w
ere
self-
sele
cted
(th
us th
ese
resu
lts m
ay n
ot a
pply
for
a ra
ndom
gro
up o
fcl
aim
ants
); o
nly
15pe
rcen
t of t
hose
of-
fere
d tr
aini
ng a
ccep
ted
it; a
nd tr
aini
ng b
en-
efite
d th
ose
who
al-
read
y ha
d m
arke
tabl
esk
ills.
Fin
ally
, ana
lysi
ssh
owed
that
the
cost
sex
ceed
ed th
e ex
pect
edbe
nefit
s.
The
ana
lysi
s w
as li
m-
ited
to th
e m
anuf
actu
r-in
g se
ctor
. Lon
g-te
rmin
vest
men
ts in
trai
ning
may
be
effe
ctiv
e in
incr
easi
ng e
arni
ngs,
but t
he tr
aini
ng is
cost
ly (
each
trai
nee
was
give
n a
$12,
000
trai
n-in
g vo
uche
r).
Tab
le A
.2.
(con
tinu
ed)
Labo
r m
arke
t pro
blem
Rel
evan
t ind
icat
ors
Inte
rven
tion
des
ign
Typ
e of
eva
luat
ion
Res
ult
Com
men
ts
Amit Dar and Indermit S. Gill 97
Long
-ter
m u
nem
-pl
oyed
wor
kers
inC
anad
a (G
oss
and
asso
ciat
es 1
989)
.
Hig
h re
gion
al u
nem
-pl
oym
ent l
evel
s in
Ger
man
y in
the
late
1970
s ( O
EC
D 1
991)
.
Indi
vidu
als
at r
isk
ofbe
ing
unem
ploy
ed in
1987
–88
in G
erm
any
(Joh
anso
n 19
94).
Com
pare
d w
ith th
eU
.S.,
unem
ploy
men
tra
tes h
ave
been
fairl
yhi
gh in
Can
ada.
How
-ev
er th
ey d
eclin
ed fr
om11
.2 p
erce
nt in
the
mid
-19
80s t
o 7.
5 pe
rcen
t in
1989
. Spu
rred
by
grow
th in
com
mer
cean
d se
rvic
es, e
mpl
oy-
men
t gre
w a
bout
3pe
rcen
t ann
ually
ove
rth
e tim
e pe
riod.
In 2
3 of
the
142
re-
gion
s in
Ger
man
y,un
empl
oym
ent l
evel
sw
ere
abov
e 6
perc
ent.
Som
e fir
ms
wer
e al
sofa
cing
ser
ious
pro
blem
sin
adj
ustin
g to
eco
-no
mic
cha
nges
.
Une
mpl
oym
ent r
ates
wer
e st
eady
bet
wee
n19
86 a
nd 1
988.
The
yfe
ll so
mew
hat b
y 19
90.
Em
ploy
men
t gre
w a
tsl
ight
ly m
ore
than
1pe
rcen
t dur
ing
this
peri
od.
The
Job
Dev
elop
men
tPr
ogra
m p
rovi
ded
clas
sroo
m tr
aini
ng a
ndO
JT. W
ages
and
dir
ect
cost
s of
cla
ssro
omtr
aini
ng w
ere
subs
i-di
zed.
The
wag
e su
b-si
dy h
elpe
d em
ploy
ers
cove
r th
e co
st o
f OJT
.
Am
ong
othe
r in
terv
en-
tions
, tra
inin
g an
dre
trai
ning
of u
nem
-pl
oyed
in fi
rms.
The
firm
s w
ho e
mpl
oyed
thes
e tr
aine
es r
ecei
ved
a su
bsid
y of
90
perc
ent
of th
e em
ploy
ees’
wag
es fo
r tw
o ye
ars.
Thi
s is
an
eval
uatio
n of
four
pro
gram
s, tw
o of
whi
ch tr
aine
d w
orke
rsfo
r ne
w o
ccup
atio
ns(o
ne o
ffer
ed fu
rthe
rtr
aini
ng fo
r em
ploy
edan
d un
empl
oyed
indi
-vi
dual
s, w
hile
the
othe
rof
fere
d re
trai
ning
for
the
unem
ploy
ed).
Qua
si-
expe
rim
enta
l
Non
scie
ntifi
c
Qua
si-
expe
rim
enta
l
Em
ploy
abili
ty o
fw
omen
ros
e w
hile
that
of m
en d
eclin
ed.
Wee
kly
earn
ings
of
wom
en w
ere
insi
gnifi
-ca
nt r
elat
ive
to th
at o
fth
e co
ntro
l gro
up,
whi
le th
ey w
ere
low
erfo
r m
ales
.
Tra
inin
g re
duce
dun
empl
oym
ent s
ome-
wha
t, bu
t it i
s es
ti-m
ated
that
mor
e th
an40
per
cent
of t
hese
hard
-to-
plac
e in
divi
du-
als
had
alre
ady
left
thei
r jo
bs b
y 19
81.
No
type
of t
rain
ing
had
any
sign
ifica
ntim
pact
on
the
flow
sou
t of s
hort
- or
long
-te
rm u
nem
ploy
men
t,or
on
the
flow
s in
toun
empl
oym
ent.
In v
iew
of t
he h
igh
prog
ram
cos
ts (
arou
nd$9
,300
per
par
tici-
pant
), th
is tr
aini
ng is
not c
ost-
effe
ctiv
e,es
peci
ally
for
men
.
In li
ght o
f the
ext
rem
ely
high
cos
t (ar
ound
$50
0m
illio
n fo
r tra
inin
g an
dot
her i
nter
vent
ions
), th
ere
sults
wer
e ve
ry d
isap-
poin
ting.
No
info
rma-
tion
was
pro
vide
d on
wag
es.
No
info
rmat
ion
onw
ages
or
cost
s of
trai
ning
.
(Tab
le c
ontin
ues o
n th
e fo
llow
ing
page
s.)
The World Bank Research Observer, vol. 13, no. 1 (February 1998)98
Tab
le A
.2.
(con
tinu
ed)
Labo
r m
arke
t pro
blem
Rel
evan
t ind
icat
ors
Inte
rven
tion
des
ign
Typ
e of
eva
luat
ion
Res
ult
Com
men
ts
Indi
vidu
als
who
wer
eei
ther
at r
isk
of b
ecom
-in
g un
empl
oyed
or
wer
e al
read
y un
em-
ploy
ed in
Ger
man
y in
the
earl
y 19
80s
(Joh
anso
n 19
94).
Hel
p fo
r lo
ng-t
erm
unem
ploy
ed a
nd d
is-
plac
ed w
orke
rs in
Eng
land
in th
e 19
80s
(Add
ison
and
Sie
bert
1994
).
Une
mpl
oym
ent r
ates
rem
aine
d fa
irly
ste
ady
at ju
st o
ver
10 p
erce
ntin
the
earl
y 19
80s.
Hig
h un
empl
oym
ent
rate
s (a
roun
d 10
–11
perc
ent)
per
sist
edth
roug
h m
uch
of th
e19
80s.
Bet
wee
n 19
83an
d 19
90 e
mpl
oym
ent
rose
by
arou
nd 1
.5pe
rcen
t.
Tra
inin
g pr
epar
edw
orke
rs fo
r ce
rtifi
ca-
tion
in o
ne o
f the
375
appr
entic
eabl
e tr
ades
.T
he p
rogr
am r
equi
red
up to
two
year
s to
com
plet
e. C
onte
nts
and
spec
ifica
tion
ofre
trai
ning
cor
resp
ond
to th
ose
of in
itial
voc
a-tio
nal t
rain
ing.
Thi
s co
mm
unity
-bas
edpr
ogra
m p
rovi
ded
voca
tiona
l tra
inin
g by
loca
l aut
hori
ties
inco
njun
ctio
n w
ith lo
cal
colle
ges,
cen
tral
gov
-er
nmen
t, an
d vo
lun-
tary
org
aniz
atio
ns.
Litt
le p
riva
te s
ecto
rin
volv
emen
t.
Non
scie
ntifi
c
Non
scie
ntifi
c
The
ove
rall
succ
ess
rate
, mea
sure
d in
term
sof
ret
entio
n ra
tes,
was
54 p
erce
nt. E
mpl
oy-
abili
ty v
arie
d by
age
—le
ss th
an h
alf o
f tho
seov
er 4
5 ye
ars
old
who
wer
e un
empl
oyed
mor
e th
an o
ne y
ear
foun
d jo
bs, w
hile
86
perc
ent o
f 25-
to 3
5-ye
ar-o
lds
did.
Litt
le im
pact
on
flow
s fr
om lo
ng-t
erm
unem
ploy
men
t.
No
data
on
cost
wer
eav
aila
ble.
The
num
ber
of w
orke
rs w
ho le
ftth
eir
jobs
was
fair
lyhi
gh: t
wo
year
s af
ter
com
plet
ing
trai
ning
only
60
perc
ent o
f the
men
and
66
perc
ent o
fth
e w
omen
wer
e st
illem
ploy
ed.
The
cos
t-ef
fect
iven
ess
of th
ese
prog
ram
s is
likel
y to
be
nega
tive.
Amit Dar and Indermit S. Gill 99
Impr
ove
skill
s an
dpr
ovid
e be
tter
acc
ess
tojo
bs fo
r th
e lo
ng-t
erm
unem
ploy
ed in
Tilb
urg,
the
Net
her-
land
s (O
EC
D 1
993a
).
Job
trai
ning
pro
gram
sfo
r un
empl
oyed
and
disp
lace
d w
orke
rs in
the
Net
herl
ands
(O
EC
D
1994
b).
Une
mpl
oym
ent r
ates
fell
from
abo
ut 1
3pe
rcen
t in
the
earl
y19
80s
to b
elow
10
perc
ent b
y th
e en
d of
the
deca
de.
Une
mpl
oym
ent r
ates
fell
from
abo
ut 1
3pe
rcen
t in
the
earl
y19
80s
to b
elow
10
perc
ent b
y th
e en
d of
the
deca
de.
Tra
inin
g pr
ovid
edha
nds-
on e
xper
ienc
e(t
hrou
gh a
sim
ulat
edw
orks
hop)
. The
em
pha-
sis w
as o
n te
chni
cal
skill
s (m
etal
wor
king
,w
oodw
orki
ng, i
nsta
lla-
tion
tech
niqu
es, a
ndap
pare
l tra
des)
as w
ell
as o
n go
od w
ork
habi
ts.
Cou
rses
ran
from
4 to
10 m
onth
s.
Cen
ters
for
adul
t voc
a-tio
nal t
rain
ing.
Non
scie
ntifi
c
Qua
si-
expe
rim
enta
l
In 1
991,
of t
he 8
2in
divi
dual
s enr
olle
d in
the
cour
se, 5
2 co
m-
plet
ed it
and
41
foun
djo
bs o
r wen
t on
for
furt
her e
duca
tion.
The
mos
t suc
cess
ful t
rain
ing
was
in m
etal
wor
king
.
Une
mpl
oyed
peo
ple
who
wer
e no
t in
the
prog
ram
foun
d jo
bs a
squ
ickl
y as
thos
e w
how
ere.
Eve
n tw
o ye
ars
late
r, th
e em
ploy
men
tsi
tuat
ion
for
the
two
grou
ps w
as n
ot s
igni
fi-ca
ntly
diff
eren
t.
Tra
inin
g w
as q
uite
expe
nsiv
e: a
nnua
lfu
ndin
g of
the
pro-
gram
was
abo
ut $
7.5
mill
ion,
or
$10,
000
per
trai
nee.
No
data
wer
e av
aila
ble
on w
ages
and
cos
ts,
but t
rain
ing
seem
s to
have
bee
n in
effe
ctiv
e in
term
s of
incr
easi
ngth
e em
ploy
abili
ty o
fpa
rtic
ipan
ts.
Not
e: JS
A, j
ob s
earc
h as
sist
ance
; OJT
, on-
the-
job
trai
ning
.
The World Bank Research Observer, vol. 13, no. 1 (February 1998)100
References
The word “processed” describes informally reproduced works that may not be commonly availablethrough library systems.
Addison, J. T., and W. S. Siebert. 1994. “Vocational Training and the European Community.”Oxford Economic Papers 46(4):696–724.
Alfthan, Torkel, and B. Janzon. 1994. Retraining Adult Workers in Sweden. Training Policy Study 3.Geneva: International Labour Office.
Anderson, Patricia, Walter Corson, and Paul Decker. 1991. “The New Jersey Unemployment In-surance Reemployment Demonstration Project: Follow up Report.” U.S. Department of LaborUnemployment Insurance Survey. U.S. Department of Labor, Washington, D.C., Processed.
Bloom, Howard. 1990. “Back to Work: Testing Reemployment Services for Displaced Workers.”W. E. Upjohn Institute for Employment Research, Kalamazoo, Mich. Processed.
Corson, Walter, Stephen Long, and Rebecca Maynard. 1985. “An Impact Evaluation of the BuffaloDislocated Worker Demonstration Program.” Mathematica Policy Research. New Jersey. Pro-cessed.
Corson, Walter, S. Dunstan, Paul Decker, and A. Gordon. 1989. “New Jersey UnemploymentInsurance Reemployment Demonstration Project.” Unemployment Insurance Occasional Paper89-3. U.S. Department of Labor, Washington, D.C. Processed.
Gill, Indermit S., and Amit Dar. 1995. “Costs and Effectiveness of Retraining in Hungary.” Inter-nal Discussion Paper IDP-155. World Bank, Europe and Central Asia Region, Washington, D.C.Processed.
Goss, Gilroy, and Associates. 1989. “Evaluation of the Job Development Program: Final Report.”Report prepared for the Program Evaluation Branch, Canadian Employment and ImmigrationOffice, Ottawa. Processed.
Grossman, Jean B. 1994. “Evaluating Social Policies: Principles and U.S. Experience.” The WorldBank Research Observer 9(2):159–80.
Heckman, James J. 1979. “Sample Selection Bias as a Specification Error.” Econometrica 47(1):153–62.
———. 1992. “Randomization and Social Policy Evaluation.” In Charles Manski and IrwinGarfinkel, eds, Evaluating Welfare and Training Programs. Cambridge, Mass.: Harvard Univer-sity Press.
Johanson, Richard. 1994. Retraining Adults in Germany. Training Policy Study 4. Geneva: Interna-tional Labour Office.
LaLonde, Robert. 1986. “Evaluating the Econometric Evaluations of Training Programs with Ex-perimental Data.” American Econometric Review 76(4):604–20.
Leigh, Duane. 1992. “Retraining Displaced Workers: What Can Developing Nations Learn fromOECD Nations?” Policy Research Working Paper 946. World Bank, Population and Human Re-sources Department, Washington, D.C. Processed.
———. 1994. Retraining Displaced Workers: The U.S. Experience. Training Policy Study 1. Geneva:International Labour Office.
Newman, John, Laura Rawlings, and Paul Gertler. 1994. “Using Randomized Control Designs inEvaluating Social Sector Programs in Developing Countries.” The World Bank Research Observer9(2):181–201.
OECD (Organisation for Economic Co-operation and Development). 1991. Evaluating Labor Mar-ket and Social Programs: The State of a Complex Art. Paris.
Amit Dar and Indermit S. Gill 101
———. 1993a. Partnerships: The Key to Job Creation. Experiences from OECD Countries. Paris.
———. 1993b. Employment Outlook. Paris.
———. 1994a. The OECD Jobs Study: Evidence and Explanations. Paris.———. 1994b. Vocational Training in the Netherlands: Reform and Innovation. Paris.
O’Leary, Christopher. 1995. “An Impact Analysis of Labor Market Programs in Hungary.” W. E.Upjohn Institute for Employment Research, Kalamazoo, Mich. Processed.
Pulay, Gyula. 1995. “Measuring the Efficiency of Employment Programs in Hungary.” Paper pre-pared for a seminar on Evaluating Active Labor Programs. World Bank, Eastern Europe Depart-ment, Washington, D.C. Processed.
Revenga, Ana, Michelle Riboud, and Hong Tan. 1994. “The Impact of Mexico’s Retraining Pro-gram on Employment and Wages.” The World Bank Economic Review 8(2):247–77.
World Bank. 1995. World Development Report 1995: Workers in an Integrating World. New York:Oxford University Press.