Upload
hathuy
View
232
Download
0
Embed Size (px)
Citation preview
Introduction Origins Origins Modern References
Hard Evidence on Soft Skills
James J. Heckman and Tim KautzUniversity of Chicago
Development Economics Vice Presidency (DEC) LectureWorld Bank
Washington, DCDecember 15, 2011
This draft, December 18, 2011
Heckman, Kautz Hard Evidence on Soft Skills 1 / 125
Introduction Origins Origins Modern References
We live in an era of widespread testing.
Achievement tests have assumed a prominent role.
They are used to:i Measure skills of persons (e.g. SAT, ACT, GRE) and certify
suitability for admission and qualifications in a variety ofdomains of life.
ii Measure the performance of schools and entire national schoolsystems and nations (e.g. PISA scores, NCLB)
These tests are not well understood.
Heckman, Kautz Hard Evidence on Soft Skills 2 / 125
Introduction Origins Origins Modern References
This lecture is about these tests and the consequences ofrelying uncritically on them.
a How predictive are they?b What do they measure?c How are they validated?d What do they miss?
Heckman, Kautz Hard Evidence on Soft Skills 3 / 125
Introduction Origins Origins Modern References
The short answer is that they miss—or perhaps better do notaccurately capture—soft skills—personality traits, goals,motivations and preferences that are valued in the labormarket, in school, and in many other domains.
Heckman, Kautz Hard Evidence on Soft Skills 4 / 125
Introduction Origins Origins Modern References
A Multiple skills are useful in predicting and causing economicand social success but with different weights in different tasks.People differ in their endowments of these skills. Thesedifferences in endowments and values of endowments acrosstasks give rise to comparative advantage and sorting in thelabor market.
B Personality traits—“soft skills”—are important skills that getneglected by the current emphasis on achievement tests inevaluating schools and screening people.
i Soft skills can be measured.ii They are predictive of many life outcomes—sometimes as
strongly as measures of cognition.iii Establishing the causal status of personality traits is not an
easy task, but there is evidence on the question.
Heckman, Kautz Hard Evidence on Soft Skills 5 / 125
Introduction Origins Origins Modern References
C Ignoring personality traits deceives. Doing so distorts statisticsand conceals major social problems.
D Traits are stable across situations. Yet they can also bechanged in a gradual way through investment. Enhancing thesetraits is an important policy option.
E No evidence for extreme situational specificity of the sortadvocated by Walter Mischel (1968), which still plays aprominent role in modern behavioral economics. There are“enduring traits” that persist across domains of economic andsocial life.
Heckman, Kautz Hard Evidence on Soft Skills 6 / 125
Introduction Origins Origins Modern References
Origins of Achievement Testing: A Brief Review
Mass application of IQ testing fostered the creation of tests ofother psychological traits and of the knowledge learned inschool.
Heckman, Kautz Hard Evidence on Soft Skills 7 / 125
Introduction Origins Origins Modern References
Many creators of the IQ test saw its limitations.
Heckman, Kautz Hard Evidence on Soft Skills 8 / 125
Introduction Origins Origins Modern References
Binet:
“[Success in school] . . .admits of other things thanintelligence; to succeed in his studies, one must havequalities which depend on attention, will, and character;for example a certain docility, a regularity of habits, andespecially continuity of effort. A child, even if intelligent,will learn little in class if he never listens, if he spends histime in playing tricks, in giggling, in playing truant.”
-Binet (1916, p. 254)
Heckman, Kautz Hard Evidence on Soft Skills 9 / 125
Introduction Origins Origins Modern References
“What are the chief personality traits which, interactingwith g, relate to individual differences in achievement andvocational success? The most universal personality trait isconscientiousness, that is, being responsible, dependable,caring, organized and persistent.”
-Jensen (1998, p. 575)
Heckman, Kautz Hard Evidence on Soft Skills 10 / 125
Introduction Origins Origins Modern References
Figure 1: Modern View of “g”: An Hierarchical Scheme of GeneralIntelligence and Its Components
General
Intelligence
Gf
(Fluid Intelligence)Sequential Reasoning
Inductive Reasoning
Quantitative Reasoning
Piagetian Reasoning
Math ReasoningQuantitative Reasoning
Math Problems
Visual PerceptionVisualization
Spatial Relations
Closure Speed
Closure Flexibility
Serial Perceptual Integration
Spatial Scanning
Imagery
ClosureClosure Speed
Closure Flexibility
Perceptual SpeedNumber Computation
RT and other Elementary Cognitive Tasks
Stroop
Clerical Speed
Digit/Symbol
Learning and MemoryMemory Span
Associative Memory
Free Recall Memory
Meaningful Memory
Visual Memory
Knowledge and AchievementGeneral School Achievement
Verbal Information and Knowledge
Information and Knowledge, Math and Science
Technical and Mechanical Knowledge
Knowledge of Behavioral Content
Ideational FluencyIdeational Fluency
Naming Facility
Expressional Fluency
Word Fluency
Creativity
Figural Fluency
Figural Flexibility
Gc
(Crystallized Intelligence)Verbal Comprehension
Lexical Knowledge
Reading Comprehension
Reading Speed
“Cloze”
Spelling
Phonetic Coding
Grammatical Sensitivity
Foreign Language
Communication
Listening
Oral Production
Oral Style
Writing
Source: Recreated from Ackerman and Heggestad (1997), based on Carroll(1993).
Heckman, Kautz Hard Evidence on Soft Skills 11 / 125
Introduction Origins Origins Modern References
Origins of the Modern Achievement Test
Achievement tests were created in the wake of the success andwidespread acceptance of the IQ test as a way to capture theknowledge acquired in schools and in general life.
Heckman, Kautz Hard Evidence on Soft Skills 12 / 125
Introduction Origins Origins Modern References
General Knowledge
Achievement test invented as a way to measure“general knowledge.”—“useful knowledge” valuablein functioning at work and in society, not specific knowledgetaught in a course.
Designed to be “objective”—not depend on teacherassessments as captured by grades. This was perceived to be away to implement meritocratic notions of education.
Iowa tests; ACT; GED; No Child Left Behind; NAEP; PISAtests are modern versions.
Heckman, Kautz Hard Evidence on Soft Skills 13 / 125
Introduction Origins Origins Modern References
A model of Academic Achievement at age t:
At︸︷︷︸Academic
Achievementat t
= φt( Kt︸︷︷︸CumulatedInvestment(Schooling,Parenting)
, Ct︸︷︷︸Cognition
(FluidIntelligenceGf and
CrystallizedIntelligence
Gc )
, Pt︸︷︷︸Personality
, RA,t︸︷︷︸Incentivesto performon the test
) t = 1, . . . ,T
(1)
Kt = ηt( Qt︸︷︷︸Quality
ofInputs
, Ct , Pt , RK ,t︸︷︷︸Incentivesto make
theinvestment
) (2)
Heckman, Kautz Hard Evidence on Soft Skills 14 / 125
Introduction Origins Origins Modern References
After age 10, Gf becomes quite stable.
Pt continues to evolve as does Gc .
Achievement tests reflect both cognitive and noncognitivetraits, incentives, and acquired knowledge.
Value added models are based on changes in scores onachievement tests.
i Attempt to estimate (1) but typically assume linearity.ii Arbitrary scale (on At).iii Any monotonic transformation of a test score is a legitimate
test score.iv To avoid (iii) anchor test scores in real world outcomes and do
so in a nonparametric fashion (see Cunha et al., 2010).
Heckman, Kautz Hard Evidence on Soft Skills 15 / 125
Introduction Origins Origins Modern References
The creators of these tests understood what they missed.
“We lean heavily on written examinations, on a few typesof objective tests, and on the subjective impressions ofteachers. Many other appraisal devices could be used, suchas records of activities in which pupils participate,questionnaires, check lists, anecdotal records andobservational records, interviews, reports made by parents,products made by the pupils, and records made byinstruments (motion pictures, eye-movement records,sound recordings, and the like).” [Tyler, 1940]
Heckman, Kautz Hard Evidence on Soft Skills 16 / 125
Introduction Origins Origins Modern References
Validation of Achievement Tests
How are these tests validated?
Usually on grades and other tests.
Ironic because the early pioneers decried the reliance ofeducational evaluations on “subjective” grades of teachers.
Heckman, Kautz Hard Evidence on Soft Skills 17 / 125
Introduction Origins Origins Modern References
Table 1: Predictive Validities of Standard IQ and Achievement Tests
Source: Almund et al. (2011).
Heckman, Kautz Hard Evidence on Soft Skills 18 / 125
Introduction Origins Origins Modern References
Table 1: Predictive Validities of Standard IQ and Achievement Tests
Source: Almund et al. (2011)
Heckman, Kautz Hard Evidence on Soft Skills 19 / 125
Introduction Origins Origins Modern References
Table 1: Predictive Validities of Standard IQ and Achievement Tests
Source: Almund et al. (2011)
Heckman, Kautz Hard Evidence on Soft Skills 20 / 125
Introduction Origins Origins Modern References
Psychometric validity of the GED
In two analyses, Means and Laurence (1984) found correlationsof .75 and .79 between mean GED subtest scores and themilitary’s Armed Forces Qualification Test (AFQT) scores.Correlations of .88 with its progenitor, the Iowa Test ofEducational Development; .80 with the American College Test(ACT); .81 with the Adult Performance Level (APL) Survey;.77 with New York’s Degrees of Reading Power (DRP) Test;.66-.68 with the Test of Adult Basic Education (TABE); and.61-.67 on the General Aptitude Test Battery (GATB).Baldwin et al. (1995): correlation of .78 between a generalGED factor and a general National Adult Literacy Survey(NALS) factor.“Correlations such as these provide evidence that the GED andother tests are measuring a common core of cognitive skills.”-Boesel, Alsalam, and Smith (1998, p. 37)
Heckman, Kautz Hard Evidence on Soft Skills 21 / 125
Introduction Origins Origins Modern References
Validities in Outcomes That Matter
Grades and achievement tests more predictive than IQ
None predict much of the variance in outcomes.
Heckman, Kautz Hard Evidence on Soft Skills 22 / 125
Introduction Origins Origins Modern References
Table 2: Validities in Labor Market Outcomes from the NationalLongitudinal Survey of Youth, 1979
Table 1: Validities in Labor Market Outcomes from the National Longitudinal Survey of Youth 1979: Our StudyNational Longitudinal Survey of Youth, 1979: Our Study
NLSY79 R2 (tests and school performance)
Males Females
Outcomes IQ GPA (10th grade) AFQT IQ GPA (10th grade) AFQT
Hourly Wage Age 35 0.03 0.05*** 0.05*** 0.11*** 0.10*** 0.13***
Hours Worked Age 35 0 10*** 0 12*** 0 21*** 0 02 0 10*** 0 17***Hours Worked Age 35 0.10*** 0.12*** 0.21*** 0.02 0.10*** 0.17***
Any Welfare Age 35 ‐0.09*** ‐0.11*** ‐0.23*** ‐0.20*** ‐0.23*** ‐0.36***
55
Source: Borghans et al. (2011)
Heckman, Kautz Hard Evidence on Soft Skills 23 / 125
Introduction Origins Origins Modern References
Personality: The Missing Ingredient
Big Five: OCEAN
OpennessConscientiousnessExtraversionAgreeablenessNeuroticism
Heckman, Kautz Hard Evidence on Soft Skills 24 / 125
Introduction Origins Origins Modern References
Table 3: Validities for Personality Tests
Source: Almlund et al. 2011
Heckman, Kautz Hard Evidence on Soft Skills 25 / 125
Introduction Origins Origins Modern References
Table 3: Validities for Personality Tests
Source: Almlund et al. 2011
Heckman, Kautz Hard Evidence on Soft Skills 26 / 125
Introduction Origins Origins Modern References
Figure 2: Association of the Big Five and intelligence with years ofschooling
-0.1 -0.05 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35
Crystalized Intelligence
Fluid Intelligence
Openness
Conscientiousness
Extraversion
Agreeableness
Emotional Stability
Standardized Regression Coefficient
Males
Unadjusted for Intelligence Adjusted for Intelligence
Source: Almlund et al. 2011
Heckman, Kautz Hard Evidence on Soft Skills 27 / 125
Introduction Origins Origins Modern References
Figure 3: Correlations of The Big Five and Intelligence with High SchoolCourse Grades
Source: Almlund et al. 2011
Heckman, Kautz Hard Evidence on Soft Skills 28 / 125
Introduction Origins Origins Modern References
Figure 4: Associations with Job Performance
Duckworth, Heckman, Almlund, and Kautz 10/22/2010 134
predictive as IQ. Conscientiousness, however, may play a more pervasive role than IQ. The
importance of IQ increases with job complexity, defined as the information processing
requirements of the job: cognitive skills are more important for professors, scientists, and senior
managers than for semi-skilled or unskilled laborers (Schmidt and Hunter [2004]). In contrast,
the importance of Conscientiousness does not vary much with job complexity (Barrick and
Mount [1991]), suggesting that it pertains to a wider spectrum of jobs. Causality remains an
open question. The raw correlations presented in Figure 14 do not account for reverse-causality,
and the authors do not clearly delineate when the measures of personality were taken.
Figure 14. Associations with Job Performance
Note. The values for personality are correlations that were corrected for sampling error, range restriction, and measurement error. Job performance was based on performance ratings, productivity data and training proficiency. The authors do report the timing of the measurements of personality relative to job performance. The value for IQ is a raw correlation. Source(s): The values reported for personality traits come from a meta-analysis conducted by Barrick and Mount [1991]. The value for IQ and job performance was reported in Schmidt and Hunter [2004].
0 0.1 0.2 0.3 0.4 0.5 0.6
Intelligence
Openness
Conscientiousness
Extraversion
Agreeableness
Emotional Stability
Correlation
Heckman, Kautz Hard Evidence on Soft Skills 29 / 125
Introduction Origins Origins Modern References
Figure 5: Correlations of mortality with personality, IQ, andsocioeconomic status (SES)
Source: Almlund et al. 2011
Heckman, Kautz Hard Evidence on Soft Skills 30 / 125
Introduction Origins Origins Modern References
Table 4: The Relative Predictive Power of Conscientiousness and SATScores for College GPA
Heckman, Kautz Hard Evidence on Soft Skills 31 / 125
Introduction Origins Origins Modern References
Personality Explains A Lot of the Variation in AFQT andOther Achievement Tests
At︸︷︷︸Academic
Achievementat t
= φt( Kt︸︷︷︸CumulatedInvestment(Schooling,Parenting)
, Ct︸︷︷︸Cognition
(FluidIntelligenceGf and
CrystallizedIntelligence
Gc )
, Pt︸︷︷︸Personality
, RA,t︸︷︷︸Incentivesto performon the test
) t = 1, . . . ,T
(1)
Kt = ηt( Qt︸︷︷︸Quality
ofInputs
, Ct , Pt , RK ,t︸︷︷︸Incentivesto make
theinvestment
) (2)
Heckman, Kautz Hard Evidence on Soft Skills 32 / 125
Introduction Origins Origins Modern References
Figure 6: Decomposing Achievement Tests and Grades into IQ andPersonality [NLSY79]
0.48
0.23
0.43
0.190.16
0.10
0.00
0.10
0.20
0.30
0.40
0.50
0.60
R-Sq
uare
d
IQ, Rosenberg, and Rotter IQ Rosenberg and Rotter
AFQT Grades
Achievement Grades
Source: Borghans, Golsteyn, Heckman et al. [2011].
Heckman, Kautz Hard Evidence on Soft Skills 33 / 125
Introduction Origins Origins Modern References
Figure 7: Decomposing Achievement Tests and Grades into IQ andPersonality [Stella Maris]
Source: Borghans, Golsteyn, Heckman et al. [2011].
Heckman, Kautz Hard Evidence on Soft Skills 34 / 125
Introduction Origins Origins Modern References
Questions
Are these relationships causal?
Can we change these traits or are they fixed? Is promotingcognition and personality a useful policy option?
What do we miss by ignoring soft skills?
Heckman, Kautz Hard Evidence on Soft Skills 35 / 125
Introduction Origins Origins Modern References
Difficulties in Establishing Causality
Reverse causality (clearly present in studies based oncontemporaneous measures)
Using lagged achievement measures creates an errors invariables problem. Current stocks determine current behavior.
Multiple traits in conjunction with incentives predict taskperformance.
Y jt︸︷︷︸
Performanceon task jat time t
= φt(Gc,t , Gf ,t , Pt , e jt︸︷︷︸
effort
) (3)
e jt︸︷︷︸
Efforton task jat time t
= χt(Gc,t , Gf ,t , Pt , R jt︸︷︷︸
Incentiveto performin task j
) (4)
Heckman, Kautz Hard Evidence on Soft Skills 36 / 125
Introduction Origins Origins Modern References
All psychological measurements are calibrated on taskperformances.
A fundamental identification problem lies at the heart of anypsychological measurement system for any particular trait —need to standardize for incentives and the effects of other traitsin performing a task.
Examples: Incentivizing IQ tests.
All psychological measurements capture performance on sometask or set of tasks.
But typically analysts do not control for all of the factors thatpromote success on the tasks.
Heckman, Kautz Hard Evidence on Soft Skills 37 / 125
Introduction Origins Origins Modern References
Causal Evidence on the Power of Soft Skills: Two Separate Sources
GED testing Program
Evidence from a social experiment
Heckman, Kautz Hard Evidence on Soft Skills 38 / 125
Introduction Origins Origins Modern References
Case Study 1:The GED as a case study of the power of soft skills and
costs of neglecting them
Evidence is strongly suggestive of the causal power ofpersonality traits.
GED is a test that secondary school dropouts can take to becertified as the equivalents of ordinary high school graduates.
Studies Of The GED Testing Program,Forthcoming, University of Chicago Press, 2012.
Use multiple data sets on outcomes, backgrounds, and abilitiesfor all major economic and social groups in the U.S.
12% of all high school certificates issued in the U.S. arecurrently given to GEDs.
GED is a group of 5 achievement tests normed against nationalsamples of high school graduates (70% can pass).
Heckman, Kautz Hard Evidence on Soft Skills 39 / 125
Introduction Origins Origins Modern References
GEDs are as smart as HSGs who do not go on to college.
GEDs who go on to college and succeed are indistinguishablefrom other college graduates in terms of annual wage income(True for AA and BA students).
Terminal GEDs perform at a level very close tothat of dropouts.
Heckman, Kautz Hard Evidence on Soft Skills 40 / 125
Introduction Origins Origins Modern References
We examine the essential life skills that GEDs lack.
Comparing GEDs to Dropouts standardizes ability (as measuredby achievement tests) and examines the importance of theother life skills and, in particular, soft skills.
Heckman, Kautz Hard Evidence on Soft Skills 41 / 125
Introduction Origins Origins Modern References
Figure 8: Cognitive ability by educational status
Source: Heckman, Humphries, Urzua, and Veramendi (2011)
Heckman, Kautz Hard Evidence on Soft Skills 42 / 125
Introduction Origins Origins Modern References
Figure 9: Cognitive ability by educational status
Source: Heckman, Humphries, Urzua, and Veramendi (2011)
Heckman, Kautz Hard Evidence on Soft Skills 43 / 125
Introduction Origins Origins Modern References
GEDs lack noncognitive traits measured in many ways:behaviors and personality test scores.
Heckman, Kautz Hard Evidence on Soft Skills 44 / 125
Introduction Origins Origins Modern References
Figure 10: Distribution of Non-Cognitive Skills by Education Group
Source: Reproduced from Heckman et al. (2011). National Longitudinal Study of Youth 1979.
Heckman, Kautz Hard Evidence on Soft Skills 45 / 125
Introduction Origins Origins Modern References
Figure 11: Distribution of Non-Cognitive Skills by Education Group
Source: Reproduced from Heckman et al. (2011). National Longitudinal Study of Youth 1979.
Heckman, Kautz Hard Evidence on Soft Skills 46 / 125
Introduction Origins Origins Modern References
Gaps in both cognitive and noncognitive skills open up earlyand persist.
Heckman, Kautz Hard Evidence on Soft Skills 47 / 125
Introduction Origins Origins Modern References
Figure 12: PIAT Scores across Ages and Education Groups - (All Races,All Post-Secondary Education)
(a) Males (b) Females
−.8
−.6
−.4
−.2
0.2
Age 6 PIAT
Age 8 PIAT
Age 10 PIAT
Age 12 PIAT
Age 14 PIAT
Dropout GED 5% Sig (GED vs.HSG)
High School S.E. 5% Sig (GED/HSG vs.Drop)
−.8
−.6
−.4
−.2
0.2
Age 6 PIAT
Age 8 PIAT
Age 10 PIAT
Age 12 PIAT
Age 14 PIAT
Dropout GED 5% Sig (GED vs.HSG)
High School S.E. 5% Sig (GED/HSG vs.Drop)
Source: Children of the National Longitudinal Survey of Youth 1979. Notes: The PeabodyIndividual Achievement Test (PIAT) is a widely childhood achievement test.
Heckman, Kautz Hard Evidence on Soft Skills 48 / 125
Introduction Origins Origins Modern References
Figure 13: Behavioral Problems Index (BPI) across Education Groups -(All Races , All Post-Secondary Education )
(a) Males (b) Females
−.5
0.5
1
Age 6
Age 8
Age 10
Age 12
Age 14
Dropout GED 5% Sig (GED vs.HSG)
High School S.E. 5% Sig (GED/HSG vs.Drop)
−.5
0.5
1
Age 6
Age 8
Age 10
Age 12
Age 14
Dropout GED 5% Sig (GED vs.HSG)
High School S.E. 5% Sig (GED/HSG vs.Drop)
Sources: Children of the National Longitudinal Survey of Youth 1979. Notes: The BehavioralProblems Index (BPI) is based on a 28 question survey given to parents about their child. The
BPI is normalized to have mean 0 and variance 1.
Heckman, Kautz Hard Evidence on Soft Skills 49 / 125
Introduction Origins Origins Modern References
The early childhood environments of dropouts and GEDs areworse than those of H.S. Grads.(Moon, 2011; Cunha et al., 2010)
Early family environments are more like those of dropouts.
Female GEDs have better early childhood environments asmeasured by parental investment—than males—close to thoseof H.S. Grads.
Males not so
Consistent pattern: Generally female GEDs have better traitsthen male GEDs—compared to dropouts.
Heckman, Kautz Hard Evidence on Soft Skills 50 / 125
Introduction Origins Origins Modern References
These traits are highly predictive of who graduates from H.S.and who does not.
Noncognitive traits do not predict GED certification.
Heckman, Kautz Hard Evidence on Soft Skills 51 / 125
Introduction Origins Origins Modern References
Figure 14: Probability of Graduating from High School (by cognitive andnon-cognitive skill decile)
Source: Reproduced from Heckman et al. (2011).
Heckman, Kautz Hard Evidence on Soft Skills 52 / 125
Introduction Origins Origins Modern References
Figure 15: Probability of GED Certification (conditional on dropping out,by cognitive and non-cognitive decile)
Source: Reproduced from Heckman et al. (2011).
Heckman, Kautz Hard Evidence on Soft Skills 53 / 125
Introduction Origins Origins Modern References
Figure 16: Post-Secondary Educational Attainment Across EducationGroups - (All Races, NLSY79)
(a) Males
0.2
.4.6
.81
Touched Coll
Some Coll
A.A.B.A.
Dropout GED 5% Sig (GED vs.HSG)
High School S.E. 5% Sig (vs.Drop)
(b) Females
0.2
.4.6
.81
Touched Coll
Some Coll
A.A.B.A.
Dropout GED 5% Sig (GED vs.HSG)
High School S.E. 5% Sig (vs.Drop)
Source: National Longitudinal Survey of Youth 1979.Notes: Variable Definitions: “Touched Coll” represents people who entered any post-secondary institution ever. “Some Coll”represents people who ever complete at least a year of post-secondary education. “A.A.” represents people who ever obtain
an Associate’s degree. “B.A.” represents people who ever obtain Bachelor’s degrees. “B.A.” also includes people with highereducation: M.A. Ph.D and professional degrees.
Heckman, Kautz Hard Evidence on Soft Skills 54 / 125
Introduction Origins Origins Modern References
If a GED gets a BA or a MA, he/she earns at that level.
However, there is usually delay.
The present value of earnings for such people is substantially(20–30%) lower.
Annual payment per unit of skill is the same for GEDs anddropouts.
Heckman, Kautz Hard Evidence on Soft Skills 55 / 125
Introduction Origins Origins Modern References
Figure 17: Annual Income Differences - By Age - NLSY79 -(Males, All Races)
(a) All Post-Secondary Education (b) No Post-Secondary Education
010
000
2000
030
000
4000
0
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
010
000
2000
030
000
4000
0
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
Source: National Longitudinal Survey of Youth 1979. Note: Estimates for hours worked andhourly wages exclude non-workers. Controls: “Raw” – age and region or state of residence;“Abil” – AFQT adjusted for schooling at time of test; “BG” – mother’s highest gradecompleted, urban status at age 14, family income in 1979, broken home status in 1979, southat age 14 and AFQT. Regressions exclude those reporting earning more than $300,000 orworking more than 4,000 hours.
Heckman, Kautz Hard Evidence on Soft Skills 56 / 125
Introduction Origins Origins Modern References
Figure 18: Hourly Wage Differences - By Age - NLSY79 -(Males, All Races)
(a) All Post-Secondary Education (b) No Post-Secondary Education
05
1015
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
05
1015
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
Source: National Longitudinal Survey of Youth 1979. Note: Estimates for hours worked andhourly wages exclude non-workers. Controls: “Raw” – age and region or state of residence;“Abil” – AFQT adjusted for schooling at time of test; “BG” – mother’s highest gradecompleted, urban status at age 14, family income in 1979, broken home status in 1979, southat age 14 and AFQT. Regressions exclude those reporting earning more than $300,000 orworking more than 4,000 hours.
Heckman, Kautz Hard Evidence on Soft Skills 57 / 125
Introduction Origins Origins Modern References
Figure 19: Employment Differences - By Age - NLSY79 -(Males, All Races)
(a) All Post-Secondary Education (b) No Post-Secondary Education
−.0
50
.05
.1.1
5
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
−.0
50
.05
.1.1
5
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
Source: National Longitudinal Survey of Youth 1979. Note: Estimates for hours worked andhourly wages exclude non-workers. Controls: “Raw” – age and region or state of residence;“Abil” – AFQT adjusted for schooling at time of test; “BG” – mother’s highest gradecompleted, urban status at age 14, family income in 1979, broken home status in 1979, southat age 14 and AFQT. Regressions exclude those reporting earning more than $300,000 orworking more than 4,000 hours.
Heckman, Kautz Hard Evidence on Soft Skills 58 / 125
Introduction Origins Origins Modern References
Figure 20: Hours Worked Differences - By Age - NLSY79 -(Males, All Races)
(a) All Post-Secondary Education (b) No Post-Secondary Education
−20
0−
100
010
020
030
0
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
−20
0−
100
010
020
030
0
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
Source: National Longitudinal Survey of Youth 1979. Note: Estimates for hours worked andhourly wages exclude non-workers. Controls: “Raw” – age and region or state of residence;“Abil” – AFQT adjusted for schooling at time of test; “BG” – mother’s highest gradecompleted, urban status at age 14, family income in 1979, broken home status in 1979, southat age 14 and AFQT. Regressions exclude those reporting earning more than $300,000 orworking more than 4,000 hours.
Heckman, Kautz Hard Evidence on Soft Skills 59 / 125
Introduction Origins Origins Modern References
Avoiding Pretest Bias Or Cherry Picking of Results
Many different specifications.
Various specifications reported as “significant” in the literature.
Mostly concentrated in the work of one author and hiscolleagues and students.
No correction for pre-testing and model selection whenreporting standard errors.
One simple procedure to avoid such bias is to display resultsacross all models and account for dependence in the estimates.
Heckman, Kautz Hard Evidence on Soft Skills 60 / 125
Introduction Origins Origins Modern References
Figure 21: Distribution of the Effect of the GED Certificate and HighSchool Graduation on Annual Income Across Models (Males)
(a) GED Recipients vs. HighSchool Dropouts (Males)
(b) High School Graduates vs.High School Dropouts
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 61 / 125
Introduction Origins Origins Modern References
Figure 22: Distribution of p-values from Tests Comparing Annual Incomeof GED Recipients and High School Graduates to High School Dropouts(Males)
(a) GED Recipients vs. HighSchool Dropouts (Males)
(b) High School Graduatesvs. High School Dropouts(Males)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 62 / 125
Introduction Origins Origins Modern References
Figure 22: Distribution of p-values from Tests Comparing Annual Incomeof GED Recipients and High School Graduates to High School Dropouts(Males)
(c) GED Recipients vs. High School Graduates (Males)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 63 / 125
Introduction Origins Origins Modern References
Figure 23: Distribution of p-values from Tests Comparing Hourly Wageof GED Recipients and High School Graduates to High School Dropouts(Males)
(a) GED Recipients vs. HighSchool Dropouts (Males)
(b) High School Graduatesvs. High School Dropouts(Males)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 64 / 125
Introduction Origins Origins Modern References
Figure 23: Distribution of p-values from Tests Comparing Hourly Wageof GED Recipients and High School Graduates to High School Dropouts(Males)
(c) GED Recipients vs. High School Graduates (Males)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 65 / 125
Introduction Origins Origins Modern References
Similar patterns found for hours of work and employment formales.
Heckman, Kautz Hard Evidence on Soft Skills 66 / 125
Introduction Origins Origins Modern References
Using R.D. method produces no GED effects(sample of 100,000 GED test takers with long histories beforeand after the taking test) (Jepsen et al., 2010).
No signalling value — GEDs at the margin of passing (maleand female) earn the same before and after certification
For people motivated enough to take the test, there is no effectof the GED on earnings.
Heckman, Kautz Hard Evidence on Soft Skills 67 / 125
Introduction Origins Origins Modern References
Women
Much of the previous literature focuses on males.
Simplifies the econometrics: avoids selection into the laborforce as an empirical issue.
But misses an important empirical phenomenon.
Controlling for ability and baseline characteristics, thereappear to be GED effects for certain groups of females.
But only for employment, hours worked, and hence earnings,but not in hourly wage rates.
Heckman, Kautz Hard Evidence on Soft Skills 68 / 125
Introduction Origins Origins Modern References
Figure 24: Annual Income Differences - By Age - NLSY79 -(Females, All Races)
(a) All Post-Secondary Education (b) No Post-Secondary Education
050
0010
000
1500
020
000
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
050
0010
000
1500
020
000
Age 20 to 24 Age 25 to 29 Age 30 to 34 Age 35 to 39Raw Abil BG Raw Abil BG Raw Abil BG Raw Abil BG
GED p<0.05(GED vs.HSG) p<0.05(HSG vs.Drop)
High School p<0.05(GED vs.Drop) S.E.
Source: National Longitudinal Survey of Youth 1979. Controls: “Raw” – age and region orstate of residence; “Abil” – AFQT adjusted for schooling at time of test; “BG” – mother’shighest grade completed, urban status at age 14, family income in 1979, broken home statusin 1979, south at age 14 and AFQT. Regressions exclude those reporting earning more than$300,000 or working more than 4,000 hours.
Heckman, Kautz Hard Evidence on Soft Skills 69 / 125
Introduction Origins Origins Modern References
Figure 25: Distribution of p-values from Tests Comparing Annual Incomeof GED Recipients and High School Graduates to High School Dropouts(Females)
(a) GED Recipients vs. HighSchool Dropouts (Females)
(b) High School Graduates vs.High School Dropouts(Females)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 70 / 125
Introduction Origins Origins Modern References
Figure 25: Distribution of p-values from Tests Comparing Annual Incomeof GED Recipients and High School Graduates to High School Dropouts(Females)
(c) GED Recipients vs. High School Graduates (Females)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 71 / 125
Introduction Origins Origins Modern References
Figure 26: Distribution of p-values from Tests Comparing Hourly Wageof GED Recipients and High School Graduates to High School Dropouts(Females)
(a) GED Recipients vs. HighSchool Dropouts (Females)
(b) High School Graduates vs.High School Dropouts(Females)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 72 / 125
Introduction Origins Origins Modern References
Figure 26: Distribution of p-values from Tests Comparing Hourly Wageof GED Recipients and High School Graduates to High School Dropouts(Females)
(c) GED Recipients vs. High School Graduates (Females)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 73 / 125
Introduction Origins Origins Modern References
Figure 27: Distribution of p-values from Tests Comparing Employment ofGED Recipients and High School Graduates to High School Dropouts(Females)
(a) GED Recipients vs. HighSchool Dropouts (Females)
(b) High School Graduates vs.High School Dropouts(Females)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 74 / 125
Introduction Origins Origins Modern References
Figure 27: Distribution of p-values from Tests Comparing Employment ofGED Recipients and High School Graduates to High School Dropouts(Females)
(c) GED Recipients vs. High School Graduates (Females)
Sources: National Longitudinal Survey of Youth 1979. Notes: Estimates for hours worked and hourly wages excludenon-workers. This graph plots the estimated p-values of GED coefficients from a series of linear regressions. All modelscontrol for region, age, year, and AFQT score. The models differ in other controls and sub-populations of the data. The setof models includes all combinations of mother’s highest grade completed, urban residence at age 14, family income, lives inthe south at age 14, smoked at 15, has had sex by 15, has committed a major crime, and 9th grade GPA. Thesub-populations are all combinations of race, post-secondary education (everyone, has some post-secondary education, nopost-secondary education), and age (measured in 5-year categories from 20 to 39) for males and females.
Heckman, Kautz Hard Evidence on Soft Skills 75 / 125
Introduction Origins Origins Modern References
Why Do Women Benefit?
Two groups of relatively bright women benefit.(relative to dropouts)
One group is bright girls who were screw-ups in high school,dropped out, GED certified, and went on to attend andcomplete college.
A second group of bright girls who drop out early (due topregnancy) but on some dimension of personality and behaviorare better than other dropouts—they GED certify late aftertheir children enroll in school.
They work hard, do not invest much on the job, have littlewage growth and do not complete any further education.
Heckman, Kautz Hard Evidence on Soft Skills 76 / 125
Introduction Origins Origins Modern References
This evidence may point to change—in preferences and/orconstraints that cause some to turn around and the GED is alifeline that promotes this.
Like “desistance” in the criminology literature. (e.g., Sampsonand Laub)
Heckman, Kautz Hard Evidence on Soft Skills 77 / 125
Introduction Origins Origins Modern References
Characteristics of GEDs Who Drop Out Due to Pregnancy
Males and females drop out of high school for very differentreasons.
Males tend to drop out because they dislike school, lack ability,or work.
Few males list marriage or parenthood as a major reason.
Many women do. 42% of all female dropouts in the NLSY79are pregnant before or during the same year that they dropout.
38% of all female GED recipients were pregnant before theydropped out of high school.
Pregnant GEDs generally have better traits than other femaleGEDs
i Better Grades in High Schoolii Better AFQTiii Better behavioral traits
Heckman, Kautz Hard Evidence on Soft Skills 78 / 125
Introduction Origins Origins Modern References
Figure 28: AFQT Scores across Education Groups -(All Races, All Post-Secondary Education)
−1.
5−
1−
.50
Oth Drop
Preg Drop
Oth GED
Preg GED
p<0.05(Prg vs. Non−Prg) S.E.
p<0.05(Drop vs. GED)
AFQT
Sources: National Longitudinal Survey of Youth 1979. Notes: The AFQT test wasadministered to the NLSY79 in 1980 when individuals were age 15 to 22. NLSY79 AFQT
scores are adjusted for years of schooling at the time of test as described in the web appendix.
Heckman, Kautz Hard Evidence on Soft Skills 79 / 125
Introduction Origins Origins Modern References
Figure 29: Crime Factor across Education Groups -(All Races, All Post-Secondary Education)High “Crime Factor” Means They Are Better
−.2
−.1
0.1
.2.3
Oth Drop
Preg Drop
Oth GED
Preg GED
p<0.05(Prg vs. Non−Prg) S.E.
p<0.05(Drop vs. GED)
Crime Factor
Sources: National Longitudinal Survey of Youth 1979. Notes: The crime factor is based onwhether the respondent committed a petty crime (damaged property, shoplifted, petty theft,robbery, fraud, fencing), major crime (auto theft, breaking and entering private property, and
grand theft), and physical crimes (fighting at work or school, assault and battery, andaggravated assault).
Heckman, Kautz Hard Evidence on Soft Skills 80 / 125
Introduction Origins Origins Modern References
Figure 30: Highest Grade Completed across Education Groups -(All Races, All Post-Secondary Education)
88.
59
9.5
10
Oth Drop
Preg Drop
Oth GED
Preg GED
p<0.05(Prg vs. Non−Prg) S.E.
p<0.05(Drop vs. GED)
Hightest Grade Completed
Sources: National Longitudinal Survey of Youth 1979.Heckman, Kautz Hard Evidence on Soft Skills 81 / 125
Introduction Origins Origins Modern References
Figure 31: 9th Grade GPA across Education Groups -(All Races, All Post-Secondary Education)
0.5
11.
52
Oth Drop
Preg Drop
Oth GED
Preg GED
p<0.05(Prg vs. Non−Prg) S.E.
p<0.05(Drop vs. GED)
9th Gr. GPA
Sources: National Longitudinal Survey of Youth 1979.Heckman, Kautz Hard Evidence on Soft Skills 82 / 125
Introduction Origins Origins Modern References
Figure 32: Sexual Intercourse Before 15 across Education Groups -(All Races, All Post-Secondary Education)
0.2
.4.6
Oth Drop
Preg Drop
Oth GED
Preg GED
p<0.05(Prg vs. Non−Prg) S.E.
p<0.05(Drop vs. GED)
Sex at 15
Sources: National Longitudinal Survey of Youth 1979.Heckman, Kautz Hard Evidence on Soft Skills 83 / 125
Introduction Origins Origins Modern References
Figure 33: Drinks Age 15 across Education Groups -(All Races, All Post-Secondary Education)
0.0
5.1
.15
Oth Drop
Preg Drop
Oth GED
Preg GED
p<0.05(Prg vs. Non−Prg) S.E.
p<0.05(Drop vs. GED)
Drinks by 15
Sources: Children of the National Longitudinal Survey of Youth 1979. Notes: The BehavioralProblems Index (BPI) is based on a 28 question survey given to parents about their child. The
BPI is normalized to have mean 0 and variance 1.
Heckman, Kautz Hard Evidence on Soft Skills 84 / 125
Introduction Origins Origins Modern References
Figure 34: Smokes by Age 15 across Education Groups -(All Races, All Post-Secondary Education)
0.2
.4.6
Oth Drop
Preg Drop
Oth GED
Preg GED
p<0.05(Prg vs. Non−Prg) S.E.
p<0.05(Drop vs. GED)
Smokes by 15
Sources: Children of the National Longitudinal Survey of Youth 1979. Notes: The BehavioralProblems Index (BPI) is based on a 28 question survey given to parents about their child. The
BPI is normalized to have mean 0 and variance 1.
Heckman, Kautz Hard Evidence on Soft Skills 85 / 125
Introduction Origins Origins Modern References
Aside from early sexual intercourse they tend to engage in lowerlevels of risky behavior. Lower than the behavior of otherpregnant dropouts who do not GED certify.
On these measures, they are much more similar to high schoolgraduates.
Some of the GED recipients would likely have graduated fromhigh school had they not become pregnant even without achange in traits.
Heckman, Kautz Hard Evidence on Soft Skills 86 / 125
Introduction Origins Origins Modern References
We have no direct measures of personality post-pregnancy, butwe have evidence on their behaviors.
However, they are persistent in the workplace after theyre-enter.
Heckman, Kautz Hard Evidence on Soft Skills 87 / 125
Introduction Origins Origins Modern References
The average age of GED receipt for the women who drop outdue to pregnancy is 24.3—about 1.3 years later than otherfemale GED recipients.
They GED certify on average when their youngest child is 3–4years old – around the age when most children start preschool.
Heckman, Kautz Hard Evidence on Soft Skills 88 / 125
Introduction Origins Origins Modern References
Female GEDs generally have better noncognitive traits, and thisshows up in their sorting into positions in the labor market.
Heckman, Kautz Hard Evidence on Soft Skills 89 / 125
Introduction Origins Origins Modern References
Figure 35: Average Occupational Factor Scores by Final Education -Males
0.8
1
0.2
0.4
0.6
-0.2
0
0.2
0 8
-0.6
-0.4
-1
-0.8
Dropout GED High School Graduate
Some College AA BAGraduate
Cognitive Traits Social Traits Physical Traits
Source: The American Community Survey 2009 and O-Net. Notes: All educational categories are final education at time ofinterview. Each factor is based on the following O-Net occupational importance scores: Cognitive - active learning, analyticalthinking, complex problem solving, critical thinking, deductive reasoning, inductive reasoning, interpretation of meaning, mathreasoning, mathematics, processing information, reading comprehension, creative thinking, updating knowledge andvisualization. Social - communicate to outside organizations, concern for others, customer or personal service, establishrelationships, leadership, oral expression, persuasion, social perceptiveness, speaking, writing, written expression, activelistening, and cooperation. Physical Traits - arm and hand steadiness, control and precision, coordination, depth perception,explosive strength, finger dexterity, gross body coordination, gross body equilibrium, manual dexterity, multi-limb coordination,reaction time, spatial orientation, stamina, static strength, stress tolerance, trunk strength, and wrist and finger speed.
Heckman, Kautz Hard Evidence on Soft Skills 90 / 125
Introduction Origins Origins Modern References
Figure 36: Average Occupational Factor Scores by Final Education -Females
0.4
0.6
0
0.2
-0.4
-0.2
1
-0.8
-0.6
-1.2
-1
Dropout GED High School Graduate
Some College AA BAGraduate
Cognitive Traits Social Traits Physical Traits
Source: The American Community Survey 2009 and O-Net. Notes: All educational categories are final education at time ofinterview. Each factor is based on the following O-Net occupational importance scores: Cognitive - active learning, analyticalthinking, complex problem solving, critical thinking, deductive reasoning, inductive reasoning, interpretation of meaning, mathreasoning, mathematics, processing information, reading comprehension, creative thinking, updating knowledge andvisualization. Social - communicate to outside organizations, concern for others, customer or personal service, establishrelationships, leadership, oral expression, persuasion, social perceptiveness, speaking, writing, written expression, activelistening, and cooperation. Physical Traits - arm and hand steadiness, control and precision, coordination, depth perception,explosive strength, finger dexterity, gross body coordination, gross body equilibrium, manual dexterity, multi-limb coordination,reaction time, spatial orientation, stamina, static strength, stress tolerance, trunk strength, and wrist and finger speed.
Heckman, Kautz Hard Evidence on Soft Skills 91 / 125
Introduction Origins Origins Modern References
Cross Section Comparisons: Potentially Dangerous
1 People can GED certify at different ages.
2 In a cross section, the experience of GEDs of the same age canbe very different.
3 Process non-ergodic.
Heckman, Kautz Hard Evidence on Soft Skills 92 / 125
Introduction Origins Origins Modern References
Longitudinal analysis generally supports the cross sectionanalysis but provides a more nuanced interpretation of it.
Using a variety of procedures, employment gains for women.
No hourly wage gains.
Heckman, Kautz Hard Evidence on Soft Skills 93 / 125
Introduction Origins Origins Modern References
Stability of Traits
Are traits stable? Can people change?
The evidence for women suggests this might be a possibility.
But as a group, post-GED turnover behavior is quite stable.
High quit rates for all who start anything: job, military,marriage.
Laurence (2012): High dropout rates in the military
Heckman, Kautz Hard Evidence on Soft Skills 94 / 125
Introduction Origins Origins Modern References
Figure 37: Cumulative Prob of Hazard Out of Employment(Males, All Races, All Levels of Post-Secondary Education)
0
.1
.2
.3
.4
Cum
ulat
ive
Pro
b of
Exi
t by
Giv
en P
d
1 2 3 4 5 6 7 8 9 10
Years Since Start of Spell
Dropouts GEDs
HSGs +− 1 S.E.
Includes Ever Jailed − Sample With and Without Post−Sec Exp. − All Ability Levels
Cumulative Prob of Hazard Out of Employment − Male − All Race
Source: National Longitudinal Survey of Youth 1979 (NLSY79),nationally representative cross sectional sample.
Heckman, Kautz Hard Evidence on Soft Skills 95 / 125
Introduction Origins Origins Modern References
Figure 38: Cumulative Prob of Hazard Out of Employment(Females, All Races, All Levels of Post-Secondary Education)
0
.2
.4
.6
Cum
ulat
ive
Pro
b of
Exi
t by
Giv
en P
d
1 2 3 4 5 6 7 8 9 10
Years Since Start of Spell
Dropouts GEDs
HSGs +− 1 S.E.
Includes Ever Jailed − Sample With and Without Post−Sec Exp. − All Ability Levels
Cumulative Prob of Hazard Out of Employment − Female − All Race
Source: National Longitudinal Survey of Youth 1979 (NLSY79),nationally representative cross sectional sample.
Heckman, Kautz Hard Evidence on Soft Skills 96 / 125
Introduction Origins Origins Modern References
Figure 39: Cumulative Prob of Hazard Out of the Same Job(Males, All Races, All Levels of Post-Secondary Education)
.2
.4
.6
.8
1
Cum
ulat
ive
Pro
b of
Exi
t by
Giv
en P
d
1 2 3 4 5 6 7 8 9 10
Years Since Start of Spell
Dropouts GEDs
HSGs +− 1 S.E.
Includes Ever Jailed − Sample With and Without Post−Sec Exp. − All Ability Levels
Cumulative Prob of Hazard Out of Same Primary Job − Male − All Race
Source: National Longitudinal Survey of Youth 1979 (NLSY79),nationally representative cross sectional sample.
Heckman, Kautz Hard Evidence on Soft Skills 97 / 125
Introduction Origins Origins Modern References
Figure 40: Cumulative Prob of Hazard Out of the Same Job(Females, All Races, All Levels of Post-Secondary Education)
.2
.4
.6
.8
1
Cum
ulat
ive
Pro
b of
Exi
t by
Giv
en P
d
1 2 3 4 5 6 7 8 9 10
Years Since Start of Spell
Dropouts GEDs
HSGs +− 1 S.E.
Includes Ever Jailed − Sample With and Without Post−Sec Exp. − All Ability Levels
Cumulative Prob of Hazard Out of Same Primary Job − Female − All Race
Source: National Longitudinal Survey of Youth 1979 (NLSY79),nationally representative cross sectional sample.
Heckman, Kautz Hard Evidence on Soft Skills 98 / 125
Introduction Origins Origins Modern References
Figure 41: Cumulative Hazard Rate into Divorce(Males, All Races, All Levels of Post-Secondary Education)
0
.2
.4
.6
.8
Cum
ulat
ive
Pro
b of
Exi
t by
Giv
en P
d
1 2 3 4 5 6 7 8 9 10
Years Since Start of Spell
Dropouts GEDs
HSGs +− 1 S.E.
Includes Ever Jailed − Sample With and Without Post−Sec Exp. − All Ability Levels
Cumulative Prob of Hazard Into Divorce − Male − All Race
Source: National Longitudinal Survey of Youth 1979 (NLSY79),nationally representative cross sectional sample.
Heckman, Kautz Hard Evidence on Soft Skills 99 / 125
Introduction Origins Origins Modern References
Figure 42: Cumulative Hazard Rate into Divorce(Females, All Races, All Levels of Post-Secondary Education)
0
.2
.4
.6
Cum
ulat
ive
Pro
b of
Exi
t by
Giv
en P
d
1 2 3 4 5 6 7 8 9 10
Years Since Start of Spell
Dropouts GEDs
HSGs +− 1 S.E.
Includes Ever Jailed − Sample With and Without Post−Sec Exp. − All Ability Levels
Cumulative Prob of Hazard Into Divorce − Female − All Race
Source: National Longitudinal Survey of Youth 1979 (NLSY79),nationally representative cross sectional sample.
Heckman, Kautz Hard Evidence on Soft Skills 100 / 125
Introduction Origins Origins Modern References
Figure 43: Cumulative Hazard Rate to First Time Incarcerated(Males, All Races, All Levels of Post-Secondary Education)
0
.05
.1
.15
Cum
ulat
ive
Pro
b of
Exi
t by
Giv
en P
d
1 2 3 4 5 6 7 8 9 10
Years Since Start of Spell
Dropouts GEDs
HSGs +− 1 S.E.
Includes Ever Jailed − Sample With and Without Post−Sec Exp. − All Ability Levels
Cumulative Prob of Hazard Into First Jail Term − Male − All Race
Source: National Longitudinal Survey of Youth 1979 (NLSY79),nationally representative cross sectional sample. Notes: The spell tofirst time being incarcerated begins in the first year that individuals
exit school.
Heckman, Kautz Hard Evidence on Soft Skills 101 / 125
Introduction Origins Origins Modern References
The stability of traits challenges the situational specificityhypothesis that is current in behavioral economics.
Claim: People adapt fully to situations.
Heckman, Kautz Hard Evidence on Soft Skills 102 / 125
Introduction Origins Origins Modern References
Situational Specificity Hypothesis
The modern origins of the situational specificity hypothesis isbased on the work of psychologist Walter Mischel:
“. . . with the possible exception of intelligence, highlygeneralized behavioral consistencies have not beendemonstrated, and the concept of personality traits asbroad dispositions is thus untenable”
-Mischel (1968, p. 146)
Heckman, Kautz Hard Evidence on Soft Skills 103 / 125
Introduction Origins Origins Modern References
Many behavioral economists hold a similar view and appeal toMischel as a guiding influence.
“The great contribution to psychology by WalterMischel [. . . ] is to show that there is no such thing asa stable personality trait.”-Thaler (2008)
The evidence from the GED and much other evidence speaksstrongly against the claims of Mischel and the behavioraleconomists. (See Almlund et al., 2011.)
Heckman, Kautz Hard Evidence on Soft Skills 104 / 125
Introduction Origins Origins Modern References
Why Do People Get a GED If Returns Are So Low?
Previous models of dropping out of school do not emphasizepermanent shocks (e.g. Eckstein-Wolpin).
Timing due to relative wage changes?
Such timing is not an important explanation.
Effects too small.
The story for women is consistent with preference shocksand/or relaxation of constraints.
i Birth of a child (stochastic)ii Activates altruism (Oxytocin and other biological changes)iii For cash-strapped women, the GED opens doors to
employment.iv Take low wage immediate payoff jobs — consistent with
short-run liquidity constraints
Heckman, Kautz Hard Evidence on Soft Skills 105 / 125
Introduction Origins Origins Modern References
Some stress the evolution of the prefrontal cortex and the racebetween intellectual ability and psychosocial maturity.
This is used to explain adolescent behavior.
May apply to the decision to take the GED.
Heckman, Kautz Hard Evidence on Soft Skills 106 / 125
Introduction Origins Origins Modern References
Figure 44: Proportion of Individuals in Each Age Group Scoring at orAbove the Mean for 26- to 30-Year-Olds on Indices of Intellectual andPsychosocial Maturity.
his conjecture about how basic intellectual ability and psychosocial maturity (related to,e.g., impulsivity, risk perception, sensation seeking, future orientation) evolve over thelife cycle.221 He argues that intellectual ability matures more rapidly than psychosocialmaturity. In his model, increases in adolescent risk taking are due to a restructuring ofthe brain’s dopaminergic system (responsible for the brain’s reward processing) in sucha way that immediate or novel experiences yield higher rewards, especially in the pre-sence of peers. He attributes declines in risk taking to the development of the brain’scognitive control system, specifically improvements in the prefrontal cortex that promoteaspects of executive function such as response inhibition, planning ahead, weighingrisks and rewards, and the simultaneous consideration of multiple information sources.Interestingly, even in his model, sensation seeking partially depends on the presence ofpeers, which corresponds to aspects of the situation (h in the framework of Section 3).This example highlights the difficulty in disentangling situational and biological changesin personality.
p1600 What factors other than preprogrammed genetic influences might account for mean-level changes in personality? Personality change in adulthood may be precipitated by majorshifts in social roles (e.g., getting a job for the first time or becoming a parent). If social rolechanges are experienced by most people in a population at the same time, we will observethe effects as mean-level changes in measured personality. If, on the other hand, these socialroles are not assumed synchronously, we will observe rank-order changes.
p1605 One difficulty with many of the studies that address this question is the problem ofreverse causality. Changes in personality may drive social role changes rather than theother way around.
5
15
25
35
45
55
% s
corin
g at
mea
n ad
ult le
vel
Age
10 − 11 12 − 13 14 − 15 16 − 17 18 − 21 22 − 25 26 − 30
Intellectual ability
Psychosocial maturity
f0140 Figure 1.27 Proportion of Individuals in Each Age Group Scoring at or Above the Mean for 26- to 30-Year Olds on Indices of Intellectual and Psychosocial Maturity.
Source: From Steinberg, Cauffman, Woolard, Graham, and Banich (2009) submitted for publication.
fn1110221 Spear (2000a,b) also finds that sensation seeking reaches its peak in adolescence.
Hanushek_2011 978-0-444-53444-6 00001
Personality Psychology and Economics 125
Source: From Steinberg, Cauffman, Woolard et al. (2009), submitted for publication.
Heckman, Kautz Hard Evidence on Soft Skills 107 / 125
Introduction Origins Origins Modern References
Story also consistent with time constraints: birth of a child =⇒dropout
Relaxed time constraints as children age coupled with demandfor income (to support child) =⇒ GED certification
This is supported by our longitudinal analysis.
The women who take the GED are also better in certainnoncognitive dimensions, closer to the margin of dropping outand also closer to margin of reentry.
But a powerful factor explained in our book is that there arestrong external incentive systems that foster GED test takingeven in high schools.
Heckman, Kautz Hard Evidence on Soft Skills 108 / 125
Introduction Origins Origins Modern References
Schooling boosts cognitive and noncognitive traits.
Heckman, Kautz Hard Evidence on Soft Skills 109 / 125
Introduction Origins Origins Modern References
Figure 45: Causal Effect of Schooling on Measures on Cognition (fromASVAB)
Source: Heckman et al. (2006).
Heckman, Kautz Hard Evidence on Soft Skills 110 / 125
Introduction Origins Origins Modern References
Figure 46: Causal Effect of Schooling on Measures on Cognition (fromASVAB)
Source: Heckman et al. (2006).
Heckman, Kautz Hard Evidence on Soft Skills 111 / 125
Introduction Origins Origins Modern References
Figure 47: Causal Effect of Schooling on Two Measures ofSocioemotional Skills
Source: Heckman et al. (2006).
Heckman, Kautz Hard Evidence on Soft Skills 112 / 125
Introduction Origins Origins Modern References
Figure 48: Causal Effect of Schooling on Two Measures ofSocioemotional Skills
Source: Heckman et al. (2006).
Heckman, Kautz Hard Evidence on Soft Skills 113 / 125
Introduction Origins Origins Modern References
Perry Preschool Study: Family Supplementation Interventions BoostNoncognitive Traits
Early childhood intervention
Perry had lasting effects on life outcomes for both boys andgirls.
With a 7–10% annual rate of return for both(Heckman et al., 2010)
Did not boost IQ
Heckman, Kautz Hard Evidence on Soft Skills 114 / 125
Introduction Origins Origins Modern References
Figure 49: Cognitive Evolution Through Time, Perry Males: MaleCognitive Dynamics
Heckman, Kautz Hard Evidence on Soft Skills 115 / 125
Introduction Origins Origins Modern References
Boosted Achievement Test Scores
Heckman, Kautz Hard Evidence on Soft Skills 116 / 125
Introduction Origins Origins Modern References
Figure 50: Perry Age 14 Total CAT Scores, by Treatment Group
CAT = California Achievement TestTreatment: N = 49; Control: N = 46Statistically Significant Effect for Males and Females (p-values 0.009, 0.021 respectively)Source: Heckman, Malofeeva, Pinto, and Savelyev (2011).
Heckman, Kautz Hard Evidence on Soft Skills 117 / 125
Introduction Origins Origins Modern References
Boosted Measured Noncognitive Traits
Heckman, Kautz Hard Evidence on Soft Skills 118 / 125
Introduction Origins Origins Modern References
Figure 51: Personal Behavior Index, by Treatment Group
Heckman, Kautz Hard Evidence on Soft Skills 119 / 125
Introduction Origins Origins Modern References
Figure 52: Socio-Emotional Index by Treatment Group
Control Treatment
Heckman, Kautz Hard Evidence on Soft Skills 120 / 125
Introduction Origins Origins Modern References
Figure 53: Decomposition of Treatment Effects
Polarization
Argument
Skills
Evidence
Critical and Sensitive Periods
Environment
Intuitive
Estimates
Illustration
Summary
Figure 1: Decompositions of Treatment Effects, Males
90 100%0 10 3020 40 50 60 70 80
Personal Behavior
Cognitive Factors
Other Factors
Socio-Emotional State
CAT total*, age 14(+)
Employed, age 19 (+)
Monthly Income, age 27 (+)
No tobacco use, age 27 (+)
# of adult arrests, age 27 (-)
Jobless for more than 2 years, age 40 (-)
Ever on welfare (-)
Total charges of viol.crimes with victim costs, age 40, (-)
Total charges of all crimes, age 40 (-)
Total # of lifetime arrests, age 40 (-)
Total # of adult arrests, age 40 (-)
Total # of misdemeanor arrests, age 40 (-)
Total charges of all crimes with victim costs, age 40 (-)
Any charges of a crime with victim cost, age 40 (-)
Source: Heckman, Malofeeva, et al. (2011).Heckman, Kautz Hard Evidence on Soft Skills 121 / 125
Introduction Origins Origins Modern References
Costs of Neglecting Soft Skills
GED induces Dropping Out
GED produces deceptive statistics
Understates High School Dropout Rate
Distorts emphasis in schools under test-based incentive systems
Heckman, Kautz Hard Evidence on Soft Skills 122 / 125
Introduction Origins Origins Modern References
Summary
1 Soft skills get neglected in current emphasis on achievementtest scores in the economics of education and in educationalpolicy evaluations.
2 Lots of correlational evidence that soft skills predict lifetimesuccess.
3 Causal evidence harder to come by. Evidence from experimentalmediation analyses are consistent — as are causal estimates ofthe effects of education on skills.
4 GED testing program affords a natural experiment as do socialexperiments.
5 Compare people as bright as ordinary HSGs who don’t go tocollege with HSGs who go to college.
6 GEDs have lower levels of noncognitive — personality skills.
Heckman, Kautz Hard Evidence on Soft Skills 123 / 125
Introduction Origins Origins Modern References
7 Most GEDs do not succeed.8 Traits generally stable.9 But some people might benefit as their circumstances change.
10 People who benefit are concentrated among above average inability and noncognitive traits females who drop out because ofpregnancy and then GED certify in mid-20s. Most do not go tocollege.
11 Benefits come mostly through enhanced employment prospects.12 Whatever benefits arise are small in present value terms,
compared to graduating high school.13 Offsetting this is costs of dropout and deceptive social
statistics.14 Evidence from social experiments and studies of the effect of
schooling on growth of measured traits show that theses traitscan be changed by schooling interventions and enriched familyenvironments.
Heckman, Kautz Hard Evidence on Soft Skills 124 / 125
Introduction Origins Origins Modern References
References
Heckman, Kautz Hard Evidence on Soft Skills 125 / 125
Introduction Origins Origins Modern References
Ackerman, P. L. and E. D. Heggestad (1997). Intelligence,personality, and interests: Evidence for overlapping traits.Psychological Bulletin 121, 219–245.
Almlund, M., A. Duckworth, J. J. Heckman, and T. Kautz (2011,February). Personality psychology and economics. IZA DiscussionPaper (No. 5500). http://ftp.iza.org/dp5500.pdf.
Baldwin, J. (1995). Who Took the GED? GED 1994 StatisticalReport. Washington, D.C.: American Council on Education, GEDTesting Service, Center for Adult Learning.
Barrick, M. R. and M. K. Mount (1991). The Big Five personalitydimensions and job performance: A meta-analysis. PersonnelPsychology 44(1), 1–26.
Binet, A. and T. Simon (1916). The development of intelligence inchildren (The Binet-Simon Scale). Psychological Science.Baltimore, MD: Williams & Wilkins Co.
Boesel, D., N. Alsalam, and T. M. Smith (1998). Educational andLabor Market Performance of GED Recipients. Washington, DC:
Heckman, Kautz Hard Evidence on Soft Skills 125 / 125
Introduction Origins Origins Modern References
U.S. Dept. of Education, Office of Educational Research andImprovement, National Library of Education.
Borghans, L., A. L. Duckworth, J. J. Heckman, and B. ter Weel(2008, Feburary). The economics and psychology of personalitytraits. IZA Discussion Paper (3333).http://ftp.iza.org/dp3333.pdf.
Borghans, L., B. H. H. Golsteyn, J. J. Heckman, and J. E.Humphries (2011). IQ, achievement, and personality. Unpublishedmanuscript, University of Maastricht and University of Chicago(revised from the 2009 version).
Carroll, J. B. (1993). Human Cognitive Abilities: A Survey ofFactor-Analytic Studies. New York: Cambridge University Press.
Conard, M. A. (2005). Aptitude is not enough: how personality andbehavior predict academic performance. Journal of Research inPersonality 40(2006), 339–346.
Cunha, F., J. J. Heckman, and S. M. Schennach (2010, May).Heckman, Kautz Hard Evidence on Soft Skills 125 / 125
Introduction Origins Origins Modern References
Estimating the technology of cognitive and noncognitive skillformation. Econometrica 78(3), 883–931.
Heckman, J. J., J. E. Humphries, S. Urzua, and G. Veramendi(2011). The effects of educational choices on labor market,health, and social outcomes. Unpublished manuscript, Universityof Chicago, Department of Economics.
Heckman, J. J., L. Malofeeva, R. Pinto, and P. A. Savelyev (2011).Understanding the mechanisms through which an influential earlychildhood program boosted adult outcomes. Unpublishedmanuscript, University of Chicago, Department of Economics(first draft, 2008). Under revision, American Economic Review.
Heckman, J. J., S. H. Moon, R. Pinto, P. A. Savelyev, A. Shaikh,and A. Q. Yavitz (2006). The Perry Preschool Project: Areanalysis. Unpublished manuscript, University of Chicago,Department of Economics.
Heckman, J. J., S. H. Moon, R. Pinto, P. A. Savelyev, and A. Q.Yavitz (2010, February). The rate of return to the HighScope
Heckman, Kautz Hard Evidence on Soft Skills 125 / 125
Introduction Origins Origins Modern References
Perry Preschool Program. Journal of Public Economics 94(1-2),114–128.
Heckman, J. J., J. Stixrud, and S. Urzua (2006, July). The effectsof cognitive and noncognitive abilities on labor market outcomesand social behavior. Journal of Labor Economics 24(3), 411–482.
Jensen, A. R. (1998). The g Factor: The Science of Mental Ability.Westport, CT: Praeger.
Means, B. and J. H. Laurence (1984). Characteristics andperformance of recruits enlisted with general educationaldevelopment (ged) credentials. Technical Report FR-PRD-84-6,Human Resources Research Organization, Alexandria, VA.
Mischel, W. (1968). Personality and Assessment. New York: Wiley.
Moon, S. H. (2011). Decomposing racial skill gaps in the U.S.Unpublished manuscript, University of Chicago, Department ofEconomics.
Noftle, E. E. and R. W. Robins (2007). Personality predictors ofHeckman, Kautz Hard Evidence on Soft Skills 125 / 125
Introduction Origins Origins Modern References
academic outcomes: Big Five correlates of GPA and SAT scores.Journal of Personality and Social Psychology 93(1), 116–130.
Schmidt, F. L. and J. Hunter (2004). General mental ability in theworld of work: Occupational attainment and job performance.Journal of Personality and Social Psychology 86(1), 162–173.
Thaler, R. H. (2008, July). A short course in behavioral economics.Edge Master Class, Sonoma, CA, July 25-27, 2008.
Tyler, R. W. (1940, September). The place of evaluation in moderneducation. The Elementary School Journal 41(1), 19–27.
Wolfe, R. N. and S. D. Johnson (1995, April). Personality as apredictor of college performance. Educational And PsychologicalMeasurement 55(2), 177–185.
Heckman, Kautz Hard Evidence on Soft Skills 125 / 125