Hard-Soft Vargas DISPLAY 2012-05-01a Jsw

8/11/2019 Hard-Soft Vargas DISPLAY 2012-05-01a Jsw

1/398

Introduction Lessons History IQ Origins Modern Stability Appendix References

Hard Evidence on Soft Skills

James J. HeckmanUniversity of Chicago;

University College Dublin;Co-Director, Human Capital and Economic Opportunity WorkingGroup, Institute for New Economic Thinking

Fundacao Getulio VargasRio de Janeiro, Brasil

May 7, 2012

Heckman Hard Evidence on Soft Skills
http://find/http://goback/


2/398


We live in an era of widespread testing.

http://find/


3/398



Achievement tests in particular have assumed a prominent role.

http://find/


4/398




They are used to:


I d i L Hi IQ O i i M d S bili A di R f
http://find/


5/398




They are used to:i

Measure skills of persons (e.g., SAT, ACT, GRE, GED) andcertify suitability for admission and qualifications in a varietyof domains of life.

http://find/


6/398



7/398


These tests are not well understood.


http://find/


8/398


These tests are not well understood.a What do they measure?


http://find/


9/398



b How are they validated? What do they predict?


http://find/


10/398



b How are they validated? What do they predict?c What do they miss?


http://find/


11/398

y Q g y pp

Achievement tests missor perhaps betterdo not accuratelycapturesoft skills.


http://find/


12/398

y Q g y pp

Achievement tests missor perhaps betterdo not accuratelycapturesoft skills.

Personality traits, goals, motivations, and preferences that arevaluable in the labor market, in school, and in many otherdomains.


http://find/


13/398

y g y

A Multiple skills are useful in predicting and causing economicand social success but with different weights in different tasks.

http://find/


14/398


15/398



16/398


i People differ in their endowments of these skills.ii These differences in endowments and values of endowments

across tasks give rise to comparative advantage and sorting in

the labor market.B Personality traitssoft skillscan be measured.


http://find/


17/398





i They are often as predictive of many life outcomes as aremeasures of cognition.


http://find/


18/398





i They are often as predictive of many life outcomes as aremeasures of cognition.

ii Personality traits have causal effects on behavioral outcomes:changing traits through interventions changes life outcomes.


http://find/


19/398

C Traits are stable across situations.


http://find/


20/398

C Traits are stable across situations.i Yet they can also be changed in a gradual way through

investment and experience.


http://find/


21/398


investment and experience.ii Enhancing these traits is an important avenue for policy.


http://find/


22/398



D Ignoring personality traits can deceive: conceals major socialproblems by using achievement test scores as the measure of

success.


http://find/


23/398



D Ignoring personality traits can deceive: conceals major socialproblems by using achievement test scores as the measure of

success.E Situational specificity that we are what we need to be in any

specific situation plays a prominent role in modernbehavioral economics.


http://find/


24/398



D Ignoring personality traits can deceive: conceals major socialproblems by using achievement test scores as the measure ofsuccess.

E Situational specificity that we are what we need to be in anyspecific situation plays a prominent role in modernbehavioral economics.

F No evidence for extreme situational specificity of the sortadvocated by Walter Mischel (1968).


http://find/


25/398



D Ignoring personality traits can deceive: conceals major socialproblems by using achievement test scores as the measure ofsuccess.

E Situational specificity that we are what we need to be in anyspecific situation plays a prominent role in modernbehavioral economics.

F No evidence for extreme situational specificity of the sortadvocated by Walter Mischel (1968).

G There are enduring traits that persist and govern behavioracross multiple domains of economic and social life.


http://find/


26/398

I draw on recent research on the GED, an achievement test thatsecondary school dropouts in the U.S. can take to certify that

they are the equivalents (in cognition) of high school graduates.


http://find/


27/398

What Are the Lessons from This Research?

Success in life requires a lot more than what is measured byacademic achievement tests.


http://find/


28/398



Soft skills matter.


http://find/


29/398



Soft skills matter.

Personalitycharacter broadly definedcan be measured.




30/398



Soft skills matter.

Personalitycharacter broadly definedcan be measured.Personality can be fostered. It is an important, but neglected,component of what schools and families produce.




31/398



Soft skills matter.

Personalitycharacter broadly definedcan be measured.Personality can be fostered. It is an important, but neglected,component of what schools and families produce.

A focus on achievement tests fostered by movements for

accountability distorts the incentives of students andteachers.




32/398

A Brief History of Testing

Goal of developers of IQ and achievement tests: to produceobjective and comparable measures of children and schools.




33/398



Pioneers recognize their limits.


http://find/


34/398



Pioneers recognize their limits.

They understood that schools produced more than simplebook learning academic knowledge expressedfrustration about how to measure it.


http://find/


35/398

Early educators lacked a conceptual framework for thinkingabout what exactly should be measured.


http://find/


36/398

Early educators lacked a conceptual framework for thinkingabout what exactly should be measured.

They lacked the technology for implementing such tests on awide spread scale.


http://find/


37/398

IQ

IQ in its modern form is an early Twentieth Century invention.


http://find/


38/398

IQ

IQ in its modern form is an early Twentieth Century invention.It was conceived as a measure of the capacity to learn.


http://find/


39/398

IQ


First IQ test was designed to screen out misfits in school.


http://find/


40/398

IQ



Role of the IQ test was broadened to measure the ability to

solve abstract problems to capture intelligence andquickly became associated with g. (Terman and Thorndike)


http://find/


41/398

IQ





IQ was tested on a wide scale.


http://find/


42/398

IQ






The successful application of the IQ test suggested the

possibility of testing achievement what was learned notjust the ability to learn on a wide scale.


http://find/


43/398

IQ






The successful application of the IQ test suggested the

possibility of testing achievement what was learned notjust the ability to learn on a wide scale.

But the creators of IQ tests realized their limitations.


http://find/


44/398

Binet:

[Success in school] . . .admits of other things thanintelligence; to succeed in his studies, one must havequalities which depend on attention, will, and character;for example a certain docility, a regularity of habits, andespecially continuity of effort. A child, even if intelligent,will learn little in class if he never listens, if he spends histime in playing tricks, in giggling, in playing truant.

-Binet (1916, p. 254)


http://find/


45/398

What are the chief personality traits which, interactingwith g, relate to individual differences in achievement andvocational success? The most universal personality trait isconscientiousness, that is, being responsible, dependable,

caring, organized and persistent.

-Jensen (1998, p. 575)


http://find/


46/398

The modern conception of intelligence has moved well beyondg to a hierarchy of traits.


http://find/


47/398

The modern conception of intelligence has moved well beyondg to a hierarchy of traits.

But g is still at the top of the hierarchy.


http://find/


48/398

Figure: Modern View of g: An Hierarchical Scheme of GeneralIntelligence and Its Components

General

Intelligence

(Fluid Intelligence)Sequential Reasoning

Inductive Reasoning

Quantitative ReasoningPiagetian Reasoning

Math ReasoningQuantitative Reasoning

Math Problems

Visual PerceptionVisualization

Spatial Relations

Closure SpeedClosure Flexibility

Serial Perceptual Integration

Spatial Scanning

Imagery

ClosureClosure Speed

Closure Flexibility

Perceptual SpeedNumber Computation

RT and other Elementary Cognitive TasksStroop

Clerical Speed

Digit/Symbol

Learning and MemoryMemory Span

Associative Memory

Free Recall Memory

Meaningful Memory

Visual Memory

Knowledge and AchievementGeneral School AchievementVerbal Information and Knowledge

Information and Knowledge, Math and Science

Technical and Mechanical Knowledge

Knowledge of Behavioral Content

Ideational FluencyIdeational Fluency

Naming FacilityExpressional Fluency

Word FluencyCreativity

Figural Fluency

Figural Flexibility

Gc

(Crystallized Intelligence)Verbal Comprehension

Lexical Knowledge

Reading Comprehension

Reading SpeedCloze

Spelling

Phonetic Coding

Grammatical Sensitivity

Foreign LanguageCommunication

Listening

Oral Production

Oral Style

Writing

g=

g

g

f

c

Source: Recreated from Ackerman and Heggestad (1997), basedonCarroll (1993).


http://find/


49/398

Origins of the Modern Achievement Test

Achievement tests were created in the wake of the IQ test as away to capture the knowledge acquired in schools, not just thecapacity to learn.


http://find/


50/398



The distinction between knowledge and the ability to acquire itis clear.




51/398



The distinction between knowledge and the ability to acquire itis clear.

How one separates these concepts empirically is not so clear,especially since the ability to acquire knowledge is not a fixed

trait but can be changed.


http://find/


52/398

General Knowledge

The inventors of the achievement test sought to measureuseful knowledge or general knowledge valuable infunctioning at work and in society, not specific knowledge of a

course.


http://find/


53/398

General Knowledge


course.Designed to be objectivenot depend on teacherassessments as captured by grades. This was perceived to be away to implement meritocratic notions of education.


http://find/


54/398

General Knowledge


course.Designed to be objectivenot depend on teacherassessments as captured by grades. This was perceived to be away to implement meritocratic notions of education.

Iowa tests; ACT; GED; No Child Left Behind; NAEP; PISAtests are modern versions.


http://find/


55/398

Iowa Test of Educational Development (ITED)

Developed in 1930s and 1940s.


http://find/


56/398



The prototype for all achievement tests.


http://find/


57/398



The prototype for all achievement tests.Achievement synonymous with educational development.



What matters most is not how many detailed facts aperson can quote but how well he can make use of what
http://find/


58/398

person can quote, but how well he can make use of whathe knows, and how straight he can think about personaland group problems.

The Iowa Tests, therefore, were planned to measure thefundamental ideas the student has acquired permanently,and what he is able to do with them.

This seemed much more important than just measuringthe specific information he had memorized in a course,which he would forget after the final exam.

They were planned to measure the real knowledge and

reasoning ability a student had, no matter how he hadacquired them . . .

-Lindquist (1948)


http://find/


59/398

Achievement tests are designed to measure acquired knowledge.


http://find/


60/398


But what motivates students to acquire knowledge?


http://find/


61/398


But what motivates students to acquire knowledge?

Cognition, personality, as well as environmental determinants,such as parents, teachers, and peers.

http://find/


62/398



63/398

What Do Schools Produce?

Is it just the knowledge captured by standard achievement

tests?Every creator of an achievement test thought otherwise.


http://find/


64/398

Mann:

Hence to value schools, by length instead of quality, is amatchless absurdity. Arithmetic, grammar, and the otherrudiments, as they are called, comprise but a small part ofthe teachings in a school. The rudiments of feeling aretaught not less than the rudiments of thinking. The

sentiments and passions get more lessons than theintellect. Though their open recitations may be less, theirsecret rehearsals are more.

-Horace Mann (1867, p. 420)



If the descriptions of educational development ofindividual students provided by tests are to be truly
http://find/


65/398

comprehensive, tests and measuring devices must bedeveloped for many more educational objectives than arenow not being measured at all. In general, satisfactorytests have thus far been developed only for objectivesconcerned with the students intellectual development, orwith his purely rational behavior. Objectives concerned

with his non-rational behavior, or with his emotionalbehavior, or objectives concerned with such things asartistic abilities, artistic and aesthetic values and tastes,moral values, attitudes toward social institutions and

practices, habits relating to personal hygiene and physicalfitness, managerial or executive ability, etc., have beenseriously neglected in educational measurement.

- Lindquist(1951,pp. 137-138)Heckman Hard Evidence on Soft Skills

http://find/http://find/


66/398

About the GED Achievement Test

It should be emphasized ... that the General EducationalDevelopment [GED] Tests do not measure all theattributes that a high school attempts to develop in itsstudents (character, attitude, interest, etc.). The

Fact-Finding Study does not suggest that the high schoollevel General Educational Development Tests are asubstitute for a formal high school education.

- American Council on Education (1956, pp. 12)


http://find/


67/398

Other mechanisms of assessment besides the objective examwere often suggested.



W l h il itt i ti f t
http://find/


68/398

We lean heavily on written examinations, on a few typesof objective tests, and on the subjective impressions of

teachers. Many other appraisal devices could be used, suchas records of activities in which pupils participate,questionnaires, check lists, anecdotal records andobservational records, interviews, reports made by parents,

products made by the pupils, and records made byinstruments (motion pictures, eye-movement records,sound recordings, and the like).

-Tyler (1940)


http://find/


69/398

But time and again, the achievement test won out as a cheaptool of assessment.

http://find/


70/398



71/398

How are IQ and achievement tests validated?

Usually on grades and other tests.


http://find/


72/398

How are IQ and achievement tests validated?

Usually on grades and other tests.

This is ironic in light of the fact that the achievement test wasinvented in order to avoid the subjectivity of grades.



Table: Predictive Validities of Standard IQ and Achievement Tests
http://find/


73/398

Cognitive Achievement and IQ Tests

Test Domain overwhich it is

validated

Estimated Validities Source Notes

SAT First year col-lege GPA

0.35 to 0.53 Kobrin et al. (2008)

ACT Grades inearly years ofcollege

0.42 ACT Incorporated(2007)

Stanford-Binet

Correlationswith otherintelligencetests

0.77 to 0.87 with WISC-R Rothlisberg (1987);Greene et al. (1990)

WISC(WechslerIntelligenceScale for

Children)

Correlationswith aca-demicachievement

WISC: 0.443 to 0.751with WRAT tests, 0.482to 0.788 with 1st gradegrades, 0.462 to 0.794 with

2nd grade grades; WISC-R:0.346 to 0.760 with WRATtests, 0.358 to 0.537 with1st grade grades, 0.420to 0.721 with 2nd gradegrades

Hartlage and Steele(1977)

WRAT =Wide RangeAchievementTest; ranges

are givenbecause cor-relations varyby academicsubject

Source: Almund et al. (2011).

http://find/


74/398



C i i A hi d IQ T


75/398


Test Domain overwhich it is

validated


ASVAB(ArmedServicesVocationalAptitudeBattery)

Performancein militarytrainingprogramsand militaryattrition rates

0.37 to 0.78 for training(mean=0.56); -0.15 for at-trition

Schmidt et al.(1988) for perfor-mance in trainingprograms; Stichtet al. (1982) forattrition rates

Large rangein trainingcorrelationsdue to avariety of jobs

GED (Gen-eral Ed-ucationalDevelop-ment)

Test difficultyis normedagainst grad-uating HSseniors. Testscores of highschool seniorsand grades ofhigh schoolseniors

0.33 to 0.49 for HS SeniorGPA

Technical Manual:2002 Series GEDTests

Other Valid-ity Studies ofthe GED

Correlationwith otherachievementtests

.75.79 AFQT, .78 NALS,

.81 Iowa TestsBoesel et al. (1998)

Source: Almund et al. (2011).Heckman Hard Evidence on Soft Skills


T bl P di ti V liditi f St d d IQ d A hi t T t
http://find/


76/398



Test Domain overwhich it isvalidated


DAT (Dif-ferentialAptitudeTests)

Correlationswith aca-demicachievement

0.13 to 0.62 for collegeGPA

Omizo (1980) Large range isdue to vary-ing validity ofeight subtests

of DAT

WIAT(WechslerIndividualAchievementTest)

Correlationwith otherachievementtests; teacherratings of studentachievement

0.80 with grade 4 CAT/2,0.69 with grade 5 CAT/2,0.83 with grade 6 CAT/2;0.67 with teacher ratings

Michalko andSaklofske (1996)

CAT=CaliforniaAchievementTest

Source: Almund et al. (2011).


http://find/


77/398

Only recently has the validity of the tests on real worldoutcomes been established.


http://find/


78/398


They predict a bit, but they leave a lot unexplained.




79/398


They predict a bit, but they leave a lot unexplained.

Achievement tests and grades are usually more predictive thanIQ.



Predictive Validities in Outcomes that Matter (Adjusted R-Squared)

IQ AFQT GPA
http://find/


80/398

Q Q

Males IQ Pers Both AFQT Pers Both GPA Pers Both

Earnings at Age 35 0.07 0.05 0.09 0.17 0.07 0.18 0.09 0.06 0.12Hourly Wage at Age 35 0.07 0.03 0.08 0.13 0.06 0.14 0.07 0.06 0.09Hours Worked at Age 35 0.01 0.03 0.04 0.03 0.02 0.03 0.02 0.01 0.02Jail by Age 35 0.03 0.02 0.04 0.06 0.06 0.09 0.03 0.03 0.04Welfare at Age 35 0.01 0.00 0.01 0.03 0.01 0.03 0.01 0.00 0.01Married at Age 35 0.01 0.05 0.05 0.04 0.03 0.06 0.03 0.03 0.04BA Degree by Age 35 0.12 0.08 0.16 0.19 0.10 0.22 0.14 0.10 0.18Depression in 1992 0.01 0.05 0.05 0.04 0.04 0.06 0.02 0.04 0.04

Adj, R2 Cog, Personality 0.07 0.17 0.11

IQ AFQT GPA

Females IQ Pers Both AFQT Pers Both GPA Pers Both

Earnings at Age 35 0.01 0.03 0.03 0.09 0.05 0.11 0.05 0.04 0.07Hourly Wage at Age 35 0.05 0.03 0.06 0.12 0.05 0.14 0.06 0.04 0.08Hours Worked at Age 35 -0.00 0.02 0.02 0.00 0.01 0.00 0.00 0.01 0.01Jail by Age 35 -0.00 0.01 0.00 0.01 0.02 0.02 0.01 0.01 0.02Welfare at Age 35 0.02 0.04 0.05 0.10 0.05 0.12 0.05 0.05 0.07Married at Age 35 0.03 0.03 0.05 0.05 0.04 0.07 0.03 0.03 0.05BA Degree by Age 35 0.10 0.08 0.14 0.17 0.09 0.20 0.10 0.08 0.13Depression in 1992 0.02 0.05 0.05 0.04 0.05 0.07 0.02 0.05 0.05


Source: National Longitudinal Survey of Youth 1979.



Personality
http://find/


81/398

Personality and Motivation: missing from IQ tests butpredictive of adult performance.

Heckman Hard Evidence on Soft Skills Introduction Lessons History IQ Origins Modern Stability Appendix References

Personality
http://find/


82/398


Some of the missing dimensions manifest in these tables thatexplain a variety of educational and labor market outcomes.


Personality


83/398



These dimensions have begun to be systematically measured.


Personality
http://find/


84/398



These dimensions have begun to be systematically measured.One popular measurement system: Big Five OCEAN


Personality
http://find/


85/398




Openness


Personality
http://find/


86/398




OpennessConscientiousness


Personality
http://find/


87/398




OpennessConscientiousnessExtraversion


Personality
http://find/


88/398




OpennessConscientiousnessExtraversionAgreeableness


Personality
http://find/


89/398




OpennessConscientiousnessExtraversionAgreeablenessNeuroticism

http://find/


90/398


Table: The Big Five Domains and Their Facets

Extraversion an orientation of ones

interests and energies

toward the outer world

of people and things

rather than the inner

Warmth (friendly)

Gregariousness

(sociable)

Assertiveness (self-

confident)

Surgency

Social dominance

Social vitality

Sensation seeking

Shyness*


91/398

rather than the inner

world of subjective

experience;

characterized bypositive affect and

sociability

confident)

Activity (energetic)

Excitement seeking

(adventurous)Positive emotions

(enthusiastic)

Shyness

Activity*

Positive emotionality

Sociability/affiliation

Agreeableness the tendency to act in a

cooperative, unselfish

manner

Trust (forgiving)

Straight-forwardness (not

demanding)

Altruism (warm)

Compliance (not

stubborn)

Modesty (not show-off)

Tender-mindedness

(sympathetic)

Empathy

Perspective taking

Cooperation

Competitiveness

Irritability*

Aggressiveness

Willfulness

Neuroticism/

Emotional Stability

Emotional stability is

predictability and

consistency in emotional

reactions, with absence

of rapid mood changes.

Neuroticism is a

chronic level of

emotional instability and

proneness to

psychological distress.

Anxiety (worrying)

Hostility (irritable)

Depression (not

contented)

Self-consciousness (shy)

Impulsiveness (moody)

Vulnerability to stress

(not self-confident)

Internal vs. External

Locus of control

Core self-evaluation

Self-esteem

Self-efficacy

Optimism

Axis I

psychopathologies

(mental disorders)

including depressionand anxiety disorders

Fearfulness/behavioral

inhibition

Shyness*

Irritability*

Frustration

(Lack of) soothability

Sadness

O C E A NSource: Table adapted from John and Srivastava(1999).

http://find/


92/398

Big Five is the most widely used system.

http://find/


93/398


Criticisms of Big Five

http://find/


94/398



95/398


Criticisms of Big Five

The five-factor model is atheoretical.The five-factor model is relatively silent on an important classof individual differences: motivation and goals.

http://find/


96/398

Defining non-cognitive traits for children

Childhood temperament is the term used by developmentalpsychologists to describe the personalities of infants andchildren.

http://find/


97/398

Defining non-cognitive traits for children

Childhood temperament is the term used by developmentalpsychologists to describe the personalities of infants andchildren.

Most of the research on temperament has examined specificlower-order traits rather than broader, higher-level factors thatcharacterize studies of adult intelligence and personality.

H k H d E id S ft Skills Introduction Lessons History IQ Origins Modern Stability Appendix References
http://find/


98/398

Validities for Personality Measures

What are the validities for personality tests?

H k H d E id S ft Skill Introduction Lessons History IQ Origins Modern Stability Appendix References
http://find/


99/398



How correlated are measures with behaviors?

http://find/


100/398



How correlated are measures with behaviors?

First consider the validities established by the creators of thepersonality measurement schemes.


Table: Validities for Personality Tests
http://find/


101/398

Source: Almlund et al. 2011

H k H d E id S f Skill Introduction Lessons History IQ Origins Modern Stability Appendix References

Table: Validities for Personality Tests
http://find/


102/398


H k H d E id S f Skill Introduction Lessons History IQ Origins Modern Stability Appendix References
http://goforward/http://find/http://goback/


103/398

How Predictive of Real World Behavior Are ThesePersonality Traits?

What do they predict?


http://find/


104/398

How Predictive of Real World Behavior Are ThesePersonality Traits?

What do they predict?

With what strength?



Figure: Association of the Big Five and intelligence with Years ofSchooling

M l
http://find/


105/398

-0.1 -0.05 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35

Crystalized Intelligence

Fluid Intelligence

Openness

Conscientiousness

Extraversion

Agreeableness

Emotional Stability

StandardizedRegressionCoefficient

Males

Unadjusted for Intelligence Adjusted for Intelligence


http://find/


106/398


107/398


Figure: Associations with Job Performance

Emotional

Stability


108/398

Note. The values for personality are correlations that were corrected for sampling error, rangerestriction, and measurement error. Job performance was based on performance ratings,

productivity data and training proficiency. The authors do report the timing of the measurementsof personality relative to job performance. The value for IQ is a raw correlation.Source(s): The values reported for personality traits come from a meta-analysis conducted by

Barrick and Mount [1991]. The value for IQ and job performance was reported in Schmidt andHunter [2004].

0 0.1 0.2 0.3 0.4 0.5 0.6

Intelligence

Openness

Conscientiousness

Extraversion

Agreeableness

Correlation



Figure: Correlations of Mortality with Personality, IQ, andSocioeconomic Status (SES)
http://find/


109/398




Ever been in jail by age 30, by ability (males)

.15
http://find/


110/398

.05

.10

.00

Cognitive

0 20 21 40 41 60 61 80 81 100

Probability

Percentile

Source: Heckman, Stixrud, and Urzua (2006).



Ever been in jail by age 30, by ability (males)

.15
http://find/


111/398

.05

.10

.00

Noncognitive

Cognitive

0 20 21 40 41 60 61 80 81 100

Probability

Percentile




Probability of being single with children (females)

.10
http://find/


112/398

.08

.04

.06

.02

Cognitive

0 20 21 40 41 60 61 80 81 100

Probability

Percentile




Probability of being single with children (females)

.10
http://find/


113/398

.08

.04

.06

.02

Noncognitive

Cognitive

0 20 21 40 41 60 61 80 81 100

Probability

Percentile



http://find/


114/398

How Predictive of Success in Post-Secondary Education AreAchievement Tests Compared with Conscientiousness?



Table: The Relative Predictive Power of Conscientiousness and SATScores for College GPA
http://find/


115/398


http://find/


116/398

What Do Grades and Scores on Achievement TestsMeasure?

How are the scores on tests and grades determined?



f
http://find/


117/398

c

http://find/


118/398


Figure: Decomposing Achievement Tests and Grades into IQ andPersonality [Stella Maris]


119/398

Source: Borghans, Golsteyn, Heckman et al. [2011].


http://find/


120/398

This explains in part the greater predictive power ofachievement tests and grades compared to IQ in explaining realworld outcomes.



Predictive Validities in Outcomes that Matter (Adjusted R-Squared)

IQ AFQT GPA

Males IQ Pers Both AFQT Pers Both GPA Pers Both

Earnings at Age 35 0.07 0.05 0.09 0.17 0.07 0.18 0.09 0.06 0.12
http://find/


121/398

Hourly Wage at Age 35 0.07 0.03 0.08 0.13 0.06 0.14 0.07 0.06 0.09Hours Worked at Age 35 0.01 0.03 0.04 0.03 0.02 0.03 0.02 0.01 0.02Jail by Age 35 0.03 0.02 0.04 0.06 0.06 0.09 0.03 0.03 0.04Welfare at Age 35 0.01 0.00 0.01 0.03 0.01 0.03 0.01 0.00 0.01Married at Age 35 0.01 0.05 0.05 0.04 0.03 0.06 0.03 0.03 0.04BA Degree by Age 35 0.12 0.08 0.16 0.19 0.10 0.22 0.14 0.10 0.18Depression in 1992 0.01 0.05 0.05 0.04 0.04 0.06 0.02 0.04 0.04


IQ AFQT GPA

Females IQ Pers Both AFQT Pers Both GPA Pers Both

Earnings at Age 35 0.01 0.03 0.03 0.09 0.05 0.11 0.05 0.04 0.07Hourly Wage at Age 35 0.05 0.03 0.06 0.12 0.05 0.14 0.06 0.04 0.08Hours Worked at Age 35 -0.00 0.02 0.02 0.00 0.01 0.00 0.00 0.01 0.01Jail by Age 35 -0.00 0.01 0.00 0.01 0.02 0.02 0.01 0.01 0.02

Welfare at Age 35 0.02 0.04 0.05 0.10 0.05 0.12 0.05 0.05 0.07Married at Age 35 0.03 0.03 0.05 0.05 0.04 0.07 0.03 0.03 0.05BA Degree by Age 35 0.10 0.08 0.14 0.17 0.09 0.20 0.10 0.08 0.13Depression in 1992 0.02 0.05 0.05 0.04 0.05 0.07 0.02 0.05 0.05


Source: National Longitudinal Survey of Youth 1979.


http://find/


122/398

Causality:

Establishing causality is essential in designing policy andinterpreting evidence.


http://find/


123/398

Causality:


Correlation is not the same as causation, although a lot ofpublic policy is based only on correlations.



C
http://find/


124/398

Causality:


Correlation is not the same as causation, although a lot ofpublic policy is based only on correlations.

The doctors in Russia.



Questions
http://find/


125/398

Are any of the predictive relationships between personality andcognition and outcomes causal?



Questions
http://find/


126/398


Can we change cognition and personality and affect outcomes?



Questions
http://find/


127/398



Are they fixed traits?



Questions
http://find/


128/398



Are they fixed traits?Is promoting cognition and personality a useful policy option?



Questions
http://find/


129/398



Are they fixed traits?Is promoting cognition and personality a useful policy option?

What do we miss by ignoring soft skills?



Difficulties in Establishing Causality
http://goforward/http://find/http://goback/


130/398


Reverse causality(A B)



http://find/


131/398


Reverse causality(A B)

Measured traits can be caused in part by the outcomes beingstudied.



Correlation vs. CausalityA is correlated with B

What is causal?
http://find/


132/398




What is causal?
http://find/


133/398




What is causal?
http://find/


134/398




What is causal?
http://find/


135/398




What is causal?
http://find/


136/398

http://find/


137/398


138/398


All psychological measurements are calibrated on taskperformances.


139/398




A fundamental interpretive problem lies at the heart of any
http://find/


140/398

u da e ta te p et e p ob e es at t e ea t o a ypsychological measurement system for any particular trait.




http://find/


141/398

p p ypsychological measurement system for any particular trait.

For these traits to be accurate measures of any particular traitor set of traits, it is necessary to standardize for incentives and

the effects of other traits in performing a task.




http://find/


142/398



the effects of other traits in performing a task.Examples: Incentivizing IQ tests.




http://find/


143/398



the effects of other traits in performing a task.Examples: Incentivizing IQ tests.

Can boost IQ by 15 points by giving candies for correct answers the Black/White gap in IQ in U.S.



Table: Incentives and Performance on Intelligence Tests

Study Sample and Study

Design

Experimental

Group

Effect size of incentive

(in standard

deviations)

Summary

Edlund

[1972]

Between subjects

study. 11 matched

pairs of low SES

children; children

were about one

M&M candies

given for each

right answer

Experimental group

scored 12 points higher

than control group

during a second testing

on an alternative form of

a carefully chosen

consequence, candy, given

contingent on each occurrence

of correct responses to an IQ

test, can result in a
http://find/


144/398

were about one

standard deviation

below average inIQ at baseline

on an alternative form of

the Stanford Binet

(about 0.8 standarddeviations)

test, can result in a

significantly higher IQ

score.(p. 319)

Breuningand Zella[1978]

Within andbetween subjectsstudy of 485

special education

high schoolstudents all took IQtests, then wererandomly assignedto control orincentive groups toretake tests.Subjects werebelow-average in

IQ.

Incentives such asrecord albums,radios (


145/398

g ( ) g pincentives in a low-stakes administration of a measure of IQincreases performance substantially among roughly one-third ofparticipants.



Responsiveness to Incentives on IQ and Achievement TestsDepends on Personality

Segal (2006) shows that introducing performance-based cash
http://find/


146/398


Less conscientious people are particularly affected by incentives.



Responsiveness to Incentives on IQ and Achievement TestsDepends on Personality

Segal (2006) shows that introducing performance-based cash
http://find/


147/398


Less conscientious people are particularly affected by incentives.

Borghans, Meijers and ter Weel (2006) show that adults spendsubstantially more time answering IQ questions when rewardsare higher, but subjects high in emotional stability and

conscientiousness are much less affected by these incentives.



Personality traits affect IQ scores indirectly through theknowledge acquired by individuals who are more open toexperience, more curious and more perseverant.
http://find/


148/398




There is a correlation between cognitive and non-cognitive
http://find/


149/398

factors.




There is a correlation between cognitive and non-cognitive
http://find/


150/398

factors.

Hansen, Heckman and Mullen (2004), Heckman, Stixrud andUrzua (2006), and Urzua (2007) show how schooling and other

acquired traits substantially causally affect measured cognitiveand non-cognitive test scores.

http://find/


151/398



152/398


http://find/


153/398


http://find/


154/398



Causal Evidence on the Power of Personality

Present causal evidence on the power of soft skills in producinglife outcomes.
http://find/


155/398

http://find/


156/398





157/398

Evidence that soft skills can be fostered.

Evidence that soft skills are a mechanism of policy for reducing

poverty and promoting productivity and social justice.




http://find/


158/398



poverty and promoting productivity and social justice.Sources of Evidence




http://find/


159/398




i The GED Program




http://find/


160/398




i The GED Programii Perry Intervention




http://find/


161/398




i The GED Programii Perry Interventioniii Causal Effect of Schooling on Test Scores



The GED as a case study of the power of soft skills andcosts of neglecting them

Draw in part on a forthcoming book.
http://find/


162/398




Draw in part on a forthcoming book.Studies Of T he GED T esting P rogram,
http://find/


163/398

University of Chicago Press, 2012.




http://find/


164/398


GED is an achievement test that secondary school dropouts

can take to certify that they are the equivalents of ordinarysecondary school graduates.




http://find/


165/398


GED is an achievement test that secondary school dropouts

can take to certify that they are the equivalents of ordinarysecondary school graduates.

GED is a group of 5 achievement tests normed against nationalsamples of high school graduates (70% can pass).

http://find/


166/398


Use multiple data sets on outcomes, backgrounds, and abilities

for all major economic and social groups in the U.S overmultiple periods.


167/398





GED d h l d h d
http://find/


168/398

GEDs are as smart as secondary school graduates who do notgo on to college.





GED d h l d h d
http://find/


169/398

GEDs are as smart as secondary school graduates who do notgo on to college.

GEDs who go on to college and succeed are indistinguishablefrom other college graduates in terms of annual wage income(True for AA and BA students).



However, terminal GEDs perform at a level closer to that of

dropouts.


170/398




dropouts.Identical to dropouts if we control for their greater cognitiveability
http://find/


171/398

ability.




http://find/


172/398

ability.

We examine what essential life skills GEDs lack.




http://find/


173/398

ability.

We examine what essential life skills GEDs lack.

Comparing GEDs to Dropouts standardizes ability (as measuredby achievement tests) and demonstrates the importance ofpersonality traits in predicting life outcomes.



Figure: Cognitive ability by educational status (no college sample, allethnic groups)
http://find/


174/398

Source: Heckman, Humphries, Urzua, and Veramendi (2010)

http://find/


175/398


GEDs lack noncognitive personality traits measured inb h i d li


176/398

many ways: behaviors and personality test scores.



Figure: Personality

.6

.4

.2

0

.2

(a) Personality, Males

.6

.4

.2

0

.2

(b) Personality, Females


177/398

SelfEsteem

(NLSY79)

Locus ofControl

(NLSY79)

Locus ofControl(NELS)

SelfConcept(NELS)

.8

SelfEsteem

(NLSY79)

Locus ofControl

(NLSY79)

Locus ofControl(NELS)

SelfConcept(NELS)

Drop p


178/398

by 15(NLSY79)

by 15(NLSY79)

by 14(NLSY97)

Gr.8(NELS)

Crime(NLSY79)

by 15(NLSY79)

by 14(NLSY97)

Gr.10(NELS)

0

.1

.2

.3

.4

MajorCrime

(NLSY79)

Arrestedby 14

(NLSY97)

Prop Crimeby 14

(NLSY97)

Theftby 14

(NLSY97)

(c) Major Crimes

0

.2

.4

.6

.8

ViolentCrime

(NLSY79)

Fightby 14

(NLSY97)

Gangby 14

(NLSY97)

SchoolFight Gr.8

(NELS)

(d) Violent Behaviors

Drop p


179/398

by 15(NLSY79)

by 15(NLSY79)

by 14(NLSY97)

Gr.8(NELS)

Crime(NLSY79)

by 15(NLSY79)

by 14(NLSY97)

Gr.10(NELS)

0

.1

.2

.

3

.4

MajorCrime

(NLSY79)

Arrestedby 14

(NLSY97)

Prop Crimeby 14

(NLSY97)

Theftby 14

(NLSY97)

(c) Major Crimes

0

.2

.4

.6

.8

ViolentCrime

(NLSY79)

Fightby 14

(NLSY97)

Gangby 14

(NLSY97)

SchoolFight Gr.8

(NELS)

(d) Violent Behaviors

Drop p


180/398


Gaps in achievement and personality deficits emerge early.


181/398



Figure: Early Cognitive and Noncognitive Tests

1.5

1

.5

0

.5

Age 6 Age 8 Age 10 Age 12

(a) PIAT, Males

1

.5

0

.5


(b) PIAT, Females

(c) BPI Males (d) BPI Females
http://find/


182/398

.2

0

.2

.4

.6


(c) BPI, Males

.2

0

.2

.4

.6


(d) BPI, Females

Drop p


183/398

(Moon, 2011; Cunha et al., 2010)



Figure: Broken Home Rates

.4

.6

(a)Broken Home, Males

.4

.6

(b) Broken Home, Females
http://find/


184/398

0

.2

BrokenAge 14

(NLSY79)

BrokenAge 2

(

Documents

Hard-Soft Vargas DISPLAY 2012-05-01a Jsw