A meta-analysis of interventions to improve social competence in early childhood

Adam S. KennedyLoyola University Chicago

Outline

I. Introduction A. Social competence B. Children with special needs C. Social competence interventions

II. Meta-analysis A. Methods B. Results C. Discussion, limitations, next steps

2

I. Social competence in action

• Healthy peer relations (Vaughn et al., 2001)

• Positive self concept • Appropriate social skills (e.g., Rose-Krasnor, 1985)

• Healthy relations with adults (e.g., Guralnick et al., 2007)

• Accurate social perceptions• Productive internal attributions• Achieving one’s social goals (Renshaw & Asher, 1983)

3

In research on preschoolers

• Initiations• Responses• Behavioral change• Play invitations• Overall competence as perceived by others

4

Benefits of social competence(Odom et al., 1995)

• Reinforces protective web of general competence

• Higher social functioning and adjustment• Entry into advanced exchanges• Acknowledgement, acceptance, achievement

5

When social competence is poor

• Emotional and behavior disorders(e.g., McElwain & Volling, 2005)

• Rates of failure in kindergarten and beyond(e.g., Hinshaw, 1992; Ladd & Coleman, 1997; Missall & Hojnoski, 2008)

• Disciplinary action in school • Engagement with law enforcement

(Entwisle & Alexander, 1999; Raver, 2002)

• Dissatisfaction with life

6

Children with special needs• Less frequent interactions/conversations

(e.g., Stanton-Chapman et el., 2007)

• Fewer successful social bids with peers• More disruptive entry

(Guralnick et al., 1995; Kopp et al., 1992)

• More often rejected (Odom et al, 1998)

• Fewer friendships (Guralnick et al, 1995; Kopp et al., 1992)

• Less positive affect• Less likely to lead group play

(Guralnick et al., 1984-98)7

Types of interventions

• Naturalistic interventions/environmental arrangement (e.g., Brown & Odom, 1995; Odom et al., 1999)

• Social skills instruction (Vaughn et al., 2003)

• Behavioral strategies (e.g., Lowenthal, 1996)

– Structured play (DeKlyen & Odom, 1989)

• Combined approaches (Hundert & Houghton, 1992)

8

Delivery of these interventions

Family-based interventions (Sheridan et al., 2008)

Peer mediated interventions (Strain & Odom, 1986)

Adult/Teacher-mediated (Lowenthal, 1996; Brestan & Eyberg, 1998; Odom et al., 1999)

Caregiver-mediated (Webster-Stratton & Hammond, 1997)

Combined9

Relevant syntheses• Beerman (1994):

– Moderate overall effect: .45

• Vaughn et al (2003):– Moderate effects: social skills instruction– Large effects: BD, autism, teacher-mediated/behavioral,

generalization– Varied outcomes: HI, delay

• Various meta-analyses of ASD studies

10

Research questions 1. What are the features of studies examining the effects of

social competence interventions?How is social competence operationalized when studying preschool-aged children with special needs?

2. What does existing evidence suggest regarding the effects of social competence interventions?

3. Does variation exist in the effects of these interventions?4. If variation exists, can it be explained by study features?5. What future research directions are suggested?

11

Literature search• Databases

– PsycINFO– ProQuest Education Complete– ERIC via EBSCO– Academic Search Premier– Digital Dissertations– Government Printing Office– PapersFirst

• Tables of contents– TECSE, JEI, JABA, additional publications previously covered

• Secondary search of reference lists/authors• Literature reviews

12

Inclusionary Criteria

• 1965 and 2008• Participants of preschool age• Group designs with control• Published or unpublished• Measures of social skills, competence or

multiple dependent measures

13

CodingStudy variables

● source● context of study● dependent measures● treatment fidelity● follow-up● generalization● differences at pre-

test● randomization

Intervention variables● type● interventionist● duration● context● setting

Participant variables● demographics● disability categories● ethnicity

14

Inter-rater reliability

15

Coding category Percent agreement

Bibliographical information 100%

Study context 100%

Participants 97.7%

Intervention 95.0%

Methods 100%

Effect size 96.0%

OVERALL 98.1%

IV. Results• Features of the published literature • Overall effect size• Effects by intervention type and subtype• Effects by participant characteristics• Summary

16

Initial search results

Search Engine Results Screened Coded/Included

ERIC 13609 178 11 / 8

PsychINFO 4583 40 8 / 4

ProQuest Dissertations & Theses 1721 60 10 / 8

EBSCO Academic Search Premier 789 80 7 / 3

ProQuest Education Complete 778 78 5 / 2

Government Printing Office Publications 42 12 1 / 0

PapersFirst 13 2 2 / 0

TOTAL 21535 450 44/25 17

Results of hand search of relevant publicationsPublication Hand search results

1965-69 1970-79 1980-89 1990-99 2000-09

S R S R S R S R S R

Child Development

4 0 3 0 0 0 2 0 0 0

Early Childhood Development and Care

First published

19710 0 3 0 3 0 2 0

Journal of Applied Dev. Psychology

First published1980 3 0 0 0 0 0

Topics in Early Child-hood Special Education

First published1981 1 0 3 0 2 0

Early Childhood Research Quarterly

First published1986 1 0 2 0 0 0

Early Childhood Research and Practice

First published 1999 0 0 0 0

Total 4 0 3 0 8 0 10 0 4 0

S=screened R=retained

18

Representation of studiesacross time periods

1965-69 1970-79 1980-89 1990-99 2000-09

Total

Number of included studies

0 3 3 12 8 25

Number of effect sizes 0 3 4 15 10 32

% of total effect sizes 0% 9% 13% 47% 31%

19

Affiliation of primary authors

Discipline # of studies % of total studies

Psychology/Child development 11 44%

Education 6 24%

Unknown 5 20%

Public health/Public policy 2 8%

Social work 1 4%

Total 25 100%

20

Contexts of included studies

Context # of studies

ParticipantN

% of total N

Urban 9 708 35%

Metropolitan area 4 606 30%

Not reported 11 605 30%

Suburban 1 100 5%

Total 25 2019 100%

21

Intervention settings in included studies

Setting # of studies

ParticipantN

% of total N

Preschool 14 913 45%

Head Start 5 772 38%

Elementary school 3 73 4%

Day care 2 229 11%

Lab school 1 32 2%

Total 25 2019 100%

22

Ethnicity of participating children

Ethnicity Total N % of total N

Caucasian 368 18%

Non-white, predominantly African-American

763 38%

Not reported 888 44%

Total 2019 (100%)

23

Disability categories of participating children

Category Treated groups Treated N % of total treated N

Hearing impairment 2 33 3%

Speech and language disorder 2 83 9%

Emotional disorder/abused 3 144 15%

At risk due to economic disadvantage and/or social isolation

15 597 61%

Diverse, containing a range of disability categories

10 119 12%

Total 32 976 (100%) 24

Duration of interventions in included studies

Duration # of treated groups

Treated N

% of total treated N

Up to 2 weeks 3 33 3%

2 weeks to 1 month 1 8 1%

1 to 3 months 15 312 32%

4 to 6 months 6 156 16%

7 to 9 months 5 467 48%

Total 32 976 (100%)

25

Findings: Random effects model• 25 included studies with 31 effect sizes• Homogeneity Q = 70.997 (p<0.001)

• Mean ES = 0.388* (0.230 to 0.546)• Range = -0.652 to 1.392

26

Funnel plot for analysis of publication bias

-2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0

0.0

0.2

0.4

0.6

0.8

Stan

dard

Error

Std diff in means

Funnel Plot of Standard Error by Std diff in means

27

Moderators: Publication type/periodModerator k d 95% confidence interval Qb

Low estimate

High estimate

Publication type 0.143 (0.471)

Published 23 0.358** (0.421**)

0.254 (0.249)

0.462 (0.594)

Unpublished 8 0.316** (0.273)

0.122 (-0.113)

0.509 (0.659)

Time period 10.036*(3.741)

Pre-1975 1 1.173 0.188 2.157

1976-1990 6 0.639 (0.508**)

0.319 (-0.137)

0.958(1.153)

1991-1997 4 0.614(0.527**)

0.293(-0.220)

0.935(1.274)

1998-2004 14 0.276**(0.279**)

0.121(0.074)

0.430(0.483)

2004-2009 6 0.297** 0.149 0.445

Moderators: Intervention duration

Moderator k d 95% confidence interval

Qb

Low estimate

High estimate

Duration of intervention 8.107(6.833)

Up to 2 weeks 3 1.162** 0.389 1.935

2 weeks to 1 month 1 -0.108(-0.165)

-0.720(-1.041)

0.504(0.711)

1 to 3 months 15 0.416**(0.439**)

0.258(0.206)

0.575(0.672)

4 to 6 months 6 0.400**(0.360)

0.143(-0.219)

0.658(0.938)

7 to 9 months 5 0.287** (0.290**)

0.158 (0.152)

0.417(0.428)

29

Moderators: Disability categories

Moderator k d 95% confidence interval

Qb

Low estimate

High estimate

Disability categories represented

8.017(2.722)

Hearing impairment 1 -0.050 -0.661 0.561

Speech/language delay/disorder

2 0.024(-0.079)

-0.397(-1.195)

0.445(1.037)

Emotional/behavioral problems

7 0.452**(0.432*)

0.236(0.091)

0.667(0.773)

At-risk 11 0.278**(0.388**)

0.142(0.163)

0.414(0.613)

Variety of disabilities 10 0.476**(0.429**)

0.307(0.114)

0.646(0.745)

Moderators: Intervention type

Moderator k d 95% confidence interval Qb

Low estimate

High estimate

Intervention type 3.256 (3.723)

Environmental arrangement

1 0.020 -0.653 0.694

Instructional 26 0.361**( 0.409**)

0.265(0.238)

0.457(0.581)

Behavioral 3 0.444**(0.537)

0.033(-0.151)

0.856(1.224)

Combined 1 -0.114 -0.744 0.516

31

Moderators: Dependent variable

Moderator k d 95% confidence interval

Qb

Low estimate

High estimate

Indicator of social competence 3.911 (0.829)

Emotional knowledge 1 0.467** 0.214 0.721

Cooperative play behaviors 10 0.372**(0.399**)

0.193(0.201)

0.551(0.597)

Overall social skills/competence

17 0.282**(0.310**)

0.159(0.076)

0.405(0.544)

Social problem-solving 3 0.626**(0.456)

0.230(-0.569)

1.023(1.481)

32

Moderators: Study quality

Moderator k d 95% confidence interval

Qb

Low estimate

High estimate

Effort to randomize 1.081 (0.236)

No 7 0.478**(0.485*)

0.217(0.022)

0.738(0.949)

Yes 24 0.330**(0.363**)

0.232(0.197)

0.428(0.530)

33

Moderators: Study quality

Moderator k d 95% confidence interval

Qb

Low estimate

High estimate

Differences at pre-test 5.702 (2.640)

Not assessed 18 0.297** (0.332**)

0.154(0.118)

0.440 (0.545)

None, or controlled for 12 0.483**(0.482**)

0.335(0.204)

0.631(0.759)

Fidelity measured 3.314 (1.437)

No 19 0.458**(0.468**)

0.309( 0.187)

0.608(0.750)

Yes 12 0.282** 0.166 0.398

Moderators: Study quality

Moderator k d 95% confidence interval

Qb

Low estimate

High estimate

Measure of generalization 0.767 (0.594)

No 30 0.343**(0.380**)

0.251(0.219)

0.436(0.541)

Yes 1 0.687 -0.076 1.450

Follow-up measure 0.018 (0.001)

No 21 0.345**(0.389*)

0.244(0.218)

0.447(0.560)

Yes 10 0.361(0.382)

0.150(-0.004)

0.572(0.768)

Revisiting previous syntheses• Beerman (1994):

– overall effect: .45 – Moderate effect: social skills– Small effect: cognition/affect, adjustment

• Vaughn et al (2003):– Small effects: individual studies– Moderate effects: social skills instruction– Large effects: autism, teacher-mediated/behavioral,

generalization– Varied outcomes: HI, delay

36

Summary of findings• Majority published studies from 1990-2009• Even distribution across contexts with exception

of rural• Primarily in Head Start and preschool

classrooms• Majority of ethnicity unknown, 38% from

primarily African-American groups• At risk, EBD/abused, diverse groups best

represented37

Summary of findings• Social competence interventions are effective

– Diverse groups with special needs• Hearing impairment/speech & language?

– Embedding into existing curricula– Instruction and supported practice

• Overall consistency with previous syntheses• Small, diverse studies despite preferred approach• Underrepresentation of intervention types,

disability categories, and contexts• Social competence = overall teacher ratings

38

Summary of findings

• Little to no evidence base for environmental arrangement, combined interventions

• Instructional and behavioral approaches• Study quality

– Efforts to address internal validity– Some efforts to address group differences and

treatment fidelity• Failure to meaningfully assess generalization

– follow-up?

Limitations in published research

• One definition fits all• One intervention fits all• Preschoolers as a hidden subset• Sensitivity of measures• Intervention types and impacts on

competence variables• Interaction of interventionist and intervention• Where are caregiver and peer voices?

40

Other limitations

• Incomplete picture of the literature

• Failure to locate earlier studies

• Competence as unitary construct

• Oversimplification of categories

41

Preschool social competence

-Observed social skills

-Teacher and parentPerceptions

-Peer perceptionsand preferences

-Social perspective taking

Social adjustment- Achievement

- Adult outcomes

Inside-Out

Outside-In

Neurology & brain development

Temperament, self-regulation, emotional competence

Gender

Cognitive and social-cognitive skills

Disability

Language-communication skills

Family

Classroom and teacher quality

Early intervention

Peer group and friendships

Culture-ethnicity and linguistic factors

Whitehurst, G.J., & Lonigan, C.J. (1997). Child development and emergent literacy. Child Development, 69, 848-872.

Influences on social competence

42

Developmentally appropriate practices

Inclusive preschool education

Social competence curriculaAffective preschool interventions

Incidental teaching of social behaviors

Friendship activities

Social integration activities

Buddy skills training

Classroomwide interventions

Naturalistic peer interaction interventionsNaturalistic peer interaction interventions

Explicit social skills interventions

From “An intervention hierarchy for promoting young children’s peer interactions in natural environments” by W.H. Brown, S.L. Odom, and M.A. Conroy, 2001, Topics in Early Childhood Special Education, 21, 162-175.

Hierarchy for promoting young children’s peer interactions

43

Future directions• Synthesize SSD studies and compare

interpretive methods• Cross-cultural comparison of competence

interventions and findings• Translating findings into practice

– Culturally-relevant interventions– Longitudinal effects

44

Stay in touch!

Adam S. Kennedy, Ph.D.

School of Education

Loyola University Chicago

820 N. Michigan Ave.Chicago, IL  60611(312) 915-6857(312) 915-6660 (Fax)akenne5@luc.edu