Improving Story Writing Skills of Students with Internalizing and

Externalizing Behaviors with Poor Writing Skills:

Preliminary Findings of Project WRITE

Kathleen Lynne Lane, Karen Harris, and Steve Graham

Vanderbilt University

Funded by IES, Project WRITE, The Effects of Strategy and Self-Regulation Instruction on Students’ Writing Performance and Behavior: A Preventative Approach

Students with EBD

The prevalence estimates of students with EBD vary between 2-20% of the school population, with most realistic estimates of 3-6% (Kauffman, 2005)

Behavioral, social, and academic deficits typical of this population become less amenable to intervention efforts over time, particularly after age 8 (Bullis & Walker, 1994; Kazdin, 1987)

Some treatment outcome studies conducted with students at-risk for EBD revealed that increased early literacy skills were associated with decreases in disruptive behavior in the classroom (Lane et al., 2001; Lane et al., 2003)

Academic Characteristics

Moderate to severe, broad academic deficits (Greenbaum et al., 1996; Mattison et al., 1998).

At best, the academic deficits characteristic of this population remain stable over time (Mattison et al., 2002); at worst, they deteriorate (Nelson et al., 2004).

“… students with EBD probably experience less school success than any other subgroup of students with or without disabilities.” (Landrum et al., 2003, p. 148).

Academic Interventions to Date

Promising outcomes, yet limited by Inadequate attention given to middle and high school

students Heavy emphasis on basic skills Absence of core components: treatment integrity; social

validity; and generalization and maintenance Incomplete intervention descriptions Few replications

(Lane, 2004)

Interventions to Date

Only a few studies have been conducted to explore how to best improve the writing skills of young students with EBD (Glomb & West, 1990; Mason & Shriner, in press; Rumsey & Ballard, 1985)

Like reading, writing is critical to school success as it provides students with a vehicle for gathering, organizing, and sharing information as well as refining one’s thoughts (Adams, Treiman, & Pressley, 200; Bangert-Drowns, Hurley, & Wilkinson, 2004; Graham, 2005)

Self-Regulated Strategies Development

Addresses three major goals:1. Students learn to carry out specific composing

process

2. Students develop the knowledge and self-regulatory procedures needed to apply the writing strategies and regulate their behavior during writing

3. Enhance specific aspects of motivation, including self-efficacy and effort

Academic Support Is Essential

The needs of students with EBD are many and the consequences of not adequately meeting these needs are deleterious.

Thus, it is necessary to identify the most effective, efficient methods for supporting students who demonstrate clear behavioral and academic deficits that impede their educational progress.

Positive Behavior Support

For students with behavioral challenges, PBS provides graduated support as needed to (a) prevent the development of behavioral problems that may lead to EBD and (b) support students with EBD via more intensive supports.

Primary Prevention

Response: 80% of Students

Secondary Prevention

Response: 10-15% of Students

Tertiary Prevention

Response: 1-5% of Students

Concerns

Many models are incomplete or underdeveloped

Little attention to: systematically identifying students who require

more focused prevention efforts conducting scientifically rigorous investigations

at the secondary level to meet the academic needs of students with or at risk for EBD (Cheney et al., 2004; Lane et al., 2003; Lane et al., 2002; Walker et al., 2005).

SSBD Screening ProcessPool of Regular Classroom Students

TEACHER SCREENINGon Internalizing and Externalizing Behavioral

Dimensions

3 Highest Ranked Pupils on Externalizing and on Internalizing Behavior Criteria

TACHER RATINGon Critical Events Index and Combined

Frequency Index

Exceed Normative Criteria on CEI of CFI

DIRECT OBSERVATIONof Process Selected Pupils in Classroom

and on Playground

Exceed Normative Criteria on AET and PSB

PASS GATE 1PASS GATE 1

PASS GATE 2PASS GATE 2

PASS GATE 3PASS GATE 3

Pre-referral Intervention(s)

Child may be referred to Child Study Team

Student Risk Screening Scale

The SRSS is 7-item mass screener used to identify students who are at risk for antisocial behavior.

Teachers evaluate each student on the following items- Steals - Low Achievement- Lies, Cheats, Sneaks - Negative Attitude- Behavior Problems - Aggressive Behavior- Peer Rejection

Student Risk is divided into 3 categories- Low 0 – 3- Moderate 4 – 8- High 9 – 21

The SRSS is validated for use at the elementary school level.

(SRSS; Drummond, 1994)

Student Risk Screening Scale

Purpose

This study examined the effectiveness of teaching second grade students at risk for EBD how to write stories using the SRSD model.

Extends the literature by: examining the effectiveness of a secondary writing

intervention in the context of an established PBS model. applying the early intervention SRSD research to a new

population – students who are at risk for EBD and have co-occurring difficulties with writing

Elementary School: Primary Plans

Mission & School-wide Expectations

Procedures for Teaching Eagle Pledge Posters

Procedures for Reinforcing Tickets Classroom Drawings and School-wide Quarterly Assemblies

Procedures for Monitoring Student Outcomes Treatment Integrity & Social Validity

Setting

Classroom Hallway Playground Cubbies Cafeteria Restrooms

Respect -Eyes on speaker

-Listen

-Follow directions

-Space & materials

-Walk

-Quiet

-Hands off walls

-Hands to self

-Follow game rules

-Respond when called

-Share/include everyone

-Use your own stuff

-Wait your turn

-4 people at a time

-Hands to self

-Low voice

-Eat your own food

-Clean up

-Clean up

-Allow privacy by only knocking on door once

Best Effort -Quality of work is accurate, neat, and complete

-Watch the person in front of you

-Zone Zero

-Walk on 3rd square

-Pick up trash

-Report unsafe objects or behavior

-Help keep area safe and clean

-Eat, then talk

-Stay in seat

-Keep floor clean

-Appropriate use of facilities

Responsibility

-Follow Eagle Expectations

-Be on time

-Be prepared

-Observe rules of tattling vs. telling

- Be Aware of personal space

-Report visitors without stickers

-Remind others to be quiet

-Be Aware of personal space

-Follow playground rules

-Play approved games

-Put materials and clothes where they belong

-Unpack all materials

-Know what to order

-Bring lunch/ lunch money

-Get all items

-Raise hand

-Wash hands

-clean up after self and others

-Report messes

Care of Property

-Respect all materials

-Pick up trash

-Hands off walls and signs

-Pick up trash

-Respect nature

-Use equipment appropriately

-Clean space

-Belongings in proper places

-Pick up trash

-Keep your area clean

-Pick up trash

-Wipe seat

-Keep walls & floor clean

-Flush

Elementary Assessment Schedule Measure Quarter 1 Quarter 2 Quarter 3 Quarter 4

Attendance X X X X

SWIS (ODR) X X X X

GEIT & SPED X X X X

IAI Reading & Math X X

STARS Reading X X

TOWL (Writing) X

TCAP X

Tickets X X X X

SRSS & SSBD X X X

Treatment Integrity X X X X

EBS Survey X X

Social Validity X X

Project WRITE Overview

Students exceeding criteria on behavioral screeners

Students exceeding criteria on

writing screeners

Screening: 2nd grade studentScreening: 2nd grade student

SRSD Instruction for each consenting student with both

writing and behavioral deficits

SRSD Instruction for each consenting student with both

writing and behavioral deficits

Participants

Participants were 13 second-grade students (8 boys, 5 girls) attending four inclusive, rural elementary schools in middle Tennessee.

Students were identified using systematic screening procedures as having either internalizing (n = 6) or externalizing (n = 7) behavioral concerns in addition to poor writing skills.

All students were Caucasian, ranging in age from 7.03 to 9.01 years.

Three students, Kate, Eva, and Chuck, had below average intellectual functioning and one student, Hector, had above average intellectual functioning as measured by a short form of the Wechsler Intelligence Scale for Children – Fourth Edition (WISC-IV; Wechsler, 2003)

Students with Internalizing Behaviors

Variable

Student (leg of study)

Ella(1) Chuck(1) Barry(2) Mandy(2) Brian(3)Amos(3

)

Demographics

Age in years 8.06 7.04 7.04 8.00 7.10 8.03

Gender Female Male Male Female Male Male

Ethnicity C C C C C C

IQ 68 83 112 94 86 88

Screening TOWL-3 5 9 25 16 9 9

SSRS-T

SS (SS) 77 107 89 83 93 91

PB (SS) 112 104 102 121 106 115

AC(SS) 71 99 85 92 72 86

SSRS-PSS (SS) 98 95 104 90 76 87

PB (SS) 105 120 110 105 137 131

SARS

Schools Attended 2 3 3 1 1 4

Days absent 3 9 * 3 11 6

Special Education No No No No No No

Note. SS=Standard Score; TOWL-3=Test of Written Language; *=information not available; C=Caucasian; IQ=intelligence quotient; SSRS-T=Social Skills Rating System–Teacher version; SSRS-P= Social Skills Rating System–Parent version; SARS=School Archival Record Search

Students with Externalizing Behaviors

Variable

Student (leg of study)

Nick(1)

Dina(1)

Kay(2)

Harry(2)

Amy (3)

Ron(3)

Ben(3)

Demographics

Age in years 7.09 7.08 9.01 8.02 7.03 7.10 7.07

Gender Male Female Female Male Female Male Male

Ethnicity C C C C C C C

IQ 97 94 68 112 120 112 103

Screening TOWL-3 25 25 9 9 16 16 9

SSRS-TSS (SS) 85 85 73 89 87 86 82

PB (SS) 127 116 130 112 127 113 113

AC (SS) 97 101 77 82 98 70 86

SSRS-P SS (SS) 76 84 69 116 86 120 98

PB (SS) 133 112 143 92 * 100 110

SARS

Schools Attended 1 1 1 1 1 2 1

Days absent 7 11 17 14 2 10 3

Special education No No Yes No No No No

Note. SS=Standard Score; TOWL-3=Test of Written Language; *=parent did not return form; C=Caucasian; IQ=intelligence quotient; SSRS-T=Social Skills Rating System–Teacher version; SSRS-P= Social Skills Rating System–Parent version; SARS=School Archival Record Search; NBC=negative behavioral comments

Identifying Nonresponsive Students with Behavior & Writing Concerns

Behavior Concern Systematic Screening for Behavior Disorders

(SSBD; Walker & Severson, 1992) Exceeding norms: Internalizing or Externalizing

Writing Difficulties Test of Written Language-3 (TOWL-3; Hammill & Larsen, 1996):

At or below 25th percentile

Consenting Obtained teacher consent (100%) Obtained parent consent (87%) Obtained student assent (100%)

Intervention Description

Self-Regulated Strategy Development(SRSD; Harris & Graham, 1996)

Behavioral Component

3 days a week; 30-min sessions delivered 1:1 by a research assistant

Fidelity collected on 35.77% of sessions

SRSD Stages & Modifications

Stages Develop Background

Knowledge Discuss It Model It Memorize It Support It Independent Performance

Modifications: Increased time and opportunities

to master the first two stages Instructional sequence: student

self-evaluation and graphing of a pre-instruction story moved from Discuss It to Support It

Verbal reinforcement & opportunities to respond

PBS Tickets Practice test

Two Mnemonic Strategies for Story Writing

1. POW for planning

Pick my idea

Organize my notes

Write and say more

2. WWW, What=2, How=2

Who?

When?

Where?

What does main character do?

What happens then?

How does the story end?

How does the main character feel?

Assessment

Writing probes Story elements (0-7) Length Quality

Social validity Treatment integrity Descriptive measures

Experimental Design & Statistical Analysis

Multiple baseline across participants design with multiple probes during baseline (Kennedy, 2005)

Data analyzed via visual inspection to examine stability, level, and trend, as well as mean changes by phase and percentage of non-overlapping data points.

Phase changes were based on number of story elements.

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

EllaChuck

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

Nu

mb

er o

f S

to

ry

E

lem

en

ts

Barry

Mandy

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

Probes

Brian

Adam

Story Elements Overtime: Students with Internalizing Behaviors

Story Elements: Students with Internalizing Behaviors Leg 1

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

Ella

Chuck

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

Nu

mb

er o

f S

tory

Ele

men

ts

Barry

Mandy

Story Elements: Students with Internalizing Behaviors Leg 2

Story Elements: Students with Internalizing Behaviors Leg 3

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

Probes

Brian

Adam

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

NickDina

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

Num

ber

of S

tory

Ele

men

ts

KayHarry

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

Probes

AmyRonBen

Story Elements Overtime: Students with Externalizing Behaviors

Story Elements: Students with Externalizing Behaviors Leg 1

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

NickDina

Story Elements: Students with Externalizing Behaviors Leg 2

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13

Num

er o

f S

tory

Ele

men

ts

KayHarry

Story Elements: Students with Externalizing Behaviors Leg 3

0

1

2

3

4

5

6

7

1 2 3 4 5 6 7 8 9 10 11 12 13Probes

AmyRonBen

Results

In addition to changes in the number of story elements present, there were also changes in the story length and, to a lesser extent, quality.

Pre Intervention Probe: Sample from Internalizing Group Brian

My big dog is so heavy that I can’t pull the wagon. I can’t even pick up my dog.

Post Intervention Probe:Sample from Internalizing Group Brian

This happened many years ago. There was a little dog and a big dog on an island. The little dog jumped out to sea. He wanted to get to the island. He swam to the island. He felt sad because the little dog jumped in the sea.

Maintenance Probe:

Sample from Internalizing Group Brian

Years ago, the birds were in a tree. They wanted to eat the dragonfly. They started fighting because the other birds wanted to eat the dragonfly. They stopped fighting. They were happy because they stopped fighting.

Pre Intervention Probe: Sample from Externalizing Group Kay

The Casper High was making a play Alex was a star Jim was the sun and Tim was the cloud.

Post Intervention Probe: Sample from Externalizing Group Kay

Tiger was having a birthday party. Joey, Lennie, Oscar, and Fritz were going to have a surprise party. They set up all the decorations. Oscar made a cake. Joey blew up the balloons. Lennie got the activities. Fritz did the final thing, he got Tiger to come over. When he got in the party started. Tiger said it was the best party he ever had.

Maintenance Intervention Probe: Sample from Externalizing Group Kay

Jordan went to Fisher Lake and caught the biggest fish in the lake. She brought the fish home. When she got home Jordan showed it to her dad. Jordan’s dad was scared when he saw the fish. Jordan had an idea. They had a big lake. Jordan threw the fish into the lake and the fish swam away.

SocialValidity

TreatmentFidelity

(Days of instruction)

Intervention Phase 

IRP-15 CIRP % (SD)

Ella (14)Baseline 90 30Intervention 95.83 (8.33)Post 90 31

Chuck (10)Baseline 85 32Intervention 100.00 (0.00)Post 89 42

Barry (11)Baseline 87 37Intervention 97.22 (5.56)Post 76 40

Mandy (11)Baseline 76 31Intervention 96.43 (7.14)Post 90 35

Brian (9)Baseline 75 32Intervention 96.67 (5.77)Post * 37

Amos (9)Baseline 58 37Intervention 100.00 (0.00)Post 74 33

Students with Internalizing Behaviors

Social Validity Tx Fidelity(Days) Phase IRP-15 CIRP % (SD)

Nick (13)Baseline 88 34Intervention 90.83 (10.17)Post 79 37

Dina (11)Baseline 89 42Intervention 100.00 (0.00)Post 82 42

Kay (9)Baseline 88 37Intervention 100.00 (0.00)Post 79 42

Harry (10)Baseline 77 31Intervention 94.44 (9.62)Post 78 32

Amy (7)Baseline 84 39Intervention 100.00 (0.00)Post 88 39

Ron (8)Baseline 85 42Intervention 100.00 (0.00)Post 88 37

Ben (10)Baseline 75 32Intervention 100.00 (0.00)Post 75 42

Students with Externalizing Behavior

Discussions/Conclusions

SRSD instruction for story writing effective in improving the number of essential elements included and the length of stories produced by students with and at-risk for EBD and limited writing skills within a PBS model

Most pronounced improvements in story elements

Improvements in quality to a lesser extent

Discussions/Conclusions

Collateral effects of SRSD instruction on behavior was measured but not established

Expands utility of SRSD for writing to students with internalizing behavior patterns

Extends Lane et al. (in press) on the utility of proving academic supports within the context of a PBS model

Project WRITE: Next Steps

Question 1: Does SRSD instruction in planning improve the writing performance of students at high-risk for EBD who have poor writing skills?

Question 2: Is both individualized SRSD instruction effective in improving the writing skills and behavior of students at high risk for EBD who have poor writing skills?

Question 3: Are there differences in how students with externalizing and internalizing behaviors and poor writing skills respond to SRSD instruction?