Assessing Narrative Skills in Children Peter de Villiers (Smith College) Frances Burns (University of Massachusetts, Amherst and Vanderbilt University)

Assessing Narrative Skills in Children

Peter de Villiers(Smith College)

Frances Burns(University of Massachusetts, Amherst

and Vanderbilt University)

Supported by NIH grant N01-DC-8-2104* web page: www.umass.edu/AAE

Acknowledgements

Jill de Villiers Smith College Elizabeth Engen Rhode Island School for the Deaf Debbie Topal Rhode Island School for the Deaf Harry Seymour University of Massachusetts Barbara Pearson University of Massachusetts Tempe Champion University of South Florida

Why Assess Narrative Skills?

Essential for continuity of personal memory, encoding of experiences, and social and cultural connection.

A major prerequisite language skill for adequate reading and writing development (Snow et al, 1998)

A test of the productive application of syntactic and semantic skills in functional communicative contexts.

A primary early form of extended discourse/ taking a sustained turn = decontextualized language with more complex syntactic forms..

What Aspects of Narrative to Assess?

1. What makes for a “well-formed narrative”?Thematic coherence on the macro-level of plot

and episode organization.Linguistic cohesion or connectivity at the micro-

level of noun phrases and clauses and their interrelationships across the discourse.

Appropriate elaboration of the different points of view of the characters.

2. What specific features of these properties of a well-formed narrative can be easily scored and will translate directly into intervention?

Narrative CoherencePlot/Episode Structure = “the landscape of action” (Bruner, 1986)

Setting/Introduction + Episode(s) + Resolution/CodaEpisode Structure: Onset/Initiation -- introducing the problem, goal, or event

that initiates and motivates the action in an episode of the story.

Unfolding/Elaboration/Action Attempts -- development of the action of the protagonists in terms of actions and attempts to solve the problem or reach the goal.

Consequences -- immediate effects of each of these actions.

Resolution -- the outcome of these endeavors.

Narrative CohesionReferential Cohesion -- introducing, maintaining

reference to, and contrasting the characters (or objects) in the story (Karmiloff-Smith, 1981).

Temporal and Causal Connectivity -- clearly marking the time and causal relationships between events (Berman & Slobin, 1994).

Foregrounding and Backgrounding -- placing the unfolding plotline events (the foreground) in the context of attendant circumstances in which they take place (Perrera, 1986).

Point of View and Evaluative Commentary

“the landscape of consciousness” (Bruner, 1986) -- talking about the mental states of the characters -- their emotional reactions, desires, and thoughts, and what they do and don’t know as events take place.

Linguistic Devices in Narrative

Referential Cohesion -- articles “a” and “the”, pronouns, names, adjectives, descriptive prepositional phrases, relative clauses.

Temporal and Causal Connectivity -- adverbs, adverbial phrases, adverbial clauses.

Foregrounding and Backgrounding -- adverbial clauses of time and place, often at the beginning of sentences.

Point of View / Evaluative Commentary -- mental state words and complement clauses.

How to Elicit NarrativesOpen-ended stories from a topic promptFamiliar “scripted” events (e.g., a birthday party)Story retellingPicture or video sequences -- long or short

How well does the elicitation technique get the child to produce language that incorporates the narrative features we have outlined AND can be easily evaluated and scored for those features?

For a more complete evaluation use more than one type.

Case Study 1: Coherence and Cohesion in the Written English Narratives of Deaf

StudentsOral subjects:

63 eight to sixteen year olds, mean age 11;10.

Average hearing loss 95dB (range 70 to 120).

Hearing loss onset prior to 18 months.Total Communication subjects:

56 eight to sixteen year olds, mean age 12;3.

14 with deaf parents (DoD), 42 with hearing parents (DoH)

Average hearing loss 99dB (range 70 to 120).

Hearing loss onset prior to 18 months.

Written Narrative Samples - 1

One multi-episode narrative based on a wordless children’s story -- “The Pirate Story.”

This was a multi-episode story chosen because it had three clear episodes, each of which depicted an initiating event or problem, an action or attempt to deal with that event, and a resolution or consequence of the action sequence.

The story was presented twice in the form of 16 color slides. Then the students wrote the story from memory.

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.





























The End



Pirate Story sample -- Age = 9;6Hearing Loss 98dB, Reading Grade 1.6

The man carrying the boat.

The man go the water.

I ride the boat. The man fighting to the boat. The man shot the gun.

The monster chseed the man.

The monster shot the gun.

The man take the monyey.

The man take bringing the boat.

The man to boat sount.

The man think. The man fixing the boat.

Pirate Story sample -- Age = 12;5Hearing Loss 93dB, Reading Grade 3;4

The men and women carrying a big ship. They threw ship on the water. One ship is good, other is bad. The ship want to go to Skull and cross bones ship. The ship shoot connon Skull and cross bones ship. The skull and crossbones ship was under the water. The ship the winner. The monster want the ship, but he didn’t. The ship shoot arrow to moster. Monster was dead. Ship are going to look island. The men climb down get golds. They put in ship. The golds was hevey in ship. The ship fell in the water. Everybody swimming off the water. Everybody sat the island. They cut the tree. They fell the tree. Everybody going to made a new ship. The everybody o.k.

The End.

Pirate Story sample -- Age = 13;3Hearing Loss 93dB, Reading Grade 5;0

There is a group of men that made a ship. They dicide to find a treasure. That day they were on the boat and travel until there was another boat. They had a war. The other ship lost and the pirate ship won. The pirate ship went to find the treasure. The Monster that was in the water heard the war and blew fire to the ship. One of the man killed the Monster. They went to find the treasure and when they got to the sandy island with a trap door, the men went down and took the treasure and left. But it was too heavy and the ship sank. The men swim to the island and live and made the ship forever.

The End

Coherence -- Episode Completeness

EpisodeStructure

SAT Grade<2.0

SAT Grade2.0 to 2.9

SAT Grade3.0 to 4.9

SAT Grade>5.0

AllIncomplete

(62.5)62.5

(5)30

PartiallyComplete

(25)18.8

(25)10

(22.2)

MostlyComplete

(12.5)6.3

(45)25

(22.2)87.5

(5.9)9.1

AllComplete 12.5

(25)30

(55.6)12.5

(94.1)90.9

( ) = oral students

Written Narrative Samples - 2

Two short narratives based on picture sequence scenarios.

These were designed to motivate the need to identify the characters in a contrastive way, to express temporal and causal relationships between events, and to refer to their mental states in explaining their actions.

They were written with the picture sequence in front of the students all the time.

The Candy Stealing Story

The Balloon Popping Story

Candy Stealing Story sample -- Age = 9;6Hearing Loss 98dB, Reading Grade 1.6

He want to the candy

The girl gave to the a penny

The girl gave to the cookies

The girl don’t went the cookies

The girl dreaming police

The girl gave penny

The woman said thank you

Candy Stealing Story sample -- Age = 12;5Hearing Loss 93dB, Reading Grade 3;4

Kerian Steal Candy Bag

Jane and Kerian went to the store. Jane like to buy some jelly bean in jar. Kerian saw candy in the shelf. Kerian want to steal some candy in the shelf. Then take candy and put in her purse. Kerian told Jane her, you want some candy, I steal candy bag in the shelf. Jane said no thank, because she learn in school. Kerian went to sleep. She dream about policeman take Kerian go to jail. Then went into the store. Kerian pay for Mrs. Williams. She said I’m sorry I steal candy bag in the shelf. Mrs. Williams said, that o.k. you won’t go to jail. Kerian feel O.K.

Candy Stealing Story sample -- Age = 13;3Hearing Loss 93dB, Reading Grade 5;0

There two girls in the store and one girl with a pocket purse. The girl was looking at the store lady and was stealing some candy on the counter, then they left. Outside the girl ask the girl who was buying the candy, and she didn’t wanted it. That night the girl had a bad dream about going to jail and the police took her. The next morning she went to the store and paid for the candy, and then the store lady was happy and pat her on the head! The girl went home happily!!

Cohesion-- Pronoun Use

PronounUsage

SATGrade<2.0

SATGrade2.0 to

2.9

SATGrade3.0 to

4.9

SATGrade>5.0

NoContrast

(50)56.3

(5)40

MostlyIncorrect

(50)31.3

(30)30

(27.8)33.3

MostlyCorrect 12.5

(55)25

(50)33.3

(17.6)18.2

AllCorrect

(10)5

(22.2)33.3

(82.3)81.9

( ) = oral students

Reference Specification

ReferenceSpecifi-cation

SATGrade<2.0

SAT Grade2.0 to 2.9

SAT Grade3.0 to 4.9

SAT Grade>5.0

None (50)6.3

Names/“other”

(12.5)43.8

(45)30

(33.3)33.3

Adjectives (37.5)50

(55)40

(5.6)22.2

(23.5)9.1

PrepositionPhrase 5

(16.7)0

(29.4)9.1

RelativeClause 25

(44.4)44.4

(47.1)81.8

( ) = oral students

Temporal Links

TemporalCohesion

SAT Grade<2.0

SAT Grade2.0 to 2.9

SAT Grade3.0 to 4.9

SAT Grade>5.0

None (37.5)12.5

And/then (50)68.8

(35)50

(27.8)

AdverbialPhrase

(12.5)12.5

(50)30

(44.4)66.7

(17.6)

AdverbialClauseFinal

(5)5

(5.6)11.1

(11.8)

AdverbialClauseInitial

(10)15

(22.2)22.2

(70.6)100.0

( ) = oral students

Partial Correlations between Reading Comprehension level and Features of Written

Narrative (controlling for Age and Hearing Loss)

GroupEpisodeStructure

TemporalCohesion

PronounUsage

ReferenceSpecification

Oral(df=56)

.55*** .61*** .67*** .54***

TC(df=46)

.43** .55*** .58*** .45**

** p<.01 *** p<.001

Assessing Narrative Skills in Children

Case Study 2: African-American English and Mainstream American

English Children

Frances Burns

University of Massachusetts, Amherst

Previous Narrative Research

Only a few studies on narrative development have focused on children who speak a dialect other than mainstream American English (MAE).

Of these studies, an even smaller number have focused on the discourse skills of young children who speak African American English (AAE) (Champion,1998; 2003).

Previous studies of young AAE speakers have focused on their overall narrative structures and the content of their stories (Champion 2003).(Champion 2003).

Narrative Style in AAE Children

Michaels (1981), described the structure of African American children’s narratives as complex but different from those of middle class European American children.

The narratives of the majority of African American children were seen as topic-associating (TA) rather than the topic-centered, linear style that dominates early schooling.

Topic associating refers to a narrative style in which the “main topic is not explicitly stated but implied via a number of loosely connected episodes.”

Topic-centered refers to “a linear progression of information with explicit lexical temporal grounding and no significant shifts in temporal-spatial perspective.”

Narrative Style Contrasts

Topic-centered

Organized around a single topic or closely related topics.

Main characters and temporal/locational grounding remain constant and are lexically explicit.

Clear thematic progression with beginning, middle and end.

Topic-associating

Organized around loosely linked topics with implied (associative) connections.

Frequent shifts in key characters and temporal/locational grounding.

Does not adhere to a linear pattern of organization.

Example of Topic Associating Narrative (8;5 - girl)

1. I live on lyme street 2. it’s a nice place 3. I got a- my auntie lives up there 4. I was gonna go to my- another school 5. this year I’m going to a different new school 6. so I might be happy there 7. but about my house 8. I just love being at my house 9. my cousins come over to play with me 10. an sleep over sometimes 11. sometimes I have slumber parties 12. great! 13. an den in the morning sometimes my mother takes us- my grandpa take us to the park 14. get us mcdonald’s or ummm all of that 15. sometimes he take us to the zoo 16. an see all the animals 17. it was fun at the zoo 18. I saw the animals, bears 19. it was great!

Example of Topic Centered Narrative (6;10 - girl)

1. one day I was going over aunt’s house. 2. then me and my cousin Jenea, we wanted to go to the liberry. 3.then we got there and I was reading books. 4. and then I wanted to um go on computers. 5. so I signed up. 6. but then we…which. 7.uh then a magic show was um startin to come on. 8. then this guy, he was just, he didn’t know where his magic hat was. 9. so he made a hat with big balloons like clowns. 10. and then after he made a hat he made um the duck out of balloons. 11. um it was like that duck that’s on Michael Jordan. 12. he made that of balloons. 13. an then he, he had helpers. 14. but he didn’t pick me. 15. an then he, whoever go, whoever did the job he gave them a wand. 16. an then when the magic show was done we, they had snacks. 17. they had cracker fishes, cookies and juice. 18. then I wanted to go a computer. 19 but I forgot that I had to go on the computer. 20. then we leff.

Further Research on Topic Associating Narrative Style

Hyon and Sulzby (1994), looked at the narrative styles of 48 African American low-income urban kindergarteners.

58.3% of the narratives were topic centered.

33.3% were topic associating.

Further Research on Topic Associating Narrative Style

Champion (1998), found that African American children, ages 6-10, produced a variety of narrative structures including the “classic” narrative structure.

In fact 66% of the narratives were classified as classic or topic-centered.As defined by Labov (1972), these narratives included an orientation, a complicating action, and a resolution, and then concluded with a coda.

Only 11% were classified as “performatives” or topic associating.

Current Research, Burns (2003)Study 1: Open-Ended Narratives

21 typically developing African American children from the Northeast aged 5;9 to 11;6 (Mean age 8;2).

The participants were video-taped telling at least three open-ended stories to one adult African American listener.

The example topics (i.e., hurt, fieldtrip, a hero, vacation) were provided in order to prompt non-fictive narratives.

Fictitious narratives were discouraged because children may be tempted to tell fairytale or story book narratives that are limited in AAE features and perhaps bias the children toward topic-centered narratives.

Study 1- Data Analysis

A total of sixty-six narratives were transcribed and analyzed for AAE dialect features, T-Units, and narrative style (topic centered vs topic associating).

The children were placed on a dialect continuum that ranged from low to high use of AAE on the basis of the frequency of appearance of several distinctive syntactic, semantic, and phonological features of AAE (Washington & Craig, 1998) in their spontaneous speech.

Study 1 - Results

Only 11% of the open-ended narratives were categorized as topic-associating, confirming Champion’s (1998) findings.

There was no relationship between depth of AAE dialect and the likelihood that the children would produce topic-associating forms of narration.

There was a trend in the data for the younger children to produce more topic-associating narratives.

The younger children also produced a higher percentage of “series of events” stories. In these there were clear topics but no orientation, complicating action or resolution.

Example of “series of events” (6;10 – boy)

1.once upon a time I saw my friends at the beach. 2. it was ??? and Carmen and Carmen’s friend. 3. den we went an we ate there. 4. there were little pointers on the floor. 5. only in some ??? 6. an I- den I we lef 7. den I rode my bike 8. den the end 9. oh, den I went in the house 10. den I got somin to drink 11. den I ate 12. den I went to bed

Conclusions

These results and those of Champion (1998) suggest that by the time African American children are aged 7 or 8, they have a range of narrative styles available to them.

By this age they predominantly produce the topic-centered, classic narrative (Labov, 1972).

This may result from code switching into the style that they are exposed to in school.

Younger African American children produce more of the topic-associating and “series of event” types of stories.

More research is needed to explore whether the topic-associating narrative style is dominant in still younger children, less than age 6.

Current Research, Burns (2003) Study 2: Picture Sequences

78 AAE (n=53) and MAE (n=25) speaking children. Ages 4 to 6. No difference between the AAE and MAE

groups in age distribution or mean age. One picture sequence narrative from the Diagnostic

Evaluation of Language Variation-Criterion Referenced (DELV-CR), San Antonio, TX: The Psychological Corporation (2003)

Assessing Reference Contrasting, Temporal Links, Mental State References, and Theory of Mind.

Narratives were audio-taped and transcribed. SEE THE DELV-CR FOR STIMULUS PICTURES

Narrative Samples 1

I want my train. I’m gonna hide the train from him. I’m gonna play out of the toy box. I’m gonna find that train. Bring that train. (C: 4;2, MAE)

He was looking for the choo choo train because the other boy was playin’. And then… and then he said, “I want that choo choo train back”, and umm… he put it in his toy box. And then he came back to find it and he looked under the bed and it wasn’t there. (SC: 4;9, MAE)

Narrative Examples 2

The big boy came into the little boy’s room and took away the little boy’s train. Then he hid it under the boy’s bed where he couldn’t get it. Then the little boy… when he left… he got out his train and put it in the toy box while the big boy was eating. Then the big boy thought about the train and he went under the bed to go see it but it wasn’t there.

(A: 6;4, MAE)

The little brother was trying to get his toy from the big brother. And the big brother hiding his toy under the bed. When he is eating his sandwich, the little boy go and get it and put it inside of his toy box. When his big brother walk in, he think about the train and he look under his bed for it. (J: 6;3, AAE)

Study 2 - Data Analysis For reference contrasting, the children were given 1 point

if they contrasted the two boys in some way in their story (e.g., “the big brother” vs “the little boy”)

For temporal links the children were given a score based on the most sophisticated type of temporal expression they used: 0 = no time links expressed, 1 = only sequencers like “then” or “and then” used, 2 = adverbial clauses of time used (e.g., “while” or “after”).

For mental state references in describing the thought balloon picture in the eliciting sequence, the children received 1 point if they referred to the intention or desire of the boy (“He wants his train.”), but 2 points if they referred to his cognitive state (“He is thinking about his train.”)

Study 2 - Data Analysis

For their answers to the final question about why the boy was looking under the bed for the train, the children again received 1 point for an answer in terms of his motivation for looking (“to find his train.”), but 2 points for a theory of mind explanation (“because he thinks his train is there.”)

So the total score on the narrative was 7 points.

Study 2 - Statistical Results (ANOVA)

For reference contrasting there was a significant age effect (p=.018), but no effect of dialect, and no interaction between age and dialect.

For temporal expressions there was a significant age effect (p=.003), but no effect of dialect, and no interaction between age and dialect.

Study 2 - Statistical Results (ANOVA)

For mental state references in Picture 5 descriptions, there was a significant age effect (p=.015), but no effect of dialect, and no interaction between age and dialect.

For theory of mind explanations there was a significant age effect (p=.003), but no effect of dialect, and no interaction between age and dialect.

Overall Picture Sequence Narrative Scores

Narrative Score

0

1

2

3

4

5

6

7

4 5 6

Age

Average Score /7

AAEMAE

DELV-CR Field Testing Study - Typically-developing and

Language-impaired Children

Narratives elicited by the train story sequence.

On-line scoring by the clinicians administering the test (a reliability check showed 87.5% agreement with audio-taped and transcribed narratives)

DELV-CR Narrative Study - Subjects

1014 four to nine year olds from all around the USA.

60% of them speakers of AAE, 40% speakers of MAE.

30% of each group (roughly equally spread across the ages) were diagnosed as being language-impaired and were receiving intervention services.

AAE and MAE groups were matched for parent education level (average level = high school).

DELV-CR Narrative Study - Results

No differences were found between the dialect groups on any of the separate measures: reference contrasting, temporal expressions, mental state references, or theory of mind.

BUT there were strong developmental growth effects for each of the measures.

AND there were clear differences between the typically-developing children and the language-impaired children on each of the measures.

Development of reference contrast in narratives (contrasting the two main characters) in typically

developing MAE and AAE speaking children.

Reference Contrast in Spoken Narrative

0

0.2

0.4

0.6

0.8

1

4.5 5.5 6.5 8 10 12

Age

Proportion of Group

AAEMAE

Development of reference contrast in narratives (contrasting the two main characters) in typically

developing and language impaired children.

Reference Contrast in Spoken Narrative

0

0.2

0.4

0.6

0.8

1

4.5 5.5 6.5 8 10 12

Age

Proportion of the Group

ImpairedTypical

Development of the expression of temporal links between events in the narratives of typically developing MAE and

AAE speaking children.

Temporal Links in Spoken Narrative

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

AAEMAE

Development of the expression of temporal links between events in the narratives of typically developing and

language impaired children.

Temporal Links in Spoken Narrative

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

ImpairedTypical

Development of mental state references to describe the “thought balloon” picture (typically developing

MAE versus AAE speaking children)

Mental State Descriptions of Picture 5

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

AAEMAE

Development of mental state references to describe the “thought balloon” picture (typically developing

versus language impaired children).

Mental State Descriptions of Picture 5

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

ImpairedTypical

Development of “theory of mind” explanations for the character’s mistaken action in the picture narrative

(typically developing MAE versus AAE speaking children).

Mental State Explanations of Action

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

AAEMAE

Development of “theory of mind” explanations for the character’s mistaken action in the picture narrative

(typically developing versus language impaired children).

Mental State Explanations of Action

0

0.5

1

1.5

2

4.5 5.5 6.5 8 10 12

Age

Average Score /2

ImpairedTypical

Development of combined narrative skills in MAE and AAE speaking children aged 4 to 12.

Spoken Narrative Score

0

1

2

3

4

5

6

7

4.5 5.5 6.5 8 10 12

Age

Average Score /7

AAEMAE

Overall narrative scores in typically developing and language impaired children aged 4 though 12.

Spoken Narrative Score

0

1

2

3

4

5

6

7

4.5 5.5 6.5 8 10 12

Age

Average Score /7

ImpairedTypical

Conclusions

The last two studies demonstrate that these picture sequences produce a dialect neutral assessment of important features of narrative cohesion and point of view.

The materials can be used for diagnosis of language impairment in both Mainstream American English speaking children and African American English speaking children.

Documents

Assessing Narrative Skills in Children Peter de Villiers (Smith College) Frances Burns (University of Massachusetts, Amherst and Vanderbilt University)