The Strange Stories Test: A Replication with High ...docs.autismresearchcentre.com/papers/1999_Jolliffe_BC_Stories.pdfThe Strange Stories Test: A Replication with High - Functioning

Journal of Autism and Developmental Disorders, Vol. 29, No. 5, 1999

The Strange Stories Test: A Replication with High-Functioning Adults with Autism or Asperger Syndrome

Therese Jolliffe1 and Simon Baron-Cohen1

Two groups of individuals, one with high-functioning autism and the other with Aspergersyndrome were tested using Happe's Strange Stories Test of a more advanced theory of mind(Happ6, 1994). This assesses the ability to interpret a nonliteral statement. Relative to nor-mal controls who were IQ and age-matched, individuals with autism or Asperger syndromeperformed less well on the task, while performing normally on a non-mentalistic control task.Individuals with autism or Asperger syndrome could provide mental state answers, but haddifficulty in providing contextually appropriate mental state answers. Rather, their answerstended to concentrate on the utterance in isolation. This replicates Happe's result. Althoughthe majority of both clinical groups provided context-inappropriate interpretations, the autismgroup had the greater difficulty. Results are discussed in relation to both weak central coher-ence and theory of mind.

INTRODUCTION

Even the most able people with autism have dif-ficulties appreciating nonliteral speech, such as indi-rect requests, sarcasm, jokes, and metaphorical ex-pressions (Happe, 1993, 1994; Ozonoff & Miller, 1996;Tantam, 1992). For example, a recent study found thathigh-functioning individuals with autism are, paradox-ically, more indirect or nonliteral in their interpretationof indirect requests than controls (Ozonoff & Miller,1996). The authors interpreted the findings as suggest-ing that individuals with autism have overlearned therule that questions beginning with "Can you . . . ?"should be interpreted in a nonliteral way. At the oppo-site extreme Baron-Cohen (1997) found that school agechildren with autism, with a mental age equivalent of6 years, had difficulty seeing that a nonliteral reference(calling a cup "a shoe") might be a joke. In contrast,normal 3-year-old children expect a speaker's inten-tions might be to joke. Such pragmatic impairments are

1 Departments of Experimental Psychology and Psychiatry, Univer-sity of Cambridge, Downing Street, Cambridge, CB2 3EB, UnitedKingdom.

395

KEY WORDS: Strange stories test; theory of mind; Asperger Syndrome; autism.

consistent with the well-established deficits in the de-velopment of a theory of mind in autism (Baron-Cohen,1995; Baron-Cohen, Leslie, & Frith, 1985).

A final example of this pragmatic impairment inrelation to the comprehension of nonliteral utteranceswas demonstrated on the Strange Stories test (Happe,1994). Verbal adolescents and adults with autism ofvarying intellectual abilities were presented with a setof vignettes, or stories, about everyday situations wherepeople say things they do not literally mean. For in-stance, someone in receipt of a birthday present says,"It's lovely, thank you. It's just what I wanted." Thiscould be said either because it was lovely and they re-ally did want it, or it could be said to spare the otherperson's feelings. In real life the different motivationsthat underlie people's utterances are distinguished bymany factors. Such factors might be the preceding con-text, emotional expression, and the relationship be-tween the speaker and hearer. The Strange Stories werewritten so that the motivation behind an utterancewould generally be interpreted by normal individualsin just one way. These stories are more natural thanstandard theory of mind (ToM) tasks, and Happ6 re-cruited participants of different ToM abilities in order

0162-3257/99/1000-0395$16.00/0 c 1999 Plenum Publishing Corporation

396 Jolliffe and Baron-Cohen

to look at their performance in relation to standard ToMtasks. She found that even the most able participantswith autism tended to give context-inappropriate men-tal state explanations.

The study presented in this paper retests the StrangeStories, since as far as we are aware there has never beenany attempt at independent replication. These storieswere reduced in number and one was modified (with per-mission from Happe) in an attempt to make them evenmore likely to be interpreted in one way rather than an-other. Furthermore, a control task was presented so asto test the understanding of physical events and checkthe generality of any comprehension deficit that mightemerge irrespective of story content. A few of these werealso modified (again with permission from the author)in an attempt to make them more likely to be interpretedin one way rather than another. These control storieswere more demanding than the control stories employedin the original version of the Strange Stories test, sincethe original version resulted in ceiling effects.

The study thus aimed to test if Happe's finding ofa tendency to give context-inappropriate mental stateexplanations would replicate. It also sought to extendher 1994 study. Thus all participants in our study wereselected for passing standard second-order ToM tasks,and were separated according to whether they had re-ceived a diagnosis of autism or Asperger syndrome. Thiswas to test whether any weakness in processing suchstories was a function of early language development,since the key difference between autism and Aspergersyndrome is that in the former there is often a historyof language delay whereas in the latter there is no clin-ically significant language delay. Second, both clinicalgroups were very high-functioning, having an IQ (Full-scale, Verbal, and Performance) which was at least av-erage. Third, a normal control group matched on age,sex, handedness, and IQ (Full-scale, Verbal, and Per-formance) was included, since the original study did notmatch participants in these respects.

If the clinical groups have mentalizing difficul-ties (Baron-Cohen et al., 1985) one would expect themnot only to have difficulty providing mental state an-swers but to produce less of these. On the other handif individuals with an autism spectrum disorder haveweak "central coherence" (Frith, 1989) they shouldshow evidence of this in their failure to use the con-text to provide the context-appropriate interpretationof an utterance. It was predicted that the individualswith autism and Asperger syndrome would show a fail-ure to give the context-appropriate interpretation of anutterance.

Participants

The last two decades has seen a great deal of dis-pute about whether or not there is a single condition ofautism which varies in severity, or whether there aredifferent types which lie along a continuum. Moreover,the last decade has seen an increase in the use of thelabel Asperger syndrome, which diagnostically requires(among other requirements) no clinically significantdelay in early language development (ICD-10; WorldHealth Organization, 1992). Thus single words haveto be used by 2 years and communicative phrases by3 years. Pragmatics is not included. In this respect thistype of autism can be distinguished from Kanner's(1943) classical cases, all of whom were clinically de-layed in early language development.

Recently there has been increasing interest in howone defines autism, and how diagnostic systems shouldbe interpreted. The DSM-IV (American Psychiatric As-sociation [APA], 1994) suggests that one can haveautism irrespective of whether or not one is delayed inearly language development. There is even a questionover whether Asperger's cases had Asperger syndrome(Miller & Ozonoff, 1996). The present study does notaim to contribute to this debate. Rather, we chose to dis-tinguish individuals with a history of autism on the basisof their early language development. This gave rise totwo groups one of which clinicians regarded as meet-ing the criteria for Asperger syndrome (ICD-10) and theother which clinicans regarded as meeting the criteriafor autism (DSM-IV). We did not attempt to distinguishindividuals on any other factor (e.g., motor clumsiness).Whereas our cases of autism resembled Kanner's clas-sical cases, their symptoms had lessened which madethem appear to function similarly to those with Aspergersyndrome and which would have put them in the resid-ual category according to the DSM-III-R (APA, 1987).

Our clinical participants were recruited nationally.Some were located via support groups or via letters sentout by the National Autistic Society. The majority werelocated through clinicians. The majority of clinical par-ticipants were or had been patients at the Maudsley Hos-pital (London) where experienced clinicians had deter-mined the individuals' diagnostic status. Some were orhad been patients at Charing Cross Hospital (London)where again an experienced clinician had made the di-agnosis. The remainder of participants came from ElliotHouse, the National Autistic Society's UK center for thediagnosis of social and communication disorders.

Irrespective of whether or not an individual was at-tending a hospital, clinicians were contacted (with the

The Strange Stories Test with Autism and Asperger Syndrome 397

patients' and parents' consent) and medical and psy-chological or psychiatric reports were inspected. Indi-viduals were excluded in cases where either the clinicanor the parents could not be sure about early language de-velopment. As part of the checking process parents weregiven the revised Howlin (1995) screening questionnaireto complete. (This, devised at the Maudsley Hospital,makes use of DSM-IV criteria and seeks to identify thepresence of autistic symptomatology and whether therewas a clinically significant delay in early language de-velopment.) In all but one case clinicians and parentsagreed on early language development. The one casewhere there was disagreement was excluded. Individu-als whose early language development was thought tobe on the borderline of what is clinically normal and ab-normal were also excluded. In our attempt to be rigor-ous about early language development we rejected adozen participants. There was thus no doubt about theearly language development of any of our participants.

The Asperger group (i.e., the group with singlewords by 2 years and phrase speech by 3) contained manyindividuals who had not received a diagnosis until adult-hood. The remainder were diagnosed in childhood or ado-lescence. For the former group clinicians made their di-agnosis retrospectively. For the latter group, in the caseof a couple of individuals, clinicians revised their diag-nosis from autism to that of Asperger syndrome on thebasis that these individuals did not have a clinically sig-nificant delay in early language development. Accordingto the DSM-IV, however, some of the Asperger partici-pants might be regarded as having autism without lan-guage delay. We have not, however, questioned the cli-nicians' diagnosis of Asperger syndrome, since from ourpoint of view this leaves unaffected the key differentia-tor between the two groups; that of early language de-velopment. Thus in the autism group all were clinicallydelayed in language development and would thereforehave been considered to have had a history of classicalautism. In the Asperger group, none were clinically de-layed in their language development.

Fifty-one adults participated in the experiment.These comprised 17 with autism, 17 with Asperger syn-drome, and 17 normal adult control participants. The nor-mal adults acted as a comparison group for the two clin-ical groups. The majority of clinical participants weretested in their place of residence, except where some pre-ferred to be tested at the university. All control partici-pants were tested in a quiet room at the university.

The 17 control participants were taken from thegeneral population of Cambridge. These control partic-ipants were chosen to match the clinical groups as

closely as possible with respect to the characteristics ofage, IQ, sex, and handedness. Table I gives the partici-pant details of chronological age (CA) verbal IQ (VIQ),performance IQ (PIQ), and full-scale IQ (FSIQ). Fourone-way ANOVAs revealed no significant-differencesbetween groups on any of these variables: CA, F(2, 48)= 0.59, p = .56; VIQ, F(2, 48) = 0.51, p = .60; PIQ, F(2,48) = 0.58, p = .57, and FSIQ, F(2, 48) = 0.10, p = .91.The sex ratio in all three groups was 15:2 (m:f), re-flecting the sex ratio found in these clinical groups inother studies (Klin, Volkmar, Sparrow, Cicchetti, &Rourke, 1995; Wing, 1981). The groups were closelymatched on handedness: 15 right-handed and 2 left-handed individuals in the normal and high-functioningautism group, and 14 right-handed and 3 left-handed inthe Asperger group. All participants were born in Eng-land and English was their first language. All threegroups contained participants from various socioeco-nomic backgrounds and the three groups were broadlyequivalent in terms of educational attainment. Severalindividuals within each group were either studying foror holding formal qualifications such as a university de-gree or diploma.

All participants were required to be of at least nor-mal intelligence (i.e., scoring > 85) on the WAIS-R(Wechsler, 1981, Full-scale, Performance, and VerbalIQ). All three groups prior to their recruitment werescreened to check whether they had any history of psy-chiatric disorder, neurological disorder, or a head in-jury. Individuals were excluded if they reported any ofthese factors, and for the clinical groups parents andprofessionals were consulted. All participants were alsorequired to be medication-free at the time of testing.There were also screening criteria specific to to the clin-

Table I

Participant group"

NormalMSDRange

AutismMSDRange

AspergerMSDRange

. Participant Characteristicsa

CA

30.009.12

18-49

30.717.84

19-46

27.777.81

18-49

VIQ

106.4710.94

87-127

107.5914.37

88-135

110.8213.51

89-130

PIQ

105.2414.00

85-134

101.7713.06

85-132

100.2914.23

85-133

FSIQ

106.3512.72

88-133

105.1213.47

90-133

107.1214.34

86-132

a n = 17 in each group.


ical and control groups. The control group had to befree of any family history of autism or Asperger syn-drome. The clinical groups were selected on the basisof their ability to pass both first- and second-order be-lief tasks2 and they were screened to exclude those whomight be depressed, since depression is much morecommon in autism and Asperger syndrome and can alsoaffect social functioning and judgment.

Materials

The stimuli presented consisted of short stories.There were 18 mentalistic stories and 6 physical con-trol stories. There were 9 types of mentalistic story, thetest containing two examples of each. The 9 story-typescomprised Double Bluff, Figure of Speech, Joke, Lie,Misunderstanding, Persuade, Pretend, Sarcasm, andWhite Lie. The 6 physical control stories did not involvemental states, nor were they social in nature. However,they did require participants to make global infer-ences that went beyond what was explicitly mentionedin the text.

The mentalistic stories contained two and, for onestory type, three test questions; the comprehensionquestion, which usually took the form, "Was it truewhat X said?" and the justification question(s) whichusually took the form, "Why did X say that?" The phys-ical control stories had just one question, asking whya particular action had been carried out.

Each of the stories with their questions appearedon white A5 sheets. The mentalistic test stories alsocontained a small black and white line drawing of thesignificant characters mentioned in the story. These linedrawings were simple, but showed emotional expres-sions and illustrated the contextual setting. All the sto-ries were laminated. All stories had an identifying wordlabel that appeared in the top right-hand corner, thepurpose of which was to facilitate scoring.

The stories that were used in this experiment weretaken from Happe (1994) and a few were excluded andamended with permission from the author. Examples ofthe mentalistic and physical stories can be found in theAppendix.

2 Participants were given first- and second-order theory of mind tests.The first-order task was a version of Perner, Frith, Leslie, andLeekam's (1989) Smarties task. The second-order task was Baron-Cohen's (1989) ice cream van test. Whereas all participants passedthe first-order task, 5 out of 51 participants failed the second-ordertask. These included 1 participant with Asperger syndrome, 2 withhigh-functioning autism, and 2 normal control participants. Theseparticipants were retested on a new variation of the second-orderbelief task and all were found to pass.

Procedure

Each participant was tested by themselves in a roomwhich was free from distractions. The experimenter satnext to the individual so that she could read them thestories.

The Test Items

The mentalistic task was always presented first, fol-lowed by the physical control task. Since the control sto-ries always came after the mentalistic stories, the latteracted as a control for fatigue and motivational deficits,as well as being a test of nonsocial global inferences.

The experimenter shuffled the mentalistic storiesthoroughly before they were given to each participant.These stories were therefore presented in a different ran-dom order to each individual. This was done to ensurethat any significant effects were not due to story orderand, as a further safeguard, the experimenter made surethat the two examples of each type of story never ap-peared together.

The stories being presented were placed face up onthe table directly in front of the participant. The partic-ipant was told that he/she were going to be read somestories. They were instructed to listen carefully as theywould be required to answer questions about the stories.

After each of the mentalistic stories had been read,the experimenter asked the comprehension and mentalstate question(s). The story remained in front of the par-ticipant not only throughout the reading but also through-out questioning. This was done in order to minimizememory requirements. The comprehension question usu-ally took the form, "Was it true what X said?" On thevery rare occasions when a participant made an error, thestory was read out again, until the participant answeredcorrectly or justified their answer and appeared to un-derstand (e.g., "well no, it's not literally true, but it is thecorrect expression to use"). The mental state question(s),usually took the form, "Why did X say that?" The par-ticipants were given as long as they felt necessary andwere encouraged to read the story for themselves a sec-ond and even a third time in order to facilitate their ex-planation for the story character's utterance.

After the mentalistic stories had been completed,participants were given the physical control stories.These stories were read out loud to each participant,and after each one had been read, its inference ques-tion was asked, asking why something happened or whya particular action had been carried out. As with thementalistic stories, the stories remained in view whilethe inference question was being asked. This was againdone to minimize memory requirements. Furthermore,


to be consistent with the mentalistic stories, partici-pants were given as long as they felt necessary to comeup with the answer, and they were encouraged to readthe story through themselves in order to facilitate theirexplanation of why something had happened or why aparticular action was carried out.

Scoring

For the mentalistic stories participants were scoredon their answers to both the comprehension and "Why"questions. For the comprehension question, partici-pants' original responses were noted, along with anyamendments. For the "Why" question, scoring was alittle more complicated, Justifications could be corrector incorrect. But a correct justification could be correctin two ways: it could be a correct physical explanationor a correct mental explanation. Similarly, an incorrectjustification could be incorrect in two ways, it could bean incorrect physical explanation or an incorrect men-tal explanation. Thus in the story where Emma is play-ing and pretends that a banana is a telephone, Emma'sstatement that the banana is a telephone can be cor-rectly justified in two ways: in the physical sense, "be-cause the banana is shaped like a telephone," and in themental sense, "because Emma is pretending that the ba-nana is a telephone." Similarly, an incorrect justifica-tion could be incorrect in two ways: in the physicalsense, an answer that is factually incorrect, such as "be-cause she is about to eat the banana" and in the men-tal sense, "because Emma is joking."

Where both a correct and incorrect justificationwere given, participants were scored on their correctjustification. Thus participants were given credit fortheir best answer. Similarly, if a participant's answerappealed to both physical and mental states, the justi-fication was scored as a mental state.

Physical state answers included terms such as big,looks like, is shaped like, to sell them, to get rid of them,to not get X (physical outcome, e.g., a filling, told off,mugged). Mental state answers included all those thatreferred to thoughts, feelings, desires, traits, and dispo-sitions. Mental state justifications included terms suchas like, want, think, know, happy, cross, joke, pretend,lie, afraid, please, hurt, expect, and to fool.

For the physical control stories, participants werescored on their answers to the global inference ques-tion, which asked why something happened or why aparticular action had been carried out. On the rare oc-casions where an individual gave more than one ex-planation for why a particular action was carried out,or amended their answer, it was always their best an-

swer that was accepted. Accenting amendments to an-swers, or giving credit for the participant's best answer,was to parallel and be consistent with the scoring ofthe mentalistic stories.

The participant's responses were noted, includingany amendments, next to the story's identification labelwhich appeared on each score sheet. The participant'sanswers were recorded in full on the score sheet so thatthey could be analyzed at a later date. There were justa few instances where participants were unable to givea response to the "Why" question of the mentalistic sto-ries, and to the action question of the physical controlstories. These omissions are examined in the Resultssection.

Judging whether an explanation is appropriate ornot is clearly subjective. However, the stories were se-lected and amended to ensure that they were unam-biguous and therefore only one explanation was rea-sonably appropriate for both the mentalistic stories andfor the justification of why a particular action occurred.Although the stimuli were relatively unambiguous,there is still a degree of subjectivity surrounding par-ticipants' statements, therefore validation of the scor-ing was undertaken in order to establish its validity. Allof the statements made by 9 of the participants in eachgroup were given to a second rater, who was blind tothe identity and diagnostic status of the participants andnaive to the hypothesis being tested. The degree of con-cordance was 98% for each of the two conditions.

RESULTS

The first performance measure was aimed at as-sessing whether the groups differed in their ability toprovide context-appropriate explanations (justifica-tions) for a story character's nonliteral utterance in thementalistic condition and whether they could make aninference or give a reasoned explanation as to why aparticular action had occurred in the physical condi-tion. The performance scores on these two conditionswere measured in percentages (so as to equate the men-talistic and physical tasks which had 18 stimuli in theformer and 6 in the latter). Therefore individuals couldobtain scores ranging from 0% to 100% for each of theconditions. The three groups mean percentage correctfor their explanations on the mentalistic and physicalstories are illustrated in Table II.

Accuracy on the Mentalistic and Physical Stories

The accuracy scores were approximately normallydistributed and the variances were approximately equal,


according to Cochrans C test, so a two-factor repeated-measures ANOVA was performed on the mean per-centage scores for each of the three groups. ThisANOVA had a between-participant variable of Group,and a within-participant variable of Condition (Men-talistic and Physical). The ANOVA revealed a signif-icant main effect of Group, F(2, 48) = 4.66, p < .05;and Condition, F(l, 48) = 12.02, p < .001, which sug-gests that the groups differed in the overall correctnessof their statements and that the type of condition(whether mentalistic or physical), had some effect onperformance. The higher order interaction of Group XCondition, was also, as predicted, significant, F(2, 48) =7.41, p<. 01.

Given that the Group effect was significant, theGroup x Condition interaction was investigated furtherto see if there were different Group effects for the twoconditions. To examine whether this group effect ap-plied to one or both of the conditions, simple effectswere examined which compared the different Groupson each Condition. As predicted analysis of simple ef-fects showed the effect of Group to be significant onlyfor the Mentalistic condition, FMentalistic (2, 48) = 15.30,p < .001; FPhysica, (2, 48) = 047, p = .63.

The source of the Group x Condition interactionwas investigated further using t tests. To reduce thefamilywise error rate only preplanned comparisonswere explored. Planned contrasts of the cell means in-dicated that the mean percentage for the autism and As-perger groups' Mentalistic condition, were as predicted,significantly different from that of the normal controlgroup, taut.(48) = 5.42, p < .001; tAsp.(48) = 3.69, p =.001, and that the mean percentage for the autismgroup's Mentalistic condition had a nonsignificanttrend to be lower than that of the Asperger group, t(48)= 1.73, p = .09. Observation of the mean percentages(see Table II and Figure 1) show the clinical groups tobe significantly worse in giving context-appropriate ex-planations for a speaker's utterance.

Given that the Condition effect was significant itwas useful to see whether there were different Condi-tion effects for the three Groups. Simple effects wereexamined which compared the two conditions for eachgroup. Analysis of simple effects showed the effect ofCondition to be significant for the normal control group,F(l, 48) = 26.01, p < .001, but not the two clinical

Table II. Percentage of Justifications that wereCorrect on each Condition

Participant groupa

NormalMSDRange

AutismMSDRange

AspergerMSDRange

Mental

99.671.35

94-100

84.3110.8056-94

89.229.31

72-100

Physical

88.1812.81

67-100

84.3110.8056-94

87.1812.49

67-100

a n = 17 for each group.

Fig. 1. Effect of Condition on justification accuracy.


groups [Faut.( 1, 48) = 0.00, p = 1.00; FAsp.( 1, 48) = 0.82,p = .37]. Observation of the means (see Table II andFigure 1) show the normal control group to be signifi-cantly better on the Mentalistic condition relative totheir own performance on the Physical condition,whereas the autism and Asperger groups showed no dif-ferential pattern of performance on the two conditions.

Nature of Performance on the Mentalistic Stories

The next performance measure assessed the natureof participants' performance on the Mentalistic condi-tion. The justifications on the mentalistic stories werenot just correct or incorrect as was previously exam-ined, they could in fact correspond to several types ofanswer. The justifications could be mentalistic or phys-ical; appealing to mental or physical states. They couldbe either a correct or incorrect mental answer, or a cor-rect or incorrect physical answer. The scores for thethree groups were examined to see whether there wereany differences in the types of justification provided.The unit of measurement for the type of statement wasthe number of answers which corresponded to the typeof statement. Therefore individuals could in theory ob-tain scores which ranged from 0 to 18 for each of thejustification types. The mean results for the three groupscan be seen in Table III. The number of participants giv-ing at least one incorrect mental state justification perstory type can be seen in Table IV.

To examine whether there were any group differ-ences for each of the justification types, a series of one-way ANOVAs were performed on the following justifi-cation types: Mentalistic Statements, Physical Statements,Correct Physical Statements, Incorrect Physical State-

ments, and Correct Mental Statements, Due to unequalvariances, Incorrect Mental Statements had to be inves-tigated with t tests using a separate variance estimaterather than a pooled variance estimate. The analysis re-vealed no difference between groups on the number ofmental state justifications F(2, 48) = 0.37, p = .69; thenumber of physical state justifications, F(2, 48) = 0.18,p = .84; the number of correct physical state justifica-tions, F(2, 48) = 0.06, p = .94; and the number of incor-rect physical state justifications, F(2, 48) = 1.56, p = .22.However, the analysis did reveal a difference betweengroups on the number of correct mental state justifica-tions, F(2, 48) = 5.31, p < .01; the source of the effectwas investigated further using t tests. The analysis re-vealed that the clinical groups made significantly fewercorrect mental state justifications than the normal cont-rol group, taut.(32) = -3.65, p = .001; tAsp.(32)= -2.18,p < .05; although the clinical groups themselves did notdiffer, t(32) = 0.86, p = 0.40. T tests also revealed a sig-nificant difference between groups on the number of in-correct mental state justifications. The clinical groupsmade significantly more incorrect mental state justifica-tions than the normal control group, taut.(147) = -4.70,p < .001; tAsp.( 17.25) = -4.46, p < .001, although againthe clinical groups themselves did not differ significantly,t(257) = 1.65, p = .11.

To determine whether the clinical groups' ten-dency to provide context-inappropriate mental state an-swers is determined by just a few individuals in eachgroup or was more widespread among the participants,the number of participants in each group scoring above(and below) the control group mean was calculated.This was compared to the numbers of participants inthe normal group scoring above (and below) their mean.

Table III. Number of each type of Justification on the Mentalistic Condition

Participantgroupa

NormalMSDRange

AutismMSDRange

AspergerMSDRange

a n = 17 in each

Mental

14.351.97

10-18

13.881.97

10-17

13.772.39

10-18

group.

Physical

3.651.970-8

3.881.761-8

4.062.280-7

Correct justifications

Mental

14.291.96

10-18

11.532.43

6-15

12.353.10

8-18

Physical

3.651.970-8

3.771.721-8

3.882.150-7

Incorrect justifications

Mental

0.060.240-1

2.352.001-8

1.4)1.230-4

Physical

0.000.000-0

0.120.330-1

0.180.390-1


The analysis revealed that the clinical groups differedsignificantly from the normal control group, X2aut. (1) =30.22, p < .001; X2Asp.(1) = 15.07, p < .001.

Comprehension and Omissionson the Mentalistic Stories

Performance on the comprehension question wasexamined. The number of comprehension errors werecalculated for each group. Three 2x2 chi-squared testswere conducted to see whether the groups differed intheir number of comprehension errors. Chi-squareanalysis revealed no significant difference between thethree participant groups (Fisher's correction for ex-pected frequencies < 5, p > .05 for all three groups).

Since only one participant in each of the clinicalgroups made an omission, it was thought unnecessary,given the large number of stimuli employed, to test forgroup differences, as omissions could not be consid-ered a factor underlying the relatively poor perform-ance of the clinical groups.

DISCUSSION

This study aimed to replicate Happe's (1994) mainfinding on the Strange Stories. Our results replicate hermain findings. Thus, on the Mentalistic condition, theclinical groups were equivalent to the control group inthe number of mental state justifications used. Whatdistinguished the clinical participants on this conditionwas not a failure to use mental state terms (includingemotion terms like "hurt") but a failure to use the ap-propriate mental state term for the story's context. Incomparison to the normal control group the clinical

groups made significantly more context-inappropriatemental state justifications, and significantly fewer con-text-appropriate mental state justifications. Althoughthe clinical groups themselves did not differ in theircorrectness or otherwise of their mental state answers,the autism group always performed at a level belowthat of the Asperger group (see Tables II, III, and IV).Although every one of the participants with autism gaveat least one (and on average two to three) context-inappropriate mental state answers, 12 out of 17 of theparticipants with Asperger syndrome gave at least one(and on average one to two) context-inappropriate men-tal state answers, whereas only one of the normal adultsmade such a mistake. The errors made by clinical par-ticipants were striking: in the sarcasm story one par-ticipant said that the lady's statement that it was alovely day for a picnic was her "pretending that every-thing was OK in order to make Tom feel happier," andanother participant explained the utterance in the pre-tend story as "a joke." As predicted, both groups ofclinical participants performed normally on the Physi-cal condition, where they were required to justify whya particular action had occurred.

Analysis of the type of justifications made on theMentalistic condition revealed that the groups did notdiffer in the number of physical state justifications andthe correctness or incorrectness of these. The findingthat the clinical groups did not differ from the normalcontrol group on the number of correct physical statejustifications is at odds with Happe's finding, where hersecond-order (ToM) autism group made more correctphysical state justifications than her normal adult con-trol group. However, there are a few factors that mightexplain this different result: (a) the clinical groups re-cruited to take part in the experiment reported heretended to be older and intellectually more able thanHappe's second-order ToM autism group. (b) Our nor-mal control group were arguably less intelligent thanthose in Happe's study because there was not such ahigh proportion of students. (c) Some of the originalmentalistic stories were excluded and one was amended.

The clinical groups' difficulties were not likely tobe because they could not comprehend the stories, sincethey did not differ in the number of errors made on thecomprehension question. Neither did the clinical groupsmake more incorrect physical state justifications, whichis what one would have expected to see had there beengeneral comprehension problems. Furthermore, theyperformed normally on the Physical condition or con-trol task which suggests they can comprehend stories.

It is pertinent to explore why the clinical groupsperformed normally on the mentalistic comprehension

Table IV. Participants giving at least one incorrect MentalState Justification

Participant group"

Double bluffFigure of speechJokeLieMisunderstandingPersuasionPretendSarcasmWhite lie

Normal

000010000

Autism

824356262

Asperger

610233241

a n = 17 in each group.


questions, since these can be viewed as a test of appre-ciating nonliteral utterances, and it is well known in thepragmatics literature that even high-functioning individ-uals with autism have problems appreciating such utter-ances (Happe,1991,1993,1995;Ozonoff & Miller, 1996;Rumsey & Hanahan, 1990; Tantam, 1991). The expla-nation for this discrepancy seems to be due to the typeof approach to or type of question asked about such state-ments. In the Strange Stories test there are two questionsabout each of the nonliteral statements. The first is theComprehension question, which asks whether the char-acter's statement is true, and the second is the Justifica-tion question, which asks why the character made thestatement. It seems that these two questions require twolevels of interpretation; the first requires a lower level ofinterpretation and the second requires a higher level ofinterpretation. Thus to answer the first question partici-pants simply have to detect that the character's statementis at odds with the content of the story. Whereas with thesecond question, participants have to integrate the char-acter's statement with the story context and therefore pro-vide a contextually appropriate explanation. It seems thatthe clinical participants had no difficulty in detecting thatthe statement was at odds with the situation, but did havedifficulty in giving a contextually appropriate explantionfor why the character said what they did.

The difference between comprehension, and theintegration of information for higher-level meaning, hasbeen noted elsewhere in the autism literature. ThusRumsey and Hamburger (1988) gave high-functioningautistic adults the Verbal Absurdity and Problem Situ-ations of the Stanford-Binet Intelligence Test (Terman& Merrill, 1973). These authors stated that the re-sponses to the Binet Verbal Absurdity and Problem Sit-uation items reflected comprehension of the linguisticaspects of these problems but failures to integrate theinformation. Their participants, while comprehendingthe information, tended to provide incorrect inferences.An example is the following Problem Situation: "Helenheard a big 'Bang' and came running outdoors. Therewere nails all over the road, and an automobile had juststopped beside the road. What was the bang?" Whereasthe correct answer was that a tire had blown, wrong an-swers included: an explosion occurred; it was a bombor sticks of dynamite; a truck with nails had a crash.Rumsey and Hamburger's (1988) evidence of intactcomprehension in the face of a failure to integrate in-formation to make an appropriate inference is consis-tent not only with the evidence from the Strange Sto-ries test but also with the findings from other pragmaticmeasures such as the finding of an impaired ability toselect contextually appropriate inferences on the Test

of Language Competence (Minshew, Goldstein, Muenz,& Payton, 1992).

In the Strange Stories test the clinical groups' dif-ficulties do not seem to be due to any tendency to beless willing to make a response in comparison to theirnormal control group (i.e., a tendency to make moreomissions) since only one participant in each of the clin-ical groups made an omission. Furthermore, whereasomissions could give rise to fewer context-appropriateanswers, they could not give rise to more context-inappropriate answers.

The fact that the clinical groups did not fail to pro-vide answers suggests that they did not have a problemin generating responses. Similarly, the fact that the clin-ical groups did not differ on the number of mental stateanswers provided suggests that they did not have aproblem in providing mentalistic answers. Instead theproblem for the clinical groups seemed to be one of pro-viding context-appropriate mentalistic answers. This sug-gests a weakness in processing mental state informationin context. Context-inappropriate mental state answerssuggest that weak central coherence might explain thespecific pattern seen on this test.

Where the clinical groups failed to use or extractmeaning from the story context they tended to focus onthe utterance in isolation. This resulted in them tendingto generate a locally coherent rather than globally co-herent answer. So, for example, a participant with autismwho explains the white lie as a joke may be failing touse the story context to inform his answer. Thus, this in-dividual would be making an inference about how a char-acter could have felt but not how he/she actually felt.This is consistent with Frith's (1989) hypothesis that in-dividuals with autism have a preference for processinglocally rather than globally. This tendency to process lo-cally rather than globally has been demonstrated in a re-cent paper, where those with autism or Asperger syn-drome attempted to justify a character's action by givinga locally rather than globally coherent inference (Jolliffe& Baron-Cohen, 1998a).

Although the clinical groups' difficulties suggest aweakness in processing mental state information in con-text, it is not entirely clear whether they could appreci-ate the mental states employed in this test. For exam-ple, it is not clear whether the participants could reallyappreciate what sarcasm means. Support for the notionthat the clinical participants might have a deficient un-derstanding of some of the mental state concepts em-ployed comes from the within-group evidence, whichfound that the normal control participants performedbetter on the Mentalistic condition relative to their ownperformance on the Physical condition, whereas the


clinical groups showed no differential pattern of per-formance on their conditions. It was possible that thenormal control group benefited from their understand-ing of different types of mental state, whereas the clin-ical groups did not. Alternatively, the within-group ev-idence might suggest something uniquely differentabout the contextual processing requirements of the twoconditions. Further evidence for the clinical groups' dif-ficulty on the Mentalistic condition being due to a lackof familiarity with the mental states employed comesfrom the disproportionate trouble they had with the con-cept of irony and double-bluff. Furthermore, the dou-ble-bluff stories needed to be understood at a third-orderToM level, since there is an extra level of embedding,that is, participants have to metarepresent "he knows theythink he will lie", rather than "he knows he will lie."

Next we need to consider why the clinical groupsnot only perform less well on the Mentalistic conditionin comparison to their normal control group, but alsowhy they did not perform worse than their normal con-trols on the Physical condition, since this condition alsorequires the making of global inferences. Examining whythe clinical groups perform normally on the Physicalcondition requires comparing the nature of the stimuliin this condition with that of the Mentalistic condition.The Mentalistic condition requires participants to inferthe meaning of the utterance from the context provided,that is, they need to integrate the utterance with the con-text and thus have to use the context to extract meaningfor the utterance. Whereas two of the six stimuli in thePhysical condition also require participants to infer themeaning of the action from the context provided (i.e.,they need to integrate the action with the context in theArmy and Car stories), the remainder of the stories donot require such processing in context. Thus the re-mainder require a knowledge of why X-rays are nor-mally taken, that the whites of eggs can be used for mak-ing meringues, that breaking security beams sets offalarms, and why glasses that correct long-sight are morelikely to have been left at a Post Office than at othertypes of places such as a flower shop. These stories tapgeneral knowledge more than processing in context. Alsoon these items more attention is perhaps drawn to thesalient elements than occurs with the mentalistic stories.However, although the clinical groups did tend to makemore errors on the two physical stories requiring thegreater contextual processing and hence integration abil-ity, the evidence for this type of processing being a prob-lem for them was extremely weak. It seems that the stim-uli tapping general knowledge tended to be performedslightly better, which may have assisted the clinicalgroups in performing at a normal level on the Physi-cal condition. Finally, it is noteworthy that the crucial

difference between the Physical and Mentalistic condi-tions is that a failure to take the context into accountwould lead to poor performance on the Mentalistic con-dition, but have a lesser effect on the Physical condition.

Although the main aim of this study was to attemptto replicate Happe's (1994) main finding of context-inappropriate explanations, a secondary aim was to seewhether early language development could differentiatethe two groups. As defined in our study, the individualswith Asperger syndrome do not exhibit a clinically sig-nificant delay in early language development, whereasthe autism group recruited all had marked languagedelay. The results on the Strange Stories test suggestsneither quantitative nor qualitative differences. The factthat there were no significant differences between groupssuggests that the presence or absence of early languagedid not differentiate the two groups. However, althoughthe clinical groups did not differ on any of the measures,it was noticeable that the autism group always performedat a level below that of the Asperger group (see TableII, III, and IV). Moreover, although 12 out of 17 of theparticipants with Asperger syndrome gave at least onecontext-inappropriate mental state answer, every one ofthe participants with autism gave at least one such an-swer. Whereas the results of the majority analysis sug-gest that this tendency to give context-inappropriate an-swers characterizes the majority of those on the autismspectrum, it is clear that this tendency was universal tothe autism group but not the Asperger group. However,despite the autism group's relatively less efficient per-formance, the failure to find significant differences be-tween the two clinical groups seems to suggest that onthis test early language development does not discrimi-nate between them. This is reminiscent of the findingsfrom these same participants in other experiments as-sessing the processing of linguistic context for meaning(Jolliffe & Baron-Cohen, 1998a, in press). However, thisfinding is not specific to linguistic material, becauserecent visual experiments with these same participantsalso suggest that individuals with autism are relativelyless efficient when it comes to noticing and identifyingand an incongruent object within a scene (Jolliffe &Baron-Cohen, 1998b) and conceptually integrating mul-tiple fragments of an object (Jolliffe & Baron-Cohen,1998c). Thus the finding that the autism group was rel-atively but not significantly less efficient on the StrangeStories test might reflect more severe symptoms in child-hood rather than language delay per se. However, thisis certainly a fruitful line of enquiry for future research.Currently it seems that performance on the Strange Sto-ries test seems to lend support to autism and Aspergersyndrome being part of the same autistic continuumrather than being discrete conditions.


Picnic

Sarah and Tom arc going on a picnic.It is Tom's idea, he says it is going to bea lovely sunny day for a picnic. Butjust as they are unpacking the food, itstarts to rain, and soon they are bothsoaked to the skin. Sarah is cross. Shesays, "Oh yes, a lovely day for a picnicalright!"

Is it true, what Sarah says?

Why docs she say this?

The Physical Stories

Army

Two enemy powers have been at war for a verylong time. Each army has won several battles, but nowthe outcome could go either way. The forces are equallymatched. However, the Blue army is stronger than theYellow army in foot soldiers and artillery. But the Yel-low army is stronger than the Blue army in air power.On the day of the final battle, which will decide theoutcome of the war, there is a heavy fog over the moun-tains where the fighting is about to occur. Low-lyingclouds hang above the soldiers. By the end of the daythe Blue army have won.

Q: Why did the Blue army win?

Glasses

Sarah is very long-sighted. She has only one pairof glasses, which she keeps losing. Today she has losther glasses again and she needs to find them. She hadthem yesterday evening when she looked up the televi-sion programmes. She must have left them somewherethat she has been today. She asks Ted to find her glasses.She tells him that today she went to her regular earlymorning keep fit class, then to the post office, and lastto the flower shop. Ted goes straight to the post office.

Q: Why is the post office the most likely place tolook?

REFERENCES

American Psychiatric Association. (1987). Diagnostic and statisti-cal manual of mental disorders, (3rd ed., Rev.) Washington, DC.

American Psychiatric Association. (1994). Diagnostic and statis-tical manual of mental disorders (4th ed.) Washington, DC.

Baron-Cohen, S. (1989). The autistic child's theory of mind: A caseof specific developmental delay. Journal of Child Psychologyand Psychiatry, 30, 285-297.

In conclusion, attempts to try to explain why theclinical groups performed poorly on the Mentalisticcondition, but not on the Physical condition, in relationto controls, are unresolved. Thus we are still left withtwo possible reasons for their difficulties on the StrangeStories task. One is the requirement that the participantmust infer the speaker's intended meaning not from theutterance but from the context in which it is embedded.The other is that there might have been a problem inappreciating some of the mental states employed in theStrange Stories test. Our conclusion is that although theStrange Stories test clearly identifies deficits in indi-viduals on the autism spectrum, these are not "pure"deficits in that they could arise for theory of mind rea-sons, or central coherence reasons, or both. [For a moresensitive measure of adult ToM (and evidence of itsimpairment in these same patients) see Baron-Cohen,Jolliffe, Mortimore, & Robertson, 1997.] Because it isnot entirely clear what the source of the clinical groups'difficulty is, future research needs to address this issue.

ACKNOWLEDGMENTS

We are grateful to a large number of people forhelping us recruit participants. Lorna Wing, WendyPhillips, the National Autistic Society, Pat Howlin,Clive Robinson, Ben Sacks, and Pam Yates. We aregrateful to Ian Nimmo-Smith for advice on methodol-ogy and Francessca Happe for allowing us to use andadapt her materials. The first author was supported byan MRC Studentship during the period of this work.The Harold Hyam Wingate Foundation also providedher with valuable financial support. Both studies re-ported here were submitted in part fulfilment of herdoctoral degree at the University of Cambridge.

APPENDIX

The Mentalistic Stories

Banana

Katie and Emma arc playing in thehouse. Emma picks up a banana fromthe fruit bowl and holds it up to hercar. She says to Katie, "Look! Thisbanana is a telephone!"

Is it true what Emma says?

Why does Emma say this?


Baron-Cohen, S. (1995), Mindblindness: An essay on autism and the-ory of mind. Cambridge, MA: MIT Press/Bradford Books.

Baron-Cohen, S. (1997). Hey! It was a joke! Understanding propo-sitions and prepositional attitudes by normally developing chil-dren, and children with autism. Israel Journal of Psychiatry, 34,174-178.

Baron-Cohen, S., Jolliffe, T., Mortimore, C., & Robertson, M. (1997).Another advanced test of theory of mind: evidence from veryhigh functioning adults with autism or Asperger syndrome. Jour-nal of Child Psychology and Psychiatry, 38, 813-822.

Baron-Cohen, S., Leslie, A. M., & Frith, U. (1985). Does the autis-tic child have a 'theory of mind'? Cognition, 21, 37-46.

Frith, U. (1989). Autism: Explaining the enigma. Oxford: Blackwell.Happe, F. G. E. (1991). Theory of mind and communication in autism.

Unpublished PhD thesis, University of London.Happe, F. G. E. (1993). Communicative competence and theory of mind

in autism: A test of relevance theory. Cognition, 48, 101-119.Happe, F. G. E. (1994). An advanced test of theory of mind: Under-

standing of story characters' thoughts and feelings by able autis-tic, mentally handicapped, and normal children and adults. Jour-nal of Autism and Developmental Disorders, 24, 129-154.

Happe, F. G. E. (1995). The role of age and verbal ability in thetheory of mind task performance of subjects with autism. ChildDevelopment, 66, 843-855.

Howlin, P. (1995). The Revised Howlin Screening Questionnaire,St. George's Hospital Medical School, University of London.

Jolliffe, T., & Baron-Cohen, S. (1998a). Linguistic processing inhigh-functioning adults with autism or Asperger syndrome: Canglobal coherence be achieved? A further test of central coher-ence theory. Unpublished manuscript, University of Cambridge.

Jolliffe, T., & Baron-Cohen, S. (in press). A test of central coherencetheory: Linguistic processing in high-functioning adults withautism or Asperger syndrome: Is local coherence impaired?Cognition.

Jolliffe, T. & Baron-Cohen, S. (1998b). A test of central coher-ence theory: Can adults with high-functioning autism or As-perger syndrome integrage objects in context? Unpublishedmanuscript. University of Cambridge.

Jolliffe, T. & Baron-Cohen, S. (1998c). A test of central coherencetheory: Can adults with high-functioning autism or Asperger

syndrome integrage fragments of an object? Unpublished man-uscript. University of Cambridge.

Kanner, L. (1943). Autistic disturbances of affective contact. Ner-vous Child, 2, 217-250.

Klin, A., Volkmar, F. R., Sparrow, S. S., Cicchetti, D. V., & Rourke,B. P. (1995). Validity and neuropsychological characterisationof Asperger syndrome: Convergence with nonverbal learningdisabilities syndrome. Journal of Child Psychology and Psy-chiatry, 36, 1127-1140.

Miller, J. N., & Ozonoff, S. (1996). Did Asperger's cases have As-perger disorder? A research note. Journal of Child Psychologyand Psychiatry, 38, 247-251.

Minshew, N. J., Goldstein, G., Muenz, L. R., & Payton, J. B. (1992).Neuropsychological functioning in non-mentally retarded autis-tic individuals. Journal of Clinical and Experimental Neu-ropsychology, 14, 749-761.

Ozonoff, S., & Miller, J. N. (1996). An exploration of right-hemispherecontributions to the pragmatic impairments of autism. Brain andLanguage, 52, 411-434.

Perner, J., Frith, U., Leslie, A. M. & Leekam, S. R. (1989). Explo-ration of the autistic child's theory of mind: Knowledge, beliefand communication. Child Development, 60, 689-700.

Rumsey, J. & Hamburger, S. (1998). Neuropsychological findingsin high-functioning men with infantile autism, residual state.Journal of Clinical and Experimental Neuropsychology, 10,201-221.

Rumsey, J. M. & Hanahan, A. P. (1990). Getting it "right": Perfor-mance of high-functioning autistic adults on a right-hemispherebattery. Journal of Clinical and Experimental Neuropsychology,12, 81.

Tantam, D. (1992). Characterizing the fundamental social handicapin autism. Acta Paedopsychiatrica, 55, 88-91.

Terman, L. M. & Merrill, M. A. (1973). Stanford-Binut IntelligenceScale Form L-M. (3rd edition). Boston: Houghton Mifflin.

Wechsler, D. (1981). Wechsler Adult Intelligence Scale - Revised.New York: Psychological Corp.

Wing, L. (1981). Asperger syndrome: A clinical account. Psycho-logical Medicine, 11, 115-129.

World Health Organization. (1994). International classification ofdiseases and related health problems (10th ed.) Geneva.

Documents

The Strange Stories Test: A Replication with High ...docs.autismresearchcentre.com/papers/1999_Jolliffe_BC_Stories.pdfThe Strange Stories Test: A Replication with High - Functioning