RESEARCH ARTICLE

Multisensory Processing in Children With Autism: High-DensityElectrical Mapping of Auditory–Somatosensory Integration

Natalie Russo, John J. Foxe, Alice B. Brandwein, Ted Altschuler, Hilary Gomes, and Sophie Molholm

Successful integration of signals from the various sensory systems is crucial for normal sensory–perceptual functioning,allowing for the perception of coherent objects rather than a disconnected cluster of fragmented features. Several prominenttheories of autism suggest that automatic integration is impaired in this population, but there have been few empirical testsof this thesis. A standard electrophysiological metric of multisensory integration (MSI) was used to test the integrity ofauditory–somatosensory integration in children with autism (N 5 17, aged 6–16 years), compared to age- and IQ-matchedtypically developing (TD) children. High-density electrophysiology was recorded while participants were presented witheither auditory or somatosensory stimuli alone (unisensory conditions), or as a combined auditory–somatosensory stimulus(multisensory condition), in randomized order. Participants watched a silent movie during testing, ignoring concurrentstimulation. Significant differences between neural responses to the multisensory auditory–somatosensory stimulus and theunisensory stimuli (the sum of the responses to the auditory and somatosensory stimuli when presented alone) served asthe dependent measure. The data revealed group differences in the integration of auditory and somatosensory informationthat appeared at around 175ms, and were characterized by the presence of MSI for the TD but not the autism spectrumdisorder (ASD) children. Overall, MSI was less extensive in the ASD group. These findings are discussed within theframework of current knowledge of MSI in typical development as well as in relation to theories of ASD.

Keywords: autism spectrum disorders; electrophysiology; multisensory integration; auditory processing; somatosensoryprocessing; development

Introduction

Anecdotal reports from parents, clinicians and indivi-

duals with autism have lead to the impression that there

are significant sensory processing atypicalities in a large

percentage of individuals with an autism spectrum

disorder (ASD) [Cesaroni & Garber, 1991; Grandin,

1992; O’Neill & Jones, 1997; Williams, 1994]. In the

1970’s, sensory processing research was central to the

field of ASD, with researchers finding evidence for

atypical sensory modulation [e.g. Stroh & Buick, 1964],

a preferential reliance on proximal rather than distal

senses [Hermelin & O’Conner, 1970], a dominance of

somatosensory over auditory stimuli [Hermelin & O’Con-

ner, 1970], a tendency to be overselective in responding

to only one aspect of a multisensory object [e.g. Lovaas &

Schreibman, 1971; Lovaas, Schreibman, Koegal, & Rehm,

1971], and a tendency to benefit less from visual cues in a

visual-motor task than typically developing (TD) children

[Hermelin & O’Conner, 1970]. Despite these findings, the

research on sensory processing in ASD fell off in the

1980’s and 1990’s, to be replaced by research on higher-

order processes, such as executive function [e.g. Ozonoff

& Strayer, 1997], language [e.g. Tager-Flusberg, 1996], and

social difficulties [e.g. Klin et al., 1999; Tager-Flusberg &

Sullivan, 1994], that are also common among persons

with ASDs. However, a paradigm shift that once again

emphasizes basic sensory processing differences among

persons with ASD has emerged in recent years, as

evidenced by a resurgence of research utilizing ques-

tionnaire methods to describe sensory processing diffi-

culties [e.g. Crane, Goddard, & Pring, 2009; Lane, Young,

Baker, & Angley, 2010; Schoen, Miller, Brett-Green, &

Nielsen, 2009; Tomchek, 2007], and an emerging litera-

ture on unisensory detection and discrimination in ASD

[e.g. Bertone, Mottron, Jelenic, & Faubert, 2003, 2005;

Bonnel et al., 2003; O’Riordan, 2004; O’Riordan &

Plaisted, 2001; O’Riordan & Passetti, 2006]. This shift is

particularly evident in the impending changes in the

diagnosis of ASD expected in the DSM-V, in which social

INSAR Autism Research 3: 1–15, 2010 1

Received July 13, 2009; accepted for publication June 25, 2010

Published online in Wiley Online Library (wileyonlinelibrary.com)

DOI: 10.1002/aur.152

& 2010 International Society for Autism Research, Wiley Periodicals, Inc.

From the City College of New York, The Children’s Research Unit, Program in Cognitive Neuroscience, Departments of Psychology & Biology, New York

(N.R., J.J.F., T.A., H.G., S.M.,); Albert Einstein College of Medicine, Children’s Evaluation and Rehabilitation Center (CERC); Departments of Pediatrics

and Neuroscience, Bronx, New York (N.R., J.J.F., A.B.B., T.A., S.M.,); Queens College of the City University of New York, Flushing, New York (J.J.F., A.B.B.,

H.G.)

Address for correspondence and reprints: Sophie Molholm, Albert Einstein College of Medicine, Children’s Evaluation and Rehabilitation Center

(CERC), Departments of Pediatrics and Neuroscience, Jack & Pearl Resnick Campus, 1300 Morris Park Avenue, Van Etten 1st floor, C-wing, Bronx, NY,

10461. E-mail: Sophie.Molholm@einstein.yu.edu

Grant sponsor: NIH; Grant number: MH 085322; Grant sponsors: Wallace Research Foundation; Cure Autism Now; Fondation du Quebec de Recherche

sur la Societe et la Culture; Canadian Institute of Health Research.

and communication difficulties are being combined into

one diagnostic criterion, thus placing equal weight on

both the negative (language, socialization, and commu-

nication) and positive symptoms (repetitive behaviors,

restricted interests, and sensory issues) of autism.

One suggestion that has gained prominence in the

autism community, but that has only recently become

the subject of empirical investigation, is that a problem

with the integration of sensory information may underlie

some of the symptoms observed in autism, including the

observed sensory hyper- and hypo-sensitivities. Despite

renewed interest in sensory issues and the prominence of

so-called sensory integration treatment centers and books

written on this topic [e.g. Ayers, 1994], surprisingly little

empirical work has been conducted to actually test the

integrity of multisensory integration (MSI) in autism.

MSI in Typical Development

In adults, redundant or mutually informative multi-

sensory inputs tend to be effortlessly integrated, and

often lead to faster and better performance on discrimi-

nation and detection tasks relative to performance on

their unisensory counterparts [e.g. Frens & Van Opstal,

1995; Molholm et al., 2002; Molholm, Ritter, Javitt, &

Foxe, 2004; Murray et al., 2005; Perrott, Saberi, Brown, &

Strybel, 1990; Spence & Driver, 1997; Stein & Meredith,

1990; Zahn, Abel, & Dell’Osso, 1978]. The same is the

case for more complex perceptual-cognitive processes,

with faster identification of objects when multisensory

inputs are provided and better speech recognition when

the speaker’s articulations can be seen compared to when

they cannot [Driver, 1996; Molholm et al., 2004; Ross,

St-Amour, Leavitt, Javitt, & Foxe, 2007]. The ability to

recognize lawful relationships between multisensory

inputs is seen from a very early age, with infants able to

recognize that audiovisual stimuli are temporally asyn-

chronous as early as 4 months of age [Lewkowicz, 2002].

Natural ‘‘sensory deprivation’’ in humans in cases of early

blindness [Hotting, Rosler, & Roder, 2004; Putzar,

Goerendt, Lange, Rosler, & Roder, 2007], and controlled

sensory deprivation studies in animals [Carriere et al.,

2007; Wallace, Perrault, Hairston, & Stein, 2004] reveal

that, as might be expected, post-natal interaction with

the environment plays a significant role in the develop-

ment of MSI. Developmental studies also reveal that for

some types of information, MSI processes tune-up to

optimal levels over the first decade or two of life

[Brandwein et al., accepted; Gori, Del Viva, Sandini, &

Burr, 2008; Nardini, Jones, Bedford, & Braddick, 2008].

MSI in ASDs

While it has been speculated that multisensory proces-

sing is impaired in autism, there has been little empirical

testing of this notion [see Foxe & Molholm, 2009]. What

studies there are have all examined interactions between

auditory and visual sensory inputs, and these studies

have yielded equivocal results. For example in one study,

temporal asynchrony judgments were impaired com-

pared to matched controls for linguistic stimuli but not

for non-linguistic stimuli [Bebko, Weiss, Demark, &

Gomez, 2006]. In another, Smith and Bennetto [2007]

found that children and adults with ASD were less

susceptible to the McGurk illusion, in which the percept

of a speech sound is changed by the simultaneous

presentation of an incongruous visual speech articula-

tion. This suggests that these auditory and visual speech

inputs were not integrated in a typical manner in ASD, a

finding receiving additional support from work by de

Gelder, Vroomen, and van der Heide [1991] who noted

that in spite of normal lip-reading ability, individuals

with ASD showed less influence of visual speech on

auditory speech perception than mental age matched TD

individuals. In contrast, van der Smagt, van Engeland,

and Kemner [2007], found that children with ASD were

just as susceptible to an audiovisual illusion as were TDs.

For both groups, hearing multiple sounds resulted in the

illusory perception of multiple flashes when only a single

flash was presented [the beep-flash illusion, see Shams,

Kamitani, & Shimojo, 2000, 2002].

Cross-sensory priming paradigms, in which a visual

stimulus precedes an auditory stimulus and its effects on

auditory processing are examined, have demonstrated

intact visual modulation of the auditory electrophysio-

logical response among persons with an ASD [Magnee, de

Gelder, van Engalend, & Kemner, 2008; Magnee, Oranje,

van Engeland, Kahns, & Kemner, 2009]. In these studies a

visual stimulus was presented 350–500 ms before the

onset of an auditory stimulus (in one case the visual

stimulus remained on for the duration of the auditory

stimulus and in the other it did not), and suppression of

the auditory-N1 response was observed. While these data

have been interpreted as evidence for intact early multi-

sensory processing in ASD, Vroomen and Stekelenburg

[2009] have convincingly demonstrated that this sort of

cross-sensory N1 suppression is due to the predictive

properties of the preceding visual stimulus on onset of

the upcoming auditory stimulus, and not multisensory

processing per say. Hence, preservation of such effects in

ASD cannot be taken as evidence for intact multisensory

processing.

The literature on multisensory processing in ASD thus

far has largely been confined to studies that have

required behavioral responses to assess multisensory

function [but see Magnee et al., 2008, 2009]. While

behavioral measures are certainly valuable, a disadvan-

tage of this approach is that it rests on the assumption

that observed differences largely reflect multisensory

processing, whereas behavioral differences could well be

due to differing interpretation of task instruction or

2 Russo et al./Auditory–somatosensory integration in ASD INSAR

differences in how attention is allocated (e.g. superior

attentional allocation for one group over the other). In

either of these cases, the implications are clearly

different. In addition, observed similarities in behavior

do not necessarily mean that the underlying brain

processes are the same. Electrophysiology provides a

metric of multisensory processing at multiple stages of

the processing hierarchy, and can be generated in the

absence of attentional demands through the use of

‘‘passive’’ paradigms in which there is no task. As a

result, the integrity of MSI can be assessed without

confounding group differences in attentional allocation

or task comprehension.

Electrophysiology of MSI in Typical Development

Both passive and active paradigms have been used to

assess the electrophysiology of auditory–somatosensory

integration in TD adults and children, with consistent

findings that MSI occurs at multiple processing stages

starting early on (circa 50 ms) in cortical areas that were

traditionally considered to be unisensory [e.g. Foxe et al.,

2000; Lutkenhoner, Lammertmann, Simoes, & Hari,

2002]. In one of the first published electrophysiological

studies of MSI in children, Brett-Green, Miller, Gavin, and

Davies [2008] examined multisensory processing in

children between the ages of 6 and 13 years. Using

median nerve stimulation and tones presented alone or

together, the authors found three time periods during

which multisensory interactions occurred: 60–80 ms over

central scalp regions of the hemisphere contralateral to

the stimulated side, 110–150 ms in the hemisphere

ipsilateral to the side of the stimulation over central-

parietal areas, and 180–220 ms over central regions

bilaterally. These latencies, though slightly later than

those found in adults [e.g. Foxe et al., 2002; Murray et al.,

2005; Sperdin, Cappe, Foxe, & Murray, 2009], support the

notion that children, like adults, integrate multisensory

information during both early and later sensory–percep-

tual time frames.

In the present study, high-density electrophysiology

was used to test the integrity of multisensory processing

of auditory and somatosensory information among

persons with an ASD. Auditory and somatosensory

stimuli were presented in randomized order, either alone

or simultaneously, in a passive paradigm in which

participants engaged in an unrelated activity (watching

a silent video). MSI was indexed by comparing electro-

physiological responses to the stimuli when presented

alone vs. when presented together [e.g. Berman, 1961;

Brett-Green et al., 2008; Foxe et al., 2000, 2002; Giard &

Peronnet, 1999; Molholm et al., 2002; Murray et al.,

2005; Talsma & Woldorff, 2005; Teder-Salejarvi, McDonald,

DiRusso, & Hillyard, 2002]. This commonly used index of

MSI entails summing the responses to the auditory and

somatosensory stimuli when presented alone and com-

paring this ‘‘sum’’ waveform to the response to the

auditory and somatosensory stimuli when presented

together. Because electrical fields sum linearly at the

scalp, any differences between the two responses can

then be attributed to the two inputs having interacted

when they were presented together.1 Our working

hypothesis was that this electrophysiological metric

would reveal differences in multisensory processing

between individuals with ASD and TD, age- and

IQ-matched children. Specifically, we predicted a reduc-

tion or absence of MSI in ASD at earlier time points,

which we assume to reflect more automatic integrative

processes, and a corresponding increase in MSI at later

time points (4200 ms) reflecting compensatory and less

automatic processing.

MethodsParticipants

Seventeen children with ASD and 17 TD children

between the ages of 6 and 16 years participated in this

study (see Table 1). In accordance with the Declaration of

Helsinki, the parents of all the participants provided a

written informed consent, and when appropriate, chil-

dren provided a written assent. All the procedures and

consent forms were approved by the Institutional Review

Board of the City College of New York.

Exclusionary criteria for both groups included uncor-

rected vision problems, a history of seizures, and the use

of psychotropic medication. TD children were excluded if

they had a history of educational, attentional, psychiatric,

or developmental difficulties as assessed by a history

questionnaire, and were also excluded if their parents

endorsed six items of inattention or hyperactivity on a

Table 1. Participant Characteristics for the Autism SpectrumDisorder (ASD) and Typically Developing (TD) Groups

Age Verbal IQ Performance IQ Full-scale IQ

TD 10.48(2.92) 110.3 (11.83) 106.53 (13.35) 108.4 (12.65)

ASD 10.36 (2.71) 102.52(18.1) 105.06 (14.24) 104.7 (14.58)

P-values 0.9 0.15 0.76 0.3

Values within parenthesis represent SD.

1The assumption of linearity no longer holds for task-related processes

that would be shared for the two unisensory inputs in an active paradigm,

such as when each would result in the recruitment of motor cortex to

make a button press response. This is because this process would be

doubly represented in the summed response and only represented once in

the multisensory response. However, in the present study, we are dealing

with a passive paradigm and focusing mainly on the sensory processing

time frame, and hence the assumption of linearity is expected to hold,

with its violation representing the occurrence of multisensory interac-

tions in the brain.

INSAR Russo et al./Auditory–somatosensory integration in ASD 3

DSM-IV behavioral checklist of attention deficit disorder

(with and without hyperactivity). Both forms were filled

out by parents on the first day of testing. The parents of

children with ASD were asked to refrain from giving their

children stimulant medication (n 5 2) before the testing

session.

Diagnoses for 15 of the 17 children with ASD were

made using both the Autism Diagnostic Interview-R

[ADI-R; Lord, Rutter, & LeCouter, 1994] and the Autism

Diagnostic Observation Schedule [ADOS-G; Lord, Rutter,

DiLavore, & Risi, 1999] by a trained administrator. For the

remaining two, who were diagnosed with Pervasive

Developmental Disorder, Not Otherwise Specified (PDD-

NOS), ADOS, and ADI, data were not available and

diagnosis was made on the basis of the diagnostic criteria

outlined in the DSM-IV-TR [APA, 2000]. Of the 17 ASD

participants, seven had autism, eight had Asperger’s

disorder, and two had a PDD-NOS. The children with

ASD and the TD children were matched on a one-to-one

basis, and within one standard deviation on Full Scale

and Performance IQ’s (FSIQ and PIQ) as measured by the

Wechsler Abbreviated Test of Intelligence [Wechsler,

1999], as well as on chronological age. The two groups

did not differ significantly on any of the matching

measures nor on Verbal IQ (see Table 1). All of the

children had IQs in the average to above average range

with the exception of one TD and one ASD child with

IQ’s in the borderline range.

Stimuli and Procedure

Testing was conducted over 2 days. In a first session,

participants completed the diagnostic and psychometric

evaluation. Event-Related Potential (ERP) recordings were

conducted on a second visit in a sound attenuated, dimly

lit electrically shielded room. Participants were seated in

a comfortable chair and watched a film of their choice

without sound during the testing protocol, which was

presented in three blocks of 10 min each.

Somatosensory stimulation. Somatosensory stimu-lation consisted of a 128-Hz vibrotactile stimulationpresented to the child’s right hand for 30ms. Thestimulation was delivered through a 2-layer Transducermanufactured by Piezo Systems, Inc, which was driven bythe soundcard of a PC through an audio cable. Thestimulator was held with gauze between the child’sthumb, index, and middle finger, and the child’s handwas covered by a black cloth, as seeing somatosensorystimulation can affect the somatosensory response [e.g.Taylor-Clarke, Kennett, & Haggard, 2002].

Auditory stimulation. Auditory stimulation consistedof a 1000Hz tone presented for 30ms through a speakerplaced on the child’s right side. The somatosensory andauditory stimuli could either be presented alone (unisensoryconditions) or simultaneously (multisensory condition) for

a total of three stimulus conditions: auditory (A: 300 trials),somatosensory (S: 300 trials) and multisensory (AS: 300trials), totaling 900 trials. The conditions were presented ina randomized order, with a randomized square-wavedistribution of Stimulus Onset Asynchrony’s (SOAs)ranging from 700 to 3,000ms. This large variance in SOAis an effective way to circumvent issues with anticipatoryslow-wave activity [see Teder-Salejarvi et al., 2002].

Measures and analyses. Sixty-eight-channel scalp EEGwas recorded, amplified, and digitized at 512 Hz usingBioSemi Systems Active Amplifiers. BioSemi uses twoelectrodes—the Common Mode Sense (CMS), which isactively recorded, and the Driven Right Leg (DRL), apassive electrode—that together form a feedback loopthat represent the reference. The acquisition of the dataoccurs referenced to the CMS-DRL ground which drivesthe average potential of the participant (i.e. the commonmode voltage) as close as possible to the AC referencevoltage of the Analog-to-Digital box (for a description ofthe BioSemi active electrode system referencing andgrounding conventions, visit www.biosemi.com/faq/cms&drl.htm).

The continuous raw EEG data were first visually

inspected to ensure sufficient data quality across our

subject population without regard for group membership

(TD or ASD). Based on this initial inspection and

preliminary exploratory analyses using preset artifact

rejection criteria, it was determined that 160 usable trials

per condition could reliably be extracted for the vast

majority of subjects and this number was then preset as

our target trial count. This strategy was motivated by a

wish to equate the number of trials sampled from

participants from both populations. Specifically, 160

trials of each of the three conditions, distributed evenly

across the three stimulation blocks (55 trials from each

block), were selected by hand from the raw EEG to be

included in the individual subject averages, with the

restriction that the signal did not exceed the artifact

rejection criterion of plus or minus 130 microvolts. Note

that in almost all cases, there were considerably more

than 160 trials that met these criteria. An exception was

made in the case of just one ASD participant for whom

we could only identify 80 trials across each of the

stimulation types that met these criteria. In turn, we

sampled just 80 trials per condition from his matched

control participant. It is important to point out that

during the hand selection procedure, there was no

possibility that the evoked brain responses could be

visualized in the raw EEG and the experimenter was blind

as to group membership. Accordingly trial selection

proceeded in an unbiased manner. A second criterion

was applied to selection of trials containing somatosen-

sory stimulation. Trials were only chosen for inclusion in

averages for those conditions where the presence of a

prominent and typical stimulus-artifact from the soma-

tosensory stimulator could be visualized in the raw EEG.

4 Russo et al./Auditory–somatosensory integration in ASD INSAR

The latter was done as an assurance that somatosensory

stimulation had definitively occurred, which could be

verified on greater than 75% of trials. This telltale

stimulus-artifact results from skin conduction of the

electrical current of the stimulator to the electrodes on

the scalp. We took this cautious approach because the

lack of such a stimulus-artifact could potentially indicate

that the stimulator had not made full contact on those

trials where it was not fully evident, and we wanted to

protect against any possibility that trials without appro-

priate stimulation might be included in the averages,

although this was likely the case on only a very small

minority of trials.

For post-processing, a band-pass filter of 2.4 Hz at 12db/

octave–30 Hz at 12 db/octave was applied to the contin-

uous data. High-pass filtering was performed to remove

any ongoing slow-wave activity remaining in the signal,

which would otherwise be doubly represented in the

summed unisensory response when assessing MSIs [see

Molholm et al., 2002; Teder-Salejarvii et al., 2002]. Low-

pass filtering served to remove high-frequency artifact

that resulted from the somatosensory stimulator. The

EEG data were epoched into 600 ms segments that

included the 100-ms preceding stimulus onset, artifact

rejected at7130 mV, and averaged based on stimulation

type using Brain Electric Source Analysis version

(www.BESA.de). The data were re-referenced to the

average reference for all analyses and figures. For analysis

of MSIs, the auditory-alone and somatosensory-alone

responses were summed for comparison with the multi-

sensory condition.

Data Analysis

Analysis of the data was guided by previous studies that

have similarly used electrophysiology to assess MSI of

simple auditory and somatosensory stimuli [e.g. Foxe

et al., 2000]. These studies have shown a series of

multisensory modulations that fall on the major peaks

of the auditory–somatosensory response over the first

200 ms of processing. To constrain our analyses, we

therefore used the average of the grand-mean auditory–

somatosensory responses from each of the ASD and TD

groups to define the peaks of the multisensory response,

and constrained our planned analyses to windows

around these peaks. The electrodes tested were selected

based on the corresponding scalp distribution of the

responses at each of the latencies. This technique allowed

us to predefine time windows and scalp regions of

analysis, and to do so in a way that did not make

reference to the dependent measure. Significant interac-

tions with the factor of group were examined by

conducting univariate Repeated Measures ANOVA’s.

The results of all the ANOVA’s are based on Green-

house–Geiser corrections.

Exploratory Analysis

In addition to the above conservative approach to data

analysis, exploratory ‘‘cluster plot’’ analyses were also

performed to fully describe the data. In this approach,

t-tests are performed for each data point for each

electrode and significance values are plotted for each of

these points and displayed in a cluster plot. To reduce the

probability of Type-1 errors, significant differences are

only considered if a criterion of Po0.05 is achieved for at

least 10 consecutive time points [Guthrie & Buchwald,

1991]. It should be pointed out that this criterion would

exclude any effects that did not last for at least 18 ms,

given our digitization rate of 512 Hz. This is a suitable

alternative to Bonferroni correction for multiple compar-

isons, which would increase the likelihood of Type II

errors through overcompensation for Type I errors. This

analysis has the advantage of providing a snapshot of all

significant differences within a single frame, and the

potential to reveal unpredicted effects that can then be

used as a hypothesis generation tool for future studies.

Using this approach, MSIs for each of the participant

groups were examined by comparing the multisensory

and the sum response. Using this same approach, each of

the auditory and somatosensory unisensory responses

were compared across the groups to probe for the

possibility of group differences in basic sensory proces-

sing.

ResultsUnisensory Responses

The electrophysiological responses to the auditory-alone

stimuli (Fig. 1) were highly similar in morphology

between the two participant groups. The scalp distribu-

tion and timing of the auditory response for both groups,

illustrated in Figure 1, represents an amalgam of typical

auditory-evoked responses for younger and older chil-

dren, with major responses both fronto-centrally and at

lateral temporal scalp regions [Ceponiene, Rinne &

Naatanen, 2002]. This is to be expected from our

relatively broad age sample. While the group responses

were highly similar, starting at 100 ms the ASD response

tended to be smaller in amplitude. This diminution of

the response can be clearly seen at the response peaks,

over fronto-central scalp at 100 ms and over bilateral

temporal scalp regions around 150 ms. The electrophy-

siological responses to the somatosensory-alone stimuli

(Fig. 2) also showed a similar morphology across the

groups, but here too responses appeared to be larger for

the TD group compared to the ASD group, starting at

about 70 ms. Somatosensory responses were the greatest

contralateral to the side of stimulation, over left central

scalp areas, with positive-going peaks at 53 and 110 ms.

This latter peak was more pronounced for the TD than

INSAR Russo et al./Auditory–somatosensory integration in ASD 5

TP7TP8

T7 T8

–100 0 100 200 300 400 500–5–4–3–2–1

012345

time in milliseconds

am

plitu

de in

mic

rovo

lts

–100 0 100 200 300 400 500

time in milliseconds

ASD

TD

F1 FZ F2

C4C3

Electrode locations

Figure 1. Auditory ERPs for the typically developing children and the children with an autism spectrum disorder at frontal, central andtemporal-parietal electrodes. [Color figure can be viewed online at wileyonlinelibrary.com.]

6 Russo et al./Auditory–somatosensory integration in ASD INSAR

the ASD group. The responses of both groups evolved

into a bilateral positive distribution over frontal areas

that showed a positive peak at 200 ms and a negative

peak at 300 ms.

Multisensory Interactions

For the analysis of multisensory interactions, time bins and

electrode locations were selected on the basis of the

averaged multisensory response collapsed across the two

groups. Inspection of the corresponding topographical map

revealed five distinct stable spatio-temporal patterns: The

first peaked at 53ms and had a positive distribution focused

over the left temporal scalp. A second pattern peaked at

110ms and had a positive focus over bilateral temporal scalp

and a negative focus over the fronto-central scalp. A third

peak was noted at 164ms, and represented a negative focus

over the right temporo-parietal scalp. A fourth peak was

positive and present at frontal regions at 190ms. The final

stable spatio-temporal configuration peaked at 273ms, with

a positive bilateral temporo-central focus and a correspond-

ing negative frontal focus. These five peaks constituted the

center of the time windows that we used in our a priori

analyses, with windows of analysis from 43 to 63, 100 to

125, 154 to 178, 175 to 205, and 255 to 299ms, respectively.

Earlier peaks were fitted with smaller time windows while

later peaks were fitted with larger windows, to reflect the

general pattern of a broadening of componentry at later

time points of the evoked response. Data from six separate

scalp regions, corresponding to the five distinct spatio-

temporal patterns and their contralateral homologs where

applicable, were tested. These corresponded to the frontal

2FZF1F

4PC3PC

–100 0 100 200 300 400 500–5–4–3–2–1012345

time in milliseconds

ampl

itude

in m

icro

volts

CP6CP5

–100 0 100 200 300 400 500

time in milliseconds

ASDTD

Electrode locations

Figure 2. Somatosensory ERPs for the typically developing children and the children with an autism spectrum disorder at frontal, central andtemporal-parietal electrodes. [Color figure can be viewed online at wileyonlinelibrary.com.]

INSAR Russo et al./Auditory–somatosensory integration in ASD 7

(F1, Fz, and F2), fronto-central (FC1, FCz, and FC2), left

central (T7, C5, and C3), right central (T8, C6, and C4), left

temporal (TP7, CP5, and CP3), and the right temporal (TP8,

CP6, and CP4) regions. The regions tested for each of the

specific time windows were as follows: left and right

temporal regions (TP7, CP5, CP3, T8, CP6, and CP4) were

tested for the 43- to 63-ms window, left and right temporal

regions (TP7, CP5, CP3, T8, CP6, and CP4), and left and

right central regions (T7, C5, C3, T8, C6, and C4) were

tested for the 100- to 125-ms time window, left and right

temporal (TP7, CP5, CP3, TP8, CP6, and CP4) were tested for

the 154- to 178-ms time window, frontal central (FC1, FCz,

and FC2) and frontal regions (F1, Fz, and F2) were tested for

the 175- to 205-ms time window, and finally, frontal central

(FC1, FCz, and FC2), left and right temporal (TP7, CP5, CP3,

T8, CP6, and CP4), and left and right central regions (T7,

C5, C3, T8, C6, and C4) and were tested for the 255- to 299-

ms window. The multisensory and ‘‘sum’’ ERPs were

compared using separate repeated measures ANOVA’s for

each time bin of analysis on the average referenced data,

with the between-subjects factor of group and the within-

subjects factors of condition (summed vs. multisensory) and

the corresponding region factors. Waveforms from repre-

sentative electrodes are presented in Figures 3 and 4 for

the TD and ASD groups respectively.

The analysis of data from 43 to 63 ms yielded no main

effect of condition or any significant interactions. In the

100- to 125-ms window, there was a main effect of

condition (F (1, 32) 5 6.121, P 5 0.019), providing

evidence of MSI. Neither group nor region interacted

significantly with condition (Ps40.53). In the 154- to

178-ms time window, there was a condition by region

interaction (F (1, 32) 5 9.91, P 5 0.004). This interaction

showed that the difference between sum and multi-

sensory responses was greater at the left (F (1, 32) 5 5.97,

P 5 0.02) than at the right temporal regions (F (1, 32) 5 4.2,

P 5 0.049). In the 175- to 205-ms time window, there was a

main effect of condition (F (1, 32) 5 4.727, P 5 0.037) and a

condition by group interaction that approached significance

(F (1, 32) 5 3.43, P 5 0.073), which was found in a

follow-up analysis to be related to the presence of significant

differences between summed and multisensory responses

for the TD group (F (1, 32) 5 8.11, P 5 0.008) but not the

ASD group (F (1, 32) 5 .051, P 5 0.822). Finally, in the 255-

to 299-ms time window, there was a condition by group

interaction (F (1, 32) 5 4.94, P 5 0.033), which in a follow-

up analysis was found to be related to the presence of

significant differences between the summed and multi-

sensory responses for the TD (F (1, 32) 5 5.3, P 5 0.028) but

not the ASD group (F(1, 32) 5 0.717, P 5 0.4).

Exploratory Analyses

Unisensory responses. In the exploratory analysis ofbetween-group differences in the processing ofunisensory stimuli, the statistical cluster plots

comparing the auditory responses revealed four timeperiods of significant difference between TD and ASDchildren (see panel A of Fig. 5). A first early difference wasapparent around 85 ms over central regions,characterized as a slightly larger P1 response for thegroup of children with ASD, a second difference around110 ms over parietal and frontal scalp regionsrepresenting a smaller N1 response in the ASD group, athird around 210 ms over more central scalp regions,which seemed to represent an N2 response that wasshifted earlier in time for the children with ASD, and afourth late difference around 350 ms over fronto-centralscalp regions, which reflected a larger P3 response for thechildren with ASD. Turning now to the between-groupdifferences in the somatosensory response (see panel B ofFig. 5), there were differences at about 70 and 100 ms,reflecting the smaller positive-going responses at theselatencies in the ASD group over the left centro-parietalregions. A difference was also seen at 150 ms due to asmaller negative going response in the ASD group overthe left centro-parietal scalp. Other differences werenoted at around 280 ms over the centro-parietal andparietal regions, with the somatosensory response in thistime frame appearing to occur earlier for the ASD thanthe TD group. Finally, late differences were noted againaround 380 ms over frontal regions, which reflected agreater response in the ASD compared to the TD group(see panel B of Fig. 5).

Multisensory responses. Exploratory analysis of MSIsusing the statistical cluster plots painted astraightforward picture of the MSIs in the ASD and TDgroups (see Panels C and D of Fig. 5). Specifically, theplots revealed a pattern of MSI from about 120–200 msthat was present in the TD group but not in the ASDgroup. This timing encompassed the initial three MSIeffects established in our a priori analysis. For the ASDgroup, a clear pattern of MSI only emerged at about300 ms (from �325 to 375 ms), a time frame that was notincluded in our a priori analysis. In the TD group, therewas also evidence for a second MSI effect at about 400 ms.Lowering the criterion to a less restrictive one of fivesuccessive significant points failed to reveal anyadditional clear patterns in the ASD group beyond thatat 300 ms, but did serve to solidify the effects at �120 and400 ms in the TD group, and also showed short-livedeffects that fell into the 250- to 300-ms range that wassignificant for this group in the ANOVA. There may evenbe a hint of an earlier effect at about 64 ms in the TDgroup that would bear replication in a follow-up study.

Discussion

The notion that fundamental differences in MSI are core

to ASD has been at the forefront of a number of

influential theories of the disorder since its original

description by Kanner in the 1940’s. In the present

study, the integrity of MSI in children with ASD was

8 Russo et al./Auditory–somatosensory integration in ASD INSAR

interrogated with scalp-recorded electrophysiology using

a passive auditory–somatosensory stimulation paradigm.

On the basis of our primary analyses, it appeared as

though both TD and ASD groups were integrating the

sensory inputs by 110 ms (as shown in main effects of

condition for the 100- to 125-ms and 154- to 178-ms

windows of analyses). However, during these time

frames, the difference between the multisensory and

summed responses were mainly driven by the TD group

(see Fig. 5, panels C and D), suggesting that we may not

have had the power to detect group interactions in this

time frame. For later time periods of analysis, at 175 and

255 ms, there was evidence for integration among the

TD but not the ASD participants. A fuller test of our

Figure 3. Multisensory vs. summed ERP waveforms for the typically developing children at frontal, centro-parietal, temporal andtemporal-parietal electrodes. [Color figure can be viewed online at wileyonlinelibrary.com.]

INSAR Russo et al./Auditory–somatosensory integration in ASD 9

data using an exploratory approach corroborated the

multisensory effects in the �100- to 200-ms range in the

TD group, whereas in the children with ASD, MSI was

only evident starting at about 310 ms. These findings

together suggest that among children with ASD, there is a

decrease in the extent of the automatic integration of

sensory inputs (i.e. MSI) compared to TD children.

We were surprised that the primary analysis failed to

reveal MSI before 100 ms in the TD group. In adults,

electrophysiological evidence of the integration of

zCzCF

T7

Multisensory

Sum

–100 0 100 200 300 400 500–5–4–3–2–1012345

time in milliseconds

amp

litu

de

in m

icro

volt

s

–100 0 100 200 300 400 500

time in milliseconds

Electrode locations

Fz

TP8TP7

CP2zPC1PC

T8

Figure 4. Multisensory vs. summed ERP waveforms for the children with an autism spectrum disorder at frontal, centro-parietal, temporaland temporal-parietal electrodes. [Color figure can be viewed online at wileyonlinelibrary.com.]

10 Russo et al./Auditory–somatosensory integration in ASD INSAR

auditory and somatosensory inputs is reliably seen

between 50 and 70 ms [Foxe et al., 2002; Murray et al.,

2005; Sperdin et al., 2009], and the one electrophysiolo-

gical study of AS integration in children, of a similar age

range to the ones tested here, found MSI between 60 and

80 ms [Brett-Green et al., 2008]. Based on our extensive

work on MSI in healthy adults [Foxe et al., 2000; Foxe &

Schroeder, 2005; Molholm et al., 2002, 2004; Molholm,

Martinez, Shpaner, & Foxe, 2007; Murray et al., 2005;

Taylor-Clarke et al., 2002], in which such effects are

consistently found for AS, AV, and visual-somatosen-

sory pairings, we have suggested that early stages of MSI

(circa. 40–90 ms) serve the function of simple coinci-

dence detection, and of response enhancement for near

threshold stimuli. Further, that this early tagging of

inputs is essential to the binding of information that is

processed though different sensory pathways with different

transmission times and numbers of relay stations [Foxe &

Schroeder, 2005; Schroeder & Foxe, 2005]. It is however

possible that these relatively transitory/short-lived effects

shift in timing over childhood, and that this rendered our

tests, on a sample that spanned a relatively broad range of

ages, insensitive to the early effect. We should note that

there was a hint of early MSI in the TD group in our

‘‘exploratory’’ analysis of the data, but since this was not

shown in our primary analysis and only appears in a

small set of electrode sites, this result needs to be

replicated in future work. Ongoing work in our labora-

tory on the developmental trajectory of AS MSI, in which

we will examine tighter age groups ranging from 2 to 3

Fronto-polarAntero-polar

Frontal

Fronto-central

Central

Centro-parietal

Parietal

Parieto-occiptal

Occipital

0.06

0.05

0.04

0.03

0.02

0.01

0.06

0.05

0.04

0.03

0.02

0.01

Fronto-polarAntero-polar

Frontal

Fronto-central

Central

Centro-parietal

Parietal

Parieto-occiptal

Occipital

Time (ms)

0 100 200 300 500400

Time (ms)

0 100 200 300 500400

Time (ms)

0 100 200 300 500400

Time (ms)

0 100 200 300 500400

A B

C D

Figure 5. Panels A and B, represent cluster plots comparing children TD and ASD children with respect to (A) auditory-alone responses,and (B) somatosensory-alone responses. Panels C and D, represent cluster plots of the difference between summed and multisensoryresponses for the (C) TD and (D) ASD children. Significance is represented here only if there were 10 consecutive time points that differedin the comparisons. The x-axis represents time (in ms). The y-axis represents individual electrode positions. Starting from the bottom ofthe graph, the electrodes are divided into sections from posterior to anterior scalp. Within a section, the electrodes are arranged from theright most lateral to the left most lateral sites. The sections are labeled based on the midline of the scalp (e.g. "central" includeselectrodes over temporal scalp).

INSAR Russo et al./Auditory–somatosensory integration in ASD 11

years, will be informative with regard to the question of

early MSI in children.

The current data indicate that children with ASD do

not automatically combine sensory inputs early in the

processing hierarchy to the same degree as TD indivi-

duals, as assessed by the absence of audio-somatosensory

MSIs in the 100- to 200-ms time frame. These findings are

in line with several theories of ASD, including the

temporal binding hypothesis [Brock, Brown, & Boucher,

2002] and the underconnectivity hypothesis [e.g. Just,

Cherkassky, Keller, Kana, & Minshew, 2007]. Both of

these theories begin from the premise that in ASD there is

a tendency to focus on the local rather than the global

aspects of information, a thesis that was originally

formulated by Frith and Happe [1994] and termed as

weak central coherence. According to the temporal

binding hypothesis, the local bias in ASD is related to a

failure to integrate information from different specialized

networks in the brain [Brock et al., 2002]. This failure to

integrate information has also received some support

from findings of local overconnectivity between prox-

imal areas as evidenced by findings of increased mini-

columnar density in postmortem studies [Casanova,

2007] and functional underconnectivity between more

distal areas of the brain as evidenced by fMRI [Just,

Cherkassy, Keller, & Minshew, 2004]. Findings of both

brain under- and over- connectivity [Mizuno, Villalobos,

Davies, Dahl, & Muller, 2006] provide a simple physio-

logical basis for proposed differences in multisensory

processing in ASD, which would be expected to require

distal brain connectivity [e.g. Falchier, Clavagnier,

Barone, & Kennedy, 2002; Rockland & Ojima, 2003;

Smiley & Falchier, 2009]. Evidence of functional under-

connectivity between frontal and visual areas in ASD has

been replicated during both rest [Cherkassky, Kana,

Keller, & Just, 2006] and a variety of experimental tasks

that include response inhibition [Kana, Keller, Minshew,

& Just, 2007], working memory [Koshino et al., 2005],

and executive function [Just et al., 2007]. In addition to

findings of underconnectivity, functional overconnectivity

between the thalamus, a key relay area in sensory

processing, and certain frontal regions that include the

left insula and right postcentral and middle frontal gyri

has also been noted [Mizuno et al., 2006]. Pathophysio-

logical findings from postmortem tissue of the brains of

persons with ASD are also consistent with atypical

connectivity. There are reliable differences in relation to

the TD brain, with findings of a smaller minicolumnar

width, a greater number of minicolumns, and increased

neuronal density in various cortical areas including the

frontal and temporal lobes [Casanova, 2004, 2006].

Together these findings indicate possible functional

and anatomical differences that may contribute to

problems with MSI, which requires rapid and accurate

communication between subcortical and cortical areas.

Crucially, evidence from anatomical studies have shown

that there are direct long range connections between the

various sensory cortical areas [Falchier et al., 2002;

Rockland & Ojima, 2003] and it may be the disruption

of this form of connectivity that leads to impairment of

early MSI in ASD.

A major question raised is how to reconcile our

findings with the few behavioral studies on multisensory

processing in autism that suggest that the integration of

basic stimuli is typical in ASD [e.g. Bebko et al., 2006; van

der Smagt et al., 2007], whereas it is the integration of

more complex stimuli, such as language that is impaired

[e.g. Mongillo et al., 2008; Smith & Bennetto, 2007]. All

these studies have involved auditory–visual stimuli, so

one possibility is that the difference lies in the sensory

modalities involved. Another possibility is that while

individuals with autism are able to benefit from multi-

sensory inputs, especially for simpler stimulus and task

configurations, the underlying brain processes never-

theless differ.

It is also possible that for persons with ASD, actively

attending the stimuli is necessary for integration to

occur. That is, that MSI processes that occur in the

absence of directed attention in TD individuals require

attention to be achieved in this population. It will be

interesting to assess if making the stimuli task-relevant

serves to ‘‘normalize’’ the neural responses, or if typical

behavioral facilitation is seen in the face of different

neural processing.

Another potential contribution to differences in the

integration of the auditory and somatosensory stimuli

might come from the small but reliable differences in

the magnitude of the unisensory responses that were

observed between the groups. Although for both groups

there were robust unisensory responses which were in the

main highly similar between the groups, these appeared

slightly smaller in the ASD group. Just how this would

impact subsequent processing is difficult to say, but we

doubt that the whole-scale differences that we see in

multisensory processing would be mitigated by ‘‘normal-

izing’’ the unisensory response. One possibility is that in

ASD multisensory processing is more (or less) mature

than for TD individuals. Only by collecting considerably

more data points, however, will we be able to paint the

developmental trajectory of auditory–somatosensory MSI

and address such questions. A number of studies have

specifically examined how the ‘‘effectiveness’’ of uni-

sensory stimulation impacts MSI. For studies that record

single cell activity, effectiveness is assessed by the number

of spikes per unit of time. Such studies have demon-

strated that the more ineffective unisensory stimuli are at

eliciting a neuronal response when presented alone, the

greater the multisensory enhancement tends to be when

they are presented together [e.g. Stein & Stanford, 2008;

Stein, Stanford, Ramachandran, Perraullt, & Rowland,

12 Russo et al./Auditory–somatosensory integration in ASD INSAR

2009]. This is referred to as the ‘‘principle of inverse

effectiveness’’. According to this principle, one could in

fact make the case that the unisensory stimuli used here

were less effective for the ASD group and therefore that

their MSIs should in fact be stronger. The relationship

between the robustness of the unisensory response and

the magnitude of the multisensory response has not been

so straightforward when tested using other methods. For

example, some fMRI studies find support for inverse

effectiveness [e.g. Werner & Noppeney, 2009], whereas

others have actually shown quite the opposite, with

enhanced integration seen as the effectiveness of uni-

sensory stimulation increased [Kim & James, 2010].

Behavioral data from our laboratory reveal that for more

complex linguistic stimuli, the largest audiovisual multi-

sensory effects are present for stimuli that are at

intermediary stages of effectiveness, suggesting that there

is not necessarily a linear relationship between the

effectiveness of unisensory stimuli and MSI [e.g. Ma,

Zhou, Ross, Foxe, & Parra, 2009; Ross et al., 2007]. Clearly

much work remains to be done to understand how the

differences in unisensory processing relate to differences

in multisensory processing.

Conclusions

The findings from this study provide preliminary con-

firmatory evidence for anecdotal and clinical reports of a

difficulty in integrating information from multiple

sensory modalities among persons with ASD. Much work,

of course, remains to be done to fully understand not

only the differences in how multisensory inputs are

integrated but also how these differences affect higher-

order processing, and the impact that a lack of early

integration has on the pathogenesis of persons with

ASDs. In addition, it will be important to determine

whether there are specific phenotypic characteristics of

autism (e.g. sensory overresponsiveness) that are more

strongly associated with differences in multisensory

processing than others, as well as whether MSI deficits

are more strongly associated with specific subtypes of the

autism spectrum than others (e.g. autism vs. Asperger’s

syndrome).

Acknowledgments

Initial support for this work came from a pilot grant from

Cure Autism Now (J.J.F.). Additional support came from

the US National Institute of Mental Health (MH 085322

to S.M. and J.J.F.) and The Wallace Research Foundation

(S.M. and J.J.F.). NR received additional support from a

Post Doctoral Research Grant from the Fondation du

Quebec de Recherche sur la Societe et la Culture and from

the Autism Research Training Program through the

Canadian Institute of Health Research.

References

American Psychiatric Association. (2000). Diagnostic and statis-

tical manual of mental disorders: DSM-IV-TR. Washington,

DC: American Psychiatric Association.

Ayers, J.A. (1994). Sensory integration and the child. Los

Angeles, CA: Western Psychological Services.

Bebko, J.M., Weiss, J.A., Demark, J.L., & Gomez, P. (2006).

Discrimination of temporal synchrony in intermodal events

by children with autism and children with developmental

disabilities without autism. Journal of Child Psychology and

Psychiatry and Allied Disciplines, 47, 88–98.

Berman, A.L. (1961). Overlap of somatic and auditory cortical

response fields in anterior ectosylvian gyrus of cat. Journal of

Neurophysiology, 24, 595–607.

Bertone, A., Mottron, L., Jelenic, P., & Faubert, J. (2003). Motion

perception in autism: A ‘‘complex’’ issue. Journal of Cogni-

tive Neurosciences, 15, 218–225.

Bertone, A., Mottron, L., Jelenic, P., &, Faubert, J. (2005).

Enhanced and diminished visuo-spatial information proces-

sing in autism depends on stimulus complexity. Brain, 128,

2430–2441.

Brett-Green, B.A., Miller, L.J., Gavin, W.J., & Davies, P.L. (2008).

Multisensory integration in children: A preliminary ERP

study. Brain Research, 1242, 283–290.

Bonnel, A., Mottron, L., Peretz, I., Trudel, M., Gallun, E., &

Bonnel, A.M. (2003). Enhanced pitch sensitivity in indivi-

duals with autism: A signal detection analysis. Journal of

Cognitive Neuroscience, 15, 226–235.

Brock, J., Brown, C.C., & Boucher, J. (2002). The temporal

binding deficit hypothesis of autism. Development and

Psychopathology, 14, 209–224.

Carriere, B.N., Royal, D.W., Perrault, T.J., Morrison, S.P.,

Vaughan, J.W., et al. (2007). Visual deprivation alters the

development of cortical multisensory integration. Journal of

Neurophysiology, 98, 2858–2867.

Casanova, M.F. (2004). White matter volume increase and

minicolumns in autism. Annals of Neurology, 56, 453.

Casanova, M.F. (2006). Neuropathological and genetic findings

in autism: The significance of a putative minicolumnopathy.

Neuroscientist, 12, 435–441.

Casanova, M.F. (2007). The neuropathology of autism. Brain

Pathology, 17, 422–433.

Ceponiene, R., Rinne, T., & Naatanen, R. (2002). Maturation of

cortical sound processing as indexed by event-related poten-

tials. Clinical Neurophysiology, 113, 870–882.

Cesaroni, L., & Garber, M. (1991). Exploring the experience of

autism through firsthand accounts. Journal of Autism and

Developmental Disorders, 21, 303–313.

Cherkassky, V.L., Kana, R.K., Keller, T.A., & Just, M.A. (2006).

Functional connectivity in a baseline resting-state network in

autism. Neuroreport, 17, 1687–1690.

Crane, L., Goddard, L., & Pring, L. (2009). Sensory processing in

adults with autism spectrum disorders. Autism, 13, 215–228.

de Gelder, B., Vroomen, J., & Van der Heide, L. (1991). Face

recognition and lip-reading in autism. European Journal of

Cognitive Psychology, 3, 69–86.

Driver, J. (1996). Enhancement of selective listening by illusory

mislocation of speech sounds due to lip-reading. Nature, 381,

66–68.

INSAR Russo et al./Auditory–somatosensory integration in ASD 13

Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2002).

Anatomical evidence of multimodal integration in primate

striate cortex. Journal of Neuroscience, 22, 5749–5759.

Foxe, J.J., & Molholm, S. (2009). Ten years at the multisensory

forum: Musings on the evolution of a field. Brain Topography,

32, 149–154.

Foxe, J.J., & Schroeder, C.E. (2005). The case for feedforward

multisensory convergence during early cortical processing.

Neuroreport, 16, 419–423.

Foxe, J.J., Morocz, I.A., Murray, M.M., Higgins, B.A., Javitt, D.C.,

& Schoreder, C.E. (2000). Multisensory auditory-somatosen-

sory interactions in early cortical processing revealed by high-

density electrical mapping. Brain Research, Cognitive Brain

Research, 10, 77–83.

Foxe, J.J., Wyliw, G.R., Martinez, A., Schoeder, C.E., Javitt, D.C.,

et al. (2002). Autidory-somatosensory multisensory proces-

sing in auditory association cortex: An fMRI study. Journal of

Neurophyisology, 88, 540–543.

Frens, M.A., & Van Opstal, A.J. (1995). A quantitative study of

auditory-evoked saccadic eye movements in two dimensions.

Experimental Brain Resarch, 107, 103–117.

Frith, U., & Happe, F. (1994). Autism: Beyond ‘‘theory of mind’’.

Cognition, 50, 115–132.

Giard, M.H., & Peronnet, F. (1999). Auditory-visual interaction

during multimodal object recognition in humans: A beha-

vioral and electrophysiological study. Journal of Cognitive

Neurosciences, 11, 473–490.

Gori, M., Del Viva, M., Sandini, G., & Burr, D.C. (2008). Young

children do not integrate visual and haptic form information.

Current Biology, 18, 694–698.

Grandin, T. (1992). An inside view of autism. In: Schopler, E., &

Mesiboc, G.B., editors. High-functioning individuals with

autism. New York: Plenum Press, pp 105–126.

Guthrie, D., & Buchwald, J.S. (1991). Significance testing of

difference potentials. Psychophysiology, 28, 240–244.

Hermelin, B., & O’Connor, N. (1970). Psychological experiments

with autistic children. Oxford, England: Pergamon Press.

Hotting, K., Rosler, F., & Roder, B. (2004). Altered auditory-tactile

interactions in congenitally blind humans: An event-related

potential study. Experimental Brain Research, 159, 370–381.

Just, M.A., Cherkassy, V.L., Keller, T.A., & Minshew, N. (2004).

Cortical activation and synchronization during sentence

comprehension in high-functioning autism: Evidence of

underconnectivity. Brain, 127, 1811–1821.

Just, M., Cherkassky, V.L., Keller, T.A., Kana, R.K., & Minshew, N.J.

(2007). Functional and anatomical underconnectivity in

autism: Evidence from an fMRI study of an executive function

task and corpus collosum morphometry. Cerebral Cortex, 27,

951–961.

Kana, R.K., Keller, T.A., Minshew, N.J., & Just, M.A. (2007).

Inhibitory control in high-functioning autism: Decreased

activation and underconnectivity in inhibition networks.

Biological Psychiatry, 62, 198–206.

Kim, S., & James, T.W. (2010). Enhanced effectiveness in visuo-

haptic object-selective brain regions with increasing stimulus

salience. Human Brain Mapping, 31, 678–693.

Klin, A., Sparrow, S.S., de Bildt, A., Cichetti, D.V., Cohen, D.J., &

Volkmar, F.R. (1999). A normed study of face recognition in

autism and related disorders. Journal of Autism and Devel-

opmental Disorders, 29, 499–508.

Koshino, H., Carpenter, P.A., Minshew, N.J., Cherkassky, V.L.,

Keller, T.A., & Just, M.A. (2005). Functional connectivity in an

fMRI working memory task in high-functioning autism.

Neuroimage, 24, 810–821.

Lane, A.E., Young, R.L., Baker, A.E., & Angley, M.T. (2010).

Sensory processing subtypes in autism: Association with

adaptive behavior. Journal of Autism and Developmental

Disorders, 40, 112–122.

Lutkenhoner, B., Lammertmann, C., Simoes, C., & Hari, R.

(2002). Magnetoencephalographic correlates of audiotactile

interaction. Neuroimage, 15, 509–522.

Lewkowicz, D.J. (2002). Heterogeneity and heterochrony in the

development of intersensory perception. Brain Research:

Cognitive Brain Research, 14, 41–63.

Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic

interview-revised: A revised version of a diagnostic interview

for caregivers of individuals with possible pervasive develop-

mental disorders. Journal of Autism and Developmental

Disorders, 24, 659–685.

Lord, C., Rutter, M., DiLavore, P.C., & Risi, S. (1999). Autism

diagnostic observation schedule. Los Angeles, CA: Western

Psychological Services.

Lovaas, O.I., & Schreibman, L. (1971). Stimulus overselectivity of

autistic children in a two stimulus situation. Behaviour

Research and Therapy, 9, 305–310.

Lovaas, O.I., Schreibman, L., Koegel, R., & Rehm, R. (1971).

Selective responding by autistic children to multiple sensory

input. Journal of Abnormal Psychology, 77, 211–222.

Ma, W.H., Zhou, X., Ross, L.A., Foxe, J.J., & Parra, L.C. (2009). Lip-

reading aids word recognition most in moderate noise:

A Bayesian explanation using high-dimensional feature space.

PLoS One 4.e4638. DOI: 10.1371/journal.pone.0004638.

Magnee, M.J., de Gelder, B., van Engeland, H., & Kemner, C.

(2008). Audiovisual speech integration in pervasive develop-

mental disorder: Evidence from event related potentials.

Journal of Child Psychology and Psychiatry, 49, 995–1000.

Magnee, M.J., Oranje, B., van Engeland, H., Kahn, R.S., &

Kemner, C. (2009) Cross-sensory gating in schizophrenia and

autism spectrum disorder: EEF evidence for impaired brain

connectivity? Neuropsychologia, 7, 1728–1732.

Mizuno, A., Villalobos, M.E., Davies, M.M., Dahl, B.C., &

Muller, R.A. (2006) Partially enhanced thalamocortical func-

tional connectivity in autism. Brain Research, 1104, 160–174.

Molholm, S., Ritter, W., Murray, M.M., Javitt, D.C.,

Schroeder, C.E., & Foxe, J.J. (2002). Multisensory auditory-

visual interactions during early sensory processing in

humans: A high-density electrical mapping study. Brain

Research: Cognitive Brain Research, 14, 115–128.

Molholm, S., Ritter, W., Javitt, D.C., & Foxe, J.J. (2004). Multi-

sensory visual-auditory object recognition in humans: A high-

density electrical mapping study. Cerebral Cortex, 14, 452–465.

Molholm, S., Martinez, A., Shpaner, M., & Foxe, J.J. (2007).

Object-based attention is multisensory: Co-activation of an

object’s representations in ignored sensory modalities.

European Journal of Neurosciences, 26, 499–509.

Mongillo, E.A., Irwin, J.R., Whalen, D.H., Klaiman, C., Carter, A.S.,

& Schultz, R.T. (2008). Audiovisual processing in children with

14 Russo et al./Auditory–somatosensory integration in ASD INSAR

and without autism spectrum disorders. Journal of Autism and

Developmental Disorders, 38, 1349–1358.

Murray, M.M., Molholm, S., Michel, C.M., Heslenfeld, D.J.,

Ritter, W., et al. (2005). Grabbing your ear: Rapid auditory-

somatosensory multisensory interactions in low-level sensory

cortices are not constrained by stimulus alignment. Cerebral

Cortex, 15, 963–974.

Nardini, M., Jones, P., Bedford, R., & Braddick, O. (2008).

Development of cue integration in human navigation.

Current Biology, 18, 689–693.

O’Neill, M., & Jones, R.S.P. (1997). Sensory-perceptual abnorm-

alities in autism: A case for more research? Journal of Autism

and Developmental Disorders, 27, 283–293.

O’Riordan, M. (2004). Superior visual search in autism. Autism,

8, 229–248.

O’Riordan, M., & Plaisted, K. (2001). Enhanced discrimination in

autism. Quarterly Journal of Experimental Psychology, 54, 961–979.

O’Riordan, M., & Passetti, F. (2006). Discrimination in autism

within different modalities. Journal of Autism and Develop-

mental Disorders, 36, 665–675.

Ozonoff, S., & Strayer, D.L. (1997). Inhibitory function in non-

retarded children with autism. Journal of Autism and

Developmental Disorders, 27, 59–77.

Perrott, D.R., Saberi, K., Brown, K., & Strybel, T.Z. (1990).

Auditory psychomotor coordination and visual search per-

formance. Perceptual Psychophysics, 48, 214–226.

Putzar, L., Goerendt, I., Lange, K., Rosler, F., & Roder, B. (2007).

Early visual deprivation impairs multisensory interactions in

humans. Nature Neuroscience, 10, 1243–1245.

Rockland, K.S., & Ojima, H. (2003). Multisensory convergence in

calcarine visual areas in macaque monkey. International

Journal of Psychophysiology, 50, 19–26.

Ross, L.A., Saint-Amour, D., Leavitt, V.M., Molholm, S., Javitt, D.C.,

& Foxe, J.J. (2007). Impaired multisensory processing in

schizophrenia: Deficits in the visual enhancement of speech

comprehension under noisy environmental conditions. Schi-

zophrenia Research, 97, 173–183.

Schoen, S.A., Miller, L.J., Brett-Green, B.A., & Nielsen, D.M.

(2009). Physiological and behavioral differences in sensory

processing: A comparison of children with autism spectrum

disorder and sensory modulation disorder. Frontiers in

Integrative Neurosciences, 29 [Electronic version].

Schroeder, C.E., & Foxe, J.J. (2005). Multisensory contributions

to low-level ‘‘unisensory’’ processing. Current Opinions in

Neurobiology, 15, 454–458.

Shams, L., Kamitani, Y., & Shimojo, S. (2000). Illusions: What

you see is what you hear. Nature, 14, 788.

Shams, L., Kamitani, Y., & Shimojo, S. (2002). Visual illusion

induced by sound. Brain Research -Cognitive Brain Research,

14, 147–152.

Smith, E.G., & Bennetto, L. (2007). Audiovisual speech integra-

tion and lipreading in autism. Journal of Child Psychiatry

and Psychology, 48, 813–821.

Smiley, J.F., & Falchier, A. (2009). Multisensory connections of

monkey auditory cerebral cortex. Hearing Research, 258,

37–46.

Spence, C., & Driver, J. (1997). On measuring selective attention

to an expected sensory modality. Perceptual Psychophysics,

59, 389–403.

Sperdin, H.F., Cappe, C., Foxe, J.J., & Murray, M.M. (2009). Early

low-level auditory-somatosensory multisensory interaction

impact reaction time speed. Frontiers in Integrative Neuros-

ciences, 3 [Epub March 11].

Stein, B.E., & Meredith, M.A. (1990). Multisensory integration.

Neural and behavioral solutions for dealing with stimuli from

different sensory modalities. Annuals of the New York

Academy of Science, 608, 51–65; discussion 65–70.

Stein, B.E., & Stanford, T.R. (2008). Multisesnory integration:

Current issues from the perspective of the single neuron.

Nature Neuroscience Reviews, 9, 255–266.

Stein, B.E., Stanford, T.R., Ramachandran, R., Perrault, Jr., T.J., &

Rowland, B.A. (2009). Challenges in quantifying multisen-

sory integration: Alternative criteria, models and inverse

effectiveness. Experimental Brain Research, 198, 113–126.

Stroh, G., & Buick, D. (1964). Perceptual development and childhood

psychosis. British Journal of Medical Psychology, 37, 291–299.

Tager-Flusberg, H. (1996). Brief report: Current theory and

research on language and communication in autism. Journal

of Autism and Developmental Disorders, 26, 169–172.

Tager-Flusberg, H., & Sullivan, K. (1994). A second look at

second-order belief attribution in autism. Journal of Autism

Developmental Disorder, 24, 577–586.

Talsma, D., & Woldorff, M.G. (2005). Selective attention and

multisensory integration: Multiple phases of effects on the evoked

brain activity. Journal of Cognitive Neuroscience, 17, 1098–1114.

Taylor-Clarke, M., Kennett, S., & Haggard, P. (2002). Vision

modulates somatosensory cortical processing. Current Biology,

12, 233–236.

Teder-Salejarvi, W., McDonald, J.J., Di Russo, F., & Hillyard, S.A.

(2002). An analysis of audio-visual crossmodal integration by

means of event-related potential (ERP) recordings. Brain

Research Cognitive Brain Research, 14, 106–114.

Tomchek, S.D. (2007). Sensory processing in children with and

without autism: A comparative study using the short sensory

profile. American Journal of Occupational Therapy, 61,

190–200.

van der Smagt, M.J., van Engeland, H., & Kemner, C. (2007). Can

you see what is not there? Low-level auditory-visual integra-

tion in autism spectrum disorder. Journal of Autism and

Developmental Disorders, 37, 2014–2019.

Vroomen, J., & Stekelenburg, J.J. (2010). Visual anticipatory

information modulates multisensory interaction of artificial

audiovisual stimuli. Journal of Cognitive Neurosciences, 22,

1583–1596.

Wallace, M.T., Perrault, Jr., T.J., Hairston, W.D., & Stein, B.E.

(2004). Visual experience is necessary for the development of

multisensory integration. Journal of Neurosciences, 24,

9580–9584.

Wechsler, D. (1999). Wechsler abbreviated scale of intelligence.

San Antonio, TX: Psychological Corporation.

Werner, S., & Noppeney, U. (2010). Superadditive responses in

superior temporal sulcus predict audiovisual benefits in

object categorization. Cerebral Cortex, 20, 1829–1842.

Williams, D. (1994). Somebody somewhere. London: Doubleday.

Zahn, J.R., Abel, L.A., & Dell’Osso, L.F. (1978). Audio-ocular

response characteristics. Sensory Processes, 2, 32–37.

INSAR Russo et al./Auditory–somatosensory integration in ASD 15