Copyright 1994 by the American Psychological Association ......Saccades Versus Manual Responses Howard C. Hughes, Patricia A. Reuter-Lorenz, George Nozawa, and Robert Fendrich Reaction

Journal of Experimental Psychology:Human Perception and Performance1994, Vol. 20, No. 1, 131-153

Copyright 1994 by the American Psychological Association, Inc.0096-1523/94/$3.00

Visual-Auditory Interactions in Sensorimotor Processing:Saccades Versus Manual Responses

Howard C. Hughes, Patricia A. Reuter-Lorenz, George Nozawa, and Robert Fendrich

Reaction times (RTs) to bimodal (visual and auditory) stimuli were examined using 3 differentresponse systems: saccades, directed manual responses, and simple manual responses. The ob-served levels of intersensory facilitation exceeded race model predictions and therefore supportsummation (coactivation) models of bimodal processing. However, response-dependent differencessuggest that the processing of bimodal targets also depends on the relevant sensorimotor pathwaysand requirements of the task. Coactivation of response mechanisms might account for the effectsfound using simple RTs. The results for saccades are consistent with known patterns of auditory-visual convergence in the oculomotor system.

Recently there has been renewed interest in understand-ing the manner in which redundant stimuli affect sensori-motor performance (e.g., Ashby & Townsend, 1986;Miller, 1982; Mordkoff & Yantis, 1991; Stein, Meredith,Honeycutt, & McDabe, 1989; Townsend & Ashby, 1983).In many cases, performance (usually reaction time, RT) tosingle-stimulus presentations is compared with perfor-mance under conditions of dual-stimulus presentations.The frequent finding is that RTs to dual-stimulus presenta-tions are faster than RTs to either stimulus presented alone(Miller, 1982, 1986; Mordkoff & Yantis, 1991; Raab,1962; van der Heijden, La Heij, & Boer, 1983). This is re-ferred to as a redundant-signals effect.

Central to an analysis of the redundant-signals effect is thequestion of whether the facilitation produced by redundanttargets is sufficiently robust to rule out the possibility thatresponses to redundant targets are simply triggered by which-ever target is detected first (equivalent to the operation of alogical OR gate). Because the detection times associated witheach modality are considered to be random variables, somereduction in RT is expected in a system that applied such anOR operation to the detection times in otherwise independentsensory channels, an effect known as probability summation(e.g., Miller, 1982; Raab, 1962; Townsend & Ashby, 1983).Probability summation assumes that a separate decision pro-cess accumulates information on each afferent channel andthat the first channel to detect the target generates the re-sponse. For this reason, such models of redundant-signalsprocessing are often called race models. Miller (1982, 1986)pointed out that the magnitude of the redundant-targets effectshould be greater than that attributable to race models if the

Howard C. Hughes and George Nozawa, Department of Psy-chology, Dartmouth College; Patricia A. Reuter-Lorenz, Depart-ment of Psychology, University of Michigan; Robert Fendrich,Center for Neurobiology, University of California, Davis.

This research was supported by Air Force Office of ScientificResearch Grant 89-0437.

Correspondence concerning this article should be addressed toHoward C. Hughes, Department of Psychology, Dartmouth Col-lege, Hanover, New Hampshire 03768.

activities of several parallel afferent channels were pooledprior to a single decision process. Combined activationwould produce RTs that are faster than those predicted byrace models (Nozawa, 1989). Thus, facilitation beyond prob-ability summation may be indicative of neural summation(coactivation) somewhere in the processing system. Thepooling of information could occur at the level of a sensorydecision (Blake, Martens, Garrett, & Westendorf, 1980; Fi-dell, 1970; Founder & Eriksen, 1990; Luce & Green, 1972;Rose, Blake, & Halpern, 1988; Wandell & Luce, 1978; We-stendorf & Blake, 1988) or at the level of response selectionor execution (e.g., Diederich & Colonius, 1987; Eriksen &Schultz, 1977; Fournier & Eriksen, 1990; Miller, 1982,1986).

Although the sensory channels in many redundant-targetsexperiments may reasonably be regarded as being organizedin parallel, the very existence of a redundant-targets effectmeans that information about the individual targets must con-verge at some point in sensorimotor processing. Recent elec-trophysiological studies have revealed a specific site ofauditory-visual convergence within the oculomotor systemsof cats and monkeys (Jay & Sparks, 1987, 1990; Meredith& Stein, 1987; Peck, 1987). These findings motivated thepresent analysis of the redundant-targets effect on the latencyof saccadic eye movements in humans.

Multimodal Convergence in the Saccadic ControlSystem

Recent electrophysiological studies have revealed a neuralmechanism that appears to enable multimodal control of sac-cades: Individual neurons within the deeper layers of thesuperior colliculus receive convergent visual and acousticinputs (e.g., Jay & Sparks, 1987; Meredith & Stein, 1987;Peck, 1987). The coordinates of the visual and auditory re-ceptive fields are usually in spatiotopic register, and spatiallyaligned bimodal inputs often elicit unit discharges that aregreater than responses evoked from either modality alone(Meredith & Stein, 1987). The responses of single cells tobimodal stimuli are sometimes close to the sum of the uni-

131

132 HUGHES, REUTER-LORENZ, NOZAWA, AND FENDRICH

modal responses (Meredith & Stein, 1987), a combinationrule referred to as superposition of impulse counting (Cox,1962; Fatt & Katz, 1952). Often, however, bimodal sum-mation effects are much greater than the superposition of thetwo unimodal responses (Peck, 1987; Stein, Meredith, &Wallace, 1993). Thus, the behavioral effects produced bybimodal stimuli might reflect an overadditive combination ofeach of the unimodal activities (e.g., Stein et al., 1989). Thisconvergence of visual and auditory inputs onto neurons in thesuperior colliculus suggests that bimodal targets might showparticularly robust facilitatory interactions in controlling theinitiation of saccades. In the present experiments, we exam-ined the degree to which such bimodal targets facilitate thelatency of saccadic eye movements relative to the latenciesassociated with either visual or auditory stimuli presentedalone. The experiments thus addressed the issue of inter-sensory facilitation in the saccadic control system and com-pared the effects with those obtained with manual responses.

Evaluating Probability Summation

An estimate of the maximal degree of intersensory facili-tation attributable to race models is derived from the workof Miller (1982) and is based on the inequality

*, =s r lS, &S2) < + P(T2 < /IS 2) , (1)

where T: and T2 are the random times associated with pro-cessing of information in Channels 1 and 2, respectively, andTmin is the minimum of T\ and T2 (Tmin = min[ri,r2]). In-equality 1 is known as Boole's inequality in probabilitytheory (Dudewicz, 1976).

Because Inequality 1 indicates an upper limit on perfor-mance attributable to race models, it can conveniently beregarded as the upper boundary of probability summation(race models). The left side of Inequality 1 represents thecumulative distribution function (CDF) of the redundant-targets condition (assuming the race model), and the otherterms represent the CDFs of the single-target conditions. Ig-noring the motor-related components of RT (i.e., the basetime), probability summation states that the CDF of theredundant-targets condition can be expressed as the sum ofthe two CDFs from the single-target conditions minus thejoint CDF:

< r lS, &S2) = P(T{ < t\Sl)

- P(Tl == t and

P(T2 < r lS 2 )

T2 < t\ 5, & S2). (2)

The joint CDF, P(Tl < t and T2 ̂ t I Sl & S2), can be writtenas a multiplication of the two single-target CDFs if (a) 7"i andT2 are stochastically independent and (b) we assume contextindependence and selective influence (i.e., the processingtimes on Channel 1 do not vary with the activity on Channel2; Colonius, 1990; Townsend & Ashby, 1983). Thus, theexpression

< t\ S, & S2) = P(T, < 1 1 S, ) + P(T2 < t\ S2)

(3)

If there is negative dependence between the random timesTI and T2, the joint CDF is less than the multiplication of twomarginal CDFs. If there is positive dependence between tworandom times, the joint CDF is greater than the multiplicationof two marginal CDFs. Regardless of the dependent structurebetween 7", and T2, Inequality 1 holds because P(Tl < t andT2 < t I S, & S2) — 0. Thus, violations of Inequality 1 indicatea redundant-targets effect that exceeds the upper limit ofprobability summation (Miller, 1986; Ulrich & Giray, 1986).We can evaluate the applicability of probability summationto obtained redundant-targets effects by comparing the CDFobtained with bimodal targets, P(RT =£ t I S: & S2), with theCDFs obtained for unimodal targets, P(RT< t I Sl)andP(RT< rlS2); that is,

P(RT < r l Sj & S2) - {P(RT < t\ 5,) + P(RT < t\S2)}. (4)

Notice that this comparison can only be evaluated over val-ues of / such that P(RT < t 15,) + P(RT < t 152) =£ 1.

Figure 1 illustrates the relationship between severalboundary conditions relevant to RT performance in theredundant-targets paradigm. These data were obtainedfrom Observer G.T. in Experiment 1. The thick solid lineillustrates the predicted CDF of saccadic RTs based on theindependent race model (independent race prediction,Equation 3). The thin solid line represents the CDF for themaximum level of performance attributable to any racemodel (i.e., the upper limit of probability summation,which is the right-hand side of the race inequality, Inequal-ity 1). The maximum of the two marginal CDFs (Frechet,1951) is indicated by the heavy dashed line. The Frechetboundary represents the slowest level of performance pos-sible in any model in which responses are determined bythe minimum of the completion times (min [T:,T2]) for

Illustration of Performance boundaries of Race Models

1.0

Visual TargetAuditory Target

Prob. Sum. Limit

Independ. race predFrechet boundary

0.0250

Time (msec)

350

represents the independent race prediction.

Figure 1. Three boundary conditions for parallel processing sys-tems. See text for details. Prob = probability; RT = response time;Prob. Sum. Limit = upper limit of probability summation; Inde-pend. race pred = independent race prediction.

VISUAL-AUDITORY INTERACTIONS 133

two parallel processes (Colonius, 1990). The region be-tween the Frechet boundary and the probability summationlimit represents levels of performance that could be pro-duced by race models in which the assumptions of stochas-tic independence and selective influence are relaxed (seeNozawa, 1989; Ulrich & Giray, 1986). As indicated previ-ously, given the assumptions of context independence andselective influence, performance that is faster than theprobability summation limit cannot be accounted for byrace models and therefore is interpreted herein as evidenceof neural summation.1 One type of neural summation (thesuperposition model, see Cox, 1962; Fatt & Katz, 1952;Schwarz, 1989) produces the linear sum of the channel in-puts. Nozawa (1989) has provided a mathematical proofthat the superposition of two neural counting processeswill produce faster RTs than the independent race model.To the extent that neural summation is an overadditivecombination of channel activities (e.g., Stein et al., 1993),predicted performance would be even faster than the su-perposition model. The degree to which observedredundant-target effects exceed the upper limit of probabil-ity summation could be related to the operator that com-bines the auditory and visual information: A multiplicativeoperator will produce greater violations of the race in-equality (Inequality 1) than will superposition.

Intersensory Facilitation as a Function of TaskRequirements

Because visual-auditory convergence in the superior col-liculus is a potentially unique architecture among sensori-motor systems, it seemed desirable to compare the magnitudeof bimodal summation observed for saccades with alternativesensorimotor tasks. Thus, we also investigated intersensoryfacilitation using two types of manual responses: directedand simple manual responses. Directed manual responsesrequired the subjects to deflect a joystick in the direction ofthe target as quickly as possible. These responses are similarto saccades in that target position must be encoded before acorrect response can be executed. The second type of manualresponse, a simple RT task, does not depend on localizationof the target.

General Method

Apparatus

The basic apparatus consisted of an array of three stimuluspanels aligned on an arc with a radius of 114 cm. Each stimu-lus panel contained a red and green light-emitting diode(LED) and a small (4 cm) speaker. Two panels positioned onthe horizontal meridian of the left and right visual fieldsprovided the targets. The green LED of the central panelserved as a fixation point. Flashes of the peripheral red LEDs(100 ms in duration) served as the visual targets. Acousticsignals consisted of bursts of white noise (100 ms in duration)delivered through the speakers. Both the amplitude of theacoustic targets and the luminance of the visual targets were

controlled by 12-bit digital-to-analog (D/A) converters.Acoustic warning signals (2000 Hz for 300 ms) presentedthrough a centrally located piezoelectric oscillator precededthe delivery of imperative targets by 1,000 ms. In order toprevent echoes that might impair sound localization, the en-tire apparatus was located in a large (1.54 m X 1.54 m X 0.9m) enclosure that was lined with a sound-absorbing foammaterial (Sonex). The apparatus was located in an isolated,completely darkened room.

Response Recording

Eye position was monitored using an infrared scleral re-flection device (Narco Biosystems Model 200 eye tracker).The output of the eye tracker was sampled using a 12-bitanalog-to-digital A/D) converter at 200 Hz, and the digitizedrecords were stored for subsequent off-line data analysis. Inaddition to measuring saccades, we included sessions inwhich the observers were required to generate directed andsimple manual responses under similar conditions. Directedmanual responses were recorded using an inductive-coil joy-stick. The subjects were simply required to push the joystickin the direction of the eccentric target as quickly as possible.The joystick position was sampled using A/D converters(200-Hz sampling rate), and the direction and latency of themovements were analyzed in the same way as saccades. Inthe simple RT condition, subjects simply depressed a mi-croswitch in response to the target onset. The microswitchwas also sampled at 200 Hz.

Response Detection

Both saccades and directed manual responses (joystickmovements) were detected using a velocity criterion. Thus,onset of a response was considered to have occurred whenthe velocity of the movement exceeded a criterion value.Although the detection of both saccades and joystick re-sponses was automated, all records were monitored by anoperator to ensure that misses or false positives were notincluded in the data set. In general, the velocity criterion forsaccades and joystick responses was set to approximately50°/s. However, the criterion was occasionally lowered tooptimize the performance of the velocity-based algorithm.This was especially true in the case of joystick responses,which tended to have a slower response onsets and lowerpeak velocities than saccades. Lowering the criterion had theeffect of detecting response onsets earlier in the waveform.

1 Whereas many authors have adopted the term coactivation torefer to violations of Inequality 1, we refer to such violations asneural summation in this article because of the parallel relationshipbetween the present results and observations of auditory-visualconvergence in the electrophysiological literature. In addition, theset of models that encompass the term coactivation is perhapslarger (e.g., Miller, 1991; Mordkoff & Yantis, 1991) than whatmight be reasonably included in the type of summation mechanismwe consider here. Thus, the term neural summation seems prefer-able in the present context, because it implies specific patterns ofconvergence in neural hardware.


Data Analysis

As previously indicated, violations of the race inequality(Inequality 1) were the principle measure of interest. Recentsimulations by Miller and Lopes (1991) have shown that"fast guesses" can bias the results against observing suchviolations. Although the unconnected (more conservative)data provide clear and robust violations of Inequality 1, wedid correct for fast guesses using a procedure derived fromthe correction suggested by Miller and Lopes. The details ofthe correction procedure are described in the Appendix.

The correction described in the Appendix serves to in-crease the magnitude of the race inequality violations, al-though the actual difference between the corrected and un-corrected CDFs was extremely small in virtually all cases.We emphasize, however, that all of the reported violationswere apparent regardless of whether this fast-guess correc-tion was applied, so the results in no way depend on thiscorrection.

Experiment 1

Procedures

Preliminary. Stimulus intensities that produced equivalent la-tencies in each observer were identified in a series of preliminarysessions in which we presented unimodal targets of varying inten-sities. There were 64 trials in each session. Each trial began witha warning tone, followed by either a visual or acoustic target (therewere no bimodal stimulus trials in these preliminary sessions).Stimulus intensity, modality, and location (left vs. right) varied ran-domly across trials. At least 4 of these preliminary sessions wererun for each response condition (saccades, directed manual re-sponses, or simple manual responses). Intensity-RT curves wereused to select visual and acoustic intensities that produced com-parable RTs for use in the formal portion of the experiment.

Data collection was always preceded by 5 min of dark adaptation,during which time the eye tracker was adjusted and eye positioncalibrated. At the viewing distance of 114 cm, the targets appearedat an eccentricity of 20°. Head movements were minimized usinga bite plate. Four naive subjects were paid for their participation.All were emmetropic (or were appropriately corrected) and hadnormal hearing.

Experimental. When intensities that produced equivalent la-tencies for the visual and acoustic targets were identified, formaldata collection began. Each observer participated in 15 blocks of 64trials each (960 trials/subject). Typically, a subject completed 2 or3 blocks/day. For saccadic and directed manual response sessions,each type of target (auditory, visual, or bimodal) occurred withequal frequency in a randomized order. For the simple manual RTsessions, bimodal targets were presented on 33.3% of the trials,unimodal targets were presented on 50.0% of the trials (25.0% vis-ual and 25.0% auditory), and no target (catch trials) was presentedon 16.7% of the trials. Targets were presented to the left or the rightof fixation with equal frequency in a randomized order. The reporteddata are based on at least 100 observations for each of the nineconditions (acoustic, visual, or bimodal stimuli for each responsecondition) in 4 naive observers.

Results

The averaged RTs for acoustic, visual, and bimodal targetsare illustrated in Figure 2. The left portion of Figure 2 shows

o%

Haie2

400

300

200

SACCADE DKECTED-MANUAL SIMPLE-MANUAL

RESPONSE CONDITION

Figure 2. Mean latencies for visual, acoustic, and bimodal tar-gets for each response condition in Experiment 1. RT = responsetime.

the saccade latencies, the middle shows the directed manualresponses, and the right shows the simple manual responsetimes. It can be seen that bimodal stimuli generally producedshorter response times than unimodal stimuli. This was con-firmed in a four-factor analysis of variance (ANOVA) (Di-rection [left vs. right] X Stimulus Modality X ResponseMode X Subject) that revealed a significant interaction be-tween response mode and stimulus modality, F(4, 12) =7.26, p < .005. Post hoc analyses (Newman-Keuls test) of themeans contributing to this interaction showed that bimodalRTs were significantly faster than either of the unimodal RTsfor each response condition (allps < .05). The lone exceptionwas that bimodal RTs were not significantly faster than au-ditory RTs in the simple manual response condition.

Average error rates for each response condition are shownin Table 1. These errors represent anticipations and falsealarms in the simple manual RT task and anticipations anddirection errors in the saccadic and directed manual responsetasks. All RTs less than 125 ms were considered anticipationerrors. These error rates were submitted to a two-factor (Re-sponse Condition X Subject) ANOVA after arcsine trans-formation. This analysis revealed no significant differencesin error rates for the three response conditions, F(2, 6) =3.25, p = .11.

Evidence for neural summation. We compared the ob-tained redundant-targets CDFs with the sum of the corre-sponding unimodal CDFs to determine whether the observedintersensory facilitation of RTs might be accounted for byprobability summation. All analyses were based on latencyhistograms with a 10-ms bin width.

Figure 3 illustrates the redundant-targets effect for atypical observer in the saccade condition. The CDFs fromthe marginal (unimodal) conditions, the sum of these mar-ginal CDFs (left side of the race inequality [Inequality 1]),


Table 1Probabilities (Ps) and Mean Reaction Times (RTs)for Correct Responses, False Alarms,and Direction Errors in Experiment 1

Response mode

Responsecategory

False alarmsPMean RT

DirectedSaccades manual

— —

Simplemanual

0.0313198.0

Direction errorsP 0.004 0.03 —MeanRT 106 214.4 —

Mean RT 213.9 315.7 267.0for correct responses

and the obtained bimodal CDF are all presented in the toppanel of the figure. Violations of the upper limit of the racemodel are indicated whenever the probability associatedwith the obtained bimodal CDF exceeds the sum of themarginal (unimodal) CDFs. These violations are indicatedby the vertical hatching in the top panel (nonviolations arerepresented by the horizontal hatching). The bottom panelshows the difference between the obtained bimodal CDFand the race inequality. Because it is much more efficientto present the data in terms of this difference between theobtained and predicted CDFs, we report most of the resultsusing this format.

Figure 4 presents the pattern of the violations of Inequality1 in all 4 observers. With one exception, the data were sub-mitted to the correction for fast guesses outlined in the Ap-pendix. The exception was the simple RT data from ObserverM.K. The reason we elected not to use the correction forM.K. is indicated in Figure 5. The magnitudes of the vio-lations of the race model inequality with and without thefast-guess correction are shown in the bottom panel. It isevident that using the correction on M.K.'s data would haveresulted in violations of the inequality that would have begunat a latency of 15 ms. This would have been entirely the resultof the fact that M.K. committed 12 false alarms on catch trialsthat had no counterpart in the signal-present trials. The CDFfor these false alarms (up to 300 ms, which included all butone of the false alarms) is shown in the top panel of Figure5. It seems obvious that, in this particular case, virtually allthe evidence for violations of the inequality comes from these12 false alarms. Given this distortion of the results producedby the fast-guess correction, we elected not to use the pro-cedure in the case of M.K.'s simple RT data.

Violations of the race inequality were apparent for eachresponse mode in all 4 observers. Notice, however, that boththe proportion of the manual RT data that violated the in-equality and the magnitude of the observed violations weresmaller than those associated with saccades. Failures to vio-late Inequality 1 do not necessarily rule out coactivationmodels. However, if selective influence and context inde-pendence hold, violations of Inequality 1 can only be realizedby neural summation (coactivation). Thus, Inequality 1 rep-

resents a very conservative test of neural summation models(Eriksen, 1988; Miller, 1991). Both superposition andoveradditive combinations of individual channel activitieswill violate Inequality 1 (see Townsend & Nozawa, 1992).In the next section, we evaluate the possibility that the like-lihood of violations varied with response mode.

Tests of the response-dependent ordering of neural sum-mation. The probability of violations of the race inequalitywas first calculated. The number of violations occurring inthe n intervals over which the inequality can be evaluated isdistributed as a binomial random variable. Thus, the prob-ability of a violation, pA = P(violation I Response Mode A)~ l/nA Bin(pA,/iA). The multiplicative factor l/nA meansthat we are dealing with the relative frequencies of the vio-lations, rather than the actual number of violations observed.As nA (the number of possible violations) increases, the bi-nomial approximates the normal distribution: l/nA Bin-(pA, nA) ~ N(pA, pA(l -p/Jn/^). The probability of observ-ing a violation can be approximated by the normaldistribution with a mean of pA and variance of pA(\ -pA)/nA.Thus, the hypothesis can be expressed as the difference of the

Obsever: GTResponse: Saccades

Visual TargetAuditory TargetRace InequalityBimodal Target

300 350

200 250 300

Time (msec)

350

Figure 3. Violation of the race model inequality. The top panelshows the cumulative distribution functions (CDFs) for the uni-modal targets, the obtained bimodal CDF, and the performancelimit of the race model inequality (sum of the unimodal CDFs).Violations of the race model inequality are indicated by the verticalhatching. The bottom panel shows the differences between theobtained bimodal CDF and the race inequality. Prob. = probabil-ity; RT = response time.

Target Location

co

—"o>_>»••"a3w<uc

uM

as5fc*O

•o3

.̂ JCMC3

e_o

5_"o>•

T̂3gsre

uegatCMou•os

,̂ JcaS

0.5

0.4

0.3

0.2

0.1

0.0

-O.I -

-0.2-

-0.3-

0.4-

-0.5 "1

0.5

0.4-

0.3-

0.2-

0.1 -

o.o --0.1 -

-0.2-

-0.3-

-0.4-

-0.5 -5

0.5 '

0.4-

0.3'

0.2-

0.1 -

o.o •-0.1 '

-0 .2 -

-0.3-

-0.4-

-0.5 "1C

0.5 •

0.4-

0.3-

0.2-

0.1 •

o.o--0.1 -

-0.2-

-0.3 -

-0.4 -

Left

_ Obs: GT

A/ \/ VAJ r-\ i f**~\.

V~* \ ; \ 1\ 1

VSaccadesDirected Manual ResponsesSimple Manual Responses

SO 200 250 300 350 400

Obs: JA

AX1^ \

r \ r^\f -=^l'/' — 'M f

v U'\i1 u

SaccadesDirected ManualSimple Manual

0 !00 150 200 250 300

Obs: LQ

yur\_4j7^

<J 1\ ISaccades VDirected Manual Responses

X) 150 200 250 300 350

Obs: MX

A/ \/ \/ \

-~ .J xxxV— *^T^ ! \

*"* >>J' 1 1ySaccadesDirected Manual ResponsesSimple Manual Responses

45

35

40(

150 200 250 300

Time (msec)

Right0.5-

0.4-

0.3-

0.2-

0.1 •

0.0- —

-0.1 -

-0.2-

-0.3-

-0.4-

-0.5 • —100

Obs: GT

^̂ — SaccadesDirected Manual ResponsesSimple Manual Responses

150 200 250 300 350 400

0.4-

0.3-

0.2-

0.1-

o.o--0.1-

-0.2-

-0.3-

-0.4-

Obs: JA

r *VVY i i

'A ;'\

"'"" SaccadesDirected Manual ResponsesSimple Manual Responses

0.45-

0.35-

0.25-

0.15-

0.05-

Obs: LQ

"̂ ™ SaccadesDirected Manual ResponsesSimple Manual Responses

0.5-

0.4-

0.3-

0.2-

0.1 •

0.0

-0.1 -

-0.2-

-0.3-

-0.4-

-0.5

Obs: MK

~~~~ SaccadesDirected Manual ResponsesSimple Manual Responses

200 250 300

Time (msec)

Figure 4. Results from Experiment 1. Positive values indicate the magnitude of the violations ofInequality 1. Data for saccades, directed manual responses, and simple manual responses are given.Obs. = observer.


two normal random variables, pA andpB, the null hypotheses,P(violation I Response Mode A) ^ P(violation I ResponseMode B) and P(violation I Response Mode A) <P(violation I Response Mode C) are distributed as N(pA -PB>PA<! -PA)/" A + Pa(l -psV^s)- Because the distributionmean and variance are known, the z score associated with thiscomparison can be calculated as follows:

PA ~ P*

~ PA) ~ PB)

Table 2Z Scores Associated With the Probabilityof Violations of Race Inequality in Experiment 1

Observer

G.T.L.Q.J.A.M.K.

Saccades vs.direct manual

response

7.36**7.98**2.75**2.12*

Saccades vs.simple manual

response

1.16.65**2.72**1.89*

*p<.05. **p<.01.

v3

n'o

1

8a

•o

0.00 50 100 150 200 250 300

Corrected for Fast GuessesUnconnected for Fast Guesses

-01 .

-0.2

Time (msec)

Figure 5. Example of the potential problem associated with thecorrection for fast guesses. The top panel shows the cumulativedistribution function for false alarms in Observer (Obs.) M.K. Atotal of 12 false alarms were recorded in the simple response timetask. The earliest of these have no counterpart in the target-presenttrials. The bottom panel illustrates the effects on computations ofrace inequality violations. Although the changes are small in mag-nitude, they dramatically affect statistical measures based on theprobability (P) of a violation.

The results of these calculations are provided in Table 2.This analysis supports the suggestion that the probability ofviolating the race inequality was greater for saccades than foreither directed manual or simple manual RTs.

Discussion

Response-dependent ordering in the magnitude of neuralsummation. All three response modes showed evidence ofneural summation between visual and auditory channels.However, the magnitudes of the violations of the race in-equality for saccadic responses were greater than those ob-served for either directed manual RTs or simple manual RTs.The magnitudes of the violations of Inequality 1 for saccadeswere quite robust, ranging from 0.23 to 0.40 in the 4 ob-servers (see Figure 4). By way of comparison, previouslyreported violations of the probability summation limit gen-erally vary between 0.05 and 0.10 (e.g., Diederich & Colo-nius, 1987; Miller, 1982; Mordkoff & Yantis, 1991). Theviolations in the present manual RT data are generally com-mensurate with these previous results. Thus, the magnitudeof race inequality violations depended on the response sys-tem. Such differences may relate to the operator that com-bines the channel activities (e.g., addition vs. multiplication).Alternatively, the number of elements receiving convergentinputs (Kimura & Tamai, 1992) might influence the mag-nitude of intersensory integration, just as the number of re-sponding elements contributes to the effects of stimulus in-tensity on RT. We returned to the possibility of response-dependent ordering in the magnitude of race inequalityviolations in Experiment 2.

The importance of central simultaneity. The response-dependent ordering suggested by these data may also relateto the quality of the matches of the unimodal RTs. Intuitively,one might think the neural summation is maximized when theactivities of all channels are contemporaneous. Miller (1986)has provided direct evidence in support of this conjecture.Although the procedure is not necessarily infallible, we triedto maximize the likelihood that all channels' activities ar-rived contemporaneously at the site of summation by match-ing the visual and auditory RTs (through manipulations ofsignal strength). The procedure worked well for both thesaccades and the directed manual responses. However, de-spite a concerted effort, the matches for the simple RTs werenot as close as we had hoped they would be (see Figure 2).This mismatch could have produced less robust violations of


the race inequality for simple RTs than might otherwise haveoccurred (e.g., Diederich & Colonius, 1987; Miller, 1986).Indeed, the present evidence for neural summation observedfor simple manual RTs may actually be a little weaker thanin some of the previously published results using similarconditions (e.g., Diederich & Colonius, 1987; Miller, 1986).Although other paradigmatic differences could play somerole in attenuating the level of neural summation relative tosome of these earlier reports (e.g., interstimulus dependen-cies, discussed in the next section; the uncertainty of thetarget location; and the presence of catch trials), the observedviolations of Inequality 1 may have been more compellinghad we succeeded in obtaining better matches between thesimple RTs in the single-target conditions. The RT matchesbetween the auditory and visual targets were quite good inthe saccadic and directed manual response conditions, how-ever, and evidence for ordering was still obtained.

It is interesting to note that recent electrophysiologicalstudies indicate that the temporal window for bimodal sum-mation in the superior colliculus is actually quite long (forindividual neurons, the integration window is often greaterthan 500 ms; see Stein & Meredith, 1990). From the eco-logical perspective, long integration times would appearquite desirable. That is, substantial differences exist betweenvisual and auditory transduction latencies, and the differ-ences in the relative velocities of sound and light would usu-ally not be expected to compensate for the differences ininternal processing time. A strict requirement of central si-multaneity for neural summation between visual and audi-tory targets would mean that neural summation could onlybe expressed with specific combinations of visual and au-ditory intensities, and the required combination of intensitieswould further depend on stimulus distance. To be generallyuseful, the summation mechanism must have a long inte-gration time (Stein & Meredith, 1990). Our second goal inExperiment 2 was to investigate the effects of auditory and

Table 3Stimulus Event Probabilities (Ps)for Each Response Condition in Experiment 1

Table 4Dependent Structures for Response Conditionsin Experiment 1

Event

A and VA and VA a n d VA and VA I VA i VV I AV I AAVAV

P

Saccadic and directedmanual response

.333

.333

.3330.0

.51.01.0.5.666.666.333.333

Simple manualresponse

.333

.25

.25

.1667

.571

.4

.4

.571

.583

.583

.417

.417Note. _A = auditory target; V = visual target; V = no visualtarget; A = no auditory target; A I V = auditory target given visualtarget; A I V = auditory target given no visual target; V I A =visual target given no auditory target; V I A = visual target givenauditory target.

Conditional MarginalEvent probability probability

SaccadicA 1 V vs. VV 1 A vs. AA 1 V vs. VV 1 A vs. V 1 AV 1 A vs. V 1 AA 1 V vs. A 1 V

and directed manual responses.5 .667.5 .6671.0 .6671.0 .667

Difference

-.1666-.1666

.333

.333-.5-.5

A I V vs. VV I A vs. AA I V vs. VV I A vs. AV I A vs. V IA i V vs. A

Simple manual response.571 .583.571 .583.6 .417.6 .417

-.0119-.0119

.183

.183-.029-.029

Note. A = auditory target; V = visual target; V = no visualtarget; A = no auditory target; A IV = auditory target given visualtarget; A I V = auditory target given no visual target; V! A =visual target given no auditory target; V I A = visual target givenauditory target.

visual detection asynchrony on the observed violations of therace inequality.

Interstimulus dependencies and coactivation. Mordkoffand Yantis (1991) recently performed an interesting analysisof the role of interstimulus contingencies and stimulus-response contingencies in experiments on coactivation ef-fects. They pointed out that in redundant-targets experi-ments, the identity of the stimulus on one channel can conferinformation concerning the identity of the stimulus presentedto the second channel.2 For example, suppose we wish todetermine whether the presence of an auditory signal con-veys information about the likelihood of a visual signal. Ifthe probability of an auditory target and that of a visual targetare independent, we have P(A 1V) = P(A) X P(V)/P(V) =P(A), and no information is conveyed. However, if P(A I V)- P(A)^ 0, then the occurrence of the visual target conveysinformation about the likelihood of an auditory target(Dudewicz, 1976). Mordkoff and Yantis referred to this dif-ference between the conditional and the marginal probabili-ties as the interstimulus contingency (denoted as 7SC[V —>A]). They observed that previous reports of coactivation wereobtained only when the 75C was greater than zero. They wenton to report the results of several experiments showing noevidence of coactivation (i.e., no violations of Inequality 1)when the ISC was zero. Tables 3 and 4 present the set of ISCvalues from the present experiment.

In all cases, targets in one modality conveyed negativeinformation with respect to the probability of a target pre-sentation on the second channel. According to Mordkoff andYantis (1991), this mitigates against violations of Inequality

2 In this context, the term information does not refer to theformal definition of information in information theory.


Table 5Stimulus Event Probabilities (Ps) in Experiment 2

Event P

A and VA and VA a n d VA and VA I VA I VV I AV I AAVAV

.25

.25

.25

.25

.5

.5

.5

.5

.5

.5

.5

.5Note. A = auditory target; V = visual target; V = no visualtarget; A = no auditory target; A I V = auditory target given visualtarget; A I V = auditory target given no visual target; V I A =visual target given no auditory target; V I A = visual target givenauditory target.

1. Nonetheless, clear evidence of neural summation wasfound. To our knowledge, these data represent the first evi-dence for violations of Inequality 1 with negative depen-dencies between the target stimuli. Mordkoff and Yantis alsodefined the interstimulus contingency benefit, ISCB(not A),as the difference between P(V I A) and P(V I not A). Thisquantity represents a benefit conferred to redundant-targetstrials over visual targets. Similarly, we can define ISCB(notV) as P(A I V ) - F(A I not V). Mordkoff and Yantis used theISCB to analyze redundant-targets paradigms in which sub-jects must make one response if either one or two targets arepresented and make a different response if two nontargets arepresented. The logical status of this measure is open in thepresent situation, however. In the present case, the only typeof nontarget was the absence of a stimulus altogether; it isdifficult to see how a lack of stimulus energy could facilitateprocessing in a different modality. In any case, the ISCBs inExperiment 1 are also provided in Table 4, and again thevalues were negative. The /SCs indicate that there was nobasis for facilitatory cross-talk between the channels in Ex-periment 1. To have the ISCBs play a role in the presentexperiment, one would have to suggest that the absence ofa target on one channel A slowed processing on Channel V.If it is assumed that the interactive race model may includeinhibitory cross-talk, it might be suggested that negativeISCBs might slow the unimodal RTs, increasing the likeli-hood of violations of Inequality 1. This suggestion acceptsthe questionable hypothesis that a lack of activity on onechannel can slow processing on another channel. In any case,the ISCBs in Experiment 2 were zero, and robust violationswere still observed.

Note that the /SCs were identical for the saccadic anddirected manual response conditions. This supports our hy-pothesis that saccades show a greater degree of intersen-sory facilitation than do directed manual responses. If any-thing, the present results may underestimate the strength ofneural summation in the saccadic and directed manual con-ditions. Because we included catch trials in the simplemanual RT condition, the negativity of the ISC was much

smaller in this condition. Thus, the /SCs in Experiment 1actually favored neural summation of simple RTs over theother response conditions. However, the fact that the ISCwas (slightly) less than zero might be another factor thatcontributed to the weak evidence of neural summationfound for simple RTs relative to earlier reports (the /SCs inmany of those reports were positive; e.g., Diederich &Colonius, 1987; Miller, 1982, 1986; Mordkoff & Yantis,1991). In Experiment 2, we attempted to replicate thesefindings under conditions in which no interstimulus depen-dencies were operating.

Experiment 2

In Experiment 2, our goal was to extend the observationsmade in Experiment 1 by (a) investigating the importance ofmatching the visual and auditory RTs in determining themagnitudes of the redundant-targets effects in the presentparadigm and (b) evaluating the redundant-targets effects inthe absence of the interstimulus dependencies that were op-erative in Experiment 1. Thus, the major differences betweenExperiments 1 and 2 were that the design of the latter in-cluded a factorial combination of high and low stimulusintensities for both the auditory and the visual targets(to produce central asynchrony between auditory andvisual detection times) in the context of complete in-dependence between the occurrences of visual and auditorytargets.

Procedures

Preliminary. Once again, we attempted to match the RTs to thevisual and auditory targets. Two levels of intensity were used forboth the visual and auditory modalities. The intensity levels thatproduced approximate matches were obtained in preliminary ob-servations. The same intensities were used by each observer. Theselected intensities of the visual targets were 0.04 cd/m2 and 12.0cd/m2. The corresponding auditory intensities were 46 decibelsstandard pressure level (dBspl) and 74 dBspI. To improve our controlover the auditory RTs, we added a constant background of whitenoise (60 dBspl), which was delivered through an overhead speaker.

Table 6Dependent Structures for Saccadicand Directed Manual Responses in Experiment 2

EventConditionalprobability

Marginalprobability Difference

A 1 V vs. VV 1 A vs. AA 1 V vs. VV 1 A vs. AV 1 A vs. V 1 AA 1 V vs. A 1 V

0.50.50.50.5

0.50.50.50.5

0.00.00.00.00.00.0

Note. A = auditory target; V = visual target; A I V = auditorytarget given visual target; V I A = visual target given auditorytarget; AI V = auditory target given no visual target; V I A =visual target given no auditory target. No differences were notedbetween the conditional and marginal probabilities in any event.


400-

350-

300-

250-

200

Visual Targets Response Mode Auditory Targets

Directed Manual •—•

Saccate •—

Simple Manual A—

Low Intensity High Intensity

Figure 6. Mean response time (RT) as a function of stimulusintensity for the unimodal targets in Experiment 2.

The general procedures were the same as in Experiment 1.However, the proportions of the trial types were altered to elimi-nate the dependent structure between the stimuli. The proportionsof each stimulus condition are provided in Tables 5, and the in-terstimulus contingency analysis is given in Table 6.

Experimental. Three observers participated in 15 experimentalsessions of 384 trials each. None had participated in Experiment 1.

Two of the observers were unaware of the issues under investigationand were paid for their participation. The third observer was one ofthe authors. Typically, a subject was tested for six blocks of 64 trialsper day (two blocks for each response condition/day for 15 days).Each response condition (saccades, directed manual, or simplemanual) was run in accordance with a Latin square to control forpossible order effects. Within each block of trials, targets were pre-sented to the left or the right of fixation with equal frequency in arandomized order. The data reported are based on 90 observationsfor each of the 16 different stimuli (4 acoustic [two intensities onthe left and right], 4 visual [two intensities on the left and right],and 8 bimodal [the factorial combination of two intensities in twomodalities for two locations]) and three response conditions (sac-cades, directed manual, and simple manual) in each observer (90 X16 X 3 = 4,320 + 1,440 catch trials = 5,760 total trials).

Results

Mean RTs averaged across the 3 observers are shown inFigure 6. The intensity effect averaged approximately 40 msand was largely independent of modality and response sys-tem. A four-factor ANOVA (Intensity X Modality X Re-sponse Mode X Subject) revealed main effects of both in-tensity, F(l, 2) = 35.06, p < .025, and stimulus modality,F(l,2) = 89.2, p<. 01. The effect of response mode did notreach statistical significance, F(2, 4) = 4.58, p = .093, butthe Stimulus Modality X Respond Mode interaction did,

Saccades

target «.4-Location

0.3

0.2-

S.2 _ 0.1«•» £

o »J o.o

-o.r

•5 -0.2-3

JS 1C

«egK

0.3

4, 0.2-TS3 £ o.i-C Ofaf Q£ o.o

•o.r

-0.2

7

Stimulus Intenilt;Obs: JZ Vtawi Avdllory

Low Low•*• Low Hi|h

AVcO. "* U*/ A\ \>4« lh Hi*b/ /7\\ \A.

'̂ T \

\f x i^ \ ;

*

10 150 200 250 300 35

^^ Stimulus lolcBsilyOb>: JZ Vitaii Avdltorj

Low Low/*. Low Hi|h

UL* mmmmm High LOW

a\ \̂ '"»s f j /w vV '»

jC*J: IV\

vH\K|\5 125 175 225 275 3

Time (msec)

0.3

0.2-

o.i-

-o.r

-0.2-

<0 1C

0.3

0.2-

0.1-

-0.1

-0.2-

!5 1(

Obj; JE

/\

/Of^^.//\* v*//? Vt \A

jfv' N- \

\f v iV \ i*» 150 200 250 300 3.

Ob.: JE ,,

IJ"$ ^j Jl 1A \Jyv n\\

^^//y Y\ \ \wv

» 150 200 250 300 3

Time (msec)

0.4

0.3

0.2-

o.r

o.o-

-o.i-

-0.2-

0 1C

0.3

0.2-

o.r

-0.1

-0.2-

» 1(

Obi:HH

xiA r-\

i/*v\' V'il vV'" '>/^L/\\ \

v '< ' •\;

» ISO 200 250 300 35

Obi:HH

4",

\

\

1

> ' Vw•

X) 150 200 250 300 35

Tim* (msec)

Figure 7. Violations of race model inequality in saccadic responses in Experiment 2. Obs. -observer.


Directed Manual Responses

Target °4

Location

e

eg

uHat

41•o

a1

-0.3250 300 350

Time (msec)

Figure 8. Violations of race model inequality in directed manual responses in Experiment 2. Obs.= observer.

F(2, 4) = 15.4, p < .02. No other interactions were signifi-cant, although the triple interaction (Intensity X Modality XResponse Mode) was close, F(2, 4) = 5.32, p < .08. Errorrates for the response conditions are shown in Table 7. Theserates appear quite small, and, as indicated previously, the RTdistributions were corrected for fast guessing. We thereforedid not analyze the error rates further.

The results, expressed as differences between the obtainedredundant-targets CDFs and the race inequality, are shown inFigure 7-9.Once again, robust violations of the race inequal-ity were observed in all observers and each response con-dition. There was little effect of mismatches between theauditory and visual detection times on the size of theredundant-targets effect. Large violations were seen amongall four combinations of auditory and visual signal strength.In addition, these violations occurred in the context of com-plete independence between the occurrence of the visual andauditory stimuli, suggesting that 75Cs appear to make littledifference in the present paradigm.

Again, however, there was a suggestion that the magni-tudes of the observed violations varied with response mode.We evaluated this difference using the methods describedearlier, and the results are provided in Table 8.

The same trend of response-dependent ordering of the vio-lations of the race inequality seen in Experiment 1 appearedin Experiment 2: Saccades were more likely than manualresponse to produce violations.

Discussion

The results of Experiment 2 replicate and extend the find-ings of Experiment 1. Thus, human saccades showed clearevidence of neural summation between the visual and au-ditory channels. Both directed manual responses and simplemanual RTs also showed evidence of neural summation, butthese latter effects may have been less robust. In all threeresponse modes, there did not appear to be any strict re-quirement of simultaneity between the visual and auditorytarget processing—asynchronies of up to 40 ms easily pro-duced summation effects. As discussed previously, relativelylong integration times for visual-auditory summation effectsare a necessary component of the processing architecture ifthe systems under investigation are to have any generallyuseful ecological validity.

The site of neural summation. Logically, neural sum-mation effects could occur at the level of sensory process-ing, response selection, or motor execution. Evidence fa-voring neural summation at each of these levels ofsensorimotor processing has been reported. Data at sensorylevel have been provided by Fournier and Eriksen (1990)and Mordkoff and Yantis (1991). Data at the response se-lection level have been provided by Fournier and Eriksen(1990); Miller (1982); Mordkoff and Yantis (1991); andSchmidt, Gielen, and van den Heuvel (1984). Data at themotor-processing-time level have been provided by Die-derich and Colonius (1987).


Target °4

Location Q 3 _

0.2Bo_ V

.2 *J o.o-

S>% -0.1

*^«S -0.2-

O"

Simple Manual Responses

Stimulus Intensity0bS' 'L V.u.1 Auditor;

Low LowLow HighHigh Low

^~— High High

rv / V

-J^^'f^\\Njr v \f^\ / i

XX>/;"\s

0.3

0.2

0.1

0.0

-o.i-

-0.2-

Obs: JE

Af- /\

/ /^ \/i \ ',

/̂ A... xV1 \ \•^fr •-/ V .;

1

0.3

0.2

0.1

0.0

-o.i-

-0.2

Obs: HH

.* /'•

1 Ji '*̂ ' ' '

*f\i / \ \ ; 11 V !<; °

V/ *

J 1 50 200 250 300 350 400 1 00 1 50 200 250 300 350 100 1 50 200 250 300 35

at,_ 0.3-oV 0.2-

•Ba.•S £ 0.1-C 0£oc •••

s-0.1-

-0.2-

Stimulus Intensityur>s: J/1 VUual Auditory

Low Low

Low High

----- High Low

-^— High High

• r> \f\ 'ii^ '"•'̂ V^V/XSr \ fat i

IfjK,]v

0.3-

0.2-

o.r

-o.i-

-0.2-

Obs: JE

r\I Uj.v~^t *t ̂^ A* i?v^\

y"vy ^jf * l/« \

/ /./' l< I

~ V-N / '•'', /

1.1

0.3"

0.2-

0.1-

-o.r

-0.2-

Obs:HH

A

/ \ N \ A*r\ / 1 * V ** '* i

^/ l/^H*1* * *> i1

// *fr ' l*\ ** '« i '^ _.{V r\ «/ * V ;

W *

150 200 250 300 350 400 100 150 200 250 300 350 100 150 200 250 300 35

Time (msec Time (msec) Time (msec)

Figure 9. Violations of race model inequality in simple manual responses in Experiment 2. Obs.= observer.

Clearly, one major locus of the visual-auditory conver-gence resides within the oculomotor system. We suggestthat the most parsimonious interpretation of the saccadedata is that neural summation results from the convergenceof visual and acoustic afferents on neurons within thedeeper layers of the superior colliculus (e.g., Jay & Sparks,1987; Peck, 1987; Stein et al., 1993). These cells sharecharacteristics of sensory neurons as well as motor neu-rons, and any attempt at a dichotomous classification

Table 7Probabilities (Ps) and Mean Reaction Times (RTs)for Correct Responses, False Alarms,and Direction Errors in Experiment 2

Response mode

Responsecategory

False alarmsPMean RT

Saccades

0.0167300.2

Directedmanual

0.009233.4

Simplemanual

0.0715261.0

Direction errorsp 0.0 0.018Mean RT 254.4

Mean RT 220.9 287.8for correct responses

seems pointless. Thus, the evidence for neural summation(coactivation) in the control of saccades seems best re-garded as occurring at the interface between sensory pro-cessing and motor execution.

Our interpretation of these findings with respect to sac-cades would receive additional support if it could be shownthat these bimodal summation effects depend on the spatialalignment of the visual and acoustic inputs in a manner simi-lar to that already described for neurons in the superior col-liculus (e.g., Meredith & Stein, 1987). Inverse relationshipsbetween the level of coactivation and target separation havebeen reported (Fournier & Eriksen, 1990; Miller, 1982). Anexamination of the effects of spatial correspondence in thepresent paradigm should provide important additional infor-mation on the mechanisms of neural summation in oculo-

Table 8Z Scores Associated With the Probabilityof Violations of Race Inequality in Experiment 2

284.8

Observer

H.H.J.E.J.Z.

Saccades vs.direct manual

response

4.88**3.84**8.6**

Saccades vs.simple manual

response

1.7*2.67**

10.03*** p < .05. ** p < .01.


Table 9Stimulus Event Probabilities (Ps)for the Two Target Conditions in Experiment 3

Table 10Dependent Structures for theTwo Target Conditions in Experiment 3

Event Auditory target Visual target

AVAVA a n d VA and VA a n d VA a n d Valigned 1 A and Vmisaligned 1 A and VA 1 VA I VV I AV I A

0.80.40.20.60.40.40.00.20.50.51.00.6670.00.5

0.40.80.60.20.40.00.40.20.50.50.50.00.06671.0

Note. A = auditory target; V = visual target; V = no visualtarget; A = no auditory target; A IV = auditory target given visualtarget; A I V = auditory target given no visual target; V I A =visual target given no auditory target; V1 A = visual target givenauditory target.

motor processing. Experiment 3 represents an initial step insuch an analysis.

Experiment 3

Although a parametric study of spatial misalignment iscertainly desirable, such a project is a major undertaking. InExperiment 3, we investigated what we expected to be a clearboundary condition for the effects of misalignment: present-ing the targets in opposing hemifields. Once again, saccades,directed manual responses, and simple manual RTs were in-vestigated. A given modality (either visual or auditory) wasassigned as the target, and, in the case of saccades and di-rected manual responses, subjects were instructed to directtheir responses toward the location of the target. They weretold that responses that were not directed toward the locationof the designated target were incorrect. For example, in thecase of out-of-register targets, it was incorrect to direct asaccade toward the visual stimulus when the auditory stimu-lus was the designated target. Notice that this requirementremoves the horse race character of the task, in that directedresponses could not simply be triggered by whichever stimu-lus was detected first. Rather, the subject had to be sure todirect responses toward the location containing the desig-nated target. This presumably required application of someform of filtering or gating of the nontarget modality. Thequestion we wished to address with respect to directed re-sponses was whether evidence of summation between spa-tially corresponding targets could be observed in spite of thisfiltering of the nontarget modality. In the case of simplemanual responses, no such filtering was necessary, becausenontarget stimuli were never presented in isolation. Thus,simple manual responses could still be triggered by which-ever target was detected first. In this case, it seemed possible

Event

Auditory targetA 1 V vs. VV 1 A vs. AA 1 A vs. VV ! A vs. AV 1 A vs. V 1 AA 1 V vs. A 1 V

Visual TargetA 1 V vs. VV 1 A vs. AA 1 V vs. VV 1 A vs. AV 1 A vs. V 1 AA 1 V vs. A 1 V

Conditionalprobability

1.00.50.6670.0

0.51.00.00.667

Marginalprobability

0.40.80.60.2

0.80.40.20.6

Difference

0.6-0.3

0.067-0.2

0.50.333

-0.30.6

-0.20.0670.3330.5

Note. A = auditory target; V = visual target; V = no visualtarget; A = no auditory target; A I V = auditory target given visualtarget; A I V = auditory target given no visual target; V I A =visual target given no auditory target; V ] A = visual target givenauditory target.

that evidence of neural summation might be obtained evenwith out-of-register targets. However, we believed it unlikelythat any evidence of neural summation between these mis-aligned targets could be attributed to auditory-visual con-vergence within a sensory pathway, there being little reasonto expect multimodal convergence between such widely dis-parate locations.

Method

In general, the methods and procedures were the same as thoseused in Experiments 1 and 2. The major differences were designchanges required by the inclusion of spatially misaligned targets.Thus, there were two basic conditions: one in which the auditorystimulus was designated as the target and one in which the visualstimulus was designated as the target. In either case, 20% of thetrials were catch trials. A target (visual or auditory, depending onthe condition) was presented on the remaining 80% of the trials.In 50% of these target trials, the target stimulus alone was pre-sented and in the remaining 50% of these trials bimodal stimuliwere presented. The auditory and visual stimuli were presented inspatial register in half of the bimodal trials and out of register (inopposite hemifields) in the remaining half. The distribution oftrial types is summarized in Tables 9 and 10.

All stimuli were presented at an eccentricity of 15°. Target lo-cation varied between the left and right locations with equal fre-quency, and the sequence of trial types was randomized.

The 3 subjects from Experiment 2 again served as observers.They participated in 10 experimental sessions, each consisting ofsix blocks of 50 trials per day (two blocks of 50 trials for each ofthe three response conditions). The order for performing each re-sponse condition was counterbalanced across days. Finally, targetmodality was blocked: Subjects participated in 5 sessions with onemodality as the target and then participated in 5 more with the othermodality as the target. J.Z. and J.E. first responded to auditory


Table 11Stimulus Intensities Used in Experiment 3

Saccades

Observer

J.E.J.Z.H.H.

Auditory

74 dB74 dB46 dB

Visual

12.0 cd/m2

12.0 cd/m2

0.04 cd/m2

Directedmanual response

Auditory

74 dB74 dB46 dB

Visual

12.0 cd/m2

12.0 cd/m2

0.04 cd/m2

Simplemanual response

Auditory

74 dB74 dB46 dB

Visual

12.0 cd/m2

12.0 cd/m2

0.04 cd/m2

targets; H.H. first responded to visual targets.The subjects were told which stimulus they should treat as the

target and to respond to the target modality as quickly as possible.Depending on response mode, these responses were, of course, tolook at the target, thrust the joystick toward the target, or depressthe microswitch. The observers knew that stimuli in the other mo-dality would sometimes appear and that they may or may not bealigned with the target. They were given no specific instructions onhow to deal with these extraneous stimuli, other than to realize thatresponses that were not directed to the designated target would betreated as errors.

The stimulus intensities were taken from Experiment 2 andwere selected on the basis of producing the closest match in uni-modal RTs obtained in that experiment. The intensities used foreach observer are provided in Table 11. As in Experiment 2,white noise was present throughout the duration of the session(60 dBspl).

As it turned out, these intensities did not produce matched uni-modal RTs in Experiment 3. Apparently, the change in conditions(having to respond to only a single intensity and having only onemodality as the target) altered the decision criteria and caused theresulting mismatches. However, these mismatches in the unimodalRTs proved fortuitous for interpreting the results, as we describe inthe next section.

ResultsAverage RTs for the bimodal trials as a function of both

spatial correspondence and target modality are illustrated inFigure 10 for each response condition. The correspondingerror rates are given in Table 12.

The RT data were submitted to a four-factor ANOVA.The factors were spatial correspondence, target modality,response mode, and subject. The analysis revealed signifi-cant main effects of correspondence, F(l, 2) = 30.22, p <.025, and response mode, F(2, 4) = 7.9, p < .05. In addi-tion, the Spatial Correspondence X Response Mode inter-action was significant, F(2, 4) = 19.51, p < .02. Examina-tion of Figure 10 shows that this interaction wasattributable to the effects of spatial correspondence on sac-cades and directed manual responses; spatial correspon-dence had little effect on simple manual responses.

To evaluate the results in terms of the race inequality, wehad to use the unimodal distributions from the two differenttarget conditions. That is, because unimodal auditory targetswere presented only during auditory target sessions and uni-modal visual targets were presented only during visual targetsessions, the unimodal CDFs used to evaluate the race in-equality had to come from different sessions. By the timethese data were collected, however, each of the observers hadperformed these tasks for a long period of time and wereproducing data that were quite stable.

The results, expressed as differences between the ob-tained bimodal RT distributions and the race inequality, areprovided in Figures 11-13. The data for the correspondingand noncorresponding bimodal stimulus trials are indicatedfor each of the three response modes. Consider first thesimple manual responses (see Figure 13). In this case, vio-lations of the race inequality were equal for both corre-sponding and noncorresponding bimodal targets. Whereas

Table 12Probabilities (Ps) and Mean Reaction Times (RTs) for Correct Responses, FalseAlarms, and Direction Errors Average Across the 3 Observers in Experiment 3

Response category SaccadesDirected

manual responseSimple

manual response

False alarmsP 0.0233 0.0133Mean RT 237.1 367.3

Direction errorsP

All trials 0.0125 0.04Corresponding stimuli: auditory target 0.003 0.023Noncorresponding stimuli: auditory target 0.073 0.16Corresponding stimuli: visual target 0.003 0.013Noncorresponding stimuli: visual target 0.003 0.093

Mean RT 279.2 276

0.0758243.1

Mean RT for correct responses 227.3 306.1 232.8


350

Otc

300-

250-

200-

150-

SaccadesDirected manual responses

Simple manual responses

Corresponding Nonconespond. Corresponding Noncorrespond.

Attend to Auditory Target Attend to Visual Target

Figure 10. Mean response time (RT) to bimodal targets pre-sented at corresponding and noncorresponding locations for sac-cades, directed manual responses, and simple manual responses.

the violations observed in H.H. were quite small, those ob-served in I.E. and J.Z. were substantial. In all cases, however,the spatial alignment of the stimuli appeared to have littleeffect.

This contrasts with the results obtained using directedresponses (see Figures 11 and 12). When the bimodalstimuli were presented at noncorresponding spatial loca-

tions, performance was substantially below that predictedby the race inequality (negative values indicate perfor-mance slower than the inequality). Indeed, there were es-sentially no violations observed in the entire data set. Incontrast, violations of the race inequality did occur whenthe bimodal targets occurred in corresponding spatial loca-tions, but they did not occur under all conditions. Con-sider, for example, Observer H.H. Violations of the raceinequality were observed for both directed manual re-sponses (see Figure 12) and saccades (see Figure 11) whenthe visual stimulus was designated as the target but notwhen the auditory stimulus was the target. The pattern ofresults found in Observer I.E. was even more obscure. I.E.showed robust violations for saccades, but only for visualtargets (see Figure 11). In contrast, J.E.'s directed manualresponses showed violations only when the auditory stimu-lus was designated as the target (see Figure 12). Anequally obscure pattern was apparent in Observer J.Z.Thus, the results indicate that evidence of neural summa-tion relies not only on the spatial correspondence betweenthe auditory and visual targets, but on some additional fac-tor(s) as well.

Examination of the actual CDFs for the various condi-tions provided important clues to the basis for these appar-ently disorderly results. Figure 14-16 illustrate the unimo-dal CDFs, the obtained bimodal CDFs, the race inequality,and the independent race prediction for the directed

Saccades

CJa

atou

•w

e

09]o

>?~Z9u

U.3

0.4

» 0.3

I 0.2-a*• o.i-ooS 0.0'

1 -0.1-

E -0.2-SU -0.3-

-0.4-

.n < •

«.. „ Tiriel ModalityObs:JZ Location v ,̂ ^^

Right ^̂ «

/\ft \

4̂ \W^C v//

.̂ "7^̂ ^

V'

100 150 200 250 300

0.5

0.4

0.3

0.2

0.1

0.0

-0.1

-0.2

-0.3

-0.4

-0.5

0.1

0.0

, -0.1

-0.2

' -0.3

-0.4

-0.5

-0.6

-0.7-

-0.8-

-0.9-

Tarfet ModilltjVisual Avditorr

.nifht —

200 250Time (msec)

300 350

100 150 200 300

0.1

0.0

-0.1

-0.2-

-0.3

-0.4-

-0.5-

-0.6-

-0.7-

-0.8-

-0.9100 200 250

Time (msec)

0.1

0.0

-0.1

-0.2-

-0.3-

-0.4

-0.5-

-0.6

-0.7-

-0.8

ObtrHH

-0.9-1350 100

Figure 11. Violations of the race inequality in saccadic responses as a function of the spatialalignment of the targets. Positive values indicate violations. Obs. = observer.

200 250 300

Time (msec)350


Directed Manual Responses

o

o>

150 200 250 300 350

at•s t

0.1

0.0

V•o

01

'

-0.1

-0.2

-0.3-

-0.4-

-0.5-

-0.6-

-0.7-

-0.8

Obi: JZ

0.2

0.1-

0.0

200 250

Tim

350 400

-o.i--0.2-

-0.3-

-0.4-

-0.5-

-0.6-

-0.7-

-0.8

0.1

0.0

100 200 250

Time (msec)

-0.1

-0.2-

-0.3-

-0.4-

-0.5-

-0.6-

-0.7-

-0.8-

-0.9200 250

Time (msec)

Figure 12. Violations of the race inequality in directed manual responses as a function of thespatial alignment of the targets. Obs. = observer.

manual and saccade responses in each of the 3 observers.Inspection of the unimodal CDFs shows that, in each case,one of the modalities produced faster RTs than the other.When we compare the obtained bimodal distributions forcorresponding stimuli with the race inequality, it becomesclear that the race inequality was violated only when thefaster of the two modalities was designated as the target.There were no exceptions.

A process of stimulus selection (based on the designatedmodality of the target) appears to precede the summationstage. When the target modality is processed slower thanthe extraneous modality, subjects must withhold their re-sponses until the stimulus location of the target modalityhas been identified (responses to the extraneous stimuluswere incorrect on half the trials, because on half of the bi-modal trials, the stimuli were presented out of spatial reg-ister). This precludes the expression of summation effectsbetween the two stimuli. Notice that, when the two stimuliare presented in noncorresponding locations, the RTs to thetarget approach but rarely exceed the unimodal CDFs. Thisspeaks to the efficiency of the filtering process and indi-cates that such filtering does involve some cost because itcan slow RTs relative to RTs to the same target presentedalone. However, when the targets appear in correspondinglocations, the bimodal CDFs are always faster than theCDF for the target modality presented alone, but they ex-ceed the race inequality only if the target modality is thefaster of the two.

Thus, the observers conform to the requirements of thetask: They make their responses contingent on the detec-tion of the designated target. If the target has a higher like-lihood of being detected first, the subject can afford to ini-tiate responses early, and a later-arriving extraneousstimulus presented in the corresponding location can stillfacilitate processing to a degree that violates the race in-equality. If the slower of the two modalities was the target,the subject must await detection of the target in order toavoid errors. They were able to do this with reasonable ef-ficiency. Some degree of intersensory facilitation is evidenteven when the slower of the two modalities is the target,but in this case delays associated with target selection pre-clude violations of the race inequality.

This pattern of results was also reflected in the errorrates (see Table 12). Notice that most of the errors occuron noncorresponding trials (they thus represent failures ofthe target selection process). The error rates for saccadesclearly contrast with those generated by the same observ-ers in Experiment 2: In Experiment 2, not a single saccadicdirection error was made by any of the subjects.

Discussion

The results of Experiment 3 indicate that summation ef-fects depend on the spatial alignment of the stimuli if the taskrequires target localization. If not, as was the case for the


Simple Manual Responses

SBaV

SBOS<*•oV

•O3

Ma

0.4-~

0.3-

» 0.2-

I 0,1

* 0.0

I-aH| -0.2-

I ~0'3'

U -0.4-

-0.5-

Target ModalityLocation yiniat Auditor,

L«n — -•

100 250 300 350

U.4'

0.3W)

& 0.2-

sS a i -C n n-u.u

1..,£ -0.2-

0 -0.3-U

g -0.4-z

-0.5-

TariM ModalityOb»:JZ Location ytaut Auditory

/~\ '''' ~'Xs — y « \^\ N

VVA. 1\ \ 1V^ N*

V ;% j\*

125 175 225Time (msec)

300 350Time (msec)

Figure 13. Violations of the race inequality in simple manual responses as a function of the spatialalignment of the targets. Obs. = observer.

simple manual responses, spatial alignment appears unim-portant. To avoid errors when the response is contingent onthe location of a designated target, subjects must (and can)efficiently gate the emission of their responses according totarget modality. Although this gating process obviously mustprecede response execution, it does not act as a simple filterthat prevents the extraneous stimulus modality from con-tributing to the activation of a response. If this were the case,the summation effects produced by spatial correspondencewould be prevented (and the large difference between theerror rates for aligned stimuli and those for misalignedstimuli would not be expected). Clearly, information from aspatially aligned extraneous target can facilitate processing,but this facilitation can generate RTs that violate the raceinequality only if the target modality is likely to be detectedfirst. Otherwise, the subject must attempt to withhold thedirected responses until the location of the target has beenidentified. This slows RTs to a level that makes violations ofthe race inequality difficult to achieve, although some degreeof intersensory facilitation is still observed.

The fact that simple RTs violated the race inequality re-gardless of whether the targets were presented in spatial reg-ister suggests that summation in this task can occur at a post-sensory stage of processing. This suggestion is based on thesupposition that a specific convergence of visual and audi-tory information originating from sources separated by 30°(15° on either side of fixation) is unlikely, especially forsources originating from opposing hemifields. Of course, it

is entirely possible that both sensory and motor coactivationplay a role in these redundant-targets effects, and their rela-tive importance depends on the particular task being inves-tigated. Here we simply point out that evidence of coacti-vation between widely separated visual and auditory targetsis not readily interpreted in terms of patterns of convergencewithin specifically sensory pathways.

Interpreting bimodal coactivation in terms of sensory con-vergence seems more clearly indicated when the effects de-pend on the spatial alignment of the stimuli. Although co-activation effects on directed responses did depend on thespatial alignment of the targets, these effects might also re-flect coactivation of motor processes rather than sensory pro-cesses. That is, aligned targets activate cooperative responsetendencies (e.g., a single saccade orients the eyes to bothtargets), whereas misaligned targets activate competing re-sponse tendencies (e.g., two different saccades). Indeed, se-lection between competing response tendencies is one likelyfactor contributing to the frequent finding that responses tomisaligned targets were often slower than responses to thetarget modality presented alone.

General Discussion

The early processing of sensory information is clearly mo-dality specific. There is, however, much to suggest that atsome point in processing, information from different mo-

Obs. JZDirectedManualResponses

Corresponding Targets Non-Corresponding Targets

400

0.00

t Unimodal Targets••"• LeftAud.— Right Vis.

225 325 375 425 225 275 325 375 425

Saccades

o£

ProbabilitySummationRaceInequalityNeuralSummation

Unimodal Target

•"• Right Aud^— Right Vis.

0.00100 225

Time (msec)

Figure 14. Unimodal cumulative distribution functions (CDFs), bimodal CDFs, the independentrace prediction, and the race model inequality for Observer (Obs.) JZ in Experiment 3. Data fromtrials presenting corresponding targets versus noncorresponding targets are indicated. Becausespatial correspondence had no effect on simple manual responses (see Figure 13), only data forsaccades and directed manual responses are shown. Aud. = auditory; Vis. = visual; Prob. =probability; RT = response time.

Obs. HH

DirectedManualResponses


VI

X

£a.

Saccades

JSO


o.oo125 175 225 275

Time (msec)

Figure 15. Unimodal cumulative distribution functions (CDFs), bimodal CDFs, the independentrace prediction, and the race model inequality for Observer (Obs.) H.H. in Experiment 3. Data fromtrials presenting corresponding targets versus noncorresponding targets are given. Because spatialcorrespondence had no effect on simple manual responses (see Figure 13), only data for saccadesand directed manual responses are shown. Aud. = auditory; Vis. = visual; Prob. = probability; RT= response time.

Obs. JE

DirectedManualResponses


VI

at

o£

Unimodal Targets

----- LeftAud.—— Left Vis.

Saccades

VI

ai

o£


0.00

300

Unimodal Targets••"•• LcftAud.^^— Left Vis.

150 200

Time (msec)

150 200

Time (msec)

250 300

Figure 16. Unimodal cumulative distribution functions (CDFs), bimodal CDFs, the independentrace prediction, and the race model inequality for Observer (Obs.) I.E. in Experiment 3. Data fromtrials presenting corresponding targets versus noncorresponding targets are given. Because spatialcorrespondence had no effect on simple manual responses (see Figure 13), only data for saccadesand directed manual responses are shown. Aud. = auditory; Vis. = visual; Prob. = probability; RT= response time.


dalities is integrated. This integration is presumably impor-tant in generating central representations of environmentalevents that are themselves often multimodal. Although it maybe natural to think that this integration of modality-specificinformation occurs at one site within the nervous system, theavailable neurophysiological evidence shows that this is notthe case. Consider, for example, the integration of visual andauditory information that occurs in the deep layers of thesuperior colliculus and appears important in generating sac-cadic eye movements. This structure is known to receiveinputs from visual, auditory, and somatosensory associationcortex. In addition, polymodal regions of the association cor-tex project to the deep layers (cf. Stein & Meredith, 1990).These latter connections could in principle support a sug-gestion that the colliculus does not itself integrate informa-tion from different modalities but is simply the recipient ofpolymodal information that has already been integrated inpolymodal cerebral cortex.

This suggestion appears to be incorrect. Electrophysiologi-cal studies indicate that all of the cortical afferents (identifiedby antidromic activation) to the deep layers of the colliculusrespond to only one modality (Wallace, Meredith, & Stein,1991). These cortical afferents provide the essential inputs tothe polymodal presaccadic burst cells, because reversiblecortical lesions abolish the responsiveness of polymodal col-licular cells (B. Stein, personal communication, January 10,1992). Thus, although there are areas in the cortex that clearlyintegrate visual and auditory inputs, polymodal cortical cellsdo not provide the source of auditory—visual convergence inthe colliculus. Because polymodal collicular cells receiveonly unimodal inputs, we conclude that the colliculus rep-resents a site of auditory-visual convergence that is distinctfrom that which occurs in the cortex (or other areas of bi-modal convergence within the central nervous system, suchas the reticular formation). It is difficult to imagine the utilityof having multiple sites of auditory-visual convergence un-less the various sites performed different functions. We notein passing one more interesting aspect of the fact that thecolliculus relies on the cortex for its essential inputs: Corticalcircuitry provides ample opportunity for the kinds of stimu-lus selection and gating that played such a conspicuous rolein Experiment 3.

The existence of multiple sites of multisensory integrationsuggests the possibility that the rules governing intersensoryintegration may vary according to the pathways used by aparticular task. In the case of saccades, neural circuitryclearly capable of producing all of the effects reportedherein is known to exist within the superior colliculus. Aswe suggested earlier, these neurons are truly at the interfacebetween sensory and motor processes, so they can be re-garded as premotor neurons as readily as anything else.Bimodal integration underlying the generation of saccadesmay have little to do with bimodal integration underlyingperformance in other tasks and may even have little to dowith achieving multimodal representations in perception it-self. Rather, it may reflect a neural architecture that is de-signed to optimize oculomotor orienting to natural environ-mental events. According to this view, the forms ofmultisensory integration revealed in other tasks (e.g., Fidell,

1970; Miller, 1991) may easily differ because they probablyuse different pathways. One can make a strong case thatmany such tasks involve cortical processes, and polymodalcortical areas have been identified (e.g., Kimura & Tamai,1992; Mistlin & Perrett, 1990; Neal, Pearson, & Powell,1990; Seltzer & Pandya, 1989). There is little to indicate thespecific role they play in sensory integration, however; cer-tainly functional correlates such as those identified in thesuperior colliculus are not yet available.

In conclusion, the present experiments show that saccadesprovide robust evidence of neural summation between au-ditory and visual inputs, evidence that can reasonably beinterpreted in terms of known patterns of auditory-visualconvergence within an important oculomotor structure: thesuperior colliculus. Different neural pathways are assumed tomediate auditory-visual summation effects observed formanual responses, and thus there may be either qualitativeor quantitative differences between intersensory integrationin the oculomotor as opposed to other systems; this is, ofcourse, an empirical issue. The suggestion of the present datathat the magnitudes of the race inequality violations mightdiffer with different response systems is in our opinion,promising, and could indicate differences in the mechanismsthat produce coactivation effects in different circumstances.Similarly, the importance of spatially aligned targets mayreflect important differences in the locus of coactivation ef-fects for different tasks. Future work involving, for example,less extreme cases of spatial misalignment should provideadditional insight on the nature and loci of coactivation ef-fects in these systems and on a range of emerging issues inthe study of coactivation effects in general.

References

Ashby, F. G., & Townsend, J. T. (1986). Varieties of perceptualindependence. Psychological Review, 93, 154-179.

Blake, R., Martens, W., Garrett, G., & Westendorf, D. (1980). Es-timating probability summation for binocular reaction time data.Perception & Psychophysics, 27, 375-378.

Colonius, H. (1990). Possibly dependent probability summation ofreaction time. Journal of Mathematical Psychology, 34, 253-275.

Cox, D. R. (1962). Renewal theory. London: Methuen.Diederich, A., & Colonius, H. (1987). Intersensory facilitation in the

motor component? Psychological Research, 49, 23-29.Dudewicz, E. J. (1976). Introduction to statistics and probability.

New York: Holt, Rinehart & Winston.Eriksen, C. W. (1988). A source of error in attempts to distinguish

coactivation from separate activation in the perception of redun-dant targets. Perception & Psychophysics, 44, 191-193.

Eriksen, C. W., & Schultz, D. W. (1977). Retinal locus and acuityin visual information processing. Bulletin of Psychonomic Soci-ety, 9, 81-84.

Fatt, P., & Katz, P. (1952). Spontaneous subthreshold activity atmotor nerve endings. Journal of Physiology, 117, 109-128.

Fidell, S. (1970). Sensory function in multimodal signal detection.Journal of the Acoustical Society of America, 47, 1009-1015.

Fournier, L. R., & Eriksen, C. W. (1990). Coactivation in the per-ception of redundant targets. Journal of Experimental Psychol-ogy: Human Perception and Performance, 16, 538-550.

Frechet, M. (1951). Sur les tableaux de correlation dont les margessont donnees [On correlation tables for which margins are given].


Annales de I'Universite de Lyon, Section A, Series 3, 14, 53-77.Jay, M. R, & Sparks, D. L. (1987). Sensorimotor integration in the

primate superior colliculus. II. Coordinates of auditory signals.Journal of Neurophysiology, 57, 35-55.

Jay, M. R, & Sparks, D. L. (1990). Localization of auditory andvisual targets for the initiation of saccadic eye movements. In M.A. Berkley & W. Stebbins (Eds.), Comparative perception: Vol.1. Basic mechanisms (pp. 351-374). New York: Wiley.

Kimura, A., & Tamai, Y. (1992). Sensory response of cortical neu-rons in the anterior ectosylvian sulcus, including the area evokingeye movement. Brain Research, 575, 181-186.

Luce, R. D., & Green, D. M. (1972). A neural timing theory forresponse times and the psychophysics of intensity. PsychologicalReview, 79, 14-57.

Meredith, M. A., & Stein, B. E. (1987). Spatial factors determinethe activity of multi-sensory neurons in the cat's superior col-liculus. Brain Research, 420, 162-166.

Miller, J. O. (1982). Divided attention: Evidence for coactivationwith redundant signals. Cognitive Psychology, 14, 247-279.

Miller, J. O. (1986). Time course of coactivation in bimodal dividedattention. Perception & Psychophysics, 40, 331-343.

Miller, J. O. (1991). Channel interaction and the redundant targetseffect in bimodal divided attention. Journal of Experimental Psy-chology: Human Perception and Performance, 17, 160-169.

Miller, J. O., & Lopes, A. (1991). Bias produced by fast guessingin distribution-based tests of race models. Perception & Psycho-physics, 50, 584-590.

Mistlin, A. J., & Perrett, D. I. (1990). Visual and somatosensoryprocessing in the macaque temporal cortex: The role of "expec-tation." Experimental Brain Research, 82, 437-^50.

Mordkoff, J. T., & Yantis, S. (1991). An interactive race model ofdivided attention. Journal of Experimental Psychology, 17, 520-538.

Neal, J. W., Pearson, R. C, & Powell, T. P. (1990). The connectionsof area PG, 7a, with cortex in the parietal, occipital and temporallobes of the monkey. Brain Research, 532, 249-264.

Nozawa, G. (1989). A study of redundant target processing: Evi-dence for parallel processing. Unpublished master's thesis, Pur-due University, West Lafayette, IN.

Peck, C. K. (1987). Auditory interactions in cat's superior collicu-lus: Their role in the control of gaze. Brain Research, 420, 162-166.

Raab, D. (1962). Statistical facilitation of simple reaction times.Transactions of the New York Academy of Science, 24, 574-590.

Rose, D., Blake, R., & Halpern, D. L. (1988). Disparity range forbinocular summation. Investigative Ophthalmology & Visual Sci-ence, 29, 283-290.

Schmidt, R. A., Gielen, S. C. A. M., & van den Heuvel, P. J. M.(1984). The locus of intersensory facilitation of reaction time.Acta Psychologica (Amsterdam), 57, 145-164.

Schwarz, W. (1989). A new model to explain the redundant-signalseffect. Perception & Psychophysics, 46, 498-500.

Seltzer, B., & Pandya, D. N. (1989). Intrinsic connections and ar-chitectonics of the superior temporal sulcus in the rhesus monkey.Journal of Comparative Neurology, 290, 451—471.

Stein, B., & Meredith, M. A. (1990). Multisensory integration. Neu-ral and behavioral solutions for dealing with stimuli from dif-ferent sensory modalities. Annals of the New York Academy ofSciences, 608, 51-65.

Stein, B., & Meredith, M. A., Honeycutt, W. S., & McDade, L.(1989). Behavioral indices of multisensory integration: Orienta-tion to visual cues is affected by auditory stimuli. Journal ofCognitive Neuroscience, 1, 12-24.

Stein, B., Meredith, M. A., & Wallace, M. T. (1993). The visually-responsive neuron and beyond: Multisensory integration in catand monkey. Progress in Brain Research, 95, 79-90.

Townsend, J. T, & Ashby, F. G. (1983). Stochastic modeling ofelementary psychological processes. Cambridge, England: Cam-bridge University Press.

Townsend, J. T, & Nozawa, G. (1992). On the spatio-temporalproperties of elementary perception: An investigation of parallel,serial, and coactivative theories. Manuscript submitted for pub-lication.

Ulrich, R., & Giray, M. (1986). Separate-activation models withvariable base times: Testability and checking of cross-channeldependence. Perception & Psychophysics, 36, 248-259.

van der Heijden, A. H. C., La Heij, W., & Boer, J. P. A. (1983).Parallel processing of redundant targets in simple visual searchtasks. Psychological Research, 45, 235-254.

Wallace, M. T., Meredith, M. A., & Stein, B. E. (1991). Corticalconvergence on multisensory output neurons of cat superior col-liculus. Society for Neuroscience Abstracts, 17, 1379.

Wandell, B. A., & Luce, R. D. (1978). Pooling peripheral infor-mation: Average versus extreme values. Journal of MathematicalPsychology, 17, 220-235.

Westendorf, D., & Blake, R. (1988). Binocular reaction times tocontrast increments. Vision Research, 28, 355-359.

Appendix

The Procedure for Correcting the Latency Distributions for the Occurrence of Fast Guesses

The cumulative distribution function (CDF) for auditory targets,corrected for fast guesses, is given by the equation:

^A.cW = (1 - g) ̂ A(') + 8 G(t),

where FA c(/) represents the CDF corrected for fast guesses, FA(0represents the unconnected auditory CDF, g is the probability of afast guess, and G(t) is the CDF of the fast guesses (which wereestimated from the distribution of false alarms on catch trials). Simi-larly, we have corrected CDFs for the visual and the bimodal targets:Fyc(f) = (1 - g)Fv(/) + gG(t) and FA&Vc« = (1 - g) FA&V

(r) + g G(t)), respectively. In this article, we present many of theresults in terms of the difference between the obtained redundant-

targets CDF and the sum of the unimodal CDFs. Let us refer to thisas the magnitude of the race inequality violation, which is given bythe equation:

Mag = - { FA,g(f) + Fyg(r)},

where the subscript g indicates the obtained CDFs uncorrected forfast guesses.

Substitution yields the corrected magnitude

v(r) + gG(t)

- {0 - 8) FM + g G(t) + (1 - g) Fv(r) + g G(f)},


and simplification gives the equation

FA&v(f) - (F.(f)Magc =

1 ~g

Although direction errors occurred, they were not included inthe present correction procedure, largely because of the lack of aformal model designed to account for their occurrence. As it

turned out, the latencies of the direction errors were often longerthan the latencies for correct responses, so they do not appear tohave been fast guesses per se.

Received February 20, 1992Revision received April 5, 1993

Accepted April 15, 1993

Six Editors Appointed, 1995-2000

The Publications and Communications Board of the American Psychological Associationannounces the appointment of six new editors for 6-year terms beginning in 1995. As ofJanuary 1, 1994, manuscripts should be directed as follows:

• For the Journal of Abnormal Psychology, submit manuscripts to Milton E. Strauss,PhD, Department of Psychology, Case Western Reserve University, 10900 EuclidAvenue, Cleveland, Ohio 44106-7123.

• For the Journal of Applied Psychology, submit manuscripts to Philip Bobko, PhD,Editor, Journal of Applied Psychology, P.O. Box 130, Skillman, New Jersey08558.

• For the Journal of Comparative Psychology, submit manuscripts to Charles T.Snowdon, PhD, Department of Psychology, University of Wisconsin, 1202 WestJohnson Street, Madison, Wisconsin 53706-1696.

• For the Attitudes and Social Cognition section of the Journal of Personality andSocial Psychology, submit manuscripts to Arie W. Kruglanski, PhD, Departmentof Psychology, University of Maryland, College Park, Maryland 20742.

• For Professional Psychology: Research and Practice, submit manuscripts toPatrick H. DeLeon, PhD, JD, Editor, Professional Psychology: Research andPractice, APA, Room 3084, 750 First Street, NE, Washington, DC 20002-4242.

• For Psychological Review, submit manuscripts to Robert A. Bjork, PhD, Psycho-logical Review, Department of Psychology, University of C alifornia, Los Angeles,California 90024-1563.

Manuscript submission patterns make the precise date of completion of 1994 volumesuncertain. The current editors, Susan Mineka, PhD; Neal Schmitt, PhD; Gordon G. Gallup,PhD; Abraham Tesser, PhD; Ursula Delworth, PhD; and Walter Kintsch, PhD, respectively,will receive and consider manuscripts until December 31,1993. Should any 1994 volumesbe completed before that date, manuscripts will be redirected to the new editors forconsideration in 1995 volumes.

Documents

Copyright 1994 by the American Psychological Association ......Saccades Versus Manual Responses Howard C. Hughes, Patricia A. Reuter-Lorenz, George Nozawa, and Robert Fendrich Reaction