r Human Brain Mapping 000:000–000 (2009) r

The Effect of Presentation Paradigm on SyntacticProcessing: An Event-Related fMRI Study

Donghoon Lee and Sharlene D. Newman*

Department of Psychological and Brain Sciences, Indiana University, Bloomington, Indiana

r r

Abstract: An event-related fMRI study was conducted to investigate the effect of two different sen-tence presentation paradigms—rapid serial visual presentation (RSVP) and whole sentence presenta-tion—on syntactic processing. During scanning, sentences were presented using one of the twopresentation paradigms and were followed by a short delay and a probe to verify sentence comprehen-sion. The delay was included in an attempt to separate sentence-related activity from probe-relatedactivity. The behavioral data showed a main effect of syntactic complexity for reaction time and accu-racy, and accuracy revealed an interaction between complexity and the presentation paradigmemployed—RSVP produced many more errors for syntactically complex sentences than did whole sen-tence presentation. The imaging data revealed a syntactic complexity effect during the sentence phasein left BA 44 and during the probe phase in left BA 44 and the left posterior MTG. In addition, time-course analysis revealed that these two regions also showed an interaction between complexity andpresentation paradigm such that there was no complexity effect during RSVP but a significant effectduring whole sentence presentation. In addition to finding that these two presentation paradigms dif-ferentially affected syntactic processing, there were main effects within the visual pathway (V1/V2 vs.V5) and the hippocampus that revealed significant differences in activation between the paradigms.Hum Brain Mapp 00:000–000, 2009. VC 2009 Wiley-Liss, Inc.

Keywords: RSVP; event-related fMRI; sentence comprehension; syntactic complexity

r r

INTRODUCTION

In recent neuroimaging studies, the syntactic complexityeffect has been localized to distinct brain areas such as theleft inferior frontal gyrus (LIFG), particularly BA 44, andthe left posterior superior/middle temporal gyrus (S/MTG) [e.g., Just et al., 1996b] and it appears to be inde-

pendent of input modality [Constable et al., 2004; Michaelet al., 2001]. Although the activation of both regions hasbeen found to be modulated by syntactic complexity, thefunctional roles assigned to each have been quite different.For example, Caplan and Waters [1999] have suggestedthat the posterior region of LIFG (Broca’s area) is special-ized for syntactic working memory processes necessary forsyntactic analysis of sentences. There have been a numberof studies presented by Caplan et al., as well as others,that have provided supportive evidence of this claim[Caplan et al., 2008; Fiebach et al., 2005; Hashimoto andSakai, 2002]. Although Wernicke’s area has traditionallybeen regarded as a semantic processing center, many stud-ies have found its activation is modulated by syntacticcomplexity [Michael et al., 2001; Newman et al., in press].In an attempt to reconcile these two ideas, a number ofhypotheses have been generated. For example, it has beensuggested that a more specific function of the posterior

Contract grant sponsors: Indiana METACyt Initiative of IndianaUniversity, Lilly Endowment, Inc.

*Correspondence to: Sharlene D. Newman, Department of Psycho-logical and Brain Sciences, Indiana University, Bloomington, IN47405. E-mail: sdnewman@indiana.edu

Received for publication 23 January 2009; Revised 27 May 2009;Accepted 28 May 2009

DOI: 10.1002/hbm.20845Published online in Wiley InterScience (www.interscience.wiley.com).

VC 2009 Wiley-Liss, Inc.

MTG is in processing phrase-level meaning [Cutting et al.,2006; Grossman et al., 2002; Vigneau et al., 2006] and inte-gration of lexico-syntactic information [see Grodzinskyand Friederici, 2006]. In this vein, syntactic manipulationsnot only affect syntactic processing but also semantic levelprocessing that explains the involvement of Wernicke’sarea.

Although many studies have found the involvement ofthe left IFG and S/MTG during syntactic processing tasks,there has been some inconsistency in the findings acrossstudies. One possible source of such inconsistency is in thevariation in the sentence presentation paradigms usedacross studies; ranging from whole sentence presentationto rapid serial visual presentation (RSVP) to a movingwindow presentation paradigm. It may very well be thatthese different presentation paradigms require differentcognitive demands which may interact with syntactic proc-essing. For example, Cooke et al. [2002] examined theeffect of syntactic complexity using serial visual presenta-tion and found an effect of complexity in the inferior fron-tal gyrus, but not in temporal cortex. Keller et al. [2001]also presented sentences one word at a time, but using amoving window paradigm, and found no syntactic effectsfor sentences containing high-frequency words in eitherthe inferior frontal gyrus or temporal cortex. In a studyusing whole sentence presentation, syntactic complexityeffects were observed in both the inferior frontal gyrusand temporal cortex [Michael et al., 2001]. Although thesestudies used different presentation paradigms, they maynot be easily comparable because of possible importantmethodological or stimulus differences. However, studiesfrom the same lab using the same stimulus materials alsoshow different patterns. For example, in an fMRI studyusing RSVP, Caplan et al. [2002] failed to observe asyntactic complexity effect in either the LIFG or the poste-rior S/MTG but instead found an effect in the right angu-lar gyrus. However, in earlier studies using the samesentence materials but different sentence presentationparadigms—auditory [Caplan et al., 1999] and whole sen-tence [Caplan et al., 1998]—an effect in the LIFG wasobserved. Caplan [2001] explained that the lack of a syn-tactic complexity effect in the LIFG was due to the cogni-tive demands associated with RSVP. Although thesediscrepancies are present in the literature, to our knowl-edge, there is no neuroimaging study that examineswhether different presentation paradigms have a differen-tial effect on syntactic processing; therefore, the currentstudy attempts to fill this gap.

RSVP, initially developed by Forster [1970], has been astaple of psycholinguistic research because it provides forcontrol of stimulus presentation time [Potter, 1984]. InRSVP reading, each word is serially presented in the cen-ter of the screen and the speed of reading is under theexperimenter’s control. Because all the words are pre-sented in the same location, eye movements are mini-mized. It has been demonstrated that with the use ofRSVP reading speed can be accelerated without severe

decrements in comprehension. For this reason, it has beenconsidered an educational tool for speed reading and as apotential technique to present text messages in small elec-tric devices such as cell phones, PDAs, and even wristwatches [Chien and Chen, 2007; Goldstein et al., 2003;Muter, 1996]. In addition, it is used as a reading aid forthose with visual limitations such as patients with bilateralmacular disease [Beccue and Vila, 2004; Rubin and Tur-ano, 1992]. RSVP has also led to the development of vari-ous experimental reading methods such as the self-pacedreading and the moving window presentation paradigm[Just et al., 1982; Kieras and Just, 1984].

Although RSVP was originally developed for use inbehavioral studies, it has been adopted for use in event-related potential (ERP) studies because it makes it possibleto time lock to a particular word within a sentence [Dit-man et al., 2007; Hagoort and Brown, 2000; King andKutas, 1995; Kutas and Hillyard, 1984; Neville et al., 1991].Recent neuroimaging studies have also employed this par-adigm for language comprehension and sometimes itallows for synchronization between ERP and fMRI data[Caplan et al., 2002; Cooke et al., 2002; Cutting et al., 2006;Fiebach et al., 2005; Hashimoto and Sakai, 2002; Kiehlet al., 2002]. Thus, RSVP has a number of advantagesforresearch as well as commercial and educationalpurposes.

Beyond its practical uses, a critical question is whetherunderlying cognitive processing (e.g., language and mem-ory processing) during RSVP is similar to whole sentencepresentation. The physical characteristics of the two arequite different in that gaze time on each word is not underthe reader’s control and regression is not allowed in RSVP.Because of these differences, RSVP reading can be uncom-fortable and may increase cognitive demands such as atten-tion and working memory. Nevertheless, RSVP is believedto not be very different from whole sentence presentation,at least in terms of language processing (for a review, seePotter [1984]). However, the evidence that has been used tosupport this idea is based on the comprehension of short,simple sentences. During the processing of these sentencesno severe decrements in comprehension have beenobserved, not even at much faster presentation rates (e.g., 12words per second (wps)) than normal reading speed (e.g. 6wps), [Beccue and Vila, 2004; Juola et al., 1982; Potter, 1984;Rubin and Turano, 1992; Ward and Juola, 1982].

Although no comprehension decrements have beenobserved for short sentences, this is not necessarily truefor longer reading materials such as passages. Potter et al.[1980] compared RSVP and whole sentence/text presenta-tion for a paragraph with the approximately same totalpresentation time, using rates of 4, 8, or 12 wps. At a slowpresentation rate, 4 wps, they reported that the percentageof idea units recalled in the first half of the paragraph inthe RSVP condition was significantly lower in the wholesentence/text presentation condition. They also examinedthe effect of topic sentence on memory by placing thetopic sentence at the beginning, middle, or end of the

r Lee and Newman r

r 2 r

paragraph. Interestingly, recall rates during RSVP weresignificantly affected by this factor, but it was not a factorduring whole sentence/text reading. When a topic sen-tence was placed at the beginning, better performance wasobserved in the RSVP condition, although performancewas still worse than that observed in whole sentence/textpresentation. Also, Rusell and Chaparro [2001] presented18 passages using RSVP at 4 wps and 11 wps with variousfont sizes and found reading comprehension at 11 wpswas significantly lowered. Recently, similar results werefound with Chinese languages [Chien and Chen, 2007; Linand Shieh, 2006]. Although Potter [1984] suggested thatthe decrement for RSVP for longer passages is due to lackof memory consolidation or deeper meaning processes[Potter, 1984; pp. 110–111], a comprehensive explanationhas not been presented.

In addition to the length of a text, linguistic complexity(e.g., syntactic complexity) of reading materials also affectsRSVP reading and it has been found to interact with read-ing speed. In fact, initial findings by Forster [1970]revealed this effect. Linguistically complex sentences (suchas passive sentences, sentences with an embedded clause)produced higher recall errors compared to simple activesentences when the number of words was equated; andthe recall errors were worse at faster presentation rates[Forster, 1970; Forster and Olbrei, 1972; French, 1981;Holmes and Forster, 1972]. This finding has importantimplications for studies examining syntactic complexity.As a matter of fact, syntactically complex sentences typi-cally increase online processing demands including work-ing memory [Caplan and Waters, 1999; Just et al., 1996a,b].Although the characteristic of the working memorydemand is still under debate (domain-general vs. domain-specific [Caplan and Waters, 1999; Fedorenko et al., 2006;Fiebach et al., 2001, 2005; Just et al., 1996a,b; King andKutas, 1995; MacDonald et al., 1992; Waters and Caplan,1996], the fact that syntactically complex sentences requiremore computational resources as does RSVP may suggestthat RSVP may interfere with syntactic processing.

In the current event-related fMRI study, we aimed toexplore the effect that the presentation paradigm employedhas on syntactic processing. To accomplish this goal, con-joined active and object-relative sentences were comparedand two presentation paradigms were examined—RSVPusing a moderate reading speed (2.5 wps) and whole sen-tence presentation. Each sentence was followed by a true/false comprehension probe. All probes were presentedusing the whole sentence presentation paradigm and theyall were simple active sentences. To examine the activationrelated to sentence processing separately from that associ-ated with the comprehension probe a 6-s delay was placedbetween the sentence and probe. On the basis of previousliterature, our focus was on two regions in particular, BA 44and the posterior S/MTG. Both of these regions have beenimplicated in syntactic processing, and the goal here was todetermine whether their activation is modulated by the dif-fering demands of the presentation mode.

METHODS

Participants

Twenty participants took part in the experiment. Theywere all Indiana University students without any historyof neurological disorders. Before scanning, all participantsgave written informed consent that was approved by theIndiana University Institutional Review Board. Data fromtwo participants were discarded due to poor behavioralperformance during the fMRI session. Data from 18 partic-ipants (9 male, 9 female, age ¼ 22 � 1.87) were used forthe current data analysis. In a training session, participantswere also introduced to the sentence comprehension taskand underwent several practice trials to familiarize themwith the experimental procedure. After the experiment,they completed a debriefing questionnaire.

Experimental Design

The current fMRI experiment used a single trial event-related design in which each trial was treated as an eventblock [Postle et al., 2000; Zarahn, 2000; Zarahn et al., 1997].A trial consisted of a sentence, a 6-s delay, and a compre-hension probe. The study was a 2 � 2 design withsentence presentation mode and syntactic complexity aswithin-subject factors. Two presentation modes; the wholesentence presentation and RSVP were used to display sen-tence materials. Sentences were presented for 5 s in theirentirety in the middle of the screen for whole sentencepresentation. During RSVP, sentences were presented oneword at a time in the middle of the screen; each sentencewas presented in 5 s with an average presentation rate of2.4 wps. This presentation rate was slightly slower thannormal reading speed. The second factor, syntactic com-plexity, was manipulated by using two sentence types,conjoined active and object-relative sentences (see later).Sentence materials were adapted from Keller et al. [2001]in which all the sentences consisted of 12 words, and fre-quency and length of words in sentences were equatedacross conditions. There were 80 sentences presented (40conjoined active sentences and 40 object-relative sentences)and half of sentences in each type were presented usingeach of the two presentation modes.

� Conjoined activeSentence: The pilot scared the escort and broke the

mirror on the closet.Probe: The pilot broke the mirror.

� Object-relativeSentence: The pilot that the escort scared broke the

mirror on the closet.Probe: The pilot broke the mirror.

To verify reading comprehension of the sentence, a shortprobe followed the sentence after a 6-s delay. The probe sen-tence had a simple SVO structure with one verb and two

r Sentence Comprehension r

r 3 r

nouns, all taken from the sentence and the probe describedone of the events in the sentence. One third of the probeswere false (e.g. ‘‘the escort broke the mirror’’). Probes werealways displayed using the whole sentence presentationparadigm regardless of sentence presentation; along withthe probe response cue (F | T) was presented in the middleof screen. Participants were to press a button with their leftindex finger to denote false and a button with the rightindex finger to denote true. So, neither syntactic complexitynor presentation mode was manipulated in the comprehen-sion probe.

One of the important elements in the current fMRI designis the 6-s delay. The 6-s delay was inserted to obtain separateestimations of the blood oxygen level dependent (BOLD)response from the sentence reading phase and the probephase, somewhat separately. Because we manipulated syn-tactic complexity and presentation mode in only the senten-ces and not the probe, we wanted to investigate these effectson the brain activation during sentence reading separatelyfrom the effects of responding to the comprehension probe.Thus, we tested two time delays (3 and 6 s) in a couple ofpilot studies and verified that the 6-s delay allowed for thebetter separation of the hemodynamic response peaks foreach phase (you can see this activation pattern in Fig. 4).With the inserted delay, the present experimental designbecame similar to the fMRI design for a delayed matchingtask [e.g. Postle et al., 2000; Zarahn et al., 1997]. Zarahn et al.[1997] described an fMRI data analysis procedure for esti-mating separate hemodynamic responses for each compo-nent in a trial. Thus, the current fMRI data analysis followedthe method in Zarahn et al. [1997]. After the probe, a 12-srest denoted by an asterisk (*) was inserted between trials toallow the BOLD signal to return to the baseline. In Figure 1,the experimental design is summarized.

The fMRI scan consisted of four functional sessions with20 trials (five trials in four conditions) presented in a pseu-dorandom order. In each session, three 28-s fixation peri-ods were included in the beginning, middle, and end ofthe session to estimate a baseline BOLD signal. To remindparticipants how to perform the task, two additional prac-tice trials were inserted in the beginning of the first ses-sion, but the scans from the practice trials were discardedfrom the data analysis.

fMRI Data Acquisition and Analysis

fMRI scanning was conducted on a 3T Siemens TRIOscanner with an 8-channel radio frequency head coil locatedin the Imaging Research Facility at Indiana University. Func-tional images were obtained in eighteen oblique axial sliceswith 5 mm thickness and a 1 mm gap (TR ¼ 1,000 ms, TE ¼25 ms, flip angle ¼ 60�, matrix size ¼ 64 � 64, FOV ¼ 240 �240 mm2) using a gradient echo planar imaging (EPI)sequence. Before statistical analysis, for all of the functionalimages, slice timing correction, head motion correction byrealignment, and spatial normalization were conductedusing the SPM99 software (Wellcome Department of Imag-ing Neuroscience; http://www.fil. ion.ucl.ac.uk/spm). Inthe spatial normalization step, all of the functional imageswere warped directly to the Montreal Neurological Institute(MNI) EPI template and resampled to the 2 � 2 � 2 voxeldimensions. The spatially normalized images were enteredfor the statistical analysis based on the general linear model(GLM) and the Gaussian random field theory, which wereimplemented in the SPM package [Friston et al., 1995]. Zar-ahn et al.’s [1997] method was applied for the statistical anal-ysis. A regressor was built for the sentence reading, thedelay, and the probe phase, respectively. Each regressor wasconstructed by convolving a canonical hemodynamicresponse function (HRF) with a stimulus delta function. Thestimulus delta function was created with stimulus onsettime and duration. For the sentence reading, the beginningtime of each sentence presentation was set for the onset andthe total presentation time, 5 s, was entered for the duration.A brief event HRF was setup in the middle of the delay forthe delay period activation. For the probe phase, the onset ofprobes and 5 s were entered for duration because the probewas also presented for 5 s. Six realignment parameters andone additional regressor for incorrect response trials, if therewere any, were entered in the model to remove contaminat-ing effects from head motion and careless responses. There-fore, only correct trials were used in the analysis.

Individual activation maps for the sentence reading andprobe phases in each condition were achieved by contrast-ing each phase to the fixation condition. On the basis ofthose contrasted images, in the second level, conjunctionanalysis and within participant ANOVAs were performed.

Figure 1.

A single event-related fMRI design and examples of sentences. Slash(/) represents the unit

of RSVP.

r Lee and Newman r

r 4 r

The conjunction analysis was conducted to examine the com-mon activation for sentence reading in both presentationmodes [Friston et al., 1999, 2005; Nichols et al., 2005]. Themain effects of syntactic complexity and presentation and theinteraction between the two factors were examined for thesentence reading and the probe phase. The main effect of pre-sentation was examined using a corrected threshold of P <0.05 using family wise error correction. To examine the syntac-tic complexity effect, the entire brain was searched using anuncorrected threshold of P < 0.001 with 25 voxel extentthreshold. Because the interaction is a relatively small effect,the entire brain was not searched. Instead the syntactic com-plexity maps were used as an inclusive mask (the maskthreshold was an uncorrected P < 0.0001). Because the clustersize found from this analysis was small, a region of interest(ROI) analysis was performed. For the ROI analysis, a 10 mmradius, spherical ROI whose center was the activation peakfrom the conjunction analysis map for sentence reading wasconstructed. The two ROIs of primary interest were the LIFGand LMTG and we ensured that they were included in theROIs generated. Then, timecourse data were extracted fromindividual datasets by using the Marsbar toolbox [Brett et al.,2002]. Individual timecourse data were truncated by each trialand sorted by conditions and averaged across trials in for eachcondition. Timecourse data from incorrect trials wereexcluded. The averaged timecourse data at each time pointwas converted into a percent signal change (PSC) value usingthe formula, (signal � baseline/baseline) � 100, where thebaseline was the mean signal of the fixation period. Finally,the PSC was baseline corrected. The timecourse analysis wasalso applied to other regions in which the main effect ofpresentation was found (see Fig. 6).

RESULTS

Behavioral Results

Behavioral responses to comprehension probes were col-lected during the scan. The average response time (RT) for

correct trials (incorrect trials were excluded for the RTanalysis) was used to calculate the RT, and the number oferrors was used to calculate accuracy. The data were ana-lyzed using a 2 � 2 (presentation mode � syntactic com-plexity) within participant ANOVA for RT and accuracy.As for RT, a main effect of syntactic complexity [F(1,17) ¼22.99, P < 0.0001], was found but neither a main effect ofpresentation nor an interaction was found. The main effectof syntactic complexity indicated that the RT was signifi-cantly faster for conjoined active than the object relativesentences (see Fig. 2). However, when examining accuracy,the main effect of syntactic complexity [F(1,17) ¼ 11.83, P< 0.003], the main effect of presentation mode [F(1,17) ¼37.7 P < 0.0001], and the interaction between the two fac-tors [F(1,17) ¼ 7.52, P < 0.014] were all found to be signifi-cant. The RSVP condition resulted in more errors onaverage and showed a larger syntactic complexity effectthan the whole sentence condition, see Figure 2. In fact,most of participants stated in their debriefings that RSVPreading was more difficult compared to whole sentencereading.

fMRI Results

The current fMRI experimental design allowed us toestimate BOLD responses for the sentence reading phaseand those for the comprehension probe phase separately(see Methods). We first examined the syntactic complex-ity effect during each phase as well as how this effect ismodulated by presentation mode, with a focus on BA 44and posterior S/MTG. Generally, we found a syntacticcomplexity effect and an interaction during the sentencephase in left BA 44 (see Table I and Fig. 3). We alsofound a similar effect during the probe phase in left pos-terior S/MTG (see Table II and Fig. 4). We then exam-ined the common and differential activation during thesentence reading phase for the two presentation para-digms. There were regions showing differential activationbetween presentation modes (Table III and Fig. 5) as well

Figure 2.

Behavioral responses to comprehension probes (error bars represent standard errors).

r Sentence Comprehension r

r 5 r

as a number of regions that revealed common activationduring sentence reading for both presentation modes (Ta-ble IV and Fig. 6). Below, we summarized each of theseeffects in detail.

Syntactic complexity effects during

the sentence reading phase

During the sentence reading phase, the only region toshow a significant effect of syntactic complexity was theleft inferior frontal gyrus (LIFG). The region also revealedan interaction between syntactic complexity and presenta-tion mode (Table I and Fig. 3A). To take a closer look atthe processing taking place in this region, the percent sig-nal change (PSC) values were retrieved for this region (seeFig. 3B). The bar graph in Figure 3 presents PSC values atthe activation peak (peaked at �48, 16, 30 in MNI coordi-nates) and Figure 4 presents the timecourse in the ROIacross conditions. These figures reveal that the lack of syn-tactic complexity effect in RSVP reading was due to rela-tively more activation in the conjoined active condition forRSVP compared to whole sentence presentation. Thisinteraction can be observed clearly when the simple main

Figure 3.

Syntactic complexity effects in the sentence reading phase. (A) Main effect of syntactic complex-

ity. (B) Interaction: more syntactic effect during whole sentence presentation than RSVP in the

IFG. (C) Simple main effect of syntactic complexity during whole sentence presentation. (D) Sim-

ple main effect of syntactic complexity (no effect) during RSVP.

TABLE I. Main effect of syntactic complexity (object-

relative minus conjoined active) and interaction

between presentation mode and syntactic complexity in

the sentence reading phase (uncorrected P < 0.001)

Area R/L Size t-value Coordinate

Main effect of syntactic complexityInferior frontal gyrus (BA44) L 139 4.77 �42, 4, 28

Interaction(More syntacticeffect in whole)Inferior frontal gyrus (BA44) L 8 2.46 �48, 16, 30

r Lee and Newman r

r 6 r

effect of syntactic complexity is calculated in each presen-tation mode (compare Fig. 3C,D). Although the left inferiorfrontal regions (BA 44 and BA45) showed the syntacticcomplexity effect during whole sentence reading, noregion showed a significant effect in RSVP.

Syntactic complexity effects during thecomprehension probe phase

All the comprehension probes were constructed using asimple SVO structure and presented using the whole sen-tence mode. As a result, all of the effects observed duringthe probe phase presumably resulted from factors manipu-lated in the sentence phase (i.e., complexity and presenta-

tion mode). Here, there was no main effect of presentationmode, but there were significant syntactic complexityeffects and a significant interaction with presentationmode (Table II) in many regions. The main effect of syn-tactic complexity during the probe phase was found at theLIFG (BA 44) again, but it was also found in other regionsincluding the left precentral gyrus, supplementary motorarea, bilateral inferior parietal lobules, the left posteriorMTG, precuneus, and right cerebellum (Table II andFig. 5A).

An interaction was observed in the left posterior middletemporal gyrus and was due to a larger syntactic effect forwhole sentence presentation compared to RSVP (Fig. 5D).The syntactic complexity effect was only significant for

TABLE II. Main effect of syntactic complexity and interaction between presentation mode

and syntactic complexity in the probe phase (uncorrected P < 0.001)

Area R/L Size t-value Coordinate

Main effect of syntactic complexityInferior frontal gyrus (BA45) L 103 4.69 �46, 30, �6

R 82 4.62 46, 15, 34Precentral gyrus (including BA44) L 1458 6.51 �38, 2, 60Insula L 133 4.65 �32, 20, �14Supplementary motor area L/R 313 5.51 �2, 8, 60Posterior middle temporal gyrus L 199 5.11 �62, �34, 2Inferior parietal lobule L 771 5.61 �28, �56, 40

R 207 4.58 38, �58, 38Precuneus (including superior parietal lobule) L/R 289 5.39 0, �70, 42Cerebellum R 212 6.55 8, �74, �30

93 4.70 40, �64, �36Interaction I (more syntactic effect in whole)Posterior middle temporal gyrus L 42 3.01 �58, �44, 2

Interaction II (More syntactic effect in RSVP)Superior parietal lobule R 20 2.83 �24, �74, 52Cerebellum R 24 3.16 36, �60, �32

Figure 4.

The extracted timecourses for each of the conditions for the two regions of interest, BA44 and

MTG. [Color figure can be viewed in the online issue, which is available at www.interscience.

wiley.com.]

r Sentence Comprehension r

r 7 r

whole sentence presentation but not for RSVP (compareFig. 5B,C). For further investigation, the averaged PSC val-ues were extracted from a spherical ROI in this region(peaked at �58, �44, 2) and plotted in Figure 5E and inFigure 4. This plot revealed the relatively greater activa-tion for the conjoined active sentences after RSVP reading,which was a very similar pattern observed in the LIFGduring reading (see Figs. 3B and 4)

Other regions such the right precuneus and the rightcerebellum also revealed an interaction, but the interac-tion was due to a larger syntactic complexity effect for

RSVP compared to whole sentence presentation (seeTable II).

Presentation mode effects during the

sentence reading phase

Presentation mode effects were revealed by subtractingestimated hemodynamic responses for RSVP from thoseassociated with whole sentence presentation and viceversa (Table III). One region that showed a critical differ-ence between the presentation paradigms was the hippo-campus. The disparate response in this region suggeststhere is differential memory processing associated withthese two presentation paradigms. The bilateral hippo-campi revealed little activation during RSVP but wasstrongly activated during whole sentence reading. In addi-tion, the hippocampus was also activated during theprobe phase in which probe statements were presentedusing the whole sentence presentation mode (Fig. 6B).Although early visual cortex (V1/V2) showed more acti-vation for whole sentence than RSVP reading, bilateralMT (V5) was uniquely activated during RSVP (Fig. 6A,C).The activation in MT (V5) indicated that the rapidly pre-sented words in the same position engaged motion-relatedprocesses [He et al., 1998]. The differential activation inthese two visual processing areas appears to be due to the

TABLE III. Presentation mode effects in the sentence

reading phase (FWE corrected P < 0.05)

Area R/L Size t-value Coordinate

Whole-RSVPHippocampus L 220 22.66 �22, �30, �4

R 209 17.17 22, �32, 4Calcarine sulcus (V1/V2) L/R 11085 20.77 4, �68, 6Superior parietal lobule R 123 8.79 22, �62, 54

RSVP-wholeMiddle temporal gyrus (V5) R 749 12.81 44, �70, 0

L 170 8.97 �46, �74, �2Superior temporal Gyrus R 43 7.30 56, �38, 20

Figure 5.

Syntactic complexity effects in the probe phase. (A) main effects of syntactic complexity. (B)

Simple main effects of syntactic complexity during whole sentence presentation. (C) Simple main

effects of syntactic complexity during RSVP. (D) Interaction (E) averaged time course data in the

left posterior MTG which showed the interaction effect.

r Lee and Newman r

r 8 r

different visual stimulus properties or differences in theamount of visual processing required for the presentationparadigms.

Common activation for sentence reading during

whole sentence and RSVP reading

Conjunction analysis revealed that most of languageregions were commonly activated during sentence reading(Table IV). The commonly activated regions included leftdominant language areas such as the inferior frontal andprecentral regions, the left anterior/posterior middle tem-poral region, and the inferior temporal region includingthe fusiform gyrus. The supplementary motor area, pri-mary visual cortex, left putamen, insula, and right angulargyrus were also involved during both presentation para-digms (see Fig. 7).

TABLE IV. Common activation during sentence

reading in the whole and RSVP modes (FWE

corrected P < 0.05)

Area R/L Size z-value Coordinate

Inferior frontal gyrus (BA45/47) L 70 7.22 �34, 28, �4Inferior frontal gyrus (BA44) L 64 6.48 �48, 16, 24Precentral gyrus L 6 5.56 �46, �2, 54

L 7 5.51 �46, �10, 58L 6 5.13 �42, �4, 56

Insula R 13 5.73 38, 18, 0Supplementary motor area L 118 6.53 �2, 0, 60

R 10 5.41 8, 10, 48Anterior middle temporal gyrus L 10 5.76 �56, �12, �14Posterior middle temporal gyrus L 66 7 �56, �44, 4Inferior temporal gyrus L 12 5.9 �40, �48, �18Angular gyrus R 7 5.53 28, �64, 46Inferior occipital gyrus L 217 6.85 �30, �90, �10Calcarine sulcus R 165 6.48 20, �98, �6Putamen L 103 6.16 �18, 8, 0

Figure 6.

Presentation effects in the sentence reading phase. Right: presentation effects overlaid in an axial

slice (z ¼ �2) of the canonical brain. Left: time course of percent signal change (PSC) in (A) the

right MT(V5) (x ¼ 44, y ¼ �70, z ¼ 0), (B) the left hippocampus (x ¼ �22, y ¼ �30, z ¼ �4),

(C) calcarine sulcus (V1) (x ¼ 4, y ¼ �68, z ¼ 6).

r Sentence Comprehension r

r 9 r

DISCUSSION

The current fMRI study aimed to determine whethersyntactic processing was modulated by the sentence pre-sentation paradigm used. Behaviorally, an expected syn-tactic complexity effect was found in response time andaccuracy to comprehension probes for both RSVP andwhole sentence presentation. However, the interactionobserved in the accuracy data indicated that RSVP elicitedmore errors, particularly for the syntactically complex sen-tences. Differential syntactic complexity effects were foundbetween the two presentation paradigms. The syntacticcomplexity effect in Broca’s area was only significant dur-ing whole sentence presentation. A similar interaction wasobserved in Wernicke’s area when processing the probes.Conversely, a larger syntactic effect was found in the pre-cuneus and right cerebellum when responding to the com-prehension probe after RSVP, seemingly corresponding tothe higher error rates elicited by this presentation para-digm. In addition, the imaging data revealed bothcommon and differential activation for RSVP and wholesentence presentation. Both presentation paradigms eli-cited activation in the left dominant language areas includ-ing Broca’s and Wernicke’s areas. However, differentialactivation was found in the hippocampus and the visualpathway. For example, the hippocampus revealed very lit-tle involvement during RSVP but was significantly acti-vated during whole sentence presentation.

Sentence Processing Phase

Most psycholinguistic research has agreed that languageprocessing is fast and autonomous. The common activa-tion across these two presentation paradigms in languageprocessing areas such as Broca’s and Wernicke’s area sup-port this claim. However, RSVP and whole sentencepresentation appears to differentially affect syntactic proc-esses. Syntactic complexity interacted with the sentencepresentation paradigm in several regions either during thereading of the sentence or when answering the compre-hension probes. The interaction during sentence reading

occurred in the LIFG. This region has been strongly associ-ated with syntactic processing, and it has consistentlyshowed syntactic complexity effects in many neuroimag-ing studies [Caplan et al., 1998, 1999; Caplan and Waters,1999; Carpenter et al., 1999; Cooke et al., 2006; Fiebach etal., 2001, 2005; Friederici et al., 2006; Just et al., 1996b; Kel-ler et al., 2001; Newman et al., 2003]. In particular, theregion has been thought to be involved in the processingof word order information and is more activated when theword order within a sentence is not the typical order (e.g.,SVO in English) [Grodzinsky and Friederici, 2006; Greweet al., 2005]. Here, we observed the syntactic complexityeffect in the LIFG during whole sentence reading but notduring RSVP. The lack of a syntactic complexity effectduring RSVP was consistent with some previous results[Caplan et al., 2002; Cooke et al., 2002]. When the signalchange measures were retrieved from the region, the sig-nal for the conjoined active sentences during RSVP waslarger than the signal in the whole sentence condition.This suggests that the computational demands associatedwith RSVP may have interfered with the readers’ ability totake full computational advantage of the simpler syntacticstructure. One possible explanation is that because the sen-tence is presented serially during RSVP, the words andtheir order must be maintained in a buffer. Therefore, theserial order processing linked with RSVP may interferewith the word order processing required during the proc-essing of syntactically complex (e.g., sentences not in thetypical SVO word order) sentences.

As implied earlier, one possible source of the increasedcomputational demand associated with RSVP is memoryprocessing. Almost 30 years ago, Potter et al. [1980]observed memory decrements when the stimuli was pre-sented using RSVP (i.e., a substantial drop in recall and aninteraction with the location of the topic within a para-graph). She later proposed that there may be an impair-ment of memory consolidation during RSVP resulting inthe meaning of the stimuli not being deeply processed[Potter, 1984]. However, to our knowledge, there is nosubsequent research on this topic. This study in some wayreplicated the interaction with memory in that here we

Figure 7.

Common activation between the RSVP and the whole sentence presentation.

r Lee and Newman r

r 10 r

found an interaction between presentation paradigm andsyntactic complexity. In addition, differential hippocampalactivation for RSVP and whole sentence presentation wasobserved such that the whole sentence condition elicited alarger response from the hippocampus than did RSVP.The hippocampus has long been implicated in memoryconsolidation and episodic memory formation [Eichen-baum, 2004; McGaugh, 2000; Squire and Alvarez, 1995;Tulving and Markowitsch, 1998]. Eichenbaum [2004]defined three elemental cognitive processes linked to thehippocampus: associative representation, sequential orga-nization, and relational networking. Davachi and Wagner[2002] found that the hippocampus was engaged duringrelational and item-based learning, and it seemed to beinvolved in the relational binding of items into an inte-grated memory system. It may be that this binding processis limited during RSVP. Because the words are presentedserially, requiring the use of a computationally, resourcedemanding internal working memory buffer to performthe relational binding process, this binding process may bemore difficult to perform or in this case may not be per-formed at all during RSVP. During whole sentence presen-tation, on the other hand, the display is an external buffer,freeing up resources to allow for this binding process tooccur. Thus, the lack of hippocampal activation in RSVPmay coincide with an impairment of memory consolida-tion and may result in poor performance, particularly forthe more computationally demanding object-relativeconstructions.

All together it seems that RSVP is more computation-ally, resource demanding and interferes with syntacticprocessing. Again, BA 44 is thought to be involved inprocessing linear word order so the word order processingrequired for RSVP may be interfering with the word orderprocessing required to comprehend more syntacticallycomplex sentences. This syntax related order processingmay interact with the binding processes associated withthe hippocampus that is required to obtain the deepersemantic meaning of the sentence. Therefore, if the syntax-based word ordering is not performed, then no binding isperformed and no activation of the hippocampus iselicited. Although this explanation fits the current data, itis speculative and requires further research to validate it.

Thus far, we have argued that the lack of a syntacticcomplexity effect observed during RSVP in Broca’s area isdue to an interaction with memory processing. However,there is an alternative explanation. Both of the critical sen-tence types used are locally ambiguous: at the position of‘‘that,’’ the relative clause constructions could continue ei-ther as an object relative clause (‘‘The pilot that the escortscared broke the mirror on the closet,’’ as in this study) oras a subject relative clause (‘‘The pilot that scared theescort and broke the mirror on the closet.’’), whereas, atthe position of ‘‘and,’’ the conjoined actives permit either aVP-coordination (‘‘The pilot scared the escort and brokethe mirror on the closet,’’ as in this study), an NP-coordi-nation (‘‘The pilot scared the escort and the admiral.’’) or

S-coordination (‘‘The pilot scared the escort and the admi-ral raised the alarm.’’). In both sentence types, the ambigu-ous region (‘‘that’’ or ‘‘and’’) is a short word and,therefore, relatively unlikely to be fixated during naturalreading [cf. Rayner, 1998]. Hence, the disambiguating in-formation, which is provided by the immediately follow-ing word, will have been available to the reader muchmore rapidly during whole sentence presentation thanduring RSVP. As a result, RSVP may have enhanced theeffects of local ambiguity during the comprehension pro-cess and elicited precisely the activation pattern observedfor the LIFG during the reading phase: a general, ambigu-ity-related activation increase for both sentence types incomparison to whole sentence reading. For whole sentencereading, by contrast, there is no (or only very little) ambi-guity, allowing the activation pattern to reflect the generalcomplexity difference between object relative and con-joined active sentences. Furthermore, this alternative ex-planation is consistent with previous reports of ambiguity-related activation in the LIFG [e.g. Bornkessel et al., 2005;Fiebach et al., 2004; Stowe et al., 2004].

Probe Processing Phase

Interestingly, the syntactic complexity effect wasobserved, and more widespread, during the processing ofthe probes. This effect is quite interesting because the probestatements were constructed with a simple SVO structureso the effect must be the result of reprocessing the sentencevia the mental representation that was generated duringsentence reading. This syntactic reanalysis taking place dur-ing the probe appears to not be affected by how the senten-ces were presented because the effect was consistent forboth presentation paradigms. In a recent study, we reportedthat syntactic complexity effects during the probe phaseinteracted with probe type [Newman et al., in press]. There,probe sentences all had the same structure, but the distance(in the preceding sentence) between the verb and noun wasmanipulated. For example, for a conjoined active sentencesuch as ‘‘the pilot scared the escort and broke the mirror onthe closet,’’ true probe sentences could be either ‘‘the pilotscared the escort’’ or ‘‘the pilot broke the mirror.’’ It wasfound that the first type of probe was easier to answer andthe brain activity in several regions including LIFG revealedlower activation levels compared to the second probe type.Although the distance was not manipulated here, the find-ing of syntactic complexity effects off-line (during theprobe) in language processing regions has been replicatedhere and provides further support for the idea that Broca’sarea is indeed associated with syntactic processing, bothduring sentence reading as well as when responding to thecomprehension probe.

While Broca’s area revealed a consistent effect of com-plexity during the probe for both sentence presentationmodes, other regions revealed an interaction between com-plexity and presentation during the probe phase. For

r Sentence Comprehension r

r 11 r

example, the left posterior MTG revealed a complexityeffect during whole sentence presentation but not duringRSVP. This interaction pattern during the probe phase inthe MTG was similar to the interaction observed in theLIFG during the sentence reading phase—a complexityeffect for whole sentence but not RSVP. Actually, the cur-rent activation patterns in the left posterior MTG is veryinteresting for several reasons. First, this region is com-monly activated for both RSVP and whole sentence presen-tation indicating this region is essential for sentenceprocessing. Second, the syntactic complexity effect in thisregion was not found during the reading phase (i.e. onlinesentence processing) but was observed during the probephase (i.e., off-line processing). This supports the claimmade by Caplan [2001] in which he argues that the syntacticcomplexity effect in Wernicke’s area is not due to the onlinesyntactic processing but may be due to off-line taskdemands. Third, the syntactic complexity effect in the probephase was only significant when sentences were presentedusing the whole sentence presentation paradigm. Accord-ing to recent neuroimaging studies, the posterior temporalregion is involved in the integration of lexical and syntacticinformation and structured knowledge such as propositio-nal or thematic concepts [Grodzinsky and Friederici, 2006;Grossman et al., 2002; Hart and Kraut, 2007; Kable et al.,2005]. On the basis of this, we may consider again the syn-tactic complexity effect observed in this region. The com-prehension probes used here queried the thematicknowledge structure of the previous sentence (e.g., who didwhat to whom). To answer the probes, the mental represen-tation of the previously read sentence should be retrieved.Given that complexity effects are observed during theprobe, this suggests that the representation generated con-tains not just sentence meaning information but also struc-tural information that may make determining thosethematic relationships a more demanding process in object-relative compared to conjoined active sentences. However,this was not the case when sentences were presented usingRSVP; there was no complexity effect when processing theprobe in temporal cortex. As stated earlier, because of thenature of RSVP it may have impeded both the word orderprocessing required during non-canonical structures in BA44, but also the binding or integration processes thought tobe associated with the hippocampus. This may mean thatresponding to the probe was based almost completely onsentence word order information instead of the ‘‘deeper’’meaning of the sentence. If the sentence word order is beingused to respond to the probe, then more errors would beexpected for the object-relative compared to the conjoinedactive sentences because the conjoined active sentences canbe answered based solely on word order.

Limitations of Current Research

and Implications for Future Research

This study would be the first neuroimaging study tocompare RSVP and whole sentence presentation para-

digms. The results indicate a critical difference betweenthe two presentation methods in terms of memory process-ing and how they interact with syntactic complexity.Although the study does account for the lack of a syntacticcomplexity effect observed in some studies, it does appearto conflict with others that do show a complexity effectusing RSVP [e.g., Cooke et al., 2002]. However, when onelooks more closely at these studies the presentation para-digm used is just one methodological difference acrossthese studies. For example, the Cooke study required ajudgment regarding the gender of the agent and allowedparticipants to respond as soon as they knew the answercausing the termination of the trial and the advancementto the next trial. Therefore, it is likely that many partici-pants did not read the entire sentence. To obtain a clearpicture of the processing taking place during sentenceprocessing and to compare across studies, it will be neces-sary to understand how all of these methodological differ-ences interact with processing.

The study provides important messages for bothresearch and commercial applications of RSVP, there aresome limitations of the study. One such limitation is withthe use of the 6-s delay between sentence and probe. Thiswas done so that we could separate the sentence-relatedand probe-related activation. Although we were able to dothat we also introduced a possible confound. It may bethat the processing taking place during sentence reading isaffected by the need to retain the sentence in memory for6 s before being presented with the comprehension probe.This additional memory processing may have contami-nated the results. However, given that the need to retainthe information is the same for both RSVP and whole sen-tence presentation it is our claim that it would not have amajor effect when examining condition effect sizedifferences.

Another limitation to this study is that we only exam-ined two sentence presentation paradigms that are rou-tinely used in psycholinguistic research. There are otherssuch as the moving window paradigm in which stimuliare presented serially but in the space to the right of theprevious word instead of the same location as the previousword. There it would be expected that the working mem-ory load would be less than with RSVP due to the abilityto make regressions back to the location of a previouslypresented word. But it is unclear how this paradigm, orothers such as phrase by phrase instead of word by wordpresentation, may interact with syntactic complexity. Fur-ther study is needed to help provide a more completecharacterization of the influence that the presentation para-digm employed has on language processing.

A final limitation of the study is related to the hippo-campus activation—the hippocampus lies incredibly closeto the LGN (lateral geniculate nucleus), part of the visualpathway. The activity of the LGN has been shown to bemodulated by saccadic eye movements [Reppas et al.,2002; Thilo et al., 2004]. Given that RSVP limits eye move-ments and whole sentence presentation allows for both

r Lee and Newman r

r 12 r

forward and regressive eye movements, activation differ-ences in the LGN are possible. That being said, the coordi-nates for the activation maxima (and the ROI center) areclearly within the hippocampus. However, we cannot ruleout the possibility that the activation attributed to thehippocampus did not extend into the LGN.

The results presented here may have several importantimplications on research and commercial usage of RSVP.First, examining more closely how the memory loadengendered by RSVP affects language processing moregenerally requires further investigation. RSVP has beenfrequently used in many research areas and has also beenconsidered for use in commercial products. If it has a seri-ous impact on memory, it must be very carefully consid-ered before being commercially used. Second, while RSVPand whole sentence presentation showed common activa-tion in the typical language areas, the current results alsoshowed that they interacted with syntactic complexity indifferent ways. One major implication is that the syntacticcomplexity effect could be reduced when using RSVP.This is important for psycholinguistic research, particu-larly for ERP and fMRI studies that use this presentationparadigm routinely. In sum, although RSVP is a conven-ient tool for research and holds additional benefits (e.g.fast reading), the advantages and disadvantages of usingRSVP should be carefully evaluated.

ACKNOWLEDGMENTS

We would like to thank Thomas Burns, Tara Muratore,Kristen Ratliff, Andrea Sampson, and Benjamin Pruce forall of their help with data collection. We would also liketo thank those who have provided wonderful insight onprevious versions of this manuscript.

REFERENCES

Beccue B, Vila J (2004): Assessing the impact of rapid serial visualpresentation (RSVP): A reading technique Adv DistributedSyst. vol. 3061, Springer Berlin: Heidelberg, 42–53.

Bornkessel I, Zysset S, Friederici AD, von Cramon Y, SchlesewskyM (2005): Who did what to whom? The neural basis of argu-ment hierarchies during language comprehension. NeuroImage26:221–233.

Brett M, Anton JL, Valabregue R, Poline JB (2002): Region of inter-est analysis using an SPM toolbox (Abstract). Presented at the8th International Conferance on Functional Mapping of theHuman Brain, Sendai, Japan.

Caplan D (2001): Functional neuroimaging studies of syntacticprocessing. J Psycholing Res 30:297–320.

Caplan D, Waters GS (1999): Verbal working memory and sentencecomprehension. Behav Brain Sci 22:77–94; discussion 95–126.

Caplan D, Alpert N, Waters G (1998): Effects of syntactic structureand propositional number on patterns of regional cerebralblood flow. J Cogn Neurosci 10:541–552.

Caplan D, Alpert N, Waters G (1999): PET studies of syntacticprocessing with auditory sentence presentation. Neuroimage9:343–351.

Caplan D, Vijayan S, Kuperberg G, West C, Waters G, Greve D,Dale AM (2002): Vascular responses to syntactic processing:Event-related fMRI study of relative clauses. Hum Brain Mapp15:26–38.

Caplan D, Chen E, Waters G (2008): Task-dependent and task-independent neurovascular responses to syntactic processing.Cortex 44:257–275.

Carpenter PA, Just MA, Keller TA, Eddy WF, Thulborn KR (1999):Time course of fMRI-activation in language and spatial net-works during sentence comprehension. Neuroimage 10:216–224.

Chien Y-H, Chen C-H (2007): The use of dynamic display toimprove reading comprehension for the small screen of a wristwatch. In: Human Interface and the Management of Informa-tion. Methods, Techniques and Tools in Information Design.vol. 4557, Springer Berlin: Heidelberg, pp 814–823.

Constable RT, Pugh KR, Berroya E, Mencl WE, Westerveld M,Ni W, Shankweiler D (2004): Sentence complexity and inputmodality effects in sentence comprehension: An fMRI study.NeuroImage 22:11–21.

Cooke A, Zurif EB, DeVita C, Alsop D, Koenig P, Detre J, Gee J,Pinango M, Balogh J, Grossman M (2002): Neural basis for sen-tence comprehension: Grammatical and short-term memory com-ponents. Hum Brain Mapp 15:80–94.

Cooke A, Grossman M, DeVita C, Gonzalez-Atavales J, Moore P,Chen W, Gee J, Detre J (2006): Large-scale neural network forsentence processing. Brain and Language 96:14–36.

Cutting LE, Clements AM, Courtney S, Rimrodt SL, Schafer JGB,Bisesi J, Pekar JJ, Pugh KR (2006): Differential components ofsentence comprehension: Beyond single word reading andmemory. NeuroImage 29:429–438.

Davachi L, Wagner AD (2002): Hippocampal contributions toepisodic encoding: Insights from relational and item-basedlearning. J Neurophysiol 88:982–990.

Ditman T, Holcomb PJ, Kuperberg GR (2007): An investigation ofconcurrent ERP and self-paced reading methodologies. Psycho-physiology 44:927–935.

Eichenbaum H (2004): Hippocampus: Cognitive processes andneural representations that underlie declarative memory. Neu-ron 44:109–120.

Fedorenko E, Gibson E, Rohde D (2006): The nature of workingmemory capacity in sentence comprehension: Evidence againstdomain-specific working memory resources. J Mem Lang54:541–553.

Fiebach CJ, Schlesewsky M, Friederici AD (2001): Syntacticworking memory and the establishment of filler-gap depend-encies: Insights from ERPs and fMRI. J Psycholinguist Res30:321–338.

Fiebach CJ, Schlesewsky M, Bornkessel I, Friederici AD (2004):Distinct neural correlates of legal and illegal word order varia-tions in German: How can fMRI inform cognitive models ofsentence processing. In: Carreiras V, Clifton C Jr, editors. TheOn-Line Study of Sentence Comprehension. New York: Psy-chology Press. pp 357–370.

Fiebach CJ, Schlesewsky M, Lohmann G, von Cramon DY, Frie-derici AD (2005): Revisiting the role of Broca’s area in sentenceprocessing: Syntactic integration versus syntactic workingmemory. Hum Brain Mapp 24:79–91.

Forster KI (1970): Visual perception of rapidly presented wordsequences of varying complexity. Perception Psychophys8:215–221.

Forster KI, Olbrei I (1972): Semantic heuristics and syntactic analy-sis. Cognition 2:319–347.

r Sentence Comprehension r

r 13 r

French P (1981): Semantic and syntactic factors in the perceptionof rapidly presented sentences. J Psycholing Res 10:581–591.

Friederici AD, Fiebach CJ, Schlesewsky M, Bornkessel ID, vonCramon DY (2006): Processing linguistic complexity andgrammaticality in the left frontal cortex. Cereb Cortex 16:1709–1717.

Friston KJ, Frith CD, Turner R, Frackowiak RS (1995): Characteriz-ing evoked hemodynamics with fMRI. Neuroimage 2:157–165.

Friston KJ, Holmes AP, Price CJ, Buchel C, Worsley KJ (1999):Multisubject fMRI studies and conjunction analyses. Neuro-image 10:385–396.

Friston KJ, Penny WD, Glaser DE (2005): Conjunction revisited.Neuroimage 25:661–667.

Goldstein M, Oquist G, Bjork S (2003):Evaluating sonified rapidserial visual presentation: An immersive reading experience ona mobile device. In: Universal Access Theoretical Perspectives,Practice, and Experience. vol. 2615, Springer Berlin: Heidel-berg, pp 508–523.

Grewe T, Bornkessel I, Zysset S, Wiese R, von Cramon DY, Schle-sewsky M (2005): The emergence of the unmarked: A newperspective on the language-specific function of Broca’s area.Hum Brain Mapp 26:178–190.

Grodzinsky Y, Friederici AD (2006): Neuroimaging of syntax andsyntactic processing. Curr Opin Neurobiol 16:240–246.

Grossman M, Koenig P, DeVita C, Glosser G, Alsop D, Detre J,Gee J (2002): Neural representation of verb meaning: An fMRIstudy. Hum Brain Mapp 15:124–134.

Hagoort P, Brown CM (2000): ERP effects of listening to speechcompared to reading: The P600/SPS to syntactic violations inspoken sentences and rapid serial visual presentation. Neuro-psychologia 38:1531–1549.

Hart J, Kraut M (2007): Neural Basis of Semantic Memory.Cambridge: Cambridge University Press.

Hashimoto R, Sakai KL (2002): Specialization in the left prefrontalcortex for sentence comprehension. Neuron 35:589–597.

He S, Cohen ER, Hu X (1998): Close correlation between activityin brain area MT/V5 and the perception of a visual motionaftereffect. Curr Biol 8:1215–1218.

Holmes VM, Forster KI (1972): Perceptual complexity and under-lying sentence structure. J Verbal Learn Verbal Behav 11:148–156.

Juola JF, Ward NJ, McNamara T (1982): Visual search and readingof rapid serial presentations of letter strings, words, and text.J Exp Psychol Gen 111:208–227.

Just MA, Carpenter PA, Woolley JD (1982): Paradigms and proc-esses in reading comprehension. J Exp Psychol Gen 111:228–238.

Just MA, Carpenter PA, Keller TA (1996a): The capacity theory ofcomprehension: new frontiers of evidence and arguments.Psychol Rev 103:773–780.

Just MA, Carpenter PA, Keller TA, Eddy WF, Thulborn KR(1996b): Brain activation modulated by sentence comprehen-sion. Science 274:114–116.

Kable JW, Kan IP, Wilson A, Thompson-Schill SL, Chatterjee A(2005): Conceptual representations of action in the lateraltemporal cortex. J Cogn Neurosci 17:1855–1870.

Keller TA, Carpenter PA, Just MA (2001): The neural bases of sen-tence comprehension: A fMRI examination of syntactic andlexical processing. Cereb Cortex 11:223–237.

Kiehl KA, Laurens KR, Liddle PF (2002): Reading anomalous sen-tences: An event-related fMRI study of semantic processing.NeuroImage 17:842–850.

Kieras DE, Just MA (1984): New Methods in Reading Comprehen-sion Research. Hillsdale, N.J.: L. Erlbaum Associates.

King JW, Kutas M (1995): Who did what and when? Using word-and clause-level erps to monitor working memory usage inreading. J Cogn Neurosci 7:376–395.

Kutas M, Hillyard SA (1984): Brain potentials during readingreflect word expectancy and semantic association. Nature307:161–163.

Lin Y-C, Shieh K-K (2006): Reading a dynamic presentation ofChinese text on a single-line display. Displays 27:145–152.

MacDonald MC, Just MA, Carpenter PA (1992): Working memoryconstraints on the processing of syntactic ambiguity. Cogn Psy-chol 24:56–98.

McGaugh JL (2000): Memory—A century of consolidation. Science287:248–251.

Michael EB, Keller TA, Carpenter PA, Just MA (2001): fMRI investiga-tion of sentence comprehension by eye and by ear: Modalityfingerprints on cognitive processes. Hum Brain Mapp 13:239–252.

Muter P (1996): Interface design and optimization of reading ofcontinuous text. In: van Oostendorp H, de Mul A, editors.Cognitive aspects of electronic text processing. Norwood, NJ:Ablex. pp 161–180.

Neville H, Nicol JL, Barss A, Forster KI, Garrett MF (1991): Syn-tactically based sentence processing classes: Evidence fromevent-related brain potentials. J Cogn Neurosci 3:151–165.

Newman SD, Just MA, Keller TA, Roth J, Carpenter PA (2003):Differential effects of syntactic and semantic processing on thesubregions of Broca’s area. Brain Res Cogn Brain Res 16:297–307.

Newman SD, Lee D, Ratliff KL: Off-line sentence processing:What is involved in answering a comprehension probe? HumBrain Mapp (in press).

Nichols T, Brett M, Andersson J, Wager T, Poline J-B (2005): Validconjunction inference with the minimum statistic. NeuroImage25:653–660.

Postle BR, Zarahn E, D’Esposito M (2000): Using event-relatedfMRI to assess delay-period activity during performance ofspatial and nonspatial working memory tasks. Brain Res Proto-cols 5:57–66.

Potter MC (1984): Rapid serial visual presentation (RSVP): Amethod for studying language processing. In: Kieras DE, JustMA, editors. New Methods in Reading ComprehensionResearch. Hillsdale, NJ: L. Erlbaum Associates. pp 91–118.

Potter MC, Kroll JF, Harris C (1980): Comprehension and memoryin rapid sequential reading. In: Nickerson R, editor. Attentionand Performance, Vol. VIII. Hillsdale, NJ: Erlbaum. pp 395–418.

Rayner K (1998): Eye movements in reading and informationprocessing: 20 years of research. Psychol Bull 124:372–422.

Reppas JB, Usrey WM, Reid RC (2002): Saccadic eye movementsmodulate visual responses in the lateral geniculate nucleus.Neuron 35:961–974.

Rubin GS, Turano K (1992): Reading without saccadic eye move-ments. Vision Res 32:895–902.

Rusell MC, Chaparro BS (2001): Exploring effects of speed andfont size with RSVP. Paper Presented at Human Factors andErgonomics Society 45th Annual Meeting, Minneapolis,Minnesota.

Squire LR, Alvarez P (1995): Retrograde amnesia and memoryconsolidation: A neurobiological perspective. Curr Opin Neu-robiol 5:169–177.

r Lee and Newman r

r 14 r

Stowe LA, Paans AMJ, Wijers AA, Zwarts F (2004): Activations of‘‘motor’’ and other non-language structures during sentencecomprehension. Brain Lang 89:290–299.

Thilo KV, Santoro L, Walsh V, Blakemore C (2004): The site ofsaccadic suppression. Nat Neurosci 7:13–14.

Tulving E, Markowitsch HJ (1998): Episodic and declarativememory: Role of the hippocampus. Hippocampus 8:198–204.

Vigneau M, Beaucousin V, Herv PY, Duffau H, Crivello F, HoudO, Mazoyer B, Tzourio-Mazoyer N (2006): Meta-analyzing lefthemisphere language areas: Phonology, semantics, and sen-tence processing. NeuroImage 30:1414–1432.

Ward NJ, Juola JF (1982): Reading with and without eye move-ments: Reply to Just, Carpenter, and Woolley. J Exp PsycholGen 111:239–241.

Waters GS, Caplan D (1996): The capacity theory of sentence com-prehension: Critique of Just and Carpenter (1992): Psychol Rev103:761–772.

Zarahn E (2000): Testing for neural responses during temporalcomponents of trials with BOLD fMRI. NeuroImage 11:783–796.

Zarahn E, Aguirre G, D’Esposito M (1997): A trial-based experi-mental design for fMRI. NeuroImage 6:122–138.

r Sentence Comprehension r

r 15 r