NeuroImage 77 (2013) 279–287

Contents lists available at SciVerse ScienceDirect

NeuroImage

j ourna l homepage: www.e lsev ie r .com/ locate /yn img

Activations of human auditory cortex to phonemic and nonphonemic vowels duringdiscrimination and memory tasks

Kirsi Harinen a, Teemu Rinne a,b,⁎a Institute of Behavioural Sciences, University of Helsinki, Finlandb Advanced Magnetic Imaging Centre, Aalto University School of Science, Finland

⁎ Corresponding author at: Institute of BehaviouralUniversity of Helsinki, Finland.

E-mail address: teemu.rinne@helsinki.fi (T. Rinne).

1053-8119/$ – see front matter © 2013 Elsevier Inc. Allhttp://dx.doi.org/10.1016/j.neuroimage.2013.03.064

a b s t r a c t

a r t i c l e i n f o

Article history:Accepted 23 March 2013Available online 6 April 2013

Keywords:Auditory cortexCategorical processingfMRIInferior parietal lobuleSpeech

We used fMRI to investigate activations within human auditory cortex (AC) to vowels during vowel discrim-ination, vowel (categorical n-back) memory, and visual tasks. Based on our previous studies, we hypothe-sized that the vowel discrimination task would be associated with increased activations in the anteriorsuperior temporal gyrus (STG), while the vowel memory task would enhance activations in the posteriorSTG and inferior parietal lobule (IPL). In particular, we tested the hypothesis that activations in the IPL duringvowel memory tasks are associated with categorical processing. Namely, activations due to categorical pro-cessing should be higher during tasks performed on nonphonemic (hard to categorize) than on phonemic(easy to categorize) vowels.As expected, we found distinct activation patterns during vowel discrimination and vowel memory tasks. Fur-ther, these task-dependent activations were different during tasks performed on phonemic or nonphonemicvowels. However, activations in the IPL associated with the vowel memory task were not stronger duringnonphonemic than phonemic vowel blocks. Together these results demonstrate that activations in human ACto vowels depend on both the requirements of the behavioral task and the phonemic status of the vowels.

© 2013 Elsevier Inc. All rights reserved.

Introduction

Human auditory cortex (AC) is believed to have an important rolein processing speech-related information (Benson et al., 2001; Binderet al., 2000; Chang et al., 2010; Dehaene-Lambertz et al., 2005;Friederici, 2011; Hickok, 2009; Jäncke et al., 2002; Liebenthal et al.,2005; Obleser et al., 2006; Raizada and Poldrack, 2007; Rauscheckerand Scott, 2009; Scott and Johnsrude, 2003; Weinberger, 2011;Woods et al., 2011). It is generally assumed that primary regions inand near Heschl's gyrus (HG) process physical features of soundswhereas regions in the anterior and posterior superior temporalgyrus/sulcus (STG/STS) are important for more speech-specific analy-sis. In addition to physical and speech-specific features of sounds, ACactivations are also strongly affected by the characteristics of thetasks performed during the presentation of speech and nonspeechsounds (Angenstein et al., 2012; Harinen et al., in press; Hickok andPoeppel, 2000, 2007; Hickok and Saberi, 2012; Husain et al., 2006;Leung and Alain, 2011; Liebenthal et al., 2005; Petkov et al., 2004).However, the effects of different listening tasks on AC activationshave not been studied systematically.

In a recent fMRI study, we compared AC activations to similar spa-tially varying sounds during spatial discrimination and spatial memory

Sciences, PO Box 9, FI-00014,

rights reserved.

tasks (Rinne et al., 2012). During amemory task, subjectswere requiredto indicate when a sound belonged to the same spatial category (left,middle or right) as the one presented 1–3 trials (depending on the dif-ficulty level) before. During a discrimination task, subjects indicatedwhen two halves of a sound had the same spatial location. We foundthat activations in the anterior AC increased during spatial discrimina-tion but not during spatial memory, while activations in the posteriorsuperior temporal gyrus (STG) and inferior parietal lobule (IPL) in-creased during spatial memory but not during spatial discrimination.These task-dependent activation patterns were almost identical tothose observed in our previous study comparing AC activations duringpitch discrimination and pitch memory tasks with spatially fixed stim-uli (Rinne et al., 2009).

In the present study we investigated AC activations during discrim-ination and memory tasks performed on phonemic or nonphonemicvowels. We hypothesized that, as several previous studies havereported results suggesting that speech is processed in specialized sys-tems in the AC (e.g., Binder et al., 2000; Chang et al., 2010; Desai et al.,2008; Hickok and Poeppel, 2007; Jäncke et al., 2002; Liebenthal et al.,2005; Obleser et al., 2006; Rauschecker and Scott, 2009; Scott andJohnsrude, 2003), activation patterns during vowel tasks could differfrom those observed in our previous studies using pitch and spatialtasks. In particular, the present study tested the possibility that the ac-tivations in the posterior STG and IPL associatedwith the pitch and spa-tial memory tasks of our previous studies are due to categoricalprocessing required in these tasks. Previous studies have shown that

280 K. Harinen, T. Rinne / NeuroImage 77 (2013) 279–287

activations in these areas increase during tasks requiring categoricalprocessing, during adverse listening conditions, and during demandingtasks (Alain et al., 2010; Chang et al., 2010; Dehaene-Lambertz et al.,2005; Desai et al., 2008; Husain et al., 2006; Leung and Alain, 2011;Obleser et al., 2007; Raizada and Poldrack, 2007; Rinne et al., 2009,2012; Sabri et al., 2008; Sharp et al., 2010; Turkeltaub and Coslett,2010). Thus, we hypothesized that activations in the posterior STGand IPL associated with categorical processing should be higher duringvowel memory tasks performed on nonphonemic (hard to categorize)than on phonemic (easy to categorize) vowels.

Our subjects were concurrently presented with auditory (vowels)and visual (Gabor gratings) stimulus streams in 15 s blocks alternat-ing with 8 s breaks with no stimuli. The auditory stream consisted ofwithin-category vowel pairs (two 200 ms vowels separated by a100 ms gap; pair onset-to-onset interval 900–1100 ms) from threephonemic (Finnish phonemes /u/, /a/ and /i/) or nonphonemic cate-gories in all conditions. Each category contained 9 different vowels(Fig. 1a). In the visual task (b), subjects were required to detectGabor orientation changes and to ignore the task-irrelevant phone-mic or nonphonemic vowels. In discrimination tasks (c), subjectswere required to indicate when the first and the second parts of thevowel pair were the same. In n-back memory tasks (d), subjects indi-cated when the vowel pair belonged to the same vowel category asthe one presented 1, 2 or 3 trials (depending on the difficulty level;3-back memory task was performed only on phonemic vowel pairs)before. Both discrimination and memory tasks were performed eitheron phonemic or nonphonemic vowel pairs in separate blocks. We

a

c

Fig. 1. In 15 s blocks (alternating with 8 s rest with no stimuli), subjects were presented wi900–1100 ms) from three Finnish phonemic or three nonphonemic vowel categories and Gacategories were defined based on typical Finnish /i/, /u/ and /a/ phonemes. The nonphonemtotypical Finnish phonemes exist. Each category contained 9 different vowels. In the visualcrimination task (c), they were required to indicate when the first and the second parts of twhen the vowel pair belonged to the same vowel category as the one, presented 1, 2 or 3 tscale on the horizontal axis is schematic.

expected that activations to vowels presented during the visual task(i.e., in the absence of an auditory task) would reveal stimulus depen-dent processing of vowels and that distinct task-dependent activationpatterns would be observed during auditory tasks performed on ei-ther phonemic or nonphonemic vowels.

Materials and methods

Subjects

Subjects (N = 22, 13 women) were right-handed, 20–41 years ofage (mean 24 years), had normal hearing (self-reported), normal orcorrected-to-normal vision, and no history of psychiatric or neurolog-ical illnesses. An informed written consent was obtained from eachsubject before the experiment. The study protocol was approved bythe Ethical Committee of the Hospital District of Helsinki andUusimaa, Finland. All subjects were native Finnish speakers. Datafrom 6 subjects were rejected due to low performance in the de-manding auditory tasks (mean d′ in auditory tasks b1.7).

Stimuli

Vowels (duration 200 ms including a linear 5 ms onset and offsetramp) were synthesized using the Praat software package (version5.1.12, www.praat.org). The vowels formed three phonemic andthree nonphonemic categories with nine vowels in each (Fig. 1a).The phonemic categories were defined based on typical Finnish /a/,

b

d

th vowel pairs (two 200 ms vowels separated by a 100 ms gap, onset-to-onset intervalbor gratings (duration 100 ms, onset-to-onset interval 300–500 ms). (a) The phonemicic categories (N1, N2 and N3) were organized in regions of F1–F2 space where no pro-task (b), subjects were required to detect Gabor orientation changes. In the vowel dis-he vowel pair were the same. In the n-back vowel memory task (d), subjects indicatedrials (depending on the difficulty level) before (2-back task is illustrated). In b–d, time

281K. Harinen, T. Rinne / NeuroImage 77 (2013) 279–287

/i/ and /u/ phonemes. The nonphonemic categories (N1, N2 and N3)were organized in regions of F1–F2 space where no prototypical Finn-ish phonemes exist. These categories were then evaluated by 3 expe-rienced listeners to make sure that the phonemic categories weresystematically associated with the phonemic vowel and that thenonphonemic categories were not systematically associated withone Finnish phoneme. Within a category, the vowels were separatedby at least 60 mel in F1–F2 space. In the /i/-category, the F1 and F2frequencies of the vowel in the center of the category were 254 and2569 Hz, respectively. The F1 and F2 frequencies of the center vowelsof the other categories were as follows: /u/ 300 and 600 Hz, /a/ 753and 1100 Hz, N1 260 and 1000 Hz, N2 570 and 1100 Hz, N3 646and 2425 Hz. The other formants (F3–F7) were fixed for all vowelsas follows: F3 3010 Hz, F4 3300 Hz, F5 3850 Hz, F6 4850 Hz, and F75850 Hz. A linear falling contour from 150 to 100 Hz was used for F0.

During fMRI, subjects were presented with concurrent streams ofvowel pairs and visual stimuli. The vowel pairs consisted of 200 msvowels separated by a 100 ms gap (900–1100 ms onset-to-onset in-terval). In the discrimination tasks the vowels in a pair were eitherthe same or separated by approximately 60 or 120 mel. In then-back tasks the vowels in a vowel pair were always the same. Phone-mic and nonphonemic vowel pairs were presented in separate blocks.Visual stimuli consisted of fourteen Gabor gratings (duration 100 ms,orientation −60–60°, 300–500 ms onset-to-onset interval).

The auditory stimuli were delivered binaurally with an UNIDESADU2a audio system (UNIDES Design, Helsinki, Finland) via plastictubes through a porous EAR-tip (ER3, Etymotic Research) acting asan earphone. The noise of the scanner was attenuated by the ear-plugs, circumaural ear protectors (Bilsom Mach 1, SNR 23 dB) andviscoelastic mattresses inside the body coil, around the head coil,and under the subject. The visual stimuli were presented in the mid-dle of a screen viewed through a mirror fixed to the head coil.

Tasks

There were three different tasks. In the visual task, subjects wererequired to detect orientation changes in the Gabor gratings(Fig. 1b). Activations during the visual task (i.e., in the absence of au-ditory attention) were analyzed to investigate stimulus-dependentprocessing of vowels. Visual task was also used as a baseline conditionto reveal general effects of the auditory tasks. In vowel discriminationtasks, subjects were required to indicate by pressing a button withtheir right index finger when the first and the second parts of thevowel pair were the same (Fig. 1c). In vowel memory tasks, subjectsindicated when the vowel pair belonged to the same vowel categoryas the one presented 1–3 trials before depending on the difficultylevel (Fig. 1d). Sound sequences presented during the visual taskswere identical to the ones presented during vowel discrimination orvowel memory tasks.

The stimuli were presented in 15 s blocks alternating with 8 sbreaks with no stimuli. During the breaks, subjects focused on a fixa-tion mark (x) presented in the middle of a screen and waited for thenext task. A graphic task instruction symbol replaced the fixationmark 3 s before the onset of the next task block and remained onthe screen until the end of the block. The order of the task conditionswas randomized. Each task block contained 2–4 targets. In the voweldiscrimination, vowel n-back memory and visual tasks there were 32,80 and 16 blocks, respectively, so that 128 blocks were presented intotal.

Pre-fMRI training

As in our previous studies (Rinne et al., 2009, 2012), the voweldiscrimination and vowel memory tasks were intentionally madevery demanding in order to reveal activations associated with these

tasks. Before fMRI, each subject was carefully trained (60–90 min)to perform the tasks.

Analysis of the behavioral performance

Mean hit rates (HRs), reaction times (RTs), false alarm rates(FaRs), and d′ were calculated separately for each condition. Re-sponses occurring between 200 and 1300 ms from the onset of thetarget stimulus (in the discrimination tasks response window startedfrom the onset of the second part of the vowel pair) were accepted ashits. Other responses (i.e., extra responses after a hit or responsesoutside the response window) were considered as false alarms. HRwas defined as the number of hits divided by the number of targets.FaR was defined as the number of false alarms divided by the numberof nontargets. HRs and FaRs were used to compute the d′ [index ofstimulus detectability, d′ = z(HR) − z(FaR)]. RTs were calculatedonly for hits. Behavioral results were analyzed using repeated mea-sures ANOVAs and t-tests.

fMRI data acquisition and analysis

fMRI imaging was performed on a 3.0 T GE Signa system retrofittedwith an Advanced NMR operating console and a 16-channel head coil.Functional images were acquired using a gradient-echo echo-planar(GE-EPI) sequence (TR 2048 ms, TE 32 ms, flip angle 90°, voxel matrix96 × 96, FOV 20 cm, slice thickness 2.1 mmwith no gap, in-plane res-olution 2.1 mm × 2.1 mm, number of slices 24). The middle EPI sliceswere aligned along the Sylvian fissures to cover the superior temporallobe, insula, and most of the inferior parietal lobe in both hemispheres(see Fig. 2 of Rinne et al., 2012).

The functional scanning was performed in one 50 min run resultingin circa 1470 volumes. After the functional scans, a fluid-attenuated in-version recovery image using the same imaging slices but with denserin-plane resolution was acquired (FLAIR; TR 10,000 ms, TE 120 ms,voxel matrix 320 × 192, FOV 20 cm, slice thickness 2.1 mm). Finally,at the end of the session, high-resolution anatomical images were ac-quired (voxel matrix 256 × 256, FOV 25.6 cm, resolution 1.0 mm ×1.0 mm × 1.0 mm).

Global voxel-wise analysis was performed using FSL (release 4.1,www.fmrib.ox.ac.uk/fsl). The first six volumes were excluded fromdata analysis to allow the initial stabilization of the fMRI signal. Thedata was motion-corrected, high-pass filtered (cutoff 100 s), and spa-tially smoothed (Gaussian kernel of 5 mm full-width half-maximum).First-level statistical analysis was carried out using FMRIB's improvedlinear model (FILM). Based on the timing information recorded dur-ing the experiment, each functional image was labeled as 1-back or2-back memory (performed on either phonemic or nonphonemicvowels), 3-back memory (phonemic only), discrimination (phonemicor nonphonemic), visual task (with phonemic or nonphonemicvowels) or baseline (8 s breaks with no stimuli). The hemodynamicresponse function was modeled with a gamma function (mean lag6 s, SD 3 s) and its temporal derivative. Finally, several contrastswere specified to create Z-statistic images testing for task and stimu-lus (phonemic vs. nonphonemic) effects.

For analysis across subjects (second level analysis), the high-resolution anatomical images were normalized in spherical standardspace using FreeSurfer (release 5.1.0, http://surfer.nmr.mgh.harvard.edu). The anatomically normalized surfaces were rotatedand projected to a two dimensional (2D) space separately for eachhemisphere using equal area Mollweide projection (Python librariesmatplotlib and basemap, http://matplotlib.sourceforge.net). Thisprocedure was then applied separately for each subject to transformthe results of the 3D second-level statistical analysis to 2D.

Due to slightly misplaced EPI slices, some IPL areas (within ourarea of interest, Fig. 3) were not imaged in all subjects. To includealso these areas in the group analysis, the data in the missing voxels

282 K. Harinen, T. Rinne / NeuroImage 77 (2013) 279–287

(in second-level statistical images) were filled with large values sothat these voxels could be detected as outliers in the group analysis.A voxel was used for group analysis if at least 15 subjects had mea-sured data for that voxel.

Finally, the group analysis (FMRIB's local analysis of mixed effectsusing automatic outlier de-weighting, N = 16) was run on the flat-tened data. Z-statistic images were thresholded using clusters deter-mined by Z > 2.3 and a (corrected) cluster significance threshold ofp b 0.05 (using Gaussian random field theory).

Regions of interest (ROIs)

Four anatomical ROIs were defined (hand drawn based on anato-my) in the flattened 2D space (see Fig. 3j) to cover the STG areas inand posterior to HG. The first ROI (med_HG_PT, blue) contained themedial two thirds of HG extending posteriorly to medial STG tocover the assumed location of A1 (Woods et al., 2010). The secondROI (lat_HG_PT, green) covered the remaining one third of HG andthe corresponding areas of superior lateral STG posterior to HG. Twoadditional ROIs were defined to cover medial (med_pST, red) and lat-eral (lat_pST, yellow) posterior STG. These ROIs contained most of thesuperior temporal areas that were activated by vowels presented dur-ing the visual task and that also showed activation enhancementsduring discrimination and memory tasks.

Results

Performance during fMRI

Mean (±s.e.m.) RTs, HRs, and d′ in each condition are shown inFig. 2. In the memory task performed on phonemic vowels (1back P,2back P and 3back P), RTs, HRs and d′ decreased with increasing taskdifficulty [RT: main effect of task difficulty, F(2,30) = 11.1, p b 0.001;linear contrast F(1,15) = 16.8, p b 0.001. HR: F(2,30) = 23.6,p b 0.001; F(1,15) = 31.8, p b 0.001. d′: F(2,30) = 38.5, p b 0.001;F(1,15) = 63.2, p b 0.001]. In the memory task performed onnonphonemic vowels (1back N and 2back N; 3-back memory taskwas performed only on phonemic vowels), HRs [t(15) = 5.7,p b 0.001] and d′ [t(15) = 7.4, p b 0.001] were lower in 2-back than1-back memory task.

In 1-back memory task, RTs were slightly longer [t(15) = 2.3,p b 0.05] and HRs lower [t(15) = 2.5, p b 0.05] in phonemic thannonphonemic blocks. In 2-back memory task, d′ was higher[t(15) = 3.7, p b 0.01] in phonemic than nonphonemic vowels. Nosignificant differences were seen in RTs, HRs or d′ between tasksperformed with phonemic or nonphonemic vowels in the discrimi-nation task. In the visual task, RTs were significantly shorter[t(15) = 3.4, p b 0.01] and HRs higher [t(15) = 2.6, p b 0.05] inblocks with (task-irrelevant) phonemic than nonphonemic vowels.

Fig. 2. Performance in the auditory and visual tasks. The bars show mean (±s.e.m.) reactionination and visual tasks with phonemic (P) or nonphonemic (N) vowels.

In addition, two-way ANOVAs with factors task (2back, discr) andVowel (P, N) revealed a significant interaction for d′ (F(1,15) = 13.6,p b 0.05). These ANOVAs were computed using data from the 2-backcondition as the 3-back task was performed only on phonemicvowels.

fMRI

As comparedwith activations during the 8 s breaks with no sounds,phonemic and nonphonemic vowels presented during the visual task(i.e., in the absence of auditory attention) activated (N = 16, thresholdZ > 2.3, cluster-corrected p b 0.05) bilateral areas of the superior tem-poral cortex including HG (Fig. 3a).

The effects of auditory tasks on activations to vowels were inves-tigated by comparing activations during each vowel task with activa-tions to identical sounds presented during the visual task. Enhancedactivations during vowel discrimination were detected bilaterally inwide areas of the anterior and posterior STG and in the anterior insula(Fig. 3b, blue and yellow). Activation increases during vowel memorytasks, in turn, were found bilaterally in the more lateral and posteriorparts of STG, IPL, and in the anterior insula (Fig. 3b, red and yellow).

Direct comparison of activations during vowel discrimination andvowel memory tasks showed stronger activations associated withdiscrimination in the anterior insula, anterior and posterior STG in-cluding HG bilaterally (Fig. 3c, blue), while during vowel memorytasks enhanced activations were seen in areas of the IPL bilaterallyand in the left anterior insula (Fig. 3c, red).

The sound sequences used in the discrimination and n-back mem-ory tasks were similar but not identical. However, a contrast of activa-tions to discrimination and n-back sound sequences presented duringthe visual task revealed no significant activations. Therefore, it is un-likely that the small acoustic difference between the discriminationand n-back blocks significantly affected the activations observed dur-ing the auditory tasks.

Effects of task-difficulty on thememory taskswith phonemic vowels(three difficulty levels)were examinedwith linear contrasts. Significantactivation increase with increasing task difficulty was detected in theanterior insula and areas of the IPL bilaterally (Fig. 3d, blue). Inverse lin-ear contrasts showed that activations decreased with increasing mem-ory load in areas extending from the anterior insula to the anteriorand posterior STG (Fig. 3d, red). Although only 1-back and 2-backmem-ory tasks were performed on nonphonemic vowels, a quite similar pat-tern of task-difficulty effects was detected in the vowel memory taskswith nonphonemic vowels (Fig. 3e).

A direct contrast between discrimination task blocks with phone-mic or nonphonemic vowels showed enhanced activations duringblocks with phonemic vowels in areas of the IPL bilaterally (Fig. 3f,blue), while activations during nonphonemic blocks were strongerin STG areas near to HG (Fig. 3f, red). In general, a similar pattern of

times, hit rates and d′ in vowel memory (1-back, 2-back and 3-back), vowel discrim-

283K. Harinen, T. Rinne / NeuroImage 77 (2013) 279–287

activation differences between phonemic and nonphonemic blockswas seen also during the 1-back and 2-back memory tasks (Figs. 3g,h). That is, STG activations were enhanced during blocks with

a

b

c

d

e

f

g

h

i

j

nonphonemic vowels, while in the IPL enhanced activations weredetected in phonemic but not in nonphonemic blocks. Comparisonof blocks with phonemic and nonphonemic vowels presented duringthe visual task revealed significantly stronger activations during pho-nemic blocks in a few scattered areas (Fig. 3i). In the left hemisphere,the areas showing stronger activations during phonemic blocksseemed to cluster in areas of the posterior superior temporal sulcus(STS), posterior STG, and IPL.

A second-level (across subjects) repeated measures ANOVA withfactors Task (discrimination, 2-back memory) and Vowel (phonemic,nonphonemic) was conducted to examine Task × Vowel interactioneffects (Fig. 4a). The ANOVA revealed a significant (N = 16, thresholdZ > 2.3, cluster-corrected p b 0.05) main effect of Task (blue) in wideareas of the insula, STG, and IPL (cf. color-coded areas in Fig. 3c). Themain effect of Vowel (red) was significant in clusters in the STG andIPL bilaterally. Task × Vowel interaction was detected only in a fewclusters in the IPL (yellow; cf. Figs. 3f, h).

ROI analysis

ROI mean signal magnitudes (±s.e.m.) relative to rest (i.e., the 8 sbreaks with no stimuli) are shown in Fig. 5. Significant differences(bootstrapped pairwise t-test using 5000 bootstrap samples) be-tween tasks with phonemic and nonphonemic vowels are indicatedin the figure (asterisks). Signal magnitudes were significantly higherwhen the task was performed on nonphonemic than on phonemicvowels in four ROIs during 2-back and in one ROI during 1-back task.

Discussion

In the present study we investigated whether (1) vowel discrimi-nation and vowel memory tasks modulate AC activations in a similarmanner as the pitch and spatial versions of these tasks used in ourprevious studies (Rinne et al., 2009, 2012), (2) whether AC activa-tions during vowel discrimination and vowel memory tasks are dif-ferent during blocks with phonemic or nonphonemic vowels, and(3), in particular, whether activations in the IPL associated with the(categorical) memory task are different when the memory task isperformed on phonemic or nonphonemic vowel categories.

Activation differences between vowel discrimination and vowel memorytasks

In line with the results of our previous studies, the vowel discrim-ination and vowel memory tasks were associated with distinct activa-tion patterns (Fig. 4a, main effect of Task). As compared withactivations to identical stimuli presented during the demanding visu-al task, activations during vowel discrimination were enhanced in

Fig. 3. Activations to vowels (N = 16, threshold Z > 2.3, cluster-corrected p b 0.05)are shown on a flattened mean 2D cortical surface. (a) Comparisons of activations dur-ing visual task with phonemic (P) vowels and activations during the 8 s breaks with nostimuli (blue). The same comparison is also shown for nonphonemic (N) vowels (red).Areas showing significant activations in both contrasts are shown in yellow. (b) Com-parisons of activations during vowel discrimination task vs. visual task with identicalstimuli (blue and yellow) and vowel memory task vs. visual task (red and yellow).(c) Areas where activations were stronger during vowel discrimination than vowelmemory tasks (blue) and areas where activations were stronger during vowel memorythan vowel discrimination task (red). (d) Areas where activations increased with in-creasing load in the memory task (1–3 back) with phonemic vowels (linear contrast,blue) and areas where activations decreased with increasing load (inverse linear con-trast, red). (e) Effects of memory load (1–2 back) during vowel memory tasks withnonphonemic vowels. (f) Areas where activations were stronger with phonemicvowels than with nonphonemic vowels (blue), or vice versa (red), during vowel dis-crimination task, (g) during 1-back vowel memory task, and (h) during 2-backvowel memory task. (i) Comparisons of activations during visual task with(task-irrelevant) phonemic and nonphonemic vowels. (j) Areas used in the ROI analy-sis (see Fig. 5) and anatomical labels. STG superior temporal gyrus, HG Heschl's gyrus,IPL inferior parietal lobule (angular gyrus and supramarginal gyrus).

a

b

Fig. 4. A second-level (across subjects) repeated measures ANOVA with factors Task(discrimination, 2-back memory) and Vowel (phonemic, nonphonemic) shown on aflattened mean 2D cortical surface. (a) The main effect of Task (blue), the main effectof Vowel (red), the interaction effect Task × Vowel (yellow). (b) Anatomical labels.STG superior temporal gyrus, HG Heschl's gyrus, IPL inferior parietal lobule.

284 K. Harinen, T. Rinne / NeuroImage 77 (2013) 279–287

areas extending from the anterior to posterior STG including HG,whereas the vowel memory task was associated with activation en-hancements in more posterior/lateral areas in the STG and IPL(Figs. 3b, c). Further, activations during the vowel memory task in-creased with increasing task difficulty in the anterior insula and IPLand decreased in wide areas of the anterior and posterior STG(Figs. 3d, e). In general, these activation patterns observed duringvowel discrimination and vowel memory tasks are quite similar tothose observed in our previous studies using analogous pitch (Rinneet al., 2009) or spatial (Rinne et al., 2012) tasks. Thus, it is evidentthat the main features of these AC activation patterns are due to char-acteristics and requirements of the discrimination and memory tasksand not due to processing of pitch, spatial, or vowel-specific informa-tion as such.

It could be argued that a general task difficulty difference couldexplain the activation patterns observed during discrimination andn-back tasks (Figs. 3b, c). For example, although the continuousacoustic noise of the scanner was attenuated by ear plugs and ear pro-tectors, it is possible that scanner noise differently modulated activa-tions during discrimination and memory tasks. Scanner noise maymask small stimulus differences especially during the discriminationtask leading to enhanced activations in the STG. However, inconsis-tent with this notion, pitch and spatial discrimination difficulty did

Fig. 5. Percentual signal magnitude in left and right hemisphere AC ROIs. The colors in theAsterisks encode the significance level of bootstrapped t-tests (N = 16) comparing phonem

not significantly modulate AC activations in our previous studies(Rinne et al., 2009, 2012). Note also that in the present and in our pre-vious studies increasing task difficulty in the n-back tasks decreasedSTG activations. Thus, it is unlikely that the enhanced STG activationsduring the present discrimination task are caused by general task dif-ficulty but rather the distinct discrimination and memory activationpatterns are due to activation of specific processing resources re-quired in these tasks.

Activation differences between phonemic and nonphonemic vowel blocks

In terms of their frequency content, the phonemic and nonphonemicvowels used in the present study were quite similar (Fig. 1a). However,we detected significant activation differences between task blockswith phonemic or nonphonemic vowels (Fig. 4, main effect ofVowel). In the vowel discrimination task, stronger activations duringnonphonemic than phonemic blocks were detected in areas in the vi-cinity of HG, whereas during phonemic blocks enhanced activationswere seen inmore posterior areas especially in the IPL (Fig. 3f). Thesedifferences are rather surprising, as in all task blocks, subjects wererequired to discriminate two nearly identical sounds based on theirphysical differences, the task did not require any speech-relatedanalysis, and there were no systematic language-level differencesbetween the two vowels to be discriminated. Note that although previ-ous studies have demonstrated that speech-related perceptual-leveldifferences (e.g., the so-called magnet effect or perceptual differencesintensified due to phoneme category boundaries) modulate activationsto speech stimuli in AC (Guenther et al., 2004; Harinen et al., in press;Kilian-Hütten et al., 2011; Myers, 2007), it is unlikely that such effectsaffected the discriminability of the phonemic or nonphonemic vowelpairs in the present discrimination task. The phonemic vowel pairswere not systematically organized around a vowel prototype, thevowels in the phonemic vowel pairs always belonged to the same pho-neme category, and the vowels in the nonphonemic pairs did not formsystematic Finnish phoneme categories. In the vowel (2-back) memorytask, a comparison between phonemic and nonphonemic vowel blocksrevealed similar activation differences than in the discrimination task,although during nonphonemic blocks, significant activation enhance-mentswere detected inmorewidespread STG areas and, during phone-mic blocks, activation enhancements in the IPL were only detected in a

panel titles refer to Fig. 3j. The bars show mean (±s.e.m.) ROI signal relative to rest.ic and nonphonemic conditions with each other (* p b 0.05, ** p b 0.01).

285K. Harinen, T. Rinne / NeuroImage 77 (2013) 279–287

small left-hemisphere cluster (Fig. 3h). Interestingly, there also wereactivation differences between phonemic and nonphonemic vowelblocks when the vowels were presented during a visual task (i.e., thevowels were irrelevant to the task). During the visual task, stronger ac-tivations to phonemic vowels were detected in areas of the STG and IPLespecially in the left hemisphere but no areas showed stronger activa-tions to nonphonemic vowels (Figs. 3i, 5).

The enhanced activations observed in areas in the vicinity of HG(approximate location of primary AC; Fig. 3f) suggest that a morethorough spectrotemporal analysis (e.g., Harinen et al., in press;Kilian-Hütten et al., 2011) was required to complete the discrimina-tion task on nonphonemic than on phonemic vowels. In addition tosuch spectrotemporal analysis, the memory task with nonphonemicvowels probably required more processing (in the STG) to achieveand maintain a category membership for each nonphonemic vowel(Fig. 3h). During the discrimination tasks, areas in the posterior STGand IPL were more activated during phonemic than nonphonemicconditions. As the difference between phonemic and nonphonemicvowels is only significant in the level of speech/language and as it iscommonly assumed that areas further away from primary auditoryareas process higher-order auditory information (e.g., Chang et al.,2010; Hickok, 2009; Obleser and Eisner, 2009; Okada et al., 2010;Rauschecker, 1998), it is likely that these activations were due tospeech-specific processing. Together the differences between phone-mic and nonphonemic vowel task blocks suggest that tasks with pho-nemic vowels require less acoustic processing in AC, that phonemicvowels are processed in dedicated networks in the posterior STGand IPL, and that even non-speech auditory tasks (e.g., discriminationof physical difference) performed on phonemic vowels benefit fromsuch speech-specific processing (for the effects of tasks on speechspecific activations in AC, see, e.g., Harinen et al., in press; Hickokand Poeppel, 2007; Tian and Poeppel, in press).

Activations in the IPL

Consistent with our previous studies using pitch and spatial tasks(Rinne et al., 2009, 2012), the vowel memory task was strongly associ-atedwith enhanced IPL activations (Figs. 3b–e). It is likely that these IPLactivations are because the memory task required acquisition and stor-age of higher-level auditory information such as categorical grouping ofvowels and sequential information about the vowel stream. However,activations in these areas were also stronger during phonemic thannonphonemic vowel blocks in the discrimination task (Figs. 3f and 4a,Task × Vowel interaction), although there is no reason to assume thatthe discrimination task required such higher-level categorical or mem-ory representations. The enhanced IPL activations during discriminationtask blocks with phonemic vowels are probably because these IPL areasare also involved in processing of information specific to the phonemicstatus of the vowels. Thus, the present results suggest that suchspeech-specific information is processed in the same or overlappingIPL areas that are also involved in processing of higher-level auditoryrepresentations during categorical memory tasks.

Activations during visual task

During the visual task, activations to phonemic vowels were en-hanced in a few scattered clusters especially in the left posterior tem-poral areas and IPL, but no enhanced activations to nonphonemicvowels were detected (Figs. 3i, 5). Together with the results observedduring discrimination and n-back tasks, this suggests that phonemicvowels are processed in specialized networks even when the vowelsare task-irrelevant, but the nonphonemic vowels are only analyzedin more detail when such analysis is required by the current task.

It could be argued that the present visual task was relatively easy sothat it was possible to attend both modalities at the same time and,thus, the activation differences between phonemic and nonphonemic

vowels are due to active processing of the vowels. Further, it is pos-sible that, although the phonemic and nonphonemic vowels werephysically quite similar (Fig. 1a), the observed activation differencesare due to acoustical differences between phonemic andnonphonemic vowels. However, the fact that activations to vowelswere significantly stronger during auditory than visual task suggeststhat the present task manipulation effectively directed subjects' at-tention. In addition, attention-related modulations and activationsdue to acoustic differences should occur in the STG areas nearer toHG and not in the posterior temporal areas. It could also be arguedthat the increased activations to phonemic vowels observed duringthe visual task are because phonemic vowels were more difficult toignore during the visual tasks, and thus the visual task blocks withphonemic vowels were associated with enhanced activations dueto involuntary auditory attention. If phonemic vowels were moredifficult to ignore than nonphonemic vowels, then presentation ofphonemic vowels should have distracted visual task performance(Alho et al., 1997; Escera et al., 2000; Weisz and Schlittmeier,2006). However, visual task performance was actually better duringphonemic blocks as RTs (to Gabor orientation changes) were fasterand HRs higher during visual task blocks with phonemic thannonphonemic vowels. Thus, the present results do not directly sup-port the notion that the enhanced activations to phonemic vowelsobserved during the visual task are due to the effects of involuntaryattention. However, the possibility that involuntary attention tovowels modulated activations during the visual task cannot be fullyruled out as even a small effect of attention (e.g., to first vowel of ablock) could be enough to differentiate processing of task-irrelevantphonemic and nonphonemic vowels.

Why should visual performance be better during presentation oftask-irrelevant phonemic than nonphonemic vowels? Previous stud-ies suggest that task-irrelevant auditory information is processed toa relatively high level (Näätänen et al., 1997; Rinne et al., 1999).One possibility is that processing of task-irrelevant nonphonemicvowels requires resources that are also required in the visual task.That is, as phonemic vowels are processed in specialized networks,there could be less competition for general processing resources,and thus less distraction, during the visual task blocks with phonemicthan with nonphonemic vowels.

Interpretation of AC activations during vowel tasks

In a recent study, we investigated AC activations to vowel pairs or-ganized around a Finnish prototype or a nonprototype vowel during asame/different discrimination task (Harinen et al., in press). In proto-type blocks, one sound of the vowel pair was always a prototypical /i/or /ae/ (separately defined for each subject) and the other sound wasa slightly different vowel. In nonprototype blocks, sound pairs weresimilarly organized around a nonprototypical vowel. We found thatactivations in areas in or near HG were stronger during vowel dis-crimination with prototype pairs than with nonprototype pairs. Wesuggested that this difference was because, due to the perceptual sim-ilarity (magnet effect), the discrimination task with prototype pairsrequired more elaborate spectrotemporal analysis of the vowels.Note that in our previous study, both the prototype and nonprototypevowels were within Finnish phonemic categories, whereas the pres-ent study compared activations to phonemic and nonphonemicvowels. In the present study, we found that activations in areas nearHG were stronger during processing of nonphonemic than phonemicvowels. This effect cannot be due to the perceptual similarity ofvowels near a prototype as the nonphonemic vowels were all far fromany Finnish prototype vowels. Rather, our present results suggest thatAC activations increase when the task cannot be completed based onprocessing in the specialized networks in the posterior temporal cortexand IPL operating on speech-specific representations. Together our pre-vious and present results suggest that enhanced analysis in AC (based

286 K. Harinen, T. Rinne / NeuroImage 77 (2013) 279–287

on lower-level auditory representations) is needed during discrimina-tion of vowels near a prototype (where two physically different vowelsare perceptually identical due to magnet effect) or when higher-levelrepresentations of speech sounds are not available (nonphonemic ornonnative vowels).

Categorical processing in the IPL?

The present study was designed in particular to investigatewhether IPL activations observed in our previous studies duringn-back memory tasks are due to categorical processing required inthese tasks. We hypothesized that if this is the case then IPL activa-tions associated with a categorical n-back task should increasewhen the stimuli are harder to categorize. Thus, we expected thatIPL activations would be higher during vowel memory task withnonphonemic than phonemic vowels. In the present study, all phone-mic vowels could easily be labeled as Finnish /i/, /u/ or /a/, whereassuch pre-learned and well-defined category labels were not availablefor nonphonemic vowels. Subjects reported that they heard at leasttwo different, ambiguous vowel qualities in every nonphonemicvowel category. However, the vowels within one nonphonemic cate-gory were more similar with each other than with the vowels in theother nonphonemic vowel categories. Consistent with the results ofour previous studies, we found that IPL activations increased with in-creasing task difficulty during memory tasks performed with vowels(Figs. 3d, e). In contrast to our expectations, however, activations dur-ing memory tasks with phonemic and nonphonemic conditions werequite similar (Figs. 3d, e). In particular, IPL activations during 2-backtasks did not significantly differ between nonphonemic and phone-mic blocks (Fig. 3h). Thus, if the requirements for categorical process-ing were higher during the 2-back memory task performed onnonphonemic than phonemic vowels, then our results suggest thatthe IPL activations observed during memory tasks are not associatedwith categorical processing as such. Alternatively, it is possible thatthe processing requirements in the 2-back memory task performedon nonphonemic or phonemic vowels were different only duringthe earlier processing stages when auditory information is analyzedto achieve category labels for each vowel (higher activations in STGareas during the memory task with nonphonemic vowels) but, afterthe vowel category is resolved, processing in the IPL proceeds similar-ly with phonemic and nonphonemic vowel categories.

Activation decreases during vowel memory task

Consistent with the results of our previous studies using categori-cal pitch and spatial memory tasks (Rinne et al., 2009, 2012), activa-tions in wide AC areas decreased with increasing memory load inthe vowel memory task (Figs. 3d, e). Previously we suggested thatthis activation decrease is due to an active interruption of default pro-cessing as soon as the pitch or spatial category is resolved. Accordingto this notion, it is possible that activation increments during discrim-ination task and activation decrements during n-back tasks are due tomodulations of the same or partially same mechanisms. The presentstudy demonstrates that activations in the anterior AC areas decreasealso during categorical memory task performed on either phonemicor nonphonemic vowels.

It should be noted that such complex dynamics of AC activationsduring active listening tasks could make it difficult to interpret the re-sults of two-condition comparisons. For example, in the presentstudy, a comparison of activations during 2-back memory tasks withphonemic or nonphonemic suggests that STG activations are higherduring 2-back task performed with nonphonemic vowels. However,it is also possible that this effect is actually caused by stronger de-crease of activations in these areas during phonemic blocks.

Conclusions

The present results extend previous studies by showing that ACactivations to vowels are dynamically modulated according to the re-quirements of the task at hand. Our results demonstrate that suchtask-dependent modulations should be carefully considered (e.g., ex-perimental designs and interpretation of the results) in studies onspeech processing in human AC as, for example, the differences be-tween activations to phonemic and nonphonemic vowels depend onthe requirements of the task. Most importantly, further studies are re-quired to better understand the dynamics and functional significance oftask-dependent effects in processing of speech and nonspeech sounds.

Acknowledgments

This work was supported by the Academy of Finland (grants#1135900, #1141006, #1141563) and Research Funds of the Universityof Helsinki.

Conflict of interest

The authors declare no competing financial interests.

References

Alain, C., Campeanu, S., Tremblay, K., 2010. Changes in sensory evoked responses coincidewith rapid improvement in speech identification performance. J. Cogn. Neurosci. 22,392–403.

Alho, K., Escera, C., Diaz, R., Yago, E., Serra, J.M., 1997. Effects of involuntary auditory at-tention on visual task performance and brain activity. Neuroreport 8, 3233–3237.

Angenstein, N., Scheich, H., Brechmann, A., 2012. Interaction between bottom-up andtop-down effects during the processing of pitch intervals in sequences of spokenand sung syllables. Neuroimage 61, 715–722.

Benson, R.R., Whalen, D.H., Richardson, M., Swainson, B., Clark, V.P., Lai, S., Liberman,A.M., 2001. Parametrically dissociating speech and nonspeech perception in thebrain using fMRI. Brain Lang. 78, 364–396.

Binder, J.R., Frost, J.A., Hammeke, T.A., Bellgowan, P.S., Springer, J.A., Kaufman, J.N.,Possing, E.T., 2000. Human temporal lobe activation by speech and nonspeechsounds. Cereb. Cortex 10, 512–528.

Chang, E.F., Rieger, J.W., Johnson, K., Berger, M.S., Barbaro, N.M., Knight, R.T., 2010. Cat-egorical speech representation in human superior temporal gyrus. Nat. Neurosci.13, 1428–1432.

Dehaene-Lambertz, G., Pallier, C., Serniclaes, W., Sprenger-Charolles, L., Jobert, A.,Dehaene, S., 2005. Neural correlates of switching from auditory to speech perception.Neuroimage 24, 21–33.

Desai, R., Liebenthal, E., Waldron, E., Binder, J.R., 2008. Left posterior temporal regionsare sensitive to auditory categorization. J. Cogn. Neurosci. 20, 1174–1188.

Escera, C., Alho, K., Schroger, E., Winkler, I., 2000. Involuntary attention and distractibilityas evaluated with event-related brain potentials. Audiol. Neurootol. 5, 151–166.

Friederici, A.D., 2011. The brain basis of language processing: from structure to function.Physiol. Rev. 91, 1357–1392.

Guenther, F.H., Nieto-Castanon, A., Ghosh, S.S., Tourville, J.A., 2004. Representation ofsound categories in auditory cortical maps. J. Speech Lang. Hear. Res. 47, 46–57.

Harinen, K., Aaltonen, O., Salo, E., Salonen, O., Rinne, T., 2013. Task-dependent activationsof human auditory cortex to prototypical and nonprototypical vowels. Hum. BrainMapp. http://dx.doi.org/10.1002/hbm.21506 (in press).

Hickok, G., 2009. The functional neuroanatomy of language. Phys. Life Rev. 6 (3), 121–143.Hickok, G., Poeppel, D., 2000. Towards a functional neuroanatomy of speech perception.

Trends Cogn. Sci. 4, 131–138.Hickok, G., Poeppel, D., 2007. The cortical organization of speech processing. Nat. Rev.

Neurosci. 8, 393–402.Hickok, G., Saberi, K., 2012. Redefining the functional organization of the planum

temporale region: space, objects, and sensory-motor integration. In: Poeppel, D.,Overath, T., Popper, A.N., Fay, R.R. (Eds.), The Human Auditory Cortex. Springer,New York, pp. 333–350.

Husain, F.T., Fromm, S.J., Pursley, R.H., Hosey, L.A., Braun, A.R., Horwitz, B., 2006. Neuralbases of categorization of simple speech and nonspeech sounds. Hum. Brain Mapp.27, 636–651.

Jäncke, L., Wüstenberg, T., Scheich, H., Heinze, H.J., 2002. Phonetic perception and thetemporal cortex. Neuroimage 15, 733–746.

Kilian-Hütten, N., Valente, G., Vroomen, J., Formisano, E., 2011. Auditory cortex encodesthe perceptual interpretation of ambiguous sound. J. Neurosci. 31, 1715–1720.

Leung, W.S., Alain, C., 2011. Working memory load modulates the auditory “What” and“Where” neural networks. Neuroimage 55, 1260–1269.

Liebenthal, E., Binder, J.R., Spitzer, S.M., Possing, E.T., Medler, D.A., 2005. Neural sub-strates of phonemic perception. Cereb. Cortex 15, 1621–1631.

Myers, E.B., 2007. Dissociable effects of phonetic competition and category typicalityin a phonetic categorization task: an fMRI investigation. Neuropsychologia 45(7), 1463–1473.

287K. Harinen, T. Rinne / NeuroImage 77 (2013) 279–287

Näätänen, R., Lehtokoski, A., Lennes, M., Cheour, M., Huotilainen, M., Iivonen, A.,Vainio, M., Alku, P., Ilmoniemi, R.J., Luuk, A., Allik, J., Sinkkonen, J., Alho, K.,1997. Language-specific phoneme representations revealed by electric and mag-netic brain responses. Nature 385, 432–434.

Obleser, J., Eisner, F., 2009. Pre-lexical abstraction of speech in the auditory cortex.Trends Cogn. Sci. 13, 14–19.

Obleser, J., Boecker, H., Drzezga, A., Haslinger, B., Hennenlotter, A., Roettinger, M.,Eulitz, C., Rauschecker, J.P., 2006. Vowel sound extraction in anterior superior tem-poral cortex. Hum. Brain Mapp. 27, 562–571.

Obleser, J., Wise, R.J.S., Dresner, M.A., Scott, S.K., 2007. Functional integration across brainregions improves speech perception under adverse listening conditions. J. Neurosci.27, 2283–2289.

Okada, K., Rong, F., Venezia, J., Matching, W., Hsieh, I.H., Saberi, K., Serences, J.T.,Hickok, G., 2010. Hierarchical organization of human auditory cortex: evidencefrom acoustic invariance in the response to intelligible speech. Cereb. Cortex20, 2486–2495.

Petkov, C.I., Kang, X., Alho, K., Bertrand, O., Yund, E.W., Woods, D.L., 2004. Attentionalmodulation of human auditory cortex. Nat. Neurosci. 7, 658–663.

Raizada, R.D., Poldrack, R.A., 2007. Selective amplification of stimulus differences dur-ing categorical processing of speech. Neuron 56, 726–740.

Rauschecker, J.P., 1998. Cortical processing of complex sounds. Curr. Opin. Neurobiol. 8(4), 516–521.

Rauschecker, J.P., Scott, S.K., 2009. Maps and streams in the auditory cortex: nonhumanprimates illuminate human speech processing. Nat. Neurosci. 12, 718–724.

Rinne, T., Alho, K., Alku, P., Holi, M., Sinkkonen, J., Virtanen, J., Bertrand, O., Näätänen, R.,1999. Analysis of speech sounds is left-hemisphere predominant at 100–150 msafter sound onset. Neuroreport 10, 1113–1117.

Rinne, T., Koistinen, S., Salonen, O., Alho, K., 2009. Task-dependent activations of humanauditory cortex during pitch discrimination and pitch memory tasks. J. Neurosci.29, 13338–13343.

Rinne, T., Koistinen, S., Talja, S., Wikman, P., Salonen, O., 2012. Task-dependent activa-tions of human auditory cortex during spatial discrimination and spatial memorytasks. Neuroimage 59 (4), 4126–4131.

Sabri, M., Binder, J.R., Desai, R., Medler, D.A., Leitl, M.D., Liebenthal, E., 2008. Attentionaland linguistic interactions in speech perception. Neuroimage 39, 1444–1456.

Scott, S.K., Johnsrude, I.S., 2003. The neuroanatomical and functional organization ofspeech perception. Trends Neurosci. 26, 100–107.

Sharp, D.J., Awad, M., Warren, J.E., Wise, R.J.S., Vigliocco, G., Scott, S.K., 2010. The neural re-sponse to changing semantic and perceptual complexity during language processing.Hum. Brain Mapp. 31, 365–377.

Tian, X., Poeppel, D., 2013. The effect of imagination on stimulation: the functionalspecificity of efference copies in speech processing. J. Cogn. Neurosci. (in press).

Turkeltaub, P.E., Coslett, H.B., 2010. Localization of sublexical speech perception com-ponents. Brain Lang. 114, 1–15.

Weinberger, N.M., 2011. Reconceptualizing the primary auditory cortex: learning,memory and specific plasticity. In: Winer, J.A., Schreiner, C.E. (Eds.), The AuditoryCortex. Springer, New York, pp. 465–491.

Weisz, N., Schlittmeier, S.J., 2006. Detrimental effects of irrelevant speech on serial re-call of visual items are reflected in reduced visual N1 and reduced theta activity.Cereb. Cortex 16, 1097–1105.

Woods, D.L., Herron, T.J., Cate, A.D., Yund, E.W., Stecker, G.C., Rinne, T., Kang, X., 2010.Functional properties of human auditory cortical fields. Front. Syst. Neurosci. 4, 155.

Woods, D.L., Herron, T.J., Cate, A.D., Kang, X., Yund, E.W., 2011. Phonological processingin human auditory cortical fields. Front. Hum. Neurosci. 5, 42.