http://pss.sagepub.com/Psychological Science

http://pss.sagepub.com/content/24/5/803The online version of this article can be found at:

 DOI: 10.1177/0956797612462140

2013 24: 803 originally published online 4 April 2013Psychological ScienceElyana Saad and Juha Silvanto

How Visual Short-Term Memory Maintenance Modulates Subsequent Visual Aftereffects  

Published by:

http://www.sagepublications.com

On behalf of: 

  Association for Psychological Science

can be found at:Psychological ScienceAdditional services and information for    

  http://pss.sagepub.com/cgi/alertsEmail Alerts:

 

http://pss.sagepub.com/subscriptionsSubscriptions:  

http://www.sagepub.com/journalsReprints.navReprints:  

http://www.sagepub.com/journalsPermissions.navPermissions:  

What is This? 

- Apr 4, 2013OnlineFirst Version of Record  

- May 10, 2013Version of Record >>

at UCSF LIBRARY & CKM on December 6, 2014pss.sagepub.comDownloaded from at UCSF LIBRARY & CKM on December 6, 2014pss.sagepub.comDownloaded from

Psychological Science24(5) 803 –808© The Author(s) 2013Reprints and permissions: sagepub.com/journalsPermissions.navDOI: 10.1177/0956797612462140pss.sagepub.com

Research Report

Prolonged visual stimulation can change the visual sys-tem’s sensitivity to external input, giving rise to percep-tual phenomena such as afterimages and aftereffects. For example, in the tilt aftereffect (TAE), the viewing of an oriented stimulus causes shifts in subsequently per-ceived orientations (Gibson & Radner, 1937; Hofmann & Bielschowsky, 1909; Kohn, 2007). Adaptation paradigms are important tools for psychologists because the result-ing aftereffects can reveal the underlying properties of perceptual systems; for this reason, adaptation is often referred to as the psychologist’s microelectrode (Frisby, 1979).

Although aftereffects based on prolonged visual stim-ulation have been examined extensively, it is not known whether active short-term memory maintenance induces such effects. Adaptation results from prolonged neuronal responding to a sensory stimulus, and electrophysiologi-cal evidence indicates that visual short-term memory (VSTM) is associated with neuronal activity in the visual cortex during the maintenance period (e.g., Pasternak & Greenlee, 2005). Thus, it may be that VSTM maintenance can induce aftereffects. In the experiments reported here, we investigated this issue by examining how holding a stimulus in short-term memory modulates the impact of

subsequent visual adaptation that takes place after memory maintenance has ended. We accomplished this by using a sequential adaptation paradigm in which the first half of each trial involved VSTM maintenance and the second half involved visual adaptation. We hypothe-sized that if VSTM maintenance induces effects similar to those induced by visual adaptation, then its effects should summate with those of a subsequently presented visual adapter, just as the effects of two sequential visual stimuli summate (Greenlee & Magnussen, 1988). Specifically, the TAE should be enhanced when a memory cue and sub-sequent adapter are of similar orientation, and reduced when the two differ (Greenlee & Magnussen, 1988). In order to dissociate any adaptation effect induced by VSTM maintenance from effects induced by purely visual processes or a general increase in cognitive load induced by memory engagement, we included a control condition in which participants passively viewed the memory cue and also controlled for the engagement of VSTM.

462140 PSSXXX10.1177/0956797612462140Saad, SilvantoVisual Short-Term Memory and the Tilt Aftereffectresearch-article2013

Corresponding Author:Juha Silvanto, O. V. Lounasmaa Laboratory, Brain Research Unit, School of Science, Aalto University, Otakaari 5I, 00076 Aalto, Finland E-mail: silvanto@neuro.hut.fi

How Visual Short-Term Memory Maintenance Modulates Subsequent Visual Aftereffects

Elyana Saad and Juha SilvantoO. V. Lounasmaa Laboratory, Brain Research Unit, School of Science, Aalto University

AbstractProlonged viewing of a visual stimulus can result in sensory adaptation, giving rise to perceptual phenomena such as the tilt aftereffect (TAE). However, it is not known if short-term memory maintenance induces such effects. We examined how visual short-term memory (VSTM) maintenance modulates the strength of the TAE induced by subsequent visual adaptation. We reasoned that if VSTM maintenance induces aftereffects on subsequent encoding of visual information, then it should either enhance or reduce the TAE induced by a subsequent visual adapter, depending on the congruency of the memory cue and the adapter. Our results were consistent with this hypothesis and thus indicate that the effects of VSTM maintenance can outlast the maintenance period.

Keywordsadaptation, tilt aftereffect, short-term memory, visual perception

Received 5/8/12; Revision accepted 8/31/12

at UCSF LIBRARY & CKM on December 6, 2014pss.sagepub.comDownloaded from

804 Saad, Silvanto

Method

Participants

Twenty-two observers (5 females, 17 males), who were naive to the aims of the study, participated in two experi-ments (11 in each experiment). All observers provided informed consent and were given a monetary reward for their participation.

Stimuli

The stimuli and task were controlled by E-Prime 2.0 (Psychology Software Tools, Inc., Pittsburgh, PA; http://www.pstnet.com/eprime.cfm). All stimuli except for control-task memory cues were sinusoidal luminance-modulated gratings (with a diameter of 6° of visual angle; generated with MATLAB, The MathWorks, Natick, MA), presented foveally on a gray background from a viewing distance of 57 cm. The spatial frequency of the gratings was 1.44 cycles/°. The memory cues, distractors, adapt-ers, and memory probes had a Michelson contrast of 0.9; the contrast of the TAE probes was 0.5 (cf. Knapen, Rolfs, Wexler, & Cavanagh, 2010). In Experiment 2, the control-task memory cues were blue or red squares that were either small (diameter = 4.5° of visual angle) or large (diameter = 6.5° of visual angle). The stimuli were pre-sented on a 15-in. screen with a resolution of 1024 × 768 pixels.

Procedure

We used a sequential adaptation paradigm in which the first half of each trial involved VSTM maintenance and the latter half involved visual adaptation (see Fig. 1 for timelines of trials in the two experiments). Each trial in the VSTM condition of Experiment 1 began with a black fixation cross appearing in the middle of the screen, fol-lowed by a blank screen and the memory cue (a grating tilted ±20° or ±40° from the vertical), which participants were instructed to hold in memory. This memory cue was followed by a maintenance period during which a distractor (a horizontal grating) was presented, 1 s after offset of the cue. The appearance of a fixation cross sig-naled the end of the maintenance period; participants were instructed to stop holding the cue in memory when the cross appeared.

The next display showed a visual adapter with an ori-entation of ±20° from vertical. Thus, the orientation dif-ference between the memory cue and the visual adapter was 0°, +20°, –40°, or –60°. Positive value indicates that the sign of tilt was the same (i.e., both the memory cue and the adapter were tilted to the left or to the right); negative values indicate that the signs were opposite. For example, an orientation difference of +20° indicates that the orientations of the memory cue and adapter were

+40° and +20°, respectively (i.e., both tilted to the right), or –40° and –20°, respectively (i.e., both tilted to the left). An orientation difference of – 40° indicates that one of these stimuli had an orientation of +20° and the other had an orientation of –20°. An orientation difference of –60° indicates that the orientations of the memory cue and adapter were – 40° and +20° or +40° and –20°, respectively. There was no combination for which the orientation difference was –20°, +40°, or +60°. After visual adaptation, a TAE probe (a grating tilted −1°, 0°, or +1° from the vertical; duration = 40 ms) was presented, and observers were asked to report the direction of the perceived tilt.

To ensure that subjects were engaged in memory maintenance, we randomly included memory catch trials. On these trials, the maintenance period was followed by a memory probe (grating tilted ±10° relative to the mem-ory cue), and participants were asked to indicate the direction of the tilt relative to the memory cue.

To dissociate effects induced by VSTM maintenance from effects induced by passive viewing of the stimuli, in both experiments we included a control condition in which participants were presented with the same sequence of stimuli as in the VSTM condition but were not asked to remember the memory cue. In addition, we designed Experiment 2 to control for memory load by asking participants in the control condition to hold in memory stimulus features that were irrelevant for the TAE (Fig. 1b). Specifically, the trials in this experiment were the same as in Experiment 1 except that the memory cue was preceded by a fixation cross and then the control-task memory cue. (In addition, some of the display dura-tions were changed; see Fig. 1.) In the VSTM condition, participants were instructed to passively view the con-trol-task memory cue and to hold the memory cue in memory. In the control condition, their task was instead to hold the colored memory-task cue in memory and to passively view the tilted memory cue. On catch trials in the control condition, to ensure that participants held the control-task memory cue in memory, we asked them to indicate whether the stimulus was blue and small, blue and large, red and small, or red and large. Catch trials in the VSTM condition were as described for Experiment 1.

Experiment 1 was run in two sessions, each session containing four blocks (two VSTM and two control blocks). Each VSTM block contained 48 trials (32 TAE tri-als, with 8 trials per orientation difference, and 16 mem-ory catch trials); each control block contained 32 trials. Experiment 2 was carried out in one session consisting of six blocks (three VSTM blocks and three control blocks, each of which consisted of 48 trials, as in the VSTM blocks in Experiment 1). The order of VSTM and control blocks was counterbalanced across participants. Each experiment was preceded by a practice block in which observers were acquainted with the VSTM task.

at UCSF LIBRARY & CKM on December 6, 2014pss.sagepub.comDownloaded from

Visual Short-Term Memory and the Tilt Aftereffect 805

Memory Cue (300 ms)

Blank (1 s)

Distractor (300 ms)

Maintenance (8 s)

Fixation (300 ms)

Adapter (2.5 s)

Fixation (300 ms)

TAE Probe (40 ms)

+

+End of Memory

Maintenance

Control-Task Memory Cue (300 ms)

Memory Cue (300 ms)

Blank (1 s)

Blank (1 s)

Distractor (300 ms)

Maintenance (6 s) 

Fixation (300 ms)

Adapter (2 s)

Fixation (300 ms)

+

TAE Probe (40 ms)+End ofMemory

Maintenance

 

 

 

a

b

Fig. 1.  Timelines of experimental trials in the two experiments. In the visual short-term memory (VSTM) condition of Experiment 1 (a), observers were instructed to maintain a memory cue in VSTM. Following the end of the memory-main-tenance period, an adapter was presented. Finally, observers indicated the direction in which a probe stimulus was tilted. In the control condition of this experiment, observers were presented with the same sequence of stimuli but were not asked to hold the memory cue in memory. In Experiment 2 (b), a fixation cross at the start of each trial was followed by a blue or red square (control-task memory cue), and then the memory cue. In the VSTM condition, observers were asked to hold the memory cue in memory and to passively view the control-task memory cue; in the control condition, they were asked to hold the control-task memory cue in memory and to passively view the memory cue. TAE = tilt aftereffect.

at UCSF LIBRARY & CKM on December 6, 2014pss.sagepub.comDownloaded from

806 Saad, Silvanto

Results

Data analysis

The magnitude of the TAE in each memory condition was calculated by averaging responses across the three TAE probes (i.e., tilt of −1°, 0°, and +1° from the vertical). As observers were asked to press “1” for perceived left-ward tilt and “2” for perceived rightward tilt, a mean value of 1 would indicate a maximal leftward bias, and a mean value of 2 a maximal rightward bias. To combine trials with leftward-tilting (i.e., −20°) and rightward-tilting (i.e., +20°) adapters, we calculated an overall TAE mea-sure by subtracting the mean response for the rightward-adapter trials from the mean response for the leftward-adapter trials. Thus, values above 0 indicate the presence of a TAE (i.e., a tendency to report that the probe’s orientation was the opposite of the adapter’s ori-entation), and the maximum value possible was 1 (a TAE of 1 would result from a maximal rightward bias for the leftward-adapter trials and a maximal leftward bias for the rightward-adapter trials). Graphs displaying the mean of participants’ responses in each memory condition as a function of the TAE probe’s orientation are in the Supplemental Material available online.

Experiment 1

Figure 2a shows the magnitude of the TAE induced by the visual adaptation as a function of the orientation difference between the memory cue and the visual

adapter, separately for the VSTM and control conditions. Two participants were removed from the analysis because of low performance on memory catch trials (accuracy of 55% and 63%, respectively). A high level of performance on these trials was important because it indicated that participants were engaging VSTM consistently through-out the task. The mean performance on memory catch trials was 84% (SD = 6%). We conducted a repeated mea-sures analysis of variance (ANOVA) with factors of mem-ory condition (VSTM, control), congruency of the memory cue and adapter (congruent direction of tilt, incongruent direction of tilt), and memory-cue orientation (±20°, ±40°). This analysis revealed a significant main effect of congruency, F (1, 8) = 10.26, p = .013, ηp

2 = .562, and an interaction between memory condition and congruency, F (1, 8) = 18.97, p = .002; ηp

2 = .703. No other main effects or interactions were significant. The interaction indicates that the TAE was modulated differently by the memory demand depending on whether the memory cue and adapter were congruent or incongruent. A pairwise com-parison showed a nonsignificant trend for the TAE to be higher in the VSTM than in the control condition when the memory cue and the adapter were congruent, t(8) = 1.64, p = .14; when they were incongruent, there was a trend in the opposite direction, t(8) = 1.55, p = .17.

Experiment 2

Figure 2b shows the magnitude of the TAE induced by the visual adaptation as a function of the orientation

a b

.20

.30

.40

.50

.60

.70

.80

20 0 –40 –60–40 –60

Mag

nitu

de o

f TAE

Experiment 2

.20

.30

.40

.50

.60

.70

.80

20 0

Mag

nitu

de o

f TAE

Orientation Difference Between the Memory Cue and Adapter (°)

Orientation Difference Between the Memory Cue and Adapter (°)

Experiment 1VSTM

Control

VSTM

Control

Fig. 2.  The mean magnitude of the tilt aftereffect (TAE) in the visual short-term memory (VSTM) and control conditions of (a) Experi-ment 1 and (b) Experiment 2 as a function of the orientation difference between the memory cue and the visual adapter. The error bars indicate ±1 SEM.

at UCSF LIBRARY & CKM on December 6, 2014pss.sagepub.comDownloaded from

Visual Short-Term Memory and the Tilt Aftereffect 807

difference between the memory cue and the visual adapter, separately for the VSTM and control conditions. We conducted a repeated measures ANOVA with factors of memory condition (VSTM, control), congruency of the memory cue and adapter (congruent, incongruent), and memory-cue orientation (±20°, ±40°). This analysis revealed a significant main effect of congruency, F(1, 10) = 12.62, p = .005, ηp

2 = .558, and an interaction between memory condition and congruency, F(1, 10) = 5.41, p = .042, ηp

2 = .351. Thus, memory demand modu-lated the TAE, and the direction of this modulation depended on congruency between the memory cue and the visual adapter. No other main effect or interaction was significant. A pairwise comparison showed that the TAE was higher in the VSTM than in the control condition when the memory cue and the adapter were congruent, t(10) = 2.22, p = .05; when they were incongruent, there was a trend in the opposite direction, t(10) = 1.69, p = .12. The mean performance on memory catch trials was 87% (SD = 6%) in the VSTM condition and 91% (SD = 5%) in the control condition.

Combined analysis of Experiments 1 and 2

To investigate the source of the interaction between memory condition and congruency in Experiments 1 and 2 (i.e., whether the results reflected facilitation in the congruent condition or inhibition in the incongruent con-dition, or both), we combined the two experiments in a single analysis in order to increase statistical power. We conducted a repeated measures ANOVA with within- subjects factors of memory condition (VSTM, control), congruency of the memory cue and adapter (congruent, incongruent), and memory-cue orientation (±20°, ±40°) to determine if there were any differences between the results of the two experiments. Experiment (Experiment 1, Experiment 2) was entered as a between-subjects factor. This analysis revealed a significant main effect of congru-ency, F (1, 18) = 22.37, p = .001, ηp

2 = .554, and an inter-action between memory condition and congruency, F (1, 18) = 10.57, p = .004, ηp

2 = .370. No other main effects or interactions were significant. No differences were found between the experiments, F(1, 18) = 0.025, p = .875, ηp

2 = .001. A pairwise comparison showed that the TAE was higher in the VSTM than in the control condition when the memory cue and the adapter were congruent, t(19) = 2.81, p = .011; when they were incongruent, the TAE was significantly lower in the VSTM than in the con-trol condition, t(19) = 2.14, p = .045.

In summary, the combined analysis of Experiments 1 and 2 indicates that VSTM maintenance significantly increased the strength of the TAE induced by a

subsequent visual adapter when the memory cue was congruent with the adapter. When the memory cue and the adapter were incongruent, the TAE was reduced by VSTM maintenance.

Discussion

Our results show that VSTM maintenance can modulate the TAE induced by subsequent visual adaptation. Specifically, an enhancement of the TAE was found when the orientations of the memory cue and the visual adapter were congruent; a reduction of the TAE was observed when they were incongruent. These results are similar to previous results for sequential adaptation using two visual adapters; adaptation was facilitated when the two adapters were of similar orientation and was reduced when they differed in orientation by 30° to 60° (Greenlee & Magnussen, 1988). These facilitatory and inhibitory effects are likely to reflect summation within and compe-tition between orientation channels, respectively (e.g., Blakemore & Campbell, 1969; Campbell & Maffei, 1971; Tolhurst & Thompson, 1975). The similarity of our results with these previous results for sequential adaptation to two visual stimuli indicates that VSTM maintenance induces adaptation-like effects.

A key aspect of our results is that a modulation of the TAE was found when we compared the VSTM condition with a condition in which the memory cue was presented without the requirement to hold it in memory. Thus, the observed effects could not have been fully driven by pas-sive memory traces induced by the memory cue. Furthermore, in Experiment 2, we controlled for memory load by asking participants to perform a visual memory task that did not involve the maintenance of orientation information (control condition) and thus was unlikely to affect the mechanisms responsible for the TAE. The inter-action between memory condition and congruency of the memory cue and adapter in this experiment indicates that the observed effects in the VSTM condition were not simply due to a higher cognitive load distracting partici-pants from the visual adapter.

Given that VSTM maintenance and visual adaptation were not concurrent, our results are unlikely to reflect attentional competition between working memory con-tent and the visual adapter. Furthermore, as VSTM main-tenance modulated the TAE differentially as a function of congruency, the observed effects cannot be explained as due to an increase in cognitive load impairing visual adaptation, a possibility also ruled out by the control condition of Experiment 2. It is also unlikely that a cost in switching attention from VSTM content to the visual adapter influenced our results, as the end of the maintenance period was signaled by the appearance of a

at UCSF LIBRARY & CKM on December 6, 2014pss.sagepub.comDownloaded from

808 Saad, Silvanto

fixation cross prior to the adapter’s onset, and partici-pants therefore had sufficient time to shift their attention to the visual adapter. Finally, the fact that these effects survived the presentation of a distractor (which would have disrupted any passive memory trace), coupled with a high level of performance on catch trials, demonstrates that active VSTM maintenance was involved.

In summary, our results demonstrate that the effects of VSTM maintenance can outlast the maintenance period: VSTM maintenance modulated the magnitude of an aftereffect induced by a subsequently presented adapter. This indicates that the prolonged neuronal activity induced by VSTM maintenance can modulate the efficacy of subsequent visual processing, as is also the case with prolonged visually induced neuronal activity.

Declaration of Conflicting Interests

The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.

Funding

J. S. is supported by the Academy of Finland (137485) and the Emil Aaltonen Foundation.

Supplemental Material

Additional supporting information may be found at http://pss .sagepub.com/content/by/supplemental-data

References

Blakemore, C., & Campbell, F. W. (1969). On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images. Journal of Physiology, 203, 237–260.

Campbell, F. W., & Maffei, L. (1971). The tilt after-effect: A fresh look. Vision Research, 11, 833–840.

Frisby, J. (1979). Seeing: Illusion, brain and mind. Oxford, England: Oxford University Press.

Gibson, J. J., & Radner, M. (1937). Adaptation, after-effect and contrast in the perception of tilted lines. I. Quantitative studies. Journal of Experimental Psychology, 20, 453–467.

Greenlee, M. W., & Magnussen, S. (1988). Interactions among spatial frequency and orientation channels adapted concur-rently. Vision Research, 28, 1303–1310.

Hofmann, F. B., & Bielschowsky, A. (1909). Ueber die Einstellung der scheinbaren Horizontalen und Vertikalen bei Betrachtung [On the adjustment of the perceived hori-zontal and vertical]. Pflügers Archiv, 126, 453–475.

Knapen, T., Rolfs, M., Wexler, M., & Cavanagh, P. (2010). The reference frame of the tilt aftereffect. Journal of Vision, 10(1), Article 8. Retrieved from http://www.journalofvision .org/content/10/1/8.full.pdf+html

Kohn, A. (2007). Visual adaptation: Physiology, mechanisms, and functional benefits. Journal of Neurophysiology, 97, 3155–3164.

Pasternak, T., & Greenlee, M. W. (2005). Working memory in primate sensory systems. Nature Reviews Neuroscience, 6, 97–107.

Tolhurst, D., & Thompson, P. G. (1975). Orientation illusions and after-effects: Inhibition between channels. Vision Research, 15, 967–972.

at UCSF LIBRARY & CKM on December 6, 2014pss.sagepub.comDownloaded from