Memory and Cognition

Memory and Memory and CognitionCognition

PSY 324PSY 324

Topic: Visual ImageryTopic: Visual Imagery

Dr. Ellen CampanaDr. Ellen Campana

Arizona State UniversityArizona State University

Visual ImageryVisual Imagery

What is Imagery?What is Imagery?

Think about the following questions….Think about the following questions…. How many windows are on front of where you How many windows are on front of where you

live?live? How is the furniture arranged in your How is the furniture arranged in your

bedroom?bedroom? Are an elephant’s ears rounded or pointy?Are an elephant’s ears rounded or pointy? What does a pumpkin pie smell like?What does a pumpkin pie smell like?

You use You use mental imagerymental imagery to answer these to answer these Mental imageryMental imagery = experiencing a sensory = experiencing a sensory

impression in the absence of sensory inputimpression in the absence of sensory input

Uses of ImageryUses of Imagery

Creative process of music Creative process of music Paul McCartney’s “Yesterday” Paul McCartney’s “Yesterday” Conductors use “inner audition” to Conductors use “inner audition” to

practice without orchestraspractice without orchestras Imagine soundsImagine sounds Imagine locations of instrumentsImagine locations of instruments

Sports PsychologySports Psychology Swimming – imagining all of the strokesSwimming – imagining all of the strokes Time to mentally rehearse linked to real Time to mentally rehearse linked to real

timestimes Jack Nicklaus, golfing, discovered an error Jack Nicklaus, golfing, discovered an error

Uses of ImageryUses of Imagery

Visual ImageryVisual Imagery (mental imagery of (mental imagery of visual info) useful in scientific visual info) useful in scientific discoveriesdiscoveries Einstein’s theory of relativity came from Einstein’s theory of relativity came from

imagining himself riding along a beam imagining himself riding along a beam of lightof light

The Study of ImageryThe Study of Imagery Early ideasEarly ideas

Aristotle: thought is impossible without an Aristotle: thought is impossible without an imageimage

Wundt: images are one of three basic Wundt: images are one of three basic elements of consciousness (others: elements of consciousness (others: sensations, feelings)sensations, feelings)

Strong link between images and thoughtStrong link between images and thought Gave rise to Gave rise to imageless-thought debateimageless-thought debate

Can people think without images?Can people think without images? Galton said yes: people who have difficulty Galton said yes: people who have difficulty

forming images can still thinkforming images can still think Wundt still said noWundt still said no

The Study of ImageryThe Study of Imagery

BehaviorismBehaviorism Rise of behaviorism shut down the Rise of behaviorism shut down the

imageless-thought debateimageless-thought debate Study of imagery is unproductive because Study of imagery is unproductive because

visual images are invisible to everyone elsevisual images are invisible to everyone else Watson (1928) called visual imagery Watson (1928) called visual imagery

“unproven” and “mythological”“unproven” and “mythological” 1920’s-1950’s: no study of imagery in 1920’s-1950’s: no study of imagery in

mainstream psychologymainstream psychology

The Study of ImageryThe Study of Imagery

The Cognitive RevolutionThe Cognitive Revolution Cognitive psychologists developed ways Cognitive psychologists developed ways

to measure behavior to to measure behavior to infer cognitive infer cognitive processes processes (like visual imagery)(like visual imagery) Memory (Paivio, 1963, 1965)Memory (Paivio, 1963, 1965) Mental chronometry (Shepard & Meltzer, Mental chronometry (Shepard & Meltzer,

1971)1971) Study of imagery returned to Study of imagery returned to

mainstream psychologymainstream psychology Research continues todayResearch continues today

Visual Imagery Visual Imagery ResearchResearch

Demonstrating that Demonstrating that Imagery ExistsImagery Exists

Paivio (1963, 1965): Paivio (1963, 1965): concreteconcrete vs. vs. abstractabstract nounsnouns Method: Method: paired-associate learningpaired-associate learning

Training: participants learn pairs of wordsTraining: participants learn pairs of words Testing: participants presented with first word of Testing: participants presented with first word of

pair, task is to recall the second (pair, task is to recall the second (recognition recognition tasktask))

Two types of word pairs (within-Ss design)Two types of word pairs (within-Ss design) Concrete: Concrete: truck – tree truck – tree (both easy to form image of)(both easy to form image of) Abstract: Abstract: truthtruth – – ideaidea (both hard to form image of) (both hard to form image of)

Result: concrete pairs were easier to recallResult: concrete pairs were easier to recall Interpretation: Interpretation: Conceptual-peg hypothesisConceptual-peg hypothesis

Demonstrating that Demonstrating that Imagery ExistsImagery Exists

Shepard and Metzler (1971)Shepard and Metzler (1971) Mental chronometryMental chronometry – general idea of – general idea of

measuring how long it takes to carry out measuring how long it takes to carry out various cognitive tasksvarious cognitive tasks

Task: participants saw two objects, had Task: participants saw two objects, had to indicate quickly whether the two to indicate quickly whether the two objects were the same or differentobjects were the same or different

Results: Time it took to indicate that they Results: Time it took to indicate that they were the same object was directly related were the same object was directly related to how far the object had to be rotatedto how far the object had to be rotated

Interpretation: Interpretation: imagery and perception imagery and perception share some of the same mechanismsshare some of the same mechanisms

Shepard & Metzler Shepard & Metzler (1971)(1971)

Imagery and PerceptionImagery and Perception

Major question: do imagery and Major question: do imagery and perception share the same perception share the same mechanisms?mechanisms? Kosslyn (1973): Kosslyn (1973): mental scanningmental scanning

Task: Memorize an image, then answer Task: Memorize an image, then answer questions about whether certain parts questions about whether certain parts appear in the imageappear in the image

Time it takes to say yes is related to distance Time it takes to say yes is related to distance between initial focus and correct partbetween initial focus and correct part

Kosslyn (1973)Kosslyn (1973)


Major question: do imagery and Major question: do imagery and perception share the same mechanisms?perception share the same mechanisms? Kosslyn (1973): Kosslyn (1973): mental scanningmental scanning

Task: Memorize an image, then answer questions Task: Memorize an image, then answer questions about whether certain parts appear in the imageabout whether certain parts appear in the image

Time it takes to say yes is related to distance between Time it takes to say yes is related to distance between initial focus and correct partinitial focus and correct part

Lea (1975): alternative explanation for resultLea (1975): alternative explanation for result Perhaps as people scan they are distracted by Perhaps as people scan they are distracted by

other partsother parts Kosslyn (1978): ruled out alternative Kosslyn (1978): ruled out alternative

explanationexplanation



Kosslyn’s experiments were Kosslyn’s experiments were convincing, but there was another convincing, but there was another alternative explanationalternative explanation

Pylyshyn (1973) – results are based Pylyshyn (1973) – results are based on on propositional mechanismspropositional mechanisms, not , not on on spatial representationspatial representation Ushered in the Ushered in the imagery debateimagery debate (still (still

going on)going on)

Propositional vs. SpatialPropositional vs. Spatial Basic idea: just because the Basic idea: just because the experienceexperience is is

spatial doesn’t mean the spatial doesn’t mean the underlying underlying representationrepresentation is is Spatial experience of mental images could be an Spatial experience of mental images could be an

epiphenomenon epiphenomenon (side effect)(side effect) Epiphenomenon:Epiphenomenon: related to / co-occurring with the related to / co-occurring with the

mechanism, but not actually part of the mechanismmechanism, but not actually part of the mechanism Information could be encoded with Information could be encoded with

language (language (propositional representationpropositional representation) ) or with images (or with images (depective depective representationsrepresentations) – can’t tell) – can’t tell Propositional rep. is like a Propositional rep. is like a semantic networksemantic network

Propositional vs SpatialPropositional vs Spatial

How Does Phlyshyn Explain How Does Phlyshyn Explain Kosslyn’s First Set of Kosslyn’s First Set of

Results?Results? Tacit-knowledge explanationTacit-knowledge explanation

People in the mental scanning task behave People in the mental scanning task behave based on what happens in a real scene (based on what happens in a real scene (tacit tacit knowledgeknowledge))

In the real world it takes longer to travel greater In the real world it takes longer to travel greater distancesdistances

Simulate this behavior in the experimentSimulate this behavior in the experiment

This explanation seems overly complicated This explanation seems overly complicated and doesn’t seem to scale to complex scenesand doesn’t seem to scale to complex scenes Phylyshyn still makes these arguments, so they Phylyshyn still makes these arguments, so they

have to be addressedhave to be addressed

Propositional vs SpatialPropositional vs Spatial

Against Propositional Against Propositional RepresentationRepresentation

Finke and Pinker (1982)Finke and Pinker (1982) Short presentation of simple display Short presentation of simple display

with 4 random dots, followed (after 2-with 4 random dots, followed (after 2-sec delay) by an arrowsec delay) by an arrow

Participants had to say whether the Participants had to say whether the arrow pointed to one of the dots in the arrow pointed to one of the dots in the first display (gone now)first display (gone now)

Finke and Pinker (1982)Finke and Pinker (1982)

Against Propositional Against Propositional RepresentationRepresentation

Finke and Pinker (1982)Finke and Pinker (1982) Short presentation of simple display with 4 Short presentation of simple display with 4

random dots, followed (after 2-sec delay) by an random dots, followed (after 2-sec delay) by an arrowarrow

Participants had to say whether the arrow Participants had to say whether the arrow pointed to one of the dots in the first display pointed to one of the dots in the first display (gone now)(gone now)

Results: time it took to respond “yes” was Results: time it took to respond “yes” was directly related to distance from arrowdirectly related to distance from arrow

Why couldn’t this be propositional?Why couldn’t this be propositional? No time to convert to propositions, no meaning No time to convert to propositions, no meaning

in the dots (except spatial relations) in the dots (except spatial relations)

Propositional vs. SpatialPropositional vs. Spatial

Phylyshyn refuses to cede, responding to Phylyshyn refuses to cede, responding to each study with each study with tacit knowledgetacit knowledge explanationsexplanations He is in the minorityHe is in the minority Most researchers think there’s Most researchers think there’s

overwhelming evidence that mental imagery overwhelming evidence that mental imagery is spatialis spatial

Let’s assume (like most people) that Let’s assume (like most people) that mental imagery is spatial, and go on mental imagery is spatial, and go on talking about how it relates to perceptiontalking about how it relates to perception

Imagery and Imagery and PerceptionPerception

Size in the Visual FieldSize in the Visual Field

As you move closer to objects in the As you move closer to objects in the real worldreal world Object fills more of your visual fieldObject fills more of your visual field Details are easier to seeDetails are easier to see

Kosslyn (1978) used these facts to Kosslyn (1978) used these facts to investigate mental imageryinvestigate mental imagery

Size in Visual FieldSize in Visual Field

Kosslyn (1978)Kosslyn (1978) TaskTask

Imagine two objects, moving close enough so Imagine two objects, moving close enough so that the bigger object fills most of visual fieldthat the bigger object fills most of visual field

Answer questions about one of the animalsAnswer questions about one of the animals Conditions (within-subjects)Conditions (within-subjects)

Question about the animal when it was Question about the animal when it was biggerbigger than the other animal than the other animal

Question about the animal when it was Question about the animal when it was smallersmaller than the other animalthan the other animal

Result: questions answered faster when it Result: questions answered faster when it was bigger was bigger Same as with a similar perceptual taskSame as with a similar perceptual task

Kosslyn (1978)Kosslyn (1978)

Kosslyn (1978)Kosslyn (1978) In addition to the question-answering In addition to the question-answering

experiment, participants did a experiment, participants did a mental mental walkwalk task task Imagine an animalImagine an animal Walk toward it until it “overflows” the visual Walk toward it until it “overflows” the visual

fieldfield How far away is it? (estimate distance)How far away is it? (estimate distance)

Let’s try:Let’s try: CatCat This buildingThis building

Result: Greater distances for larger objectsResult: Greater distances for larger objects

Interactions Between Interactions Between Perception and ImageryPerception and Imagery

Visual images can interact with perceptionVisual images can interact with perception Perky (1910) – subliminal presentationPerky (1910) – subliminal presentation

Participants imagined an object and described their Participants imagined an object and described their mental image to the experimenter mental image to the experimenter

While they did this, an image of the same object While they did this, an image of the same object was projected very dimlywas projected very dimly

No participants reported seeing the image No participants reported seeing the image Descriptions of mental images matched details of real Descriptions of mental images matched details of real

imageimage

Farah (1985) – priming taskFarah (1985) – priming task Mental images prime performance in a perception Mental images prime performance in a perception

tasktask

Farah (1985)Farah (1985)

Imagine the letter Imagine the letter ______

Indicate whether it Indicate whether it was in the first or was in the first or second white squaresecond white square

H

Results: participants were more Results: participants were more accurate when the letter that accurate when the letter that appeared matched the letter they appeared matched the letter they had imaginedhad imagined

Imagery and the Imagery and the BrainBrain

Brain’s Response to Brain’s Response to ImageryImagery

Imagery neurons (Kreiman & coworkers, 2000)Imagery neurons (Kreiman & coworkers, 2000) Respond to both perceiving and imagining an objectRespond to both perceiving and imagining an object Single cell recording, but in humansSingle cell recording, but in humans

fMRI & PETfMRI & PET most studies show overlap for perceiving and most studies show overlap for perceiving and

imagining (visual cortex, frontal lobes)imagining (visual cortex, frontal lobes) More activity toward the back of the brain for More activity toward the back of the brain for

perception than imageryperception than imagery Nonvisual areas deactivated (Amedi & coworkers, Nonvisual areas deactivated (Amedi & coworkers,

2005)2005)


Transcranial Magnetic StimuluationTranscranial Magnetic Stimuluation Method for temporarily disrupting function Method for temporarily disrupting function

in part of the brainin part of the brain Lets researchers demonstrate causality, not Lets researchers demonstrate causality, not

just correlation (if you disrupt it and people just correlation (if you disrupt it and people can still do the task it cannot be causal)can still do the task it cannot be causal)

Kosslyn and coworkers (1999) used this Kosslyn and coworkers (1999) used this method to show that areas involved in both method to show that areas involved in both perception and imagery were causally perception and imagery were causally linked to imagery (not an epiphenomenon)linked to imagery (not an epiphenomenon)


Neuropsychological case studiesNeuropsychological case studies Some evidence of related mechanismSome evidence of related mechanism

MGS: mental imagery changed after removal of MGS: mental imagery changed after removal of occipital lobe tissueoccipital lobe tissue

Perceptual problems accompanied by problems with Perceptual problems accompanied by problems with imageryimagery

Unilateral neglect (Map of part of Milan study)Unilateral neglect (Map of part of Milan study)

Some evidence of independenceSome evidence of independence Dissociations between imagery and perceptionDissociations between imagery and perception

R.M. Could see and draw but not imagine and draw, etc.R.M. Could see and draw but not imagine and draw, etc. C.K. Visual agnosia (could not name pictures) but OK C.K. Visual agnosia (could not name pictures) but OK

drawing from imaginationdrawing from imagination

Putting it all TogetherPutting it all Together

Some evidence for common Some evidence for common mechanism BUT also some evidence mechanism BUT also some evidence for separate mechanismsfor separate mechanisms Behrmann & Coworkers (1994) have an Behrmann & Coworkers (1994) have an

account that brings them together (top-account that brings them together (top-down vs. bottom-up)down vs. bottom-up)

No true “answer,” just continuing No true “answer,” just continuing debatedebate

Other stuff…Other stuff…

Using Images to Improve Using Images to Improve MemoryMemory

Method of LociMethod of Loci ““Placing” objects at places within a Placing” objects at places within a

familiar location to remember them – familiar location to remember them – walking through that location when walking through that location when rememberingremembering

Peg-word TechniquePeg-word Technique Learning a rhyme (one is a bun, two is a Learning a rhyme (one is a bun, two is a

shoe…) and then imagining things you shoe…) and then imagining things you want to remember asssociated with those want to remember asssociated with those thingsthings

Mental Representation of Mental Representation of Mechanical SystemsMechanical Systems

People can use visual imagery to People can use visual imagery to solve problemssolve problems Five-gear problemFive-gear problem Water-pouring problemWater-pouring problem Pulley problemPulley problem

It has also been important for It has also been important for scientific discoveryscientific discovery Einstien’s theory of relativityEinstien’s theory of relativity

The EndThe End

Documents

Memory and Cognition