Chapter 2: Modeling mental imagery

Chapter 2:Modeling mental imagery

Cognitive Science José Luis Bermúdez / Cambridge University Press 2010

The ingredients

•Encountered some of the basic ideas feeding into cognitive science

• move away from associationist models of learning and behavior

• information theory as a tool for exploring the nature and limits of cognitive abilities

• development of “boxological” accounts of how cognitive tasks can be performed

• theory of computation as a model for information-processing


Putting them together: 3 case studies

• Terry Winograd and SHRDLU [TODAY]

• The imagery debate [MONDAY]

• Marr’s theory of vision [WEDNESDAY]


Earlier themes

•The nature of mental representation

– Miller and chunking information-processing depends on how information is coded

– Winograd and procedural semantics representation of “knowledge” in terms of algorithmic routines


Common assumptions about information

•Information is amodal

• Miller’s suggestion that the sensory systems all have the same channel capacity

•Information is coded in a digital/propositional format

• based on the formal languages used to program computers


Digital information-coding

•Information is coded in a format that has the basic properties of a language

• Basic constituents are individual symbols

• Compositionality – complex structures are built up from individual symbols according to formation rules

• Arbitrary connections between symbolic structures and what they represent


Digital information-processing

•The model for thinking about digital information-processing are formal languages (e.g. logical languages and computer programming languages)

•Model information-processing on, e.g.

• proofs in logical languages

• implementation of instructions in a production system


Imagistic information-coding

•Non-symbolic: images are not built up from basic elements

•Not compositional

• The parts of images cannot reoccur in other images

• No rules for building up images from their parts


Representation in images

•Representation depends upon systematic correlation between properties of representation and properties of what it represents

• pictorial depiction depends upon resemblance

• can be schematic resemblance, as in a map


Tricky issues

•Imagistic representations can exploit symbols (e.g. maps)• need to distinguish between the representation

and the labeling of the representation

•Imagistic representations ≠ analog representations• a representation is analog just if it permits

continuous variation• there are examples of analog representartions

that are not imagistic and imagistic representations that are not analog


Imagistic information-processing

•The real issue comes with how information is extracted from imagistic representations

• scanning images

• manipulating images (e.g. rotation)

•Certain types of information are much easier to extract from images than from digital representations


The issues for cognitive science

•Is information always encoded in a digital format - or are there cases of imagistically encoded information?

•How can we explore this experimentally?

• By looking at how subjects carry out information-processing tasks involving images

• Seeing whether their behavior provides indirect evidence that they are scanning/manipulating images


Brooks 1968

F• Form a memory image of a capital F• Trace around the image, starting at the bottom left corner and working clockwise• Indicate for each corner whether it is on a top edge of the figure

• Performance is impaired when responses are made visually (i.e. by pointing to the word ‘Yes’), rather than by saying ‘yes’



Cooper 1975


Scanning mental images


The strong interpretation

• Subjects perform the task by rotating/scanning mental images in their “mind’s eye”

• The process of mental rotation/scanning has is structurally similar to physical processes of rotation/scanning

• Seems to match evidence from introspection


Problems with the strong interpretation

• Dennett’s “Cartesian theatre”

• Who or what is doing the scanning/rotating?

• Where is the image projected?

• Threat of regress if we take the metaphor of the “mind’s eye” literally

• Not clear how these mental images relate to “phenomenal images”


Kosslyn’s theory

• Develops metaphor of images as spatial displays on cathode ray tube

• Mental images are temporarily generated from propositionally encoded information in long-term memory

• Mental images “projected” onto visual buffer (which is where perceptual representations also appear)


Solving a problem


Ambiguity

• Personal-level phenomena• phenomenal images• conscious experience of the world• accessible to introspection (not always reliable)

• Subpersonal information-processing explains our personal-level phenomena and abilities

Documents

Chapter 2: Modeling mental imagery