Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
PART-WHOLE RELATIONSHIPS
IN VISUAL PERCEPTION
JOHAN WAGEMANS
LABORATORY OF EXPERIMENTAL PSYCHOLOGY UNIVERSITY OF LEUVEN
PRINCETON, NOVEMBER 22, 2011
Some examples
Some examples
Some examples
Some examples
Overview
• Part 1: historical and conceptual background
• Part 2: some recent empirical studies – bistable diamonds
• Murray et al. (2002) • Fang et al. (2008)
– motion silencing • Suchow & Alvarez (2011) • Poljac et al. (subm.)
– configural-superiority effect • Kubilius et al. (2011)
• Part 3: general discussion
HISTORICAL AND CONCEPTUAL BACKGROUND
PART 1
How Gestalt psychology started
Wertheimer, M. (1912). Experimentelle Studien über das Sehen von Bewegung. Zeitschrift für Psychologie, 61, 161-265. phi motion
Steinman, R. M., Pizlo, Z., & Pizlo, F. J. (2000). Phi is not beta, and why Wertheimer’s discovery launched the Gestalt revolution. Vision Research, 40, 2257-2264.
http//psych.purdue.edu/magniphi/ key role
phi as pure motion, not a displacement between two objects
phi as a process, “an across in itself”, that cannot be composed from the usual optical contents
Max Wertheimer (1880-1943)
A radical vision
emerging Gestalt theory not Gestalt qualities added to the primary sensations not Gestalts as more than the sum of the parts but Gestalts as different from the sum of the parts often the whole is grasped even before the individual parts
enter consciousness a structured unit emerges as a whole
psychological facts and physiological hypotheses went hand-in-hand continuous whole-processes rather than associated
combinations of elementary excitations specifically: some kind of physiological short circuit, and a
flooding back of the current flow, creating a unitary continuous whole-process
Some early Gestalt history
Koffka, K. (1915). Beitrage zur Psychologie der Gestalt. III. Zur Grundlegung der Wahrnehmungspsychologie. Eine Auseinandersetzung mit V. Benussi. Zeitschrift für Psychologie, 73, 11-90. implications of this view primary relations
no longer stimulus ~ sensation but stimulus pattern ~ perceived whole
perceived wholes not constructed in the mind from elementary sensations but direct experience-correlates emerging in the brain
Kurt Koffka (1886-1941)
Some early Gestalt history
Köhler, W. (1920). Die physischen Gestalten in Ruhe und im stationären Zustand. Eine natur-philosophische Untersuchung. Braunschweig, Germany: Vieweg.
decisive step: real physical Gestalts in the brain
strong Gestalts the mutual dependence among the parts is so great that
no displacement or change of state is possible without influencing all the other parts of the system
in fact: there are no parts at all, only interacting moments of structure that carry one another
psychophysical isomorphism psychological facts and the brain events that underlie
them are similar in all of their structural characteristics specifically: visual Gestalts result from a single Gestalt
process in which the whole optic sector from the retina onwards is involved, including transverse functional connections
in fact: the brain described as a self-organizing physical system
Wolfgang Köhler
(1887-1967)
From speculation to facts
Köhler, W., & Held, R. (1949). The cortical correlate of pattern vision. Science, 110, 414-419.
first recordings of visual currents, picked up by an electrode at the scalp of human observers
“electrical field theory”
Lashley, K. S., Chow, K. L., & Semmes, J. (1951). An examination of the electrical field theory of cerebral integration. Psychological Review, 58, 123-136.
more direct test of electrical field theory rationale: insulate part of a cortical field and test for consequent
disturbances of function metallic strips and pins inserted in macaque cortex almost no effect on post-operative retention of object discrimination devastating blow to
electrical field theory basic isomorphism postulate of Gestalt theory
Single-neuron doctrine
Hubel & Wiesel: big success
Barlow, H. (1972): Single units and sensation: A neuron doctrine for perceptual psychology. Perception, 1, 371-394.
our perceptions are caused by the activity of a small number of neurons the activity of a single neuron is related quite simply to our subjective
experience
reductionist, elementalist approach which Gestalt theorists had banished
mapping of responses of single neurons in LGN, striate and extrastriate cortex in cat and monkey
tuning to simple stimulus attributes (e.g., orientation)
single neurons interpreted as “feature detectors” (e.g., line detectors, edge detectors)
Further developments
single-unit recording flourished tuning properties of different types of cells in different areas of
the brain functional specialization hierarchical organization
retinotopy decreases invariance increases complexity of the “features” increases
confirmed in human fMRI (modules, maps) standard view
Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1, 1-47.
Grill-Spector, K., & Malach, R. (2004). The human visual cortex. Annual Review of Neuroscience, 27, 649-677.
Serre, T., Oliva, A., & Poggio, T. (2007). A feedforward architecture accounts for rapid categorization. Proceedings of the National Academy of Science of the USA, 104, 6424-6429.
Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1, 1-47.
Grill-Spector, K., & Malach, R. (2004). The human visual cortex. Annual Review of Neuroscience, 27, 649-677.
Serre, T. et al. (2007). A feedforward architecture accounts for rapid categorization. PNAS, 104, 6424-6429.
Re-emergence of Gestalt issues
surround influences from outside classic receptive field (cRF) Allman, J., Miezin, F., & McGuinness, E. (1985). Direction- and
velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception, 14, 105-126.
neural responses to illusory contours
von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260-1262.
neural responses to figure-ground organization
Zhou, H., Friedman, H. S., & von der Heydt, R. (2000). Coding of border-ownership in monkey visual cortex. Journal of Neuroscience, 20, 6594-6611.
Allman, J. et al. (1985). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT).
Perception, 14, 105-126.
Allman, J. et al. (1985). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT).
Perception, 14, 105-126.
von der Heydt, R. et al. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260-1262.
Zhou, H. et al. (2000). Coding of border-ownership in monkey visual cortex. Journal of Neuroscience, 20, 6594-6611.
A modern view on a radical vision
more general models Hochstein, S., & Ahissar, M. (2002). View from the top:
Hierarchies and reverse hierarchies in the visual system. Neuron, 36, 791-804.
Bar, M. et al. (2006). Top-down facilitation of visual recognition. Proceedings of the National Academy of Science of the USA, 103, 449-454.
interesting characteristics from viewpoint of Gestalt theory “wholes” come first highly interactive highly dynamic not limited to visual areas
Hochstein, S., & Ahissar, M. (2002). View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron, 36, 791-804.
Bar, M. et al. (2006). Top-down facilitation of visual recognition. PNAS, 103, 449-454.
Interim conclusions
• “parts” versus “wholes” constitutes a prominent theme in vision science, from Gestalt psychology up until today
• the pendulum seems to have swung back-and-forth
• new attempts towards a synthesis are emerging
• new research inspired by these views
Some conceptual points
• “parts” and “wholes” are used generically (not
just parts of objects)
• parts: features, components, constituents • wholes: objects, compositions, configurations,
Gestalts
• parts are generally smaller than wholes
• hierarchical relationship: – parts belong to wholes – wholes consist of parts
BISTABLE DIAMONDS: MURRAY ET AL. (2002) FANG ET AL. (2008)
PART 2A
Murray et al.: Key papers
• Murray, S. O., Kersten, D., Olshausen, B. A.,
Schrater, P., & Woods, D. L. (2002). Shape perception reduces activity in human primary visual cortex. Proceedings of the National Academy of Sciences, 99, 15164-15169.
• Fang, F., Kersten, D., & Murray, S. O. (2008).
Perceptual grouping and inverse fMRI activity patterns in human visual cortex. Journal of Vision, 8(7):2, 2-9. doi: 10.1167/8.7.2
Demonstration
Demonstration
Demonstration
Nice features
• perceptual bi-stability: – “parts” seen to move vertically – “whole” seen to move horizontally
• switching relatively slow, perceptual states rather clear
• stable individual differences
• studied rather extensively at psychophysical level, e.g. – Lorenceau, J. & Shiffrar, M. (1992). The influence of
terminators on motion integration across space. Vision Research, 32, 263-273.
Murray et al.: Design
• present bistable diamonds
• ask observers to indicate perception of “parts”
(line segments) or “whole” (diamond)
• record BOLD responses (fMRI) in different areas and relate these to the reported percepts
Murray et al.: Results
Murray et al.: Results
Murray et al.: Results
Murray et al.: Results
Interim conclusions
• convincing demonstration of inverse activity patterns in V1 and LOC
• interpretation? – perception of “parts” suppressed by perception of
“whole” – direct interaction between V1 and LOC? – are reductions of V1 activity caused by feedback from
LOC? – are reductions in V1 necessary for the perception of the
whole?
MOTION SILENCING: SUCHOW & ALVAREZ (2011) POLJAC, DE-WIT, & WAGEMANS (SUBM.)
PART 2B
Suchow & Alvarez (2011)
• Suchow, J. W., & Alvarez, G. A. (2011). Motion silences awareness of visual change. Current Biology, 21(2), 140-143. doi:10.1016/j.cub.2010.12.019
• “Best Illusion of the Year 2011”
Demonstration
Demonstration
More demonstrations
Methods
• Stimuli: – 100 dots – first stationary, then rotating back and forth for 30° – 2 phases alternating every 3 s
• Task: – observers had to adjust the rate of change during the
stationary phase to match the apparent rate of change in the moving phase
– rate of change (“silencing factor”) between 0.1 (static perceived as changing slower) and 10 (static perceived as changing faster)
Results
Interpretation
• Suchow & Alvarez:
– local mechanisms with small receptive fields – because a fast-moving object spends little time at any
one location, a local detector is afforded only a brief window in which to assess the changing object
• alternative interpretation: – objecthood – when a good “whole” is formed, the details of the
“parts” are fundamentally less accessible to conscious perception
Larger framework
• this idea is consistent with
– rapid access to the gist of a given image (e.g., Thorpe et al., 1996)
– sometimes surprisingly limited access to parts that make up the interpretation (e.g., Rensink et al., 1997)
• perhaps these reflect 2 sides of the same coin: providing only the most critical information (Hochstein & Ahissar, 2002)
Our study
• motivation: to test this alternative interpretation and more general idea with biological motion
• Poljac, E., de-Wit, L., & Wagemans, J. (submitted). Perceptual wholes can reduce the awareness of their changing parts.
Why biological motion?
• a prototypical case of a complex hierarchical stimulus (Johansson, 1973; Cutting & Proffitt, 1982) – multiple elements, each with their own spatio-temporal
trajectories – organized quickly and efficiently in a hierarchical configuration,
in which the motion of the local elements are coded relative to a more global structural description
• the perceptual Gestalt is constructed automatically by the
visual system (Thornton & Vuong, 2004)
• the construction of the perceptual whole implies a more efficient representation of the relationships between the parts (Tadin et al., 2002)
• inversion allows control over low-level motion trajectories (Sumi, 1984; Pavlova & Sokolov, 2000)
Demonstrations
Demonstrations
Demonstrations
Methods
• Stimuli:
– motion captured point-light treadmill walkers (Vanrie & Verfaillie, 2004)
– 70 colored dots (“confetti walker”) – upright, inverted, phase-scrambled
• Task: adjust rate of change in the test figure until it matches the range of change in the comparison figure
• Experiment 1: all static – comparison figure: scrambled – test figures: upright, inverted or scrambled
• Experiment 2: dynamic and static – comparison figure: scrambled – test figures: upright or inverted
Results: Experiment 1
Results: Experiment 2
Discussion
• on top of the effect of static vs moving, there is
a clear effect of configurality (“goodness” of the whole percept)
• cost of objecthood: the more strongly the parts are integrated into the perception of a whole object, the less accessible the changing features of the parts are (e.g., also embedded figures)
Interim conclusions
• two sides of the same coin:
– the rapid extraction of a perceptual Gestalt – the inaccessibility of the parts that make up
that Gestalt
• general principle: – human vision provides only the most useful
level of abstraction to conscious awareness
CONFIGURAL-SUPERIORITY EFFECT KUBILIUS ET AL. (2011)
PART 2C
Point of departure
• configural-superiority effect
– Pomerantz, J. R., Sager, L. C., & Stoever, R. J. (1977). Perception of wholes and their component parts: Some configural superiority effects. Journal of Experimental Psychology: Human Perception and Performance, 3, 422-435.
– Pomerantz, J. R., & Portillo, M. C. (2011). Grouping and emergent features in vision: Toward a theory of basic Gestalts. Journal of Experimental Psychology: Human Perception and Performance, 37, 1331-1349.
Kubilius et al. (2011)
• Kubilius, J., Wagemans, J., & Op de Beeck, H. P. (2011). Emergence of perceptual Gestalts in the human visual cortex: The case of the configural superiority effect. Psychological Science, 22(10), 1296-1303.
• behavioral results
• fMRI decoding results
Behavioral results
parts corner whole
Scanning protocol
fMRI results: Retinotopic mapping
MVPA: decoding
fMRI results: decoding
fMRI results: decoding
Interim conclusions
• behavioral configural-superiority effect
• neural configural-superiority effect: – better coding of “wholes” than “parts” in higher shape-
selective regions – better coding of “parts” than “wholes” in lower-level
retinotopic regions
• general conclusions: – at least some Gestalts emerge only at higher stages of
visual information processing – feedforward processing may be sufficient to produce
some Gestalts
GENERAL DISCUSSION
PART 3
Some tentative conclusions
• first set of studies: – “wholes” dominate and “parts” disappear from experience – “wholes” emerge in higher areas of the brain and encoding of “parts” is
then suppressed
• second set of studies: – functional “wholes” also arise spontaneously and “parts” become less
functional – but still, the encoding of these “wholes” at higher levels of the cortical
hierarchy does not suppress the encoding of the “parts”
• more generally: – not all Gestalts are created equal – probably not all Gestalts are created equally – we seem to get better handles to understand the relations between
“parts” and “wholes” in visual perception – a lot of work remains to be done
Announcement: VSS symposium
Friday, May 11, 2012 “Part-whole relationships
in visual cortex”
• Johan Wagemans: “Part-whole relationships in vision science: A brief historical review and conceptual analysis”
• Charles E. Connor: “Ventral pathway visual cortex: Representation by parts in a whole object reference frame”
• Scott O. Murray: “Long-range, pattern-dependent contextual effects in early human visual cortex”
• James R. Pomerantz: “The computational and cortical bases for configural superiority”
• Jacob Feldman: “Computational integration of local and global form” • Shaul Hochstein: “The rise and fall of the Gestalt gist”
THANK YOU
WWW.GESTALTREVISION.BE
Part-Whole Relationships �in Visual Perception�Some examplesSome examplesSome examplesSome examplesOverviewhistorical and conceptual backgroundHow Gestalt psychology startedA radical visionSome early Gestalt historySome early Gestalt historyFrom speculation to factsSingle-neuron doctrineFurther developmentsFelleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1, 1-47.Grill-Spector, K., & Malach, R. (2004). The human visual cortex. �Annual Review of Neuroscience, 27, 649-677.Serre, T. et al. (2007). A feedforward architecture accounts for rapid categorization. PNAS, 104, 6424-6429.Re-emergence of Gestalt issuesAllman, J. et al. (1985). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception, 14, 105-126.Allman, J. et al. (1985). Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception, 14, 105-126.von der Heydt, R. et al. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260-1262.Zhou, H. et al. (2000). Coding of border-ownership in monkey visual cortex. Journal of Neuroscience, 20, 6594-6611.A modern view on a radical visionHochstein, S., & Ahissar, M. (2002). View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron, 36, 791-804.Bar, M. et al. (2006). Top-down facilitation of visual recognition. �PNAS, 103, 449-454.Interim conclusionsSome conceptual pointsBistable diamonds:�Murray et al. (2002)�Fang et al. (2008)Murray et al.: Key papersDemonstrationDemonstrationDemonstrationNice featuresMurray et al.: DesignMurray et al.: ResultsMurray et al.: ResultsMurray et al.: ResultsMurray et al.: ResultsInterim conclusionsMotion silencing:�Suchow & alvarez (2011)�Poljac, De-wit, & wagemans (subm.)Suchow & Alvarez (2011)DemonstrationDemonstrationMore demonstrationsMethodsResultsInterpretationLarger frameworkOur studyWhy biological motion?DemonstrationsDemonstrationsDemonstrationsMethodsResults: Experiment 1Results: Experiment 2DiscussionInterim conclusionsConfigural-superiority effect�Kubilius et al. (2011)Point of departureKubilius et al. (2011)Behavioral resultsScanning protocolfMRI results: Retinotopic mappingMVPA: decodingfMRI results: decodingfMRI results: decodingInterim conclusionsGeneral discussionSome tentative conclusionsAnnouncement: VSS symposiumThank You