Visually-induced Visually-induced auditory spatial adaptation auditory spatial adaptation
in monkeys and humansin monkeys and humans
Norbert Kopčo, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh
Center for Cognitive Neuroscience, Duke UniversityHearing Research Center, Boston University
Technical University, Košice, Slovakia
2Nov 6, 2007 SFN 07 San Diego
Introduction
Visual stimuli can affect the perception of sound location
e.g. the Ventriloquism Effect
Way to go
Red Sox! Way to go
Red Sox!
But does effect persist?
3Nov 6, 2007 SFN 07 San Diego
Introduction
Visual stimuli can affect the perception of sound location
e.g. the Ventriloquist Effect
Way to go
Red Sox!
But does effect persist?- barn owls: prism adaptation (Knudsen et al.)- monkeys: “ventriloquism aftereffect” (Woods and Recanzone, Curr. Biol. 2004)
4Nov 6, 2007 SFN 07 San Diego
GOALS
1. Ventriloquism “aftereffect” in saccade task, in monkeys and humans?1. - well-defined sensory-motor paradigm
2. - bridge to barn owl prism adaptation studies (on different time scale)
2. Reference frame of plasticity?1. - Visual, auditory, or oculomotor reference frame?
5Nov 6, 2007 SFN 07 San Diego
Methods
Basic idea:
1. Pre-adaptation baseline: Measure auditory saccade accuracy
2. Adaptation phase: Present combined visual-auditory stimuli, with visual location shifted
3. Compare auditory saccade accuracy pre- and post-adaptation
6Nov 6, 2007 SFN 07 San Diego
Methods
Initial experiment: Does it work?
Design:MonkeyPre-adaptation baseline – ~100 Auditory-only trialsAdaptation phase –
80% V-A stimuli, visual stimulus shifted 6 deg. Left or Right
20% Auditory-onlyCompare Auditory-only trials from adaptation phase to pre-
adaptation phase
Sounds: Loudspeakers
Visual stimuli: LEDs
7Nov 6, 2007 SFN 07 San Diego
RESULTS
8Nov 6, 2007 SFN 07 San Diego
RESULTS
9Nov 6, 2007 SFN 07 San Diego
RESULTS
10Nov 6, 2007 SFN 07 San Diego
RESULTS
11Nov 6, 2007 SFN 07 San Diego
Methods: reference frame
Eye-centered?
Head (ear) -centered?
Oculomotor?
?
?
12Nov 6, 2007 SFN 07 San Diego
Method
Audiovisual display Expected behavior
Stimulus Location (°)
Mag
nitu
de (
°)
Induce shift: - in only one region of space- from a single fixation point
Test to see if shift generalizes to the same sub-region in:- head-centered space- eye-centered space
Experiment divided into 1-hour blocks:(12 for humans, 16
monkeys)
Within a block, 3 types ofrandomly interleaved trials:- AV 50%, - A-only, trained FP- A-only, shifted FP
13Nov 6, 2007 SFN 07 San Diego
Results: HumansAudiovisual display
Expected Responses
FPLEDs
Speakers
Stimulus Location (°)
Mag
nitu
de o
f Ind
uced
Shi
ft (°
)
or
Trained FP A-onlyresponses:- Shift induced in trained sub-region- Generalization to untrained regions (asymmetrical)
Shifted FP A-onlyresponses:- Shift reduced in center region
Head-centeredrepresentation,modulated by eye position
14Nov 6, 2007 SFN 07 San Diego
Results: MonkeysAudiovisual display
Expected Responses
FPLEDs
Speakers
Stimulus Location (°)
Mag
nitu
de o
f Ind
uced
Shi
ft (°
)
or
Trained FP A-onlyresponses:- Shift in trained sub- region weaker- Generalization to untrained regions stronger (asymmetry oppo- site to humans)
Shifted FP A-onlyresponses:- Shift decreases on the right- Shift increases on the leftHumans:
Representation
more mixed
15Nov 6, 2007 SFN 07 San Diego
SummaryThe main results are consistent across species:
Locally induced ventriloquist effect results in short-term adaptation, causing shifts in responses to A-only stimuli from trained sub-region.
The induced shift generalizes outside the trained sub-region, with gradually decreasing strength (However, the pattern of generalization differs across the species)
The pattern of induced shift changes as the eyes move. But, overall, it appears to be in a representation frame that is more head-centered than eye-centered.
16Nov 6, 2007 SFN 07 San Diego
DiscussionPosterior ParietalCortex
Neural adaptation could have beeninduced at several stages along thePathway (IC, MGB, AC, PPX, MC,SC).
In humans, multiple effects observed at different temporalscales likely adaptation atmultiple stages
Future workExtend Examine temporal and
spatial factors influencing the eye-centered modulation.
Look at other trained sub-regions.
Midbrain
Pons
Cerebrum
Thalamus
Midbrain
Pons
Thalamus
17Nov 6, 2007 SFN 07 San Diego
Summary and discussionthe main results are consistent across species:
Locally induced ventriloquist effect results in short-term adaptation, causing shifts in responses to A-only stimuli from trained sub-region.The pattern of induced shift is modified as the eyes move.
Bad news – there is a lot of differences between species:
Humans MonkeysRepresentation head-centered, eye-modulated eye-centered
Generalizationto untrained sub-regions more on the side away from FP opposite
Difference betweenhyper- and hypometricshifts no yes
Representation whenshift induced on side (data not shown) head-centered, no eye modulation eye-centered
18Nov 6, 2007 SFN 07 San Diego
Results: HumansAudiovisual display
Expected Responses
FPLEDs
Speakers
Human Behavior
Mean+SE
Stimulus Location (°)
Mag
nitu
de o
f Ind
uced
Shi
ft (°
)
19Nov 6, 2007 SFN 07 San Diego
Results: Humans
Stimulus Location (°)
Audiovisual display Expected Responses
FPLEDs
SpeakersHuman Behavior Data collapsed across
direction of inducedshift
Trained FP A-onlyresponses:- Shift induced in trained sub-region- Generalization to untrained regions (asymmetrical)
Shifted FP A-onlyresponses:- Shift reduced in center regionHead-centered repre-sentation, modulated by eye position
Mean+SE
20Nov 6, 2007 SFN 07 San Diego
Results: Humans vs. MonkeysAudiovisual display
FPLEDs
SpeakersHuman Behavior Monkey data
(only hypometric)
AV responses:- as expected
Trained FP A-onlyresponses:- Shift in trained sub- region weaker- Generalization to untrained regions stronger (asymmetry opposite to humans)
Shifted FP A-onlyresponses:- Shifted with eyes
Representation moreeye-centered
Mean+SE
Monkey Behavior
Mean
6
6-24 0 24
-24 0 24
-24 0 24
Mean+individualsMean+SE
Mean+SE Mean+individuals