Upload
silas
View
17
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Connecting Sound with the Mind’s Eye: Multisensory Interactions in Music Conductors. W. David Hairston, Ph.D Advanced Neuroscience Imaging Research Lab Department of Radiology; Wake Forest University School of Medicine, Winston-Salem, NC 27157. - PowerPoint PPT Presentation
Citation preview
Connecting Sound with the Mind’s Eye: Multisensory Interactions in Music
Conductors
W. David Hairston, Ph.D
Advanced Neuroscience Imaging Research LabDepartment of Radiology; Wake Forest University School
of Medicine, Winston-Salem, NC 27157
Multisensory enhancement within a number of paradigms, including:
• Simple reaction times
• Saccadic response latencies
• Signal detection
• Orientation/localization behavior in cat
General conditions for enhancements to be observed
-spatial alignment
-temporal congruence
-minimal efficacy (“inverse effectiveness”)
Stimuli:
Vis (LED)
Aud (broadband)
Vis-Aud
50 ms, 30 reps
What about human localization ability
Hairston et al, J. Cog. Neuro, 2003
0
10
20
30
40
50
60
70
80
-15 -13 -11 -9 -7 -5 -3 -1 1 3 5 7 9 11 13 15
Error (degrees)
Pe
rce
nt
Visual
Multisensory
Auditory
0
5
10
15
20
25
30
35
-15 -10 -5 0 5 10 15
Error (degrees)
Pe
rce
nt
Evaluating Precision
Precision ~ Std Dev
0
2
4
6
8
10
12
-40 -20 -10 0 10 20 40
Target Position (deg.)
Lo
cali
zati
on
Var
iab
ilit
y (d
eg.)
Visual
Auditory
Multisensory
Hmmmm…. Why?
A
C
B
S1 S2 M
The total amount of gain/ enhancement observed is determined by the relative contribution of each sense, based upon its own perceptual acuity
How do you increase auditory acuity?
Conductors as auditory localizers
Daily, career experience requires rapid and accurate assessment of auditory scene
Also requires coordination of multisensory information – “who” played “what” wrong note, etc
20 conductors, min 7 years “podium experience” (ave 10.2)
Matched on age, education, sex, etc
(Hodges, Hairston & Burdette ’06)442.8
454.2
440
445
450
455
460
465
470
440 Hz Base Tone
Fre
quency
(H
z)Conductors
Non-Conductors
A#
A
0
1
2
3
4
5
6
7
8
0 10 20 30 40
Target Location (+/- deg.)
Pre
cisi
on
(+
/- d
eg.)
Auditory
Multisensory
Visual
Visual and multisensory performance very similar
Non-musicians
Multisensory performance enhanced
0
1
2
3
4
5
6
7
8
0 10 20 30 40
Target Location (+/- deg.)
Pre
cisi
on
(+
/- d
eg.)
Auditory
Multisensory
Visual
Control
0
1
2
3
4
5
6
7
8
0 10 20 30 40
Target Location (+/- deg.)
Pre
cisi
on
(+
/- d
eg.)
Auditory
ControlImproved auditory performance
0
2
4
6
8
10
12
-7 -2 3 8 13
% Auditory Improvement (> Control)
% M
ulti
sen
sory
E
nh
ance
men
t (>
Vis
ual
)
10
40
3020
Conclusions from this…
• While untrained subjects gain little from an additional auditory cue, music conductors appear to benefit from additional auditory signals, for which their spatial acuity is much better.
• The degree of multisensory gain in this unique population appears to be tied to their improved auditory performance
• These results suggest that the specialized training and experience of these individuals has a profound affect within both the auditory and multisensory realms
But what if…
• Multisensory “enhancements” are beneficial when the task can make use of additional information
• BUT - When do not match or are not relevant – can be detrimental or inhibiting….
-Slower RT
-Biased localization
-Illusions and misperceptions
Many times the task at hand required focusing on one sense and ignoring other.
“Tuning out” irrelevant information…
Current studies…
• Physiological effects of “tuning in” to one sense over another
• “Cross-modal deactivation”: Decreased activity within one sensory cortex related to stimulation of another
E.g., decreased BOLD signal in occipital cortex during auditory task
• Some evidence for relation to selective attention
Unclear whether the extent of deactivation observed is related to the difficulty of the task at hand
MethodsGoals:
Does changing task difficulty affect cross-modal deactivation?
Does extreme acuity within one modality, and unique multisensory training affect this process?
Subjects- Non-musicians – various careers, no formal musical training, 25-40 y/o- Conductors – min. 5 yrs podium experience
Matched on age, gender, education, etcWhy conductors?
- Heightened auditory acuity - Conducting activities require NOT inhibiting other
(e.g., visual) information
Methods: Tasks
Pitch Discrimination
440 Hz 660 Hz
440 Hz 660 Hz
Easier
Difficult
60 ms
20 ms
Temporal Discrimination
440 Hz 456 Hz
440 Hz 443 Hz
Easier
Difficult(Time)
Base Test
Subjects’ thresholds for each task acquired prior to fMRI scanning
Methods: Thresholding
Acquired threshold
0
20
40
60
80
100
120
1 6 11 16 21 26 31 36 41 46 51
Presentation orderS
OA
(m
s)
Allows compensation for variability in perceptual acuity between subjects and groups
2 down/1 up rule
Perceptual acuity, non-musicians vs. conductors
430
440
450
460
470
Conductors Non-musicians
Fre
qu
ency
(H
z)
0
20
40
60
80
100
Conductors Non-musicians
SO
A (
ms)
A
A#
Base comparison
Conductors
Non-Musicians
Non-musicians need a significantly larger difference to discriminate tones than conductors.
Methods: fMRI
Each task performed at 2 levels
- at threshold (“difficult”)- above threshold (“moderate”)
Also performed visual temporal discrimination (2 circles) Visual
Temporal discrimination
silence TRTR(Scanner ON) (Scanner ON)(Scanner OFF)
Trial Trial Trial
Sparse sampling (10 s pause), block design
Schmuel et al, Nature Neuroscience, 2006
Positive BOLD
Positive BOLD
Analysis
• Activity in task (ON) blocks contrasted against resting baseline (OFF)
Baseline: eyes open on fixation, no task• ROI of visual-responsive occipital cortex
2.78 7.0
-2.78 -7.0 Used to generate summary stats
Non-musicians: Moderate
Deactivation of visual cortex
+/-3.95 +/-8.0
Vis ROI
Non-musicians: Difficult
Robust deactivation of visual cortex
+/-3.95 +/-8.0
Vis ROI
Non-musicians: Difficult vs. Moderate
EasierDifficult
-3
-2
-1
0Voxel (Y coord)
Me
an
Sig
na
l
-3
-2
-1
0Voxel (Y coord)
Me
an
Sig
na
l
-1
-0.5
0
0.5
1
Mea
n S
igna
l (%
)
-650
-450
-250
-50
150
Tota
l Sig
nal M
agni
tude
0
200
400
600
800
1000
# of
Vox
els
Non-musicians: stats
ROI A.Cort.Post. Cing.
V Cort.
ROI
A.Cort.
Post. Cing. V Cort.
# significant voxels
Total Signal Magnitude
Mean signal
Conductors
DifficultModerate
Vis ROIVis ROI
+/-3.95 +/-8.0
Only slight deactivation of visual cortex in both cases
Conductors: Difficult vs. Moderate
Easier Difficult
0
200
400
600
800
1000
# o
f V
ox
els
-650
-450
-250
-50
150
To
tal S
ign
al M
ag
nit
ud
e
ROI A.Cort.Post. Cing.
V Cort.
# significant voxels
Total Signal Magnitude
Non-musicians vs. Conductors
Difference seen when task is difficult
Non-Musicians Conductors
0
40
80
120
160
200
Easier Difficult
N S
ign
if V
oxe
ls
Non-musicians
Conductors
n.s.
**
Difficult Task
0
20
40
60
80
High M oderate
Acc
ura
cy(%
)
Conclusions
• The degree and extent of cross-modal deactivation observed specifically depend of the task difficulty; when the same task is easier, cross-modal deactivations are attenuated.
• Contrary to non-musicians, conductors show only minor cross-modal deactivation, even when the task is very difficult to perform.
• This suggests that the role of functional deactivations is dynamic, and adapts to fit the needs of the individual and situation at hand.
Conductors
• Conductors are highly-trained individuals with unique daily activties and specialization
• This experience leads to not only high auditory acuity, but altered interactions when dealing with multiple senses
• Two theories:
- High auditory acuity negates the need for visual suppression.
- Daily experience has led to familiarity with completing complex auditory tasks while also monitoring visual information.
0
15
30
45
60
75
90
Conductors Non-musicians
0
20
40
60
80
High M oderate
Acc
ura
cy(%
)