7/31/2019 ASP Hearing 2007
1/136
Meena Ramani4/2/2007
EEL 6586Automatic Speech Processing
7/31/2019 ASP Hearing 2007
2/136
7/31/2019 ASP Hearing 2007
3/136
Best example of speech recognition Mimic human speech processing
Speech coding e.g. mp3Hearing aids/ Cochlear implants
7/31/2019 ASP Hearing 2007
4/136
7/31/2019 ASP Hearing 2007
5/136
Audible frequency range: 20Hz-20kHzIntensity dynamic range: 0-130 dBJND frequency: 5 cents *JND intensity: ~1dBSize of cochlea: D=2mm L uc=27mm
~Tip of your tiny finger
* one octave=12 equally tempered semi tonesone equally tempered semi tone=100 cents
7/31/2019 ASP Hearing 2007
6/136
7/31/2019 ASP Hearing 2007
7/136
A
N
A
T
O
M
Y
7/31/2019 ASP Hearing 2007
8/136
Function: Focuses sound pressure waves into the ear canal Sound localizationHuman Pinna structure: Symmetrical, curved and pointed forward More sensitive to sounds in front Pinna size: Inverse square lawOthers: Dogs/ Cats: Movable Pinna Elephants: Hear LF sound from up to 5 miles away Barn Owl: Left ear opening is higher than the right
Improvement in localization
Pinna
Auditory Canal
7/31/2019 ASP Hearing 2007
9/136
Interaural differences
1. Horizontal localization2. Vertical localization
PinnaPinna sound localizationsound localization
Is the sound on your right or left side?
7/31/2019 ASP Hearing 2007
10/136
Signal needs to travel further to the more distant earTime difference
More distant ear is partially occluded by the headIntensity differenceInteraural time difference (ITD)Interaural intensity difference (IID)
7/31/2019 ASP Hearing 2007
11/136
Illustration of Illustration of interauralinteraural differencesdifferences
Left
ear
Rightear
soundonset
left right
time
7/31/2019 ASP Hearing 2007
12/136
LeftLeft
earear
RightRightearear
soundsoundonsetonset
timetime
arrival timedifference
Illustration of interaural differencesIllustration of interaural differences
7/31/2019 ASP Hearing 2007
13/136
Left
ear
Rightear
soundonset
time
ongoing timedifference
Illustration of Illustration of interauralinteraural differencesdifferences
7/31/2019 ASP Hearing 2007
14/136
Left
ear
Rightear
soundonset
time
i n t e n s i t y d i f
f e r e
n c e
Illustration of Illustration of interauralinteraural differencesdifferences
7/31/2019 ASP Hearing 2007
15/136
Interaural time differences (ITDs)Threshold ITD 10-20 ms (~ 0.7 cm)
Interaural intensity differences (IIDs)Threshold IID 1 dB
Just noticeable thresholdsJust noticeable thresholds
7/31/2019 ASP Hearing 2007
16/136
Interaural time differences (ITDs)Used for low frequency sound localizationValid up to around 1000 HzSensitivity declines rapidly above 1000 Hz
Smallest phase difference corresponds to true ITD
Interaural intensity differences (IIDs)Used for high frequency sound localizationLess than 500 Hz, IIDs are negligible (due to diffraction)Attenuation varies with frequencyIIDs can reach up to 20 dB at high frequencies
DD
UU
PPLL
EE
XX
TT
HH
EE
OO
RR
YY
Horizontal localizationHorizontal localization
7/31/2019 ASP Hearing 2007
17/136
Pinna directional filtering
Pinna amplifies sound above and below differently Selective spectral amplification
Is sound above or below?
1. Horizontal localization2. Vertical localization
PinnaPinna sound localizationsound localization
7/31/2019 ASP Hearing 2007
18/136
Function: Boosts energy between 2-5Khz by 15dB 2-5kHz region is important for understanding speechStructure:
Auditory canal length: 2.7cm Closed tube resonance: wave resonator Broad peak since the closed end is a pliant ear drum Resonance frequency: ~ 3Khz
Pinna
Auditory Canal
7/31/2019 ASP Hearing 2007
19/136
A
N
A
T
O
M
Y
7/31/2019 ASP Hearing 2007
20/136
Impedance matching Acoustic impedance of the fluid is 4000 x that of air
Need amplification
Amplification By lever action ~ 3x Area amplification [55mm 2 3.2mm 2] 15xStapedius reflex Protection against low frequency loud sounds
Tenses muscles stiffens vibration of Ossicles Reduces sound transmitted (20dB)
Eardrum
Ossicles
Oval windowTranslation of sound wave to vibrations in cochlea
Eardrum Ossicles Oval WindowOssicles: Malleus, Incus, Stapes
7/31/2019 ASP Hearing 2007
21/136
A
N
A
T
O
M
Y
7/31/2019 ASP Hearing 2007
22/136
Semicircular Canals
Cochlea
Function: Keeps body in balance
Canals: Accelerometers in 3 perpendicular planesStructure:
3 perpendicular fluid-filled canals Hair cells in canal detect fluid movements Connected to the auditory nerve
7/31/2019 ASP Hearing 2007
23/136
Function: Cochlea is the body's microphone Converts mechanical movement to electrical action potentialsStructure:
Cochlea is a snail-shell like structure 2.5 turns
(1) Organ of Corti (fluid-filled)
(2) Scala tympani (fluid-filled)
(3) Scala vestibulli (fluid-filled)
(4) Spiral ganglion
(5) Auditory nerve fibers
Semicircular Canals
Cochlea
Inner earInner ear
7/31/2019 ASP Hearing 2007
24/136
[16,000 -20,000] Inner hair cells and Outer hair cells along BM 1 row of IHC, 3 rows of OHCs
IHC: Transmit signals to the brain via auditory nerves ( Afferent )OHC: Transmit feedback signals from the brain via auditory nerves ( Efferent )
7/31/2019 ASP Hearing 2007
25/136
Each hair cell has 100s of tiny stereocilia
7/31/2019 ASP Hearing 2007
26/136
Vibrations of the oval window causes the cochlear fluid to vibrate
BM vibration produces a traveling wave OHCs amplify and sharpen the vibration of the BM Bending of the IHC cilia (in one direction-HWR) produces action potentials Action potentials travel via the auditory nerve to the brain
7/31/2019 ASP Hearing 2007
27/136
7/31/2019 ASP Hearing 2007
28/136
Tonotopic mapping along the BM and for the AN fibers
7/31/2019 ASP Hearing 2007
29/136
Each position along the BM has a characteristic frequency formaximum vibrationFrequency of vibration depends on the place along the BMAt the base , the BM is stiff and thin (more responsive to high Hz )At the apex , the BM is wide and floppy (more responsive to low Hz )
32-35 mm long
Base Apex
7/31/2019 ASP Hearing 2007
30/136
7/31/2019 ASP Hearing 2007
31/136
Human ear resolves 1500 pitches
One pitch every .002cm!Possible by OHC feedback results in sharper response peaks along BM
7/31/2019 ASP Hearing 2007
32/136
The ear produces some sounds! By-product of electro-motile vibrations of the OHCsUses: To test hearing for infants To check if patient is feigning a loss
7/31/2019 ASP Hearing 2007
33/136
Each hair cell has about 10 AN fibersAN fibers carry impulses from cochlea and semicircular canalsAN makes connections with both auditory areas of the brain
Tonotopic mappingNeurons encode Steady state sounds (phase, frequency, intensity) Onsets
Auditory Area of Brain
7/31/2019 ASP Hearing 2007
34/136
Why do your ears pop while on an airplane?Does the pinna continue to grow?Why do you hear waves when you couple a sea-shell to
your ear?How do you perceive pitch while on a telephone?
7/31/2019 ASP Hearing 2007
35/136
Meena Ramani4/4/2007
EEL 6586Automatic Speech Processing
7/31/2019 ASP Hearing 2007
36/136
Anatomy e.g. Cochlea
Psychoacoustics e.g. MaskingEngineering Applications e.g. Hearing aids and Cochlear implants
7/31/2019 ASP Hearing 2007
37/136
7/31/2019 ASP Hearing 2007
38/136
Outer ear: Pinna and Auditory canal Horizontal localization: IID, ITD Vertical localization: Pinna spectral amplification Auditory canal resonance ~3 kHz
Middle ear: Tympanic membrane, Ossicles, Oval window Translates sound pressure to cochlea fluid vibration Ossicles- Lever (15x) and Area amplification (3x) Stapedius reflexInner ear: Cochlea, Semicircular canals Semi-circular canals keep the body in balance Cochlear hair cells convert BM vibration to electrical impulses Impulses are transmitted via auditory nerve to the brain
7/31/2019 ASP Hearing 2007
39/136
7/31/2019 ASP Hearing 2007
40/136
The study of how sounds entering the ear are processed by the ear and the brain in order to give the listener useful information
Measurement of the sensitivity of listeners to changes in theauditory spectrum
The study of the relationship between acoustics and perception
The study of the structures and processes which convert soundsinto sensations and then into our perceptions
7/31/2019 ASP Hearing 2007
41/136
Auditory nerve codingHearing thresholdsPsychoacoustics basic phenomena
7/31/2019 ASP Hearing 2007
42/136
Each hair cell has about 10 auditory nerve fibers Bending of the IHC cilia produces action potentials Action potentials travel via the auditory nerve to the brain Tonotopic mapping for AN response
Auditory Area of Brain
7/31/2019 ASP Hearing 2007
43/136
Neurons encode Steady state sounds (phase, frequency, intensity) Onsets
At stimulus onset, AN firing rate increases rapidly For constant stimulus, the rate decreases exponentially Spontaneous rate: AN firing rate in the absence of stimulus
Neuron is more responsive to changes than to steady inputs
7/31/2019 ASP Hearing 2007
44/136
To determine the tonotopic map of AN: Apply 50ms tone bursts every 100ms Increase SPL till spike rate increases by 1
Repeat for all frequencies AN characteristic frequency
~=BM resonance frequency
Tuning curves are BPFs with almostconstant Q (=f 0/BW)
7/31/2019 ASP Hearing 2007
45/136
Minimum intensity of sound needed at a particular frequency to just stimulate an AN above spontaneous activity
HF slopes are very steep (c. 300 dB/oct) LF slopes generally have a steep tip followed by a flatter base Damage to the cochlea abolishes the tip
7/31/2019 ASP Hearing 2007
46/136
Play a tone above a particular (CF) ANs threshold, it will fire If a second tone is played, at a frequency and level in the shaded
area, then the firing rate of the first neuron will be reduced Auditory system is non-linear Contention for same neurons to encode both frequencies
7/31/2019 ASP Hearing 2007
47/136
ANs tend to fire at a particular phase of LF stimulus Inter Spike Intervals (ISI) occur at integer multiples of the tone period >3kHz phase locking gets weaker because of neuron refractory period
7/31/2019 ASP Hearing 2007
48/136
As the amplitude of a tone increases, the firing rate of a AN neuronat that CF increases up to saturation High spontaneous rate neurons code low level intensity changes Low spontaneous rate neurons code high level intensity changes High spontaneous rate neurons are more in number
7/31/2019 ASP Hearing 2007
49/136
Coding of frequencyPhase-locking for low frequenciesPlace (tonotopic) theory
Coding of phasePhase-lockingbinaural localization (ITD)
Coding of intensityRate of firing above thresholdSpread of firing
7/31/2019 ASP Hearing 2007
50/136
Matlab code available:http://www.ece.mcmaster.ca/~ibruce/Input: wav fileOutput: PSTH [post stimulus time histogram]
AuditoryAuditory- -periphery modelperiphery model
(Zhang et al. ~2001)
7/31/2019 ASP Hearing 2007
51/136
Auditory nerve codingHearing thresholdsPsychoacoustics basic phenomena
7/31/2019 ASP Hearing 2007
52/136
Threshold of hearing at 1,000 Hz is 0 dB
7/31/2019 ASP Hearing 2007
53/136
Loudness is not simply sound intensity!
Factor of ten increase in intensity for the sound to be twice as loud Unit of loudness: Phon [Reference: SPL at 1,000 Hz] The Bass loss problem: discrimination against LF
7/31/2019 ASP Hearing 2007
54/136
Presbycusis: hearing loss due of aging Hearing sensitivity decreases especially at HFs Threshold of pain remains the same Reduced dynamic range
7/31/2019 ASP Hearing 2007
55/136
a) Normal hearing b) Severe hearing loss
c) Fused cilia of IHC-Noise induced loss
7/31/2019 ASP Hearing 2007
56/136
7/31/2019 ASP Hearing 2007
57/136
Occupational Safety & Health Administration (OSHA)OSHA Noise exposure computation standard- 1910.95
Noise dosimeters: Microphone and noise processor Memory for storing results and time history
7/31/2019 ASP Hearing 2007
58/136
Auditory nerve codingHearing thresholdsPsychoacoustics basic phenomena
7/31/2019 ASP Hearing 2007
59/136
Critical bands and loudnessMonoaural beatsMasking effect Frequency masking, temporal masking
7/31/2019 ASP Hearing 2007
60/136
Psychoacoustic experimentsEqually loud, close in frequency ( A,B)
Same IHCs Slightly louder
Equally loud, separated in freq. ( C,D) Different IHCs Twice as loud
Critical bands of frequency(Proposed by Fletcher )
7/31/2019 ASP Hearing 2007
61/136
Center Freq(Hz)
Critical BW(Hz)
100 90
200 90
500 110
1000 150
2000 280
5000 700
10000 1200
How to measure?Signal threshold in noise vs. BW CB ~= 1.5mm spacing on BM 24 such band pass filters
BW of the filters increases with f c Weber: Logarithmic relationship Psychoacoustical scale: bark scale
7/31/2019 ASP Hearing 2007
62/136
Loudness increases when bandwidth exceeds a critical band
7/31/2019 ASP Hearing 2007
63/136
7/31/2019 ASP Hearing 2007
64/136
Critical bands and loudnessMonoaural beatsMasking effect Frequency masking, temporal masking
7/31/2019 ASP Hearing 2007
65/136
If two tones are presented monaurally with a smallfrequency difference, a beating pattern can be heard
500 & 502 Hz 500 & 520 Hz
Interaction of the two tones in the same auditory filter Beating arises from neural interaction Only perceived if the tones are sufficiently close in frequency
7/31/2019 ASP Hearing 2007
66/136
Meena Ramani4/6/2007
EEL 6586Automatic Speech Processing
7/31/2019 ASP Hearing 2007
67/136
Critical bands and loudnessMonoaural beatsMasking effect Frequency masking, temporal masking
7/31/2019 ASP Hearing 2007
68/136
A loud sound makes a weaker sound imperceptibleCategories and aspects of masking
Frequency masking Temporal maskingFrequency selectivity of the auditory system
Psychophysical tuning curves
MAS KING
7/31/2019 ASP Hearing 2007
69/136
Masking occurs because two frequencies lie within a critical band and the higher amplitude one masks the lower amplitudesignalTypes of maskers:
Broad band noise, narrowband noise, pure & complex tonesLow frequency broad band sounds mask the mostMasking threshold : Amt of dB for test tone to be just audible in noise
MAS KING
MAS KING
7/31/2019 ASP Hearing 2007
70/136
White noiseMasked thresholds are a function of frequencyTOQs frequency dependence almost completely disappearsLow and very high frequency almost same as TOQ Above 500Hz, thresholds increase with increase in frequency by10dB/decade
M AS KING
MAS KING
7/31/2019 ASP Hearing 2007
71/136
Narrow band
7/31/2019 ASP Hearing 2007
72/136
Noise (Varying Amplitude, Fixed Frequency) 1KHz noise; 20-100dBSlope of rise seems independent of amplitudeBut slope of fall is dependent on amplitudeNon-Linear frequency dependenceStrange effect at high masker amplitudes: At high amplitudes ear begins to listen to anything audible!! Begin to hear difference noise (noise and testing tone)
M AS KING
MAS KING
7/31/2019 ASP Hearing 2007
73/136
Previously assume long lasting test and masking soundsSpeech has a strong temporal structure Vowels= Loudest parts; Consonants=faint parts Consonants are masked by preceding loud vowel
M AS KING
MAS KING
7/31/2019 ASP Hearing 2007
74/136
Post-stimulus/Forward/Post-masking 1st Masker 2 nd test tonePre-Stimulus/Backward/Pre-masking 1st test tone 2 nd MaskerSimultaneous Masking Test tone and Masker together
M AS KING
MAS KING
7/31/2019 ASP Hearing 2007
75/136
Simultaneous masking Duration >200ms, constant test tone threshold Assume hearing system integrates over a period of 200ms
Postmasking Decay in effect of masker for 100ms More dominant effect
Premasking Takes place 20ms before masker is on!! Each sensation is not instantaneous, requires build-up time
Quick build up for loud maskers; Slower build up for softer maskers Less dominant effect
M AS KING
MAS KING
7/31/2019 ASP Hearing 2007
76/136
Procedure
Masker: A narrowband noise of variable center freq is the maskerTarget: A fixed freq and fixed level pure toneLevel of masker that just masks the tone for different masker freqs.
Inference:
Masking curves tell much about auditory selectivity Psychophysical tuning curves match with physiological curves
M AS KING
7/31/2019 ASP Hearing 2007
77/136
Anatomy e.g. CochleaPsychoacoustics e.g. MaskingEngineering Applications e.g. Hearing aids and Cochlear implants
7/31/2019 ASP Hearing 2007
78/136
7/31/2019 ASP Hearing 2007
79/136
Normal Hearing
Sensorineural Hearing LossMild to Severe Loss[10 20 30 60 80 90] dB HL
Time (s)
F r e q u e n c y
( H z
)
Cell phone speech for normal hearing
0 0.5 1 1.5 2
0
500
1000
1500
2000
2500
3000
3500
4000
-250
-200
-150
-100
-50
0
Time (s)
F r e q u e n c y
( H z
)
Cell phone speech for SNHL
0 0.5 1 1.5 20
500
1000
1500
2000
2500
3000
3500
4000
-250
-200
-150
-100
-50
0
What do the hearingWhat do the hearingimpaired hear?impaired hear?
7/31/2019 ASP Hearing 2007
80/136
One in every ten (28 million) Americans has hearing loss
95% of those with hearing loss can have their hearing losstreated with hearing aids
Only 6 million use HAs
Factors for HAs not being used: stigma, cost and inefficiency
7/31/2019 ASP Hearing 2007
81/136
1. Decreased audibility2. Loudness recruitment3. Decreased frequency resolution4. Decreased temporal resolution
A good hearing loss (HL) compensation algorithm shouldcompensate for all the above four factors but often not possible
Most HAs compensate for 1 and 2
7/31/2019 ASP Hearing 2007
82/136
Effect:More loss at HFPossible solution:Frequency dependent gainMeasured by:Audiograms
2) Loudness recruitment2) Loudness recruitment
102
103
104-1 0
0
10
20
30
40
50
60
70
80
90
Frequency (Hz)
T h r e s
h o
l d o
f h e a r i n g
( d B
S P L )
Thresholds of hearing for normal & HI listeners
Normal hearingHearing impaired
20 40 60 80 100 120
Very soft
Soft
Comfortable
Loud
Very loud
Too loud
Input level (dB SPL)
L o u
d n e s s r a
t i n g
Loudness growth curves for normal & HI listeners
Normal hearingHearing impaired
Effect:Loud sounds are just as loudPossible solution:CompressionMeasured by:Loudness growth curves
7/31/2019 ASP Hearing 2007
83/136
3) Frequency resolution3) Frequency resolution 4) Temporal resolution4) Temporal resolution
Effect:Upward spread of masking increasesPossible solution:Sharper and narrower filter banksMeasured by:Psychoacoustic tuning curve
103
1040
10
20
30
40
50
60
70
80
90
Desired tone frequency (Hz)
D e s
i r e
d t o n e
t h r e s
h o
l d ( d B
S P L )
4 kHz tuning curve for normal & HI listeners
Masker Normal hearingHearing impaired
0 20 40 60 80 100 120 1400
10
20
30
40
50
60
70
80
Desired-Masker tone separation (ms)
D e s
i r e
d t o n e
t h r e s
h o
l d ( d B
S P L )
Temporal resolution at 4 kHz for normal & HI listeners
Normal hearingHearing impaired
Effect:Temporally masking of weak soundsPossible solution:Vary gain to get normal masking thresholdMeasured by:Psychoacoustic tuning curve
7/31/2019 ASP Hearing 2007
84/136
Behind The ear In the Ear
In the Canal Completely in the canal
7/31/2019 ASP Hearing 2007
85/136
MicrophoneTone hook
Volume controlOn/off switch
Battery compartment
d fd f
7/31/2019 ASP Hearing 2007
86/136
Ear mold measurements
Hearing aid fittingHearing aid fitting
7/31/2019 ASP Hearing 2007
87/136
So, do you want your HA to:1) Be comfortably loud2) Have loudness Equalization3) Have loudness Normalization
?
?
Which fitting methodology is theWhich fitting methodology is the bestbest ??
7/31/2019 ASP Hearing 2007
88/136
O/P
H1(w )
LevelDetector
GainComputation
H2(w )
LevelDetector
GainComputation
Hn(w )
LevelDetector
GainComputation
O/P LimitingCompression
I/P
HA parameters optimizationHA parameters optimization
7/31/2019 ASP Hearing 2007
89/136
HA parameters optimizationHA parameters optimization
Factors contributing to optimization complexity: Dimensionality of the parameter space is large The determinants of hearing-impaired user satisfaction are unknown Satisfaction evaluation through listening tests is costly and unreliable Acclimitization
7/31/2019 ASP Hearing 2007
90/136
Speech Intelligibility
Objective Measures
AI, STI
Speech Quality
Objective MeasuresPESQ
Subjective MeasuresMOS
Speech Intelligibility (SI): The degreeto which speech can be understood
Performance metricsPerformance metrics
Subjective Measures
HINT
Speech Quality: Does the speech
match your expectations?
Spectrograms and sound filesSpectrograms and sound files
7/31/2019 ASP Hearing 2007
91/136
Sensorineural Hearing Loss [10 20 30 60 80 90] dB HLNoise level =65 dBA and Speech level=65dBA
Spectrograms and sound filesSpectrograms and sound files
HI with Linear gainNormal hearing
HI with RBC gain HI with NR-RBC gain
7/31/2019 ASP Hearing 2007
92/136
7/31/2019 ASP Hearing 2007
93/136
First functional Brain Machine Interface (BMI)First functional Brain Machine Interface (BMI)
Definition:
A device that electrically stimulates the auditory nerve
of patients with severe-to-profound hearing loss toprovide them with sound and speech information
7/31/2019 ASP Hearing 2007
94/136
Severe-to profound sensorineural hearing lossLimited benefit from hearing aidsHearing loss did not reach severe-to-profoundlevel until after acquiring oral speech andlanguage skills
7/31/2019 ASP Hearing 2007
95/136
Worldwide: Over 100,000 multi-channel implants
At Univ of Florida: Implanted first patient in 1985 Currently follow over 400 cochlear patients
Magnetic Resonance Imaging
Surgical issues
Technical and Safety IssuesTechnical and Safety Issues
7/31/2019 ASP Hearing 2007
96/136
7/31/2019 ASP Hearing 2007
97/136
Parts of a CIParts of a CI
7/31/2019 ASP Hearing 2007
98/136
7/31/2019 ASP Hearing 2007
99/136
7/31/2019 ASP Hearing 2007
100/136
7/31/2019 ASP Hearing 2007
101/136
1. Electrode design Number of electrodes, electrode configuration
2. Type of stimulation
Analog or pulsatile3. Transmission link
Transcutaneous or percutaneous
4. Signal processing
Waveform representation or feature extraction
7/31/2019 ASP Hearing 2007
102/136
Compressed Analog (CA)Continuous Interleaved Sampling (CIS)Multiple Peak (MPEAK )
Spectral Maxima Sound Processor (SMSP)Spectral Peak (SPEAK)
7/31/2019 ASP Hearing 2007
103/136
7/31/2019 ASP Hearing 2007
104/136
7/31/2019 ASP Hearing 2007
105/136
7/31/2019 ASP Hearing 2007
106/136
6 channel PR: 100-2500 pulses/sec
7/31/2019 ASP Hearing 2007
107/136
7/31/2019 ASP Hearing 2007
108/136
F0/F1/F2 +HFs Unvoiced: 250 pps 4 electrodes
7/31/2019 ASP Hearing 2007
109/136
7/31/2019 ASP Hearing 2007
110/136
16 bands [16/22 electrodes] 6 max O/P every 4ms PR: 250 pps
7/31/2019 ASP Hearing 2007
111/136
7/31/2019 ASP Hearing 2007
112/136
Strategy: MPEAK or SPEAK Vary selected maxima: 5-10 Vary PR: 180-300pps
For BB: more maxima, less PR For NB: less maxima, more PR Preserves temporal and spectral data
7/31/2019 ASP Hearing 2007
113/136
7/31/2019 ASP Hearing 2007
114/136
Wide range of successMost score 90-100% on AV sentence materials
Majority score > 80% on high contextPerformance more varied on single word tests
7/31/2019 ASP Hearing 2007
115/136
7/31/2019 ASP Hearing 2007
116/136
Definition:The ABI is a surgically implanted device that bypasses thedamaged auditory nerves by providing electrical stimulation tothe cochlear nucleus of the brainstem
Who is a candidate?ABI is designed to provide sound to people withNeurofibromatosis Type II (NF2) who become deaf whentumors are removed from their ANBe at least twelve years of ageHave the ABI placed during tumor removal surgeryHave appropriate expectations and be prepared to participatein a follow-up program
7/31/2019 ASP Hearing 2007
117/136
The first patient was implanted with the MultichannelABI in 1992.So far over 100 NF2 patients have received the device.
How does an ABI work?How does an ABI work?
7/31/2019 ASP Hearing 2007
118/136
How does an ABI work?How does an ABI work?
How does an ABI work? (contd )How does an ABI work? (contd )
7/31/2019 ASP Hearing 2007
119/136
Ventral cochlear nucleus (VCN) transmitssound frequencyinformation to higher auditory centers VCN is tonotopically organized
How does an ABI work? (contd.)How does an ABI work? (contd.)
7/31/2019 ASP Hearing 2007
120/136
Approved October 20, 2000Uses the Nucleus 24 system processorsPlate array with 21 electrodesMost commonly used: Mode: Monopolar Coding strategy: SPEAK No of electrodes: 8
7/31/2019 ASP Hearing 2007
121/136
82% of the implanted subjects were able to perceivesound and use the device postoperatively85% of the subjects demonstrated statistically significantimprovements in open-set sentence understanding whenusing the Nucleus ABI in conjunction with lip-reading.
7/31/2019 ASP Hearing 2007
122/136
7/31/2019 ASP Hearing 2007
123/136
Speaker tracking is not possible with singlemicrophoneMultiple microphones facilitate spatiotemporalfilteringSetup consists of two microphones with the first
microphone assumed as originDistance of the wavefront from microphone is:
The direction of source is given by:
sk dF m
Nc
)1(sin
k d sin
7/31/2019 ASP Hearing 2007
124/136
Differentiate speech source from noise sourceOvercome problems of signal distortion due to noisePrevent loss of accuracy due to room reverberations
Types of DOA estimation aTypes of DOA estimation algorithmslgorithms
7/31/2019 ASP Hearing 2007
125/136
DOA Algorithms
Spatial Correlationmethods
Subspace decompositionmethods
MUSICMultiple Signal Estimation
ESPRITEstimation of Signal parametersusing rotational invariance
Delay and Sum Minimum Variance
Coherent MUSIC
Root MUSIC
ypyp g g
DOA implementation equationsDOA implementation equations
7/31/2019 ASP Hearing 2007
126/136
DOA Method Equation for Implementation
Delay and Sum
MinimumVariance
MUSIC
CoherentMUSIC
Root MUSIC
ESPRIT
*( ) ( ) ( ) P a Sa
*
1( )
( ) ( ) ( ) P
a inv s a
*
* *
( ) ( )( )( ) ( ) N N
a a P a EEa
'( )
' ' N N
a a P
a E E a
1 0sin . ( ) /( ) K c angle z d
1 0sin arg( ) /( ) K K c d
p qp q
7/31/2019 ASP Hearing 2007
127/136
Responsible factorsSampling rateBaseline distanceNumber of microphones
7/31/2019 ASP Hearing 2007
128/136
1. Delay-and-sum:Require prior knowledge of the direction of propagationof signals i.e. delay per microphone
2. Minimum variance:These algorithms adapt their computations to thecharacteristics of the signal
3. Generalized side-lobe cancellers:Uses the delay-and-sum technique for beamforming and
adaptively cancels noise
7/31/2019 ASP Hearing 2007
129/136
The oldest and simplest beamforming technique
Add incoming signals with appropriate delay to reinforce signal withrespect to noise
))(()( 01
0
t ywt z m M
mm
)(1 t y
7/31/2019 ASP Hearing 2007
130/136
Setup similar to delay-and-sum beamformer
Weights are chosen to minimize the weighted array power output fornon-desired look directions
The weights maintain unity gain along desired direction andconstructively add the signal
11)( e Re P e Re
e Rw 11
0
7/31/2019 ASP Hearing 2007
131/136
Target-SignalFilter [H 1]
Signal Blo cking [B] Adaptive Filtering[H2]
Speech
Delay Compensation
Noise wave front
+
-
Speech wave front
Best performance among the three algorithms Adaptive filtering decreases throughput Mismatch in delay lowers array gain Reverberations reduce the effectiveness of adaptive filtering
7/31/2019 ASP Hearing 2007
132/136
Poor performance at low frequenciesSensitivity decreases as inter-microphone distance decreasesPoor noise cancellation at small baseline
7/31/2019 ASP Hearing 2007
133/136
Phase lag of border frequencies isBeampattern chosen to have null at90 degree and unity gain atbroadside
The weights are calculated atindividual frequency binsSignal is multiplied by weights infrequency domain and broughtback to time domain via IFFT
cd L
L
sinc
d U U
sin
S N
N
jkd jkd
jkd
eee
w sinsin
sin
1
S N jkd jkd eew sinsin2
1
t jnd
j M
mm eewt y
sin21
0)(
7/31/2019 ASP Hearing 2007
134/136
7/31/2019 ASP Hearing 2007
135/136
Frequency independent beamformer outperforms othertechniques in terms of recognition improvement.
7/31/2019 ASP Hearing 2007
136/136
Anatomy e.g. Pinna, Ossicles, CochleaPsychoacoustics e.g. AN coding, MaskingEngineering Applications e.g. Hearing aids, Cochlear implants, ABI
http://www.cnel.ufl.edu/~meena/ppt.htmContact: meena[@]cnel.ufl.edu