Upload
ella-summers
View
220
Download
2
Tags:
Embed Size (px)
Citation preview
Physical and perceptual evaluation of the Physical and perceptual evaluation of the
Interaural Wiener Filter algorithmInteraural Wiener Filter algorithm
Simon Doclo1, Thomas J. Klasen1, Tim van den Bogaert2,
Marc Moonen1, Jan Wouters2
1Dept. of Electrical Engineering (ESAT-SCD), KU Leuven, Belgium2Laboratory for Exp. ORL, KU Leuven, Belgium
IHCON, Aug 19 2006
Slides available at http://homes.esat.kuleuven.be/~doclo/presentations.html
22
OverviewOverview
• Binaural hearing aids: noise reduction and preservation of binaural cues
• Overview of binaural noise reduction algorithms
• Binaural multi-channel Wiener filter:
o Estimate of speech component at both hearing aids
o Speech cues are preserved – noise cues may be distorted
• Preservation of binaural cues:
o Extension of cost function with ITD-ILD-ITF expressions
• Experimental results:
o Physical evaluation (SNR, ITD, ILD)
o Perceptual evaluation (SRT, localisation)
• Audio demonstration
33
PL PR
ITD
ILD L
R
P
P
• Binaural auditory cues:
o Interaural Time Difference (ITD) – Interaural Level Difference (ILD)
o Binaural cues, in addition to spectral and temporal cues, play an important role in binaural noise reduction and sound localization
Problem statementProblem statement
• Hearing impairment reduction of speech intelligibility in background noise
o Signal processing to selectively enhance useful speech signal
o Many hearing impaired are fitted with hearing aid at both ears
o Multiple microphones available: spectral + spatial processing
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
44
Problem statementProblem statement
• Bilateral system:
o Independent processing of left and right hearing aid
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
55
Problem statementProblem statement
• Bilateral system:
o Independent processing of left and right hearing aid
o Localisation cues are distorted
• Binaural system:
o Cooperation between left and right hearing aid (e.g. wireless link)
o Assumption: all microphone signals are available at the same timeObjectives/requirements for binaural
algorithm:
1. SNR improvement: noise reduction, limit speech distortion
2. Preservation of binaural cues (speech/noise) to exploit binaural hearing advantage
3. No assumption about position of speech source and microphones
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
[Van den Bogaert, 2006]
66
Binaural noise reduction Binaural noise reduction techniquestechniques
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
77
Binaural noise reduction Binaural noise reduction techniquestechniques
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
88
Binaural noise reduction Binaural noise reduction techniquestechniques
• Fixed beamforming: spatial selectivity + binaural speech cues
o Maximize directivity index while restricting speech ITD error
o Superdirective beamformer using HRTFS
[Desloge, 1997]
[Lotter, 2004]
low computational complexity
limited performance, known geometry, broadside array, only speech cues
[Desloge, 1997]
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
99
Binaural noise reduction Binaural noise reduction techniquestechniques
• CASA-based techniques
o Computation and application of (real-valued) binaural mask based on binaural and temporal/spectral cues
[Kollmeier, Peissig, Wittkop, Dong, Haykin]
perfect preservation of binaural cues of speech/noise component
mostly for 2 microphones, “spectral-subtraction”-like problems
[Wittkop, 2003]
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
1010
• Adaptive beamforming: based on GSC-structure
o Divide frequency spectrum: low-pass portion unaltered to preserve ITD cues, high-pass portion processed using GSC
Binaural noise reduction Binaural noise reduction techniquestechniques
[Welker, 1997]
preserves binaural cues to some extent
substantial reduction in noise reduction performance, known geometry
[Welker, 1997]
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
1111
Binaural noise reduction Binaural noise reduction techniquestechniques
• Binaural multi-channel Wiener filter
o MMSE estimate of speech component in microphone signal at both ears
[Doclo, Klasen, Wouters, Moonen]
speech cues are preserved, no assumptions about position of speech source and microphones
noise cues may be distorted
Extension of MWF :
preservation of binaural speech and noise cues without substantially compromising noise
reduction performance
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
1212
Design of hearing aid SP algorithm
requires some mathematics
but perceptual evaluation in a couple of minutes…
1313
Configuration and signalsConfiguration and signals
• Configuration: microphone array with M microphones at left and right hearing aid, communication between hearing aids
• Vector notation: ( ) ( ) ( ) Y X V
noise component
0, 0, 0, 0( ) = ( ) , 0) =( 1m m mVY X m M 00, 0,, 0( ) = ( )( 0) , = 1mm mY V m MX
speech component
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
0 0 1 1( ) = ( ) ( ), ( ) = ( ) ( )H HZ Z W Y W Y
• Use all microphone signals to compute output signal at both ears
1414
Overview of cost functionsOverview of cost functions
Multi-channel Wiener filter (MWF): MMSE estimate of speech component in microphone signal at both ears
trade-off noise reduction and speech
distortion
Speech-distortion weighted multi-channel Wiener filter
(SDW-MWF)[Doclo 2002, Spriet 2004]binaural cue
preservation of speech + noise
Partial estimation of noise component
[Klasen 2005]
Extension with ITD-ILD or Interaural Transfer
Function (ITF)
[Doclo 2005, Klasen 2006]
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
1515
• Binaural SDW-MWF: estimate of speech component in microphone signal at both ears (usually front microphone) + trade-off between noise reduction and speech distortion
Binaural multi-channel Wiener Binaural multi-channel Wiener filterfilter
0
1
= , = ,x v M xx y v
M x v x
R R 0 rR r R R R
0 R R r
0
1
2 2
0, 0 0
11, 1
( )H H
r
HHr
XJ E
X
W X W VW
W VW X1=SDWW R r
trade-off parameter
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
estimate
o Depends on second-order statistics of speech and noise
o Estimate Ry during speech-dominated time-frequency segments, estimate Rv during noise-dominated segments, requiring robust voice activity detection (VAD) mechanism
o No assumptions about positions of microphones and sources
1616
Binaural multi-channel Wiener Binaural multi-channel Wiener filterfilter
• Binaural cues (ITD-ILD) :
Perfectly preserves binaural cues of speech component
Binaural cues of noise component speech component !!(cf. physical and perceptual evaluation)
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
• Extension of SDW-MWF with binaural cues
o Add term related to binaural cues of noise (and speech) component to SDW cost function
o Possible cues: ITD, ILD, Interaural Transfer Function (ITF)
o Weight factors and can be frequency-dependent
( ) = ( ) ( ) ( )x vtot SDW cue cueJ J J J W W W W
1717
Interaural Wiener FilterInteraural Wiener Filter
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
• Preserve binaural cues between input and output
o ITD: phase of cross-correlation
o ILD: power ratio
o ITF: Interaural transfer function (incorporates ITD and ILD)
0 0
1 1
Hv vout H
v
ZITF
Z
W V
W V
0 10
1 1 1
*0, 1,0, 0 1
*1, 1 11, 1,
( , )
( , )
r rrv vin
r vr r
E V VV r rITF
V r rE V V
R
Re.g.
0
1
2 2
0, 0 0
11, 1
2 2
0 1 0 1
( ) =H H
r
tot HHr
H x H H v Hin in
XJ E
X
E ITF E ITF
W X W VW
W VW X
W X W X W V W V
ITF preservation speech ITF preservation noise
o Closed form expression!
o large changes direction of speech component to noise component increase weight (cf. physical and perceptual evaluation)
1818
Overview of batch algorithmOverview of batch algorithmLeft input
signalsRight input
signals
( ) ( ) ( ) Y X VFFT FFT
0 0 0x vZ Z Z 1 1 1x vZ Z Z Left output Right output
IFFT IFFT
Frequency-domain filtering
Off-line computation of
statistics
VAD
( ), ( )v x R R
Calculate binaural input cues and
filter 0
1
( )( ) =
( )
WW
W, ,
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
1919
Experimental resultsExperimental results
• Identification of HRTFs:
o Binaural recordings on CORTEX MK2 artificial head
o 2 omni-directional microphones on each hearing aid (d=1cm)
o LS = -90:15:90, 90:30:270, 1m from head
o Conditions: T60=140 ms, fs=16 kHz, L=1366 taps
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results- Physical- Perceptual
Audio demo
Conclusions
2020
Experimental resultsExperimental results
• Speech and noise material:
o Dutch sentences (VU list)
o Stationary speech-weighted noise with same long-term spectrum as speech material spatial aspects
o S0N60 , SNR=0 dB
o fs=16 kHz, FFT-size N=256, =1
• Physical evaluation:
o Speech intelligibility: SNR
o Localisation: ITD / ILD
• Perceptual evaluation:
o Preliminary study
o Speech intelligibility: SRT
o Localisation: localise S and N
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results- Physical- Perceptual
Audio demo
Conclusions
2121
Physical evaluationPhysical evaluation
• Performance measures:
o Intelligibility weighted SNR improvement (left/right)
o ILD error (speech/noise component) power ratio
x xx out i in i
i
ILD ILD ILD
o ITD error (speech/noise component) phase of cross-correlation
x i x ii
ITD I ITD 1
* *0, 1, 0 10
{ } { }x i r r x xITD E X X E Z Z
L i L ii
SNR I SNR
importance of i-th frequencyfor speech intelligibility
low-pass filter 1500 Hz
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results- Physical- Perceptual
Audio demo
Conclusions
Physical evaluation: SNRPhysical evaluation: SNR
00.05
0.10.15
0.2
00.1
0.20.3
0.40.5
0
5
10
15
20
25
SNR improvement left ear
SN
Rw [d
B]
00.05
0.10.15
0.2
00.1
0.20.3
0.40.5
0
5
10
15
20
25
SNR improvement right ear
S
NR
w [d
B]
Physical evaluation: ILD-ITDPhysical evaluation: ILD-ITD
00.05
0.10.15
0.2
0
0.2
0.4
0
5
10
15
ILD error speech component
IL
D [
dB]
00.05
0.10.15
0.2
0
0.2
0.4
0
5
10
15
ILD error noise component
IL
D [
dB]
00.05
0.10.15
0.2
0
0.2
0.4
0
0.5
1
1.5
ITD error speech component
IT
D [
rad]
00.05
0.10.15
0.2
0
0.2
0.4
0
0.5
1
1.5
ITD error noise component
IT
D [
rad]
2424
Physical evaluationPhysical evaluation
• Conclusions: increases: ITD-ILD error of noise component decreases
… BUT… ITD-ILD error of speech component increases
increases: ITD-ILD error of speech component decreases
… BUT… ITD-ILD error of noise component increases
o Compromise between speech and noise localisation error possible (cf. localisation experiments)
o SNR improvement only slightly degraded (cf. SRT experiments)
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results- Physical- Perceptual
Audio demo
Conclusions
2525
• Speech intelligibility: SRT
o How does parameter affect speech intelligibility ?
o Two effects: increasing reduces SNR improvement, but preserves binaural noise cues better, enabling binaural speech intelligibility advantage
• Localisation performance
o How do parameters and affect localisation of processed speech and noise components ?
: preservation of speech cues, : preservation of noise cues
Perceptual evaluationPerceptual evaluation
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results- Physical- Perceptual
Audio demo
Conclusions
2626
• Measurement procedure:
o SRT = SNR where 50% of speech is intelligible
o adaptive procedure (2 dB/step)
o headphone experiments, using HRTFs
o S0N60 (Dutch VU sentences – stationary noise)
o presentation level = 65 dB SPL
o 5 normal-hearing subjects
o fs=16 kHz, FFT-size N=256, =1, =0
o Reference condition = no processing
Perceptual evaluation: SRTPerceptual evaluation: SRT
HRTFx
HRTFv
speech
noise
GBinaural
filter
Mic L
R
Headphones
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results- Physical- Perceptual
Audio demo
Conclusions
2727
Perceptual evaluation: SRTPerceptual evaluation: SRT
VU noise 60 deg, alpha=0
9,00
11,00
13,00
15,00
0,0 0,1 0,3 1,0 10,0
Beta
SR
T im
pro
vem
ent
• Results:
o average SRT without processing = -9.2 dB
o SRT improvements in the range 11-13 dB
o Binaural speech intelligibility advantage does not seem to compensate for loss in SNR improvement
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results- Physical- Perceptual
Audio demo
Conclusions
2828
• Sum of localisation errors Sx and N0
• Parameters can be tuned to achieve better overal localization performance at the cost of some noise reduction
• Good correlation between physical and perceptual evaluation
Perceptual evaluation: Perceptual evaluation: localisationlocalisation
Loc error Sx + Loc error N0 5 subjects SxN0
0
10
20
30
40
50
60
70
80
0 0,1 0,3 1 10 100
beta
(°)
a l p h a = 0
a l p h a = 0 , 5
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results- Physical- Perceptual
Audio demo
Conclusions
2929
Audio demonstrationAudio demonstration
• Speech and noise material:
o HINT sentences, speech source in front (0)
o Multi-talker babble noise at 60
o SNR=0 dB, fs=16 kHz, FFT-size N=256, =1, =0
Noisy Speech Noise
Input
Output (=0)
Output (=0.05)
Output (=10)
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
3030
• Speech enhancement for binaural hearing aids:
o Improve speech intelligibility
o Localisation: preserve binaural speech and noise cues
o No assumptions about position speech source and microphones
• Suitable algorithm: multi-channel Wiener filter
speech cues are preserved noise cues may be distorted
• Preservation of binaural noise cues:
Interaural Wiener filter: extension with Interaural Transfer Function of noise (and speech) component
• Perceptual evaluation:
o S0N60: SRT improvements in the range 11-13 dB
o Binaural speech intelligibility advantage does not seem to compensate for (small) loss in SNR improvement
o Parameters can be tuned to achieve better overal localization performance at the cost of some noise reduction
ConclusionsConclusions
Problem statement
Binaural noise reduction
Multi-channel Wiener filter
Preservation of binaural cues
Experimental results
Audio demo
Conclusions
3131
AcknowledgmentsAcknowledgments