Design and low-cost implementation of a robust multichannel noise reduction scheme for cochlear implants Simon Doclo 1, Ann Spriet 1,2, Marc Moonen 1,

Design and low-cost Design and low-cost

implementation of a robust implementation of a robust

multichannel noise reduction multichannel noise reduction

scheme for cochlear implantsscheme for cochlear implants

Simon Doclo1, Ann Spriet1,2, Marc Moonen1, Jan Wouters2

1Dept. of Electrical Engineering (ESAT-SCD), KU Leuven, Belgium2Laboratory for Exp. ORL, KU Leuven, Belgium

SPS-2004, 15.04.2004

22

OverviewOverview

• Problem statement: hearing in background noise

• Adaptive beamforming: GSC

o not robust against model errors

• Design of robust noise reduction algorithm

o robust fixed spatial pre-processor

o robust adaptive stage

• Experimental results

• Low-cost implementation of adaptive stage

o stochastic gradient algorithms

o computational complexity + memory

• Conclusions

33

Problem statementProblem statement

• Hearing problems effect more than 10% of population

• Digital hearing instruments allow for advanced signal processing, resulting in improved speech understanding

• Major problem: (directional) hearing in background noiseo reduction of noise wrt useful speech signalo multiple microphones + DSPo current systems: simple fixed and adaptive

beamforming o robustness important due to small inter-microphone

distance

hearing aids and cochlear implants

design of robust multi-microphone noise reduction scheme

Introduction -Problem statement

-State-of-the-art -GSC Robust spatial pre-processor

Adaptive stage Implementation Conclusions

44

State-of-the-art noise reductionState-of-the-art noise reduction

• Single-microphone techniques:

o spectral subtraction, Kalman filter, subspace-based

o only temporal and spectral information limited performance

• Multi-microphone techniques:

o exploit spatial information

o Fixed beamforming: fixed directivity pattern

o Adaptive beamforming (e.g. GSC) : adapt to differentacoustic environments improved performance

o Multi-channel Wiener filtering (MWF): MMSE estimate of speech component in microphones improved robustness

Sensitive to a-priori assumptions

Robust scheme, encompassing both GSC and MWF




55

Adaptive beamforming: GSCAdaptive beamforming: GSC

• Fixed spatial pre-processor:o Fixed beamformer creates speech reference o Blocking matrix creates noise references

• Adaptive noise canceller:o Standard GSC minimises output noise power

Spatial pre-processing][0 ku

][1 ku

][1 kuN

Fixed beamformer

A(z)

Speechreferenc

e ][0 ky

Blocking matrixB(z)

Noise references

][1 ky

][2 ky

][1 kyN

Adaptive Noise Canceller

][kz

][1 kw

][2 kw

][1 kwN

(adaptation during noise)

][0 ky

noisespeech

][][][ kvkxky iii

2

0][

][][][min kkkvE T

kvw

w

TT

NTT

TTN

TT

kkkk

kkkk

][][][][

][][][][

121

121

vvvv

wwww




66

Robustness against model errorsRobustness against model errors

• Spatial pre-processor and adaptive stage rely on assumptions (e.g. no microphone mismatch, no reverberation,…)

• In practice, these assumptions are often not satisfied

o Distortion of speech component in speech reference

o Leakage of speech into noise references, i.e.

• Design of robust noise reduction algorithm:

1. Design of robust spatial pre-processor (fixed beamformer), e.g. using statistical knowledge about microphone characteristics

2. Design of robust adaptive stage, e.g. by taking speech distortion explicitly into account in optimisation criterion

Speech component in output signal gets distorted

][0 kx

0x ][k

][][][][ 0 kkkxkz Tx xw

Limit distortion both in and ][0 kx ][][ kkT xw




77

• Small deviations from assumed microphone characteristics (gain, phase, position) large deviations from desired directivity pattern, especially for small-size microphone arrays

• In practice, microphone characteristics are never exactly known

• Consider all feasible microphone characteristics and optimise

o average performance using probability as weight

– requires statistical knowledge about probability density functions

– cost function: least-squares, eigenfilter, non-linear

o worst-case performance minimax optimisation problem

Robust spatial pre-processorRobust spatial pre-processor

101010 )()(),,(0 1

NNN

A A

mean dAdAAfAfAAJJN

Incorporate specific (random) deviations in design

position

/cos

phase

),(

gain

),(),( cfjjnn

snn eeaA

Measurement or calibration procedure

Introduction Robust spatial pre-processor


88

SimulationsSimulations

• N=3, positions: [-0.01 0 0.015] m, L=20, fs=8 kHz

• Passband = 0o-60o, 300-4000 Hz (endfire)Stopband = 80o-180o, 300-4000 Hz

• Robust design - average performance:Uniform pdf = gain (0.85-1.15) and phase (-5o-10o)

• Deviation = [0.9 1.1 1.05] and [5o -2o 5o]

• Non-linear design procedure (only amplitude, no phase)



99

Non-robust design Robust design

No d

evia

tions

Devia

tions (g

ain

/phase

)

SimulationsSimulations

Angle (deg)

Frequency (Hz)

dB

Angle (deg)

Frequency (Hz)

dB

Angle (deg)

Frequency (Hz)

dB

Angle (deg)

Frequency (Hz)

dB



1010

Design of robust adaptive stageDesign of robust adaptive stage

• Distorted speech in output signal:

• Robustness: limit by controlling adaptive filter

o Quadratic inequality constraint (QIC-GSC):

= conservative approach, constraint f (amount of leakage)

o Take speech distortion into account in optimisation criterion(SDW-MWF)

– 1/ trades off noise reduction and speech distortion

– Regularisation term ~ amount of speech leakage

][][][][ 0 kkkxkz Tx xw

][][ kkT xw ][kw

][kw

22

0][

][][1

][][][min kkEkkkvE TT

kxwvw

w

noise reduction speech distortion

Limit speech distortion, while not affecting noise reduction performance in case of no model errors QIC


Adaptive stage -SP SDW MWF -Experimental validation

Implementation Conclusions

1111

Wiener solutionWiener solution

• Optimisation criterion:

• Problem: clean speech and hence speech correlation matrix are unknown!

Approximation:

• VAD (voice activity detection) mechanism required!

22

0][

][][1

][][][min kkEkkkvE TT

kxwvw

w

][][][][][][1

][ 0

1

kvkEkkEkkEk TT vvvxxw

][kx

][][ kkE Txx

][][][][][][ kkEkkEkkE TTT vvyyxx

11

11

][

kw ][][ kkE Txx ][][ 0 kvkE v ][][ kkE Tvv

speech-and-noise periods noise-only periods




1212

Spatially-preprocessed SDW-Spatially-preprocessed SDW-MWFMWF

][0 kw

Spatial preprocessing][0 ku

][1 ku

][1 kuN

Fixed beamformer

A(z)

Speechreferenc

e ][0 ky

Blocking matrixB(z)

Noise references

][1 ky

][2 ky

][1 kyN

Multi-channel Wiener Filter (SDW-MWF)

][kz

][1 kw

][2 kw

][1 kwN

• SP-SDW-MWF encompasses both GSC and SDW-MWF:

o No filter speech distortion regularised GSC (SDR-GSC)– regularisation term added to GSC: the larger the speech leakage,

the larger the regularisation

– special case: 1/ = 0 corresponds to traditional GSC

o Filter SDW-MWF on pre-processed microphone signals– in absence of model errors = cascade of GSC + single-channel

postfilter

– Model errors do not effect its performance!

][0 kw

][0 kw




1313

Experimental validation (1)Experimental validation (1)

• Set-up:o 3-mic BTE on dummy head (d = 1cm, 1.5cm)

o Speech source in front of dummy head (0)o 5 speech-like noise sources: 75,120,180,240,285o Microphone gain mismatch at 2nd microphone

• Performance measures:o Intelligibility-weighted signal-to-noise ratio

– Ii = band importance of i th one-third octave band

– SNRi = signal-to-noise ratio in i th one-third octave band

o Intelligibility-weighted spectral distortion

– SDi = average spectral distortion in i th one-third octave band

2

i

I

iiI SNRSNR

1intellig

i

I

iiI SDSD

1intellig

ic

f

f x

i f

dffGic

ic

,6/16/1

2

2 10

22

)(log10SD

,6/1

,6/1

)(

)()(

2

2

fXE

fZEfG x

x

(Power Transfer Function for speech component)




1414

Experimental validation (2)Experimental validation (2)• SDR-GSC:

o GSC (1/ = 0) : degraded performance if significant leakageo 1/ > 0 increases robustness (speech distortion noise

reduction)

• SP-SDW-MWF: o No mismatch: same as SDR-GSC, larger

due to post-filtero Performance is not degraded by mismatch

0w 0

0w 0

intelligSNR intelligSD




1515

Experimental validation (3)Experimental validation (3)

• GSC with QIC ( ) : QIC increases robustness GSC

• For large mismatch: less noise reduction than SP-SDW-MWF

][kw

QIC f (amount of speech leakage) less noise reduction than SDR-GSC for small mismatch

SP-SDW-MWF achieves better noise reduction than QIC-GSC, for a given maximum speech distortion

level




1616

Audio demonstrationAudio demonstration

Algorithm No deviationDeviation (4dB)

Noisy microphone signal

Speech reference

Noise reference

Output GSC

Output SDR-GSC

Output SP-SDW-MWF




1717

Low-cost implementation (1)Low-cost implementation (1)

• Algorithms (in decreasing order of complexity):

o GSVD-based – chic et très cher

o QRD-based, fast QRD-based – chic et moins cher

o Stochastic gradient algorithms – chic et pas cher

• Stochastic gradient algorithm (time-domain) :

o Cost function

results in LMS-based updating formula

o Allows transition to classical LMS-based GSC by tuning some parameters (1/, w0)

22

0 ][][1

][][][)( kkEkkkvEJ TT xwvww

][][][][][]1[ 0 kkkvkkk T wvvww

regularisation term

][][][1

kkk T wxx

Classical GSC


Adaptive stage Implementation -Stochastic gradient

-Complexity and memory

Conclusions

1818

Low-cost implementation (2)Low-cost implementation (2)

• Stochastic gradient algorithm (time-domain):

o Regularisation term is unknown

– Store samples in memory buffer during speech-and-noise periods and approximate regularisation term

– Better estimate of regularisation term can be obtained by smoothing (low-pass filtering)

• Stochastic gradient algorithm (frequency-domain):

o Block-based implementation: fast convolution and fast correlation

Complexity reduction Tuning of and 1/ per frequency Still large memory requirement due to data buffers

o Approximation of regularisation term in FD allows to replace data buffers by correlation matrices memory + computational complexity reduction (cf. poster)

][][][1

kkk T wxx

Large buffer required




Conclusions

1919

Complexity + memoryComplexity + memory

• Parameters: N = 3 (mics), M = 2 (a), M = 3 (b), L = 32, fs = 16kHz, Ly = 10000

• Computational complexity:

• Memory requirement:

Algorithm Complexity (MAC) MIPS

QIC-GSC (3N-1)FFT + 16N - 9 2.16

SDW-MWF (buffer)

(3M+5)FFT + 30M + 10 3.22 (a), 4.27 (b)

SDW-MWF (matrix)

(3M+2)FFT + 10M2 + 15M + 4

2.71 (a), 4.31 (b)

Algorithm Memory kWords

QIC-GSC 4(N-1)L + 6L 0.45

SDW-MWF (buffer)

2MLy + 6LM + 7L 40.61 (a), 60.80 (b)

SDW-MWF (matrix)

4LM2 + 6LM + 7L 1.12 (a), 1.95 (b)

Complexity comparable to FD implementation of QIC-GSC

Substantial memory reduction through FD-approximation




Conclusions

2020

ConclusionsConclusions

• Design of robust multimicrophone noise reduction algorithm:

o Design of robust fixed spatial preprocessor

need for statistical information about microphones

o Design of robust adaptive stage

take speech distortion into account in cost function

• SP-SDW-MWF encompasses GSC and MWF as special cases

• Experimental results:

o SP-SDW-MWF achieves better noise reduction than QIC-GSC, for a given maximum speech distortion level

o Filter w0 improves performance in presence of model errors

• Implementation: stochastic gradient algorithms available at affordable complexity and memory

Spatially pre-processed SDW Multichannel Wiener Filter



Documents

Design and low-cost implementation of a robust multichannel noise reduction scheme for cochlear implants Simon Doclo 1, Ann Spriet 1,2, Marc Moonen 1,