View
231
Download
2
Tags:
Embed Size (px)
Citation preview
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
Formant Track Restoration in Train Noisy Speech
Qin Yan
Communication & Multimedia Signal Processing Group
Dept of Electronic & Computer Engineering, Brunel University
25 May, 2004
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
Main Progress
• Restore the formant tracks from the noisy speech.
• Initial progress of the speech enhancement system
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
Formant Tracking by 2D HMM in Noise Conditions
SNR F1 F2 F3 F4 F5
0 51.3 12.5 6.3 3.7 2.6
5 42 9.7 4.6 2.7 1.8
10 32.3 7.4 3.4 2 1.4
15 23.1 5.8 2.6 1.5 1.1
20 15.6 4.6 2.1 1.2 1
Table : Average errors (%) of formant tracks in train noisy speech by
2D HMM at different SNR conditions
• 2D HMM is not robust to formant tracking in noise conditions
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
LP Based Formant Tracking
Noise Model
LP-basedSpectral
Subtraction
Formant CandidatesSelection
LP Pole Analysis
Kalman Filter based
Formant Tracker
Noisy Speech
Formant tracks
VAD
Figure : Procedure of LP formant Tracking
• High LP order is to over-model the LP spectrum to split the poles from formants and noise.
• Formant candidate selection rejects spurious candidates.
• Kalman filter smoothes formant tracks.
• Formant tracks are fed back to reclassification according to the distance to the initial tracks
Reclassifier
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
LP Spectral Subtraction
• Noise is modelled by a low LP order but speech is modelled by a high order.
• Computation efficiency
• Disadvantage :
• Noise variance absence.
• A hard-decision needs to be employed to avoid the subtracted values going below a noise-floor.
• The spectral trajectory across time is not modeled and used in the denoising process.
)()()(ˆ fYfWfX LPSSSSLP
))(exp(
)(/)(ˆ)(1)(
SNRfSNRFf
fYfNffW
Thresh
LPSS
If ThreshSNRFf SNRf>
other
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
Performance of LP Spectra Subtraction
Figure : Improvement by LP spectra subtraction
-2-10123456789
0 5 10 15 20Global SNR(dB)
Imp
rovem
nt
SN
R(d
B)
LPSS Improvment
2
2
))()(ˆ)(
10log10fXfX
fXSNRframeNote : Improvement is calculated between average frame SNRs as:
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
LPC Spectrogram of speech in noisy train (SNR= 0)
LPC Spectrogram of Speech in noisy train after spectral subtraction
Performance I
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
• R is the measurement covariance matrix, updated by variance of differences between noisy observation and estimated tracks. • The process matrix Q is set to 0.16 experimentally.
Kalman Filter
1|ˆˆ
kkk FF
QPP k
1 )ˆ(ˆˆ kkkkk FZKFF
1)( RPPK kkk
kkk PKIP )()(ˆˆ
11| ikFcF k
P
ikikk
Time Update EquationsMeasurement Update Equations
“CORRECT”
“PREDICT”
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
Performance II
Figure : Comparison of clean formant tracks (solid) and cleaned formant tracks (dash dot) and noisy formant tracks (dot).
SNR=0 CleanedF1 51.3 18.1F2 12.5 11.8F3 6.3 6.2F4 3.7 2.7F5 2.6 2.5
Table : Average errors (%) of formant tracks in train noisy speech and cleaned speech.
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
Noise Model
LP-basedSpectral
Subtraction
Formant CandidatesSelection
LP Pole Analysis
Kalman Filter based
Formant Tracker
Noisy Speech
Formant tracks
VADReclassifier
Wiener Filter
Speech Reconstruction
Enhanced Speech
Initial Speech Enhancement system
Initial Speech Enhancement System
-2
0
2
4
6
8
10
0 5 10 15 20
Global SNR(dB)
Imp
rovem
nt
SN
R(d
B)
Overall Improvement
LPSS Improvment
Wiener Improvement
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
Speech enhancement with Speech enhancement with restored formant trajectoriesrestored formant trajectories
Future Work
Noise Model
LP-basedSpectral
Subtraction
Formant CandidatesSelection
LP Pole Analysis
Kalman Filter based
Formant Tracker
Noisy Speech
Formant tracks
VADReclassifier
Wiener Filter
Speech Reconstruction
Enhanced Speech
Initial Speech Enhancement system
Pitch Track Pitch Track
RestorationRestorationResidual
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
Speech enhancement with Speech enhancement with restored formant trajectoriesrestored formant trajectories
Future Work
Noise Model
LP-basedSpectral
Subtraction
Formant CandidatesSelection
LP Pole Analysis
Kalman Filter based
Formant Tracker
Noisy Speech
Formant tracks
VADReclassifier
Wiener Filter
Speech Reconstruction
Enhanced Speech
Speech Enhancement System
Pitch Track Pitch Track
RestorationRestorationResidual
Formant Tracks Restoration System
Co
mm
un
icat
ion
s &
Mu
ltim
edia
Sig
nal
Pro
cess
ing
The End