View
217
Download
1
Tags:
Embed Size (px)
Citation preview
ESTIMATION OF A PHYSICAL MODEL OF THE VOCAL FOLDS VIA DYNAMIC PROGRAMMING TECHINQUES
E. Marchetto1, F. Avanzini1, and C. Drioli2
1Dept. of Information Engineering, University of Padova, Italy2Dept. of Computer Science, University of Verona, Italy
MAVEBA 2007 Firenze, 13-15 Dec. 2007
Summary The physical model and its control A codebook between articulatory vectors
and acoustical vectors The inverse problem and its codebook Non-univocity issue, cost function and
dynamic programming Applications of the RBFNs by clustering Results with a resynthesis example Conclusions
The physical model We refer to the two-
mass vocal folds model presented in [1]
One-dimensional, quasi-stationary and incompressible flow
Time-varying separation point
Vocal tract modeled as an inertive load [2]
sz
Control of the physical model Low-level physical parameters are not independently
controlled by a speaker: more physiologically motivated control spaces are needed.
In [4] a set of rules, derived from [3], was used to control a two-mass model.
The rules link vocal fold geometry to the activation levels of three muscles: cricothyroid , thyroarytenoid and cricoarytenoid . We also consider the subglottal pressure .
Values normalized in [0-1], except in [0.5-1.5]kPa.
TAaLCa
ps
The physical model is completely controlledby a set of only four articulatory parameters:
CTa
CTa TAa LCa ps
ps
The direct codebook The glottal pulse is characterized by means of a
set of well-known acoustic parameters: Foundamental frequency (F0) Open, Speed, Return Quotients (OQ, SQ, RQ) Normalized Amplitude Quotient (NAQ)
Direct codebook as a Dictionary: Articulatory vectors are the keys Acoustical vector are the values Only one value for each key
Articulatory vector Acoustical vector
The direct codebook Large number of
numerical simu-lations of the two-mass model (about 100k)
86125 vectors in the codebook
The figure shows the distributions of the acoustical parameters
The inverse problem Given a glottal flow we want to estimate the
articulatory vectors which, used as input to the simulator, lead to a re-synthesis of the given glottal flow
The problem is in principle non-unique We build an inverse codebook
Each acoustical vector is associated to one or more articulatory vectors
How to tackle the non-uniqueness problem during the inverse lookup process?
Dynamic programming techniques
Dynamic programming Rather than work on single vectors, we sub-
divide the acoustical input sequence in frames In each frame we find the optimal sequence of
articulatory vectors by minimizing a cost function
Three terms: Acoustical distance between input vector and its
discretized companion in the codebook Articulatory effort: distance between each consecutive
articulatory vector in the output sequence Accumulation term: provides a way to find the global
minimum for the entire frame, but causes exponential complexity
)(min)( ,12,1,
2
2
1,
kkjki
kjk
ki ff vvvcxv
Dynamic programming We have N acoustical vectors in the frame, each
associated with Vk possible articulatory vectors Lookup process in brief (for each frame):
Forward: Compute the cost function for each path Backward: Minimize the cost function and choose the
optimal output sequence for the frame
Dynamic Prog. cuts down the complexity from expo-nential to polynomial Exploiting the optimal sub-
structure we are able to store many values instead of recalculate them
RBFNs are defined for functions, not for multi-maps
Radial Basis Function Networks A way to interpolate the articulatory space
The input vectors are rarely present in the codebook The output can only be the nearest approximation
We apply the RBFNs to interpolate from the acoustical space to the articulatory one[5]:
46 RR
Need to overcomethe non-uniqueness
Clusters and subclusters The algorithm avoids the non-
uniqueness problem Subdivide the acoustical space in
clusters Associate to each cluster oneor more subclusters in thearticulatory space
Cluster
Acoustical space
Subcluster
Articulatory space
Subclusters are built joining the nearest vectors Find a sort of hyperplanes in
the articulatory space and put together the nearest vectors
Create as many subclusters as are necessary to put every non-unique vector in a different subcluster
Results We apply the descripted techniques to a
complete resynthesis example The process in brief:
Starting from a recorded utterance, we estimate the glottal flow and characterize it by means of the acoustical parameters before descripted
The obtained vectors are used as input for dynamic programming and eventually RBFNs
The output articulatory vectors drive the numerical simulator, which outputs a full synthetic flow
Filtering the obtained flow with tempo-variant formants (from recorded utterance) we are able to obtain the resynthetized speech
Results / Articulatory vectors
About 160 vectors retrieved by dynamic programming. Notice the smoothness of the RBFNs vectors.
Without RBFNs
With RBFNs
Legend
Results / Acoustical vectors
Without RBFNs
With RBFNs
Legend
Reference
Comparison between the reference (input) acoustical vectors and the ones obtained by a look-up in the direct codebook using the vectors of the previous slide as keys
Conclusions We develop an effective approach to cope with the
inverse problem, with reference to the glottal source The cost function seems to adequately model the
physiological facts RBFNs have proved as a good tool in this context, but
some work remains to be done (weights determination and other peculiarities)
The resynthesis is perceptually good Also the time-varying vectors are almost well followed
Usually NAQ is followed with good accuracy We recall the relation between NAQ and voice quality
References [1] N. J. C. Lous, G. C. J. Hofmans, R. N. J. Veldhuis, and A.
Hirschberg, “A symmetrical two-mass vocal-fold model coupled to vocal tract and trachea, with application to prothesis design”, Acta Acustica united with Acustica, vol. 84 pp. 1135-1150, 1998
[2] I. R. Titze and B. H. Story, “Acoustic interactions of the voice source with the lower vocal tract”, J. Acoust. Soc. Am., vol. 101(4) pp. 2234-2243, Apr. 1997
[3] -, “Rules for controlling low-dimensional vocal fold models with muscle activation”, J. Acoust. Soc. Am., vol. 112(3) pp. 1064-1027, Sep. 2002
[4] F. Avanzini, S. Maratea and C. Drioli, “Physiological control of low-dimensional glottal models with applications to voice-source parameter matching”, Acta Acustica united with Acustica, vol. 92 suppl. 1 pp. 731-740, Aug. 2002
[5] T. Poggio and F. Girosi, “Networks for approximation and learning”, Proceedings of the IEEE, vol. 78(9) pp.1481-1497, Sep. 1990