MANDARIN TONE RECOGNITION USING AFFINE-INVARIANT PROSODIC FEATURES AND TONE POSTERIORGRAM

Yow-Bang Wang, Lin-Shan LeeINTERSPEECH 2010

Speaker: Hsiao-Tsung Hung

1.INTRODUCTION

Introduction

Tone recognition are definitely influenced by as least the following:1. Speaker2. The “prosodic state”3. Co-articulation effect

Introduction

Although the tones depend heavily on many intra-syllabic and prosodic behaviors which are definitely speaker dependent, the native speaker of Mandarin can easily recognize the tones

This implies the tones should be classified by some “robust” prosodic cues, which remain useful across many different conditions.

Introduction

in this paper we try to introduce robustness into prosodic features by different feature normalization schemes, based on the concept of affine invariance property proposed in recent years

We also incorporate the prosodic features with the context information by tone posteriorgram analogous to the TANDEM system for speech recognition.

2.PROPOSED APPROACH

Prosodic feature set

NumPitch Mean and slop

(3 segments)6

Mean and slop (Preceding and following syllable)

4

First frame, last frame, minimal, maximal pitch value

4

The last voiced frame pitch of preceding syllable

1

The first voiced frame pitch of following syllable

1

1Duration

Duration ratio with two adjacent syllables 2

Energy Log-energy difference with two adjacent syllables

2

Affine Invariance property Consider an n-dimensional feature

vector sequence along the time axis. If a certain change of condition over these feature vectors is stationary within some period of time, and can be represented as an affine translation:

Affine Invariance property There may exist some features

obtained from which remain invariant under such change of conditions:

,where is the feature function.

Affine invariance for normalized pitch features Assume the transformation between

the pitch contours for the same syllable for two speakers, and , can be approximated by an affine transform:

(assume here)

Affine invariance for normalized pitch features relationship between the utterance-

level means and standard deviation:

Affine invariance for normalized pitch features

Any feature function M() applied to this normalized pitch contour is automatically affine-invariant.

Invariance of duration and energy features Duration

Energy difference for two adjacent syllables

Pitch contour normalization schemes

Tone recognition

21-dimensional prosodic feature vector

SVM

Enh1 : current syllableEnh2 : current, preceding and following syllable

EXPERIMENTS

Corpus and experiment setup Sinica Continuous Speech Prosody

Corpora (COSPRO) Contained 4672 utterances (more

than 60,000 syllables), produced by 38 male and 40 female native speakers.

SVM tone recognizers.

Experimental results

Experimental results