Beyond F0: sentence modality and speech rate

Francesco CangemiLaboratoire Parole et Langage & Université de Provence

Aix-en-Provence

Outline

1. Introduction2. Material3. Discrete analysis (phone durations)

3a. Hypotheses 3b. Method 3c. Results3d. Discussion

4. Continuous analysis (local phone rate)4a. Hypotheses 4b. Method 4c. Results4d. Discussion

5. Conclusions

1. Introduction• Until recent years, researchers mainly regarded speech rate as a

phonetic feature falling outside the scope of the core form-function relations in language.

• In many cases, e.g. in the study of intrinsic phone durations, speech

rate has been considered as a source of noise to be controlled for or, in worst cases, simply normalized.

• Other scholars see speech rate as an idiosyncratic feature (thus potentially useful in speaker verification applications) or as related to paralinguistic dimensions (as in the case of emotional speech).

• Speech rate has also been considered as an acoustic cue to stylistic variation or, from the perspective of conversational analysis, as a resource for turn management.

Introduction ●○ Material Discrete Continuous Conclusions

1. Introduction• Efforts to put speech rate in direct relation with core modules of

language structure remain quite rare and, moreover, they usually lack explicitness.

• For example, the hypothesis of a link between speech rate and pragmatic meaning has only been asistematically foreshadowed in isolated studies [1], on few languages [2,3], or as a byproduct of analyses focused on other acoustic cues [1,3,4].

• In this paper, a production experiment on the effects of sentence modality (i.e. declarative vs. yes/no question) on speech rate in Neapolitan Italian (which is also the variety examined in [1,4]) is presented.[1] Maturi, P. (1988), L’intonazione delle frasi dichiarative ed interrogative nella varietà napoletana dell’Italiano, Rivista Italiana di Acustica, 12: 13-30

[2] van Heuven, V. & van Zanten, E. (2005), Speech rate as a secondary prosodic characteristic of polarity questions in three languages, SpeechCom, 47: 87–99[3] Smith, C. L. (2002), Prosodic Finality and Sentence Type in French, Language and Speech, 45 (2): 141–178[4] Petrone, C. (2008), Le rôle de la variabilité phonétique dans la représentation des contours intonatifs et de leur sens, PhD Thesis, Université Aix-Marseille I

Introduction ●● Material Discrete Continuous Conclusions

S V O

FOCUS

x3 repetit.x30 speak.

MODALITYQues Stat

2. MaterialNeapolitan Italian

read speech in sound-treated booth

30 speakers x 54 utterancespreceded by contextualization paragraph

from 3 highly controlled sentences:

sentence

s

Since focalization is relevant both to pragmatic interpretation and speech rate (through accenting and consequent lengthening phenomena), different focus patterns were also included in the experimental design.

Contrastive Narrow Focus on {S,V,O}.

E.g. Ralego doma Boveda

[σ.σ�.σ]S [σ�.σ]V [σ.σ�.σ]O

σ = CVentirely voicedno diphthongs

no “rare phones” S and O are fantasy names(lexical frequency control)

Segmentation performed using ASSI (Cangemi et alii, 2011) - 14h30 talk

Introduction Material ● Discrete Continuous Conclusions

Outline

1. Introduction2. Material3. Discrete analysis (phone durations)

3a. Hypotheses 3b. Method 3c. Results3d. Discussion

4. Continuous analysis (local phone rate)4a. Hypotheses 4b. Method 4c. Results4d. Discussion

5. Conclusions

Duration: S ? Q Language Utterance U-Start U-End Remarks

Maturi 1988 Neap. Italian >Smith 2002 French = < in Inverted/Partial

van Heuven and Manado Malay > >van Zanten 2005 Dutch > = = > in central portion

Petrone 2008 Neap. Italian < > /< >/< speaker dependent

3a. Hypotheses

Previous studies (on various languages with various methods) yield a very fragmented picture of duration across modality.

No clear effects, no unified theories

Introduction Material Discrete ●○○○ Continuous Conclusions

H1: duration difference between S/Q on utterance levelH2: duration differences on phrase/syllable/phone level

3b. Method

X-axis: phone position(see table above)

Y-axis: normalized duration (on duration of entire utterance)

Example across Focus conditions:

Introduction Material Discrete ●●○○ Continuous Conclusions

W R a l e g o d o m a B o v e d a

S 1 2 3 4 5 6 7 8

P C V C V C V C V C V C V C V C V

X C1 V1 C2 V2 C3 V3 C4 V4 C5 V5 C6 V6 C7 V7 C8 V8

3c. Results

Introduction Material Discrete ●●●○ Continuous Conclusions

Duration: S ? Q Language Utterance U-Start U-End

Maturi 1988 Neap. Italian >Smith 2002 French =

van Heuven and Manado Malay > >van Zanten 2005 Dutch > = =

Petrone 2008 Neap. Italian < > /< >/<

H1: Utterance

H2: Phone

3c. Results

Introduction Material Discrete ●●●○ Continuous Conclusions

Duration: S ? Q Language Utterance U-Start U-End

Maturi 1988 Neap. Italian >Smith 2002 French =

van Heuven and Manado Malay > >van Zanten 2005 Dutch > = =

Petrone 2008 Neap. Italian < > /< >/<

H1: Utterance

H2: Phone

3d. Discussion• Focus patterns and Sentence modality both seem to cause

lengthening. A full control of these factors is needed if comparisons with results from the literature are to be done.

• Utterance duration is the same in statements and questions, but final vowel duration is longer in questions. Initial segments are longer in statements, but this doesn’t apply to S-Focus condition.

These results clearly point to the need for global rather than local metrics:

DURATION SPEECH RATE

Introduction Material Discrete ●●●● Continuous Conclusions

Outline

1. Introduction2. Material3. Discrete analysis (phone durations)

3a. Hypotheses 3b. Method 3c. Results3d. Discussion

4. Continuous analysis (local phone rate)4a. Hypotheses 4b. Method 4c. Results4d. Discussion

5. Conclusions

4a. Hypotheses• For these reasons, in the second part of the study a different metric for the

assessment of speech rate was employed, in order to capture global patterns of variation rather than punctual differences localized on specific parts of the utterance.

• This is in line with current developments in the analysis of other acoustic cues, as shown by recent quantitative studies which used Functional Data Analysis on F0 data [5,6].

• Global representation of speech rate variations should be more useful in disentangling Focus-induced and Modality-induced lengthening. Separate analysis of focus conditions is crucial.

H3: Q and S show globally different speech rate patterns(qualitative assessment)

Introduction Material Discrete Continuous ●○○○ Conclusions

[5] Gubian, M., Cangemi, F. & Boves, L. (2010), Automatic and Data Driven Pitch Contour Manipulation with Functional Data Analysis, Proceedings of 5th Speech Prosody Conference (Chicago, May 11-14)[6] Gubian, M., Cangemi, F. & Boves, L. (2011), Joint analysis of F0 and speech rate with FDA, Proceedings of 36th ICASSP Conference (Prague, May 22-27)

4b. Method

LOCAL PHONE RATE: a continuous representation of variations in phone durations was calculated by revising some of the algorithms proposed in [7]

[7] Pfitzinger, H. (2001), Phonetische Analyse der Sprechgeschwindigkeit, Forschungs-berichte des Instituts für Phonetik und Sprachliche Kommunikation der Universität München, pp. 117-264.

Introduction Material Discrete Continuous ●●○○ Conclusions

X-Axis: Normalized Utterance DurationY-Axis: Local Phone Rate

Example across Focus conditions:

4c. ResultsH3 confirmed: Q and S do show different Local Phone Rate curves

Moreover, a comparison between S-Focus (left) and O-Focus (right) shows an INTERACTION

between Focus- and Modality-induced lengthening.

Introduction Material Discrete Continuous ●●●○ Conclusions

4d. DiscussionThe results of duration (§3) and speech rate (§4) analyses allow us to draw conclusions at different levels:

• First of all, and most importantly, the existence of a link between speech rate and pragmatic contrasts is confirmed.

• Speech rate in an utterance seem to be affected by both Focus and Modality. Focus-induced lengthening seem to be stronger in Declarative Modality (“interaction”).

• In conclusion, it seems that sentence modality affects speech rate in a global (rather than local) way. Local Phone Rate extraction could be more suited than discrete (utterance/phrase/syllable/phone) duration measurements for research in this field.

Introduction Material Discrete Continuous ●●●● Conclusions

5. ConclusionsWe still need to master reliable statistical techniques for the analysis of functional data in linguistics

New studies are directlyaddressing this issue [5,6]

Production studies as the one presented here ought to be complemented by perception studies in order to achievea better understanding of speech processing

Phonetic Detail reinforcing cue

offline/online tasks

The use of more spontaneous speech material could also represent an important phase of the theory validation process

Corpus DanSer as a first step

Could the exploration of this link between pragmatics (sentence modality) and phonetics (speech rate) benefit from a more abstract phonological representation?

[5] Gubian, M., Cangemi, F. & Boves, L. (2010), Automatic and Data Driven Pitch Contour Manipulation with Functional Data Analysis, Proceedings of 5th Speech Prosody Conference (Chicago, May 11-14)[6] Gubian, M., Cangemi, F. & Boves, L. (2011), Joint analysis of F0 and speech rate with FDA, Proceedings of 36th ICASSP Conference (Prague, May 22-27)

Introduction Material Discrete Continuous Conclusions ●

Beyond F0: sentence modality and speech rate

Francesco CangemiLaboratoire Parole et Langage & Université de Provence

Aix-en-Provence