26
Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological Support: Many thanks to: Dafydd Gibbon Emmanuel Dupoux Franck Ramus

Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

  • View
    220

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Automatic identification of vocalic intervals in speech signal

Jesus Garcia Antonio Galves

Flaviane FernandesJanaisa ViscardiUlrike Gut

Phonological Support:

Many thanks to:

Dafydd GibbonEmmanuel DupouxFranck Ramus

Page 2: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

ProblemWe have:

and we want:

0.179 0.301 v 0.301 0.390 c 0.390 0.440 v0.440 0.498 c0.498 ....

Page 3: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Identification

Cues to the segmentation:

acoustic signal plotspectrogram plotlistening

Page 4: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Acoustic Signal

Spectrogram

Noise

Vowel

Cons.

Vowel.

“A autoridade do governador diminuiu”

Example:

Page 5: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Marca

Noise

Vowel

Cons.

Vowel.

Page 6: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

vowels are recurrent

Page 7: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

“t” signal:

Page 8: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

c(i,t) = energy for the frequency i in time t

pi,t c i,t k1

N

ck,t

Page 9: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Relative entropy of the column t respect to the column s.

h t,s p i,t ln p i,t p i,s

Page 10: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

we use h(t-1,t), h(t-2,t) e h(t-3,t).

plot for h(t):

h(t)=h(t-1,t)+h(t-2,t) +h(t-3,t)

Page 11: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological
Page 12: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Distance between consecutives columns

Page 13: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

d t,t1 p i,t pi,t12

Page 14: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological
Page 15: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological
Page 16: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological
Page 17: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological
Page 18: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological
Page 19: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Spectrogram for the sentence:' A hurricane was announced this afternoon on the TV.'

Page 20: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Signal

Relative Entropy800-3000 hz

Relative Entropy0-800 hz

Page 21: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Spectrogram and segmentation for' A hurricane was announced this afternoon on the TV.'

Page 22: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological
Page 23: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Box Plots for the integral of the relative entropy for each language

Page 24: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Spectrogram and R. entropy for the sentence with the lower integral

Page 25: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological

Ramus classification (1999).

%V x Delta C

jap

catspa ita

fre

dutpol

eng

3

3,5

4

4,5

5

5,5

40 45 50 55

Del

taC

%V

%V = vowel percent

DeltaC = standard deviation of consonantal intervals

Page 26: Automatic identification of vocalic intervals in speech signal Jesus Garcia Antonio Galves Flaviane Fernandes Janaisa Viscardi Ulrike Gut Phonological