Improvements in Speech Synthesis

8/18/2019 Improvements in Speech Synthesis

1/406


2/406


3/406


4/406


5/406


6/406


7/406


8/406


9/406


10/406


11/406


12/406


13/406


14/406


15/406


16/406


17/406


18/406


19/406


20/406


21/406


22/406


23/406


24/406


25/406


26/406


27/406


28/406


29/406


30/406


31/406


32/406


33/406


34/406


35/406


36/406


37/406


38/406


39/406

0 1000 2000

S p e c t r u m (

d B )

3000 4000

Frequency (Hz)

5000 6000 7000 80000

20

40

60

80

0 1000 2000

S p e c t r u m (

d B )

3000 4000

Frequency (Hz)

5000 6000 7000 80000

20

40

60

80


40/406

100

−24

−22

−20

−18

−16

−14

−12

−10

−8

120 140 160 180

Basic frequency (Hz)(a)

d B

200 220 240 260 280 300

100

−24

−22

−20

−18

−16

−14

−12

−10

−8

120 140 160 180Basic frequency (Hz)

d B

200 220 240 260 280 300(b)


41/406

100

−24

−22

−20

−18

−16

−14

−12

−10

−8

120 140 160 180

Basic frequency (Hz)

d B

200 220 240 260 280 300

(c)


42/406

010

20

30

40

50

60

d B

0.2 0.4 0.6 0.8 1 1.2

0−5000

0

5000

A m p l i t u d e

10000

0.2 0.4 0.6

Sec

0.8 1 1.2

Pitchmarking

Harmonic/stochasticdecomposition

PS-modulation

LPC analysis

PS-ABS

WSS discretecepstrum

j n dct j ncep

dctAncep

T 0

P npol

a nlpc

w 0

An


43/406

0−4000

−2000

0

2000

4000

6000

50 100 150 200 250 300 350 400 450

0−4000

−2000

0

2000

4000

6000

50 100 150 200 250 300 350 400 450


44/406

0

−20

0

20

d B

40

60

80

1000 2000 3000 4000

Hz

5000 6000 7000 8000

0

−3

−2

−1

0 R a d

1

2

3

1000 2000 3000 4000

Hz

5000 6000 7000 8000


45/406

−20

−10

0

10

20

AK

d B

30

40

0 2 4

kHZ kHZ kHZ

6−20

−10

0

10

20

LSP

d B

30

40

0 2 4 6−20

−10

0

10

20

DCT

d B

30

40

0 2 4 6


46/406

2000 4000 6000−2

−1

0

1

2x 104

2000 4000 6000−2

−1

0

1

2x 104

100 300 400200 5000

20

40

60

80

100 300 400200 5000

20

40

60

80

0 0.2 0.4 0.6S

sonagram

0.8 1 1.20

2000

4000

6000

8000

H z

(a)


47/406

0 0.2 0.4 0.6S

sinusoidal sonagram

0.8 1 1.20

2000

4000

6000

8000

H

z

(b)

20.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8−2

0

2

4 x 104

20.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8−2

0

0

1

2

3

10

2

4x 104 x 104

x 104

20.2 0.4 0.6 0.8 1

1.2 1.4 1.6 1.8

x 104(e)

(d)

(c)


48/406

−2000

−1000

0

1000

2000

0.505 0.51 0.515 0.52 0.525

−2000

−1000

0

1000

2000 + + + + + + + + + + + + + +

0.505 0.51 0.515 0.52 0.525


49/406

−5000

0

5000

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−5000

0

5000

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

−5000

0

5000

0.1

A m p l i t u d e

A

m p l i t u d e

A m p l i t u d e

0.2 0.3 0.4 0.5

Temps (sec)

0.6 0.7 0.8 0.9 1

White noisegenerator

PS-modulation LPC filter

+

Trilinear interpolation

j n ′dct j ncep

dctAncep

′T 0 P npol

′w 0

′An


50/406


51/406


52/406


53/406


54/406

Analysis

Prosodicdeviations

Covariationmodel

Synthesis

Modified paramtetricrepresentation

Original parametricrepresentation

off-line


55/406


56/406


57/406

pca

$

10000

0

0 1000 2000 ech

$ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $$ $^ ^ ^ ^ ̂ ^ ^ ̂ ^ ̂ ^ ̂ ^ ^ ^ ̂ ^ $ $

0

0

200

31.13 41.62

s

66.54 Syl

a−

Syl

Inf

SENTIMENTALISER

PHR

PRNCGN

Hz

10

seg

20 ech


58/406


59/406


60/406

0

100

200

−1

0

1x 104

−1

0

1x 104

Distortion

SSC output

Target


61/406

ORIGIN

FD

EM

AT30DB0

20DB0

10DB0

c4_1c4_0

c2_1c1_1

c2_0

c3_0

VO

TDPICP0 c1_0TDP

First component

S e c o n d c o m p o

n e n t


62/406

0

0

0

2000

4000

6000

8000

Hz

P P[ P@ @

0.2 0.4 0.6 s

T] [

(a)

0

0

0 0.2 0.4 0.6 s

2000

4000

6000

Hz

8000

@ @[ ]T] [

(b)


63/406

0

0

2000

4000

6000

8000

Hz

P T] [P[ ]@ @

(c)


64/406


65/406


66/406


67/406


68/406


69/406


70/406


71/406


72/406


73/406


74/406


75/406


76/406


77/406


78/406


79/406


80/406


81/406


82/406


83/406


84/406


85/406


86/406


87/406


88/406


89/406


90/406


91/406


92/406

0 100 200 300 400 500 600

0

Voiced residual signal

a)

samples0 100 200 300 400 500 600

0

Unvoiced residual signal

b)

samples

0 100 200 300 400 500 6000

Energy of voiced residual samples

0 100 200 300 400 500 6000

Energy of unvoiced residual samples


93/406

0.12 0.122 0.124 0.126 0.128 0.13 0.132 0.134 0.136 0.138

0

Time (s)


94/406


95/406

Inventory

LPC-Filter

bla blI

1/f0(t) 1/f0(t) 1/f0(t)

t

s res

f0(t)

t

f0 contour

s out

t


96/406


97/406

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16

0

@ d i t

Time (s)

Time (s)

F r e q u e n c y ( H z )

@ d i t

a)

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.160

2000

4000

6000

8000

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16

0

@ d i t

Time (s)

Time (s)


@ d i t

b)

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.160

2000

4000

6000

8000

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16

0

@ d i t

Time (s)

Time (s)


@ d i t

c)

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.160

2000

4000

6000

8000

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16

0

@ d i t

Time (s)

Time (s)


@ d i t

d)

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.160

2000

4000

6000

8000


98/406


99/406


100/406


101/406


102/406


103/406


104/406


105/406


106/406


107/406


108/406


109/406


110/406


111/406


112/406


113/406


114/406


115/406


116/406


117/406


118/406


119/406


120/406


121/406

Italian, Spanish, Portugues

French

Aucune de ces raisons ne regardaient son pouse

AU

cunede ces

raisons ne re gar daient son

pouse


122/406

Italian

Nessuna di queste ragioni riguar dava la moglie.

Nes

suna

diques

tera

gioni

ri

guarda va la mo glie

Spanish

Ninguno de estos motivos a su mujerconcern a

Nin guno de

es tos mo ti convos

cer n a a su mu

Portuguese

Nenhuma destas raz es dizia respeito a sua mulher

Nenhu

ma

des tas razões di zia res

peitoa

sua mu lher


123/406

Natural


Aucune

de ces rai sons ne re gar daientson

pouse


124/406

MONS


Au cune de

ces rai sons ne re gar daient son pouse

Bell Labs


Aucune de

ces rai sonsne

regar

daient sonpouse

Elan


Au cune de ces raisons ne re gar daient son

pouse


125/406

LATL


Au cune de

cesrai sons ne re gar daient son

pouse

LAIPTTS


Au cune de cesrai

sons ne re gar daient son pouse

SYNTAIX


Au cune de ces raisons

ne re gardaient son

pouse


126/406

L & H


Au cunede ces

rai ne re gar daient son

pouse

Natural

Un groupe de chercheurs allemands a r solu l' nigme.

Ungroupe de cher cheurs alle mands a so

nigmer l'


127/406

MONS

Un groupe de chercheurs allemands

Un groupe de cher cheursalle mands a so lu nigmer

l'

a r solu l' nigme.

Bell Labs


Un groupede

chercheurs

alle mands a sol u nigmer l'

a r solu l' nigme.

Elan


Ungroupe de cher cheurs alle mands a so lu

nigme

rl'

a r solu l' nigme.


128/406

LATL


Un groupe de chercheurs alle mands a so lu

nigmerl'

a r solu l' nigme.

LAIPTTS


Ungroupe

de cher cheurs allemands

a so lu nigmer l'

a r solu l' nigme.

SYNTAIX


Ungroupe de cher

cheurs allemands

a solu

nigmer l'

a r solu l' nigme.


129/406

L & H


Ungroupe de cher cheurs alle

mands a

so lunigme

r l'

a r solu l' nigme.

Natural

Alcuni edifici si sono rivelati pericolosi

Al

cuni

e

difi

ci si so no ri vela

ti

pe ri co

lo si


130/406

Bell Labs


Al cuni e di fi ci si so no ri ve la ti

pe ri co lo si

ELAN


Alcu ni e di

fi ci si so no ri ve lati

pe ri co lo si

L & H


Al cu ni edi

fici si so no ri ve la ti pe ri co

lo si


131/406

Un grupo de investigadores alemanes ha resuelto l'enigma

Natural

Un grupo de investigadores alemanes ha resuelto l' enigma.

Un gru pode in ves ga

tido

resa

lema

nes ha re sue lto l'enig

ma

Elan


Un gru po de in ves gati dores a le ma

nes ha re sue lto l'e nig ma


132/406

L & H


Un gru

pode in ves ga

ti do res a le manes ha re sue lto l'e nig

ma


133/406


134/406


135/406


136/406

Position of wordin the phrase

Position of tonic in theword

Standard Deviationin %

I s o l a t e d

E n d

E n d

0.0

5.0

10.0

15.0

20.0

25.0

30.0

M i d d l e

M i d d l e

B e g i n n i n g

B e g i n n i n g

Isolated Word

Word in theBeginning

Word in theMiddle

% o

f d u r a t i o

n

Word at the End

1. Beginning

2.Middle

3. End

4. Beginning

5.Middle

6. End

7. Beginning

8.Middle

9. End

10. Beginning

11.Middle

12. End1

0.0

50.0

100.0

150.0

200.0

250.0300.0

350.0

400.0

450.0

2 3 4 5 6 7 8 9 10 1211


137/406


138/406

Isolated Word

Word in theBeginning

Word in theMiddle

d B

Word at the End

1. Beginning2.Middle3. End


7. Beginning

8.Middle9. End


1−5.0

−10.0

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

2 3 4 5 6 7 8 9

Documents

Improvements in Speech Synthesis