The Phonetic Filter Hypothesis: How phonology impacts speech perception (and vice versa) Emmanuel Dupoux Ecole des Hautes Etudes en Sciences Sociales,

The Phonetic Filter Hypothesis:The Phonetic Filter Hypothesis:How phonology impacts speech How phonology impacts speech

perception (and vice versa)perception (and vice versa)

Emmanuel DupouxEmmanuel DupouxEcole des Hautes Etudes en Sciences Sociales, ParisEcole des Hautes Etudes en Sciences Sociales, Paris

Sharon PeperkampSharon PeperkampUniversité de Paris VIIIUniversité de Paris VIII

Laboratoire de Sciences Cognitives et Psycholinguistique EHESS-ENS-CNRSLaboratoire de Sciences Cognitives et Psycholinguistique EHESS-ENS-CNRS

CNtrast in Phonology: Toronto 2002

Perception - Production Production

Phonetic Phonetic encodingencoding

Phonological encoding

Surface formSurface form

Phonetic planPhonetic plan

Underlying formUnderlying form

Lexeme retrieval

LemmaLemma

Perception - ProductionPerception - Production






Lexeme retrieval

LemmaLemma

Phonetic Phonetic decodingdecoding

Phonological decoding


Acoustic/Phonetic codeAcoustic/Phonetic code


Lemma retrieval

LemmaLemma






Lexeme retrieval

LemmaLemma






Lemma retrieval

LemmaLemma

Central ExecutiveCentral ExecutiveDecision MakingDecision Making

ResponseResponse

??

Modeling the TaskModeling the Task






Lexeme retrieval

LemmaLemma






Lemma retrieval

LemmaLemma

Central ExecutiveCentral ExecutiveAAAX: no variabilityAAAX: no variability


ResponseResponse






Lexeme retrieval

LemmaLemma






Lemma retrieval

LemmaLemma

Central ExecutiveCentral ExecutiveABX: talker changeABX: talker change


ResponseResponse






Lexeme retrieval

LemmaLemma






Lemma retrieval

LemmaLemma

Central ExecutiveCentral ExecutiveLexical DecisionLexical Decision


ResponseResponse

Perception - ProductionPerception - Production






Lexeme retrieval

LemmaLemma






Lemma retrieval

LemmaLemma

I. SegmentsI. Segments

Continuous signal - finite segment inventoriesContinuous signal - finite segment inventories

Continuous signal - finite segment inventoriesContinuous signal - finite segment inventories

92 consonants92 consonants

6 consonants6 consonants

XUXU

Language-specific segment detectorsLanguage-specific segment detectorsTRACE, McClelland & Elman, 1986, SHORLIST, Norris et al., 2000; Kuhl, 1996; Best, 1994 …TRACE, McClelland & Elman, 1986, SHORLIST, Norris et al., 2000; Kuhl, 1996; Best, 1994 …

English Japanese

||a|a| |la||la|

[[a]a]

ttppmm ss ii kkmm ttaa ee . . . . . .

||a|a| |la||la|

[ra][ra] [la][la]

ttppmm ss ii kkmm ttaarr ll Kuhl (1996)Kuhl (1996)

Early Acquisition of segmentsEarly Acquisition of segments

Werker & Tees (1984)Werker & Tees (1984)

Maye and Gerken (2002)Maye and Gerken (2002)

• Mechanism: statistical prototype extractionMechanism: statistical prototype extraction

Plasticity : L2 acquisitionPlasticity : L2 acquisitionSpanish-Catalan bilinguals (L2 started at age 4) (Pallier et al, 1998)

[e] vs [] classification Lexical decision: long term repetition priming

|pera| ... |pera|

|pra| ... |pera|

Plasticity : L2 acquisitionPlasticity : L2 acquisitionSpanish-Catalan bilinguals (L2 started at age 4) (Pallier et al, 1998)

[e] vs [] classification Lexical decision: long term repetition priming

|pera| ... |pera||pra| ... |pera|

Phonetic Phonetic DecodingDecoding

Phonological Decoding

Surface form

Underlying form


Phonetic decoding Phonetic decoding (I)(I)

Consonants & vowels

|la, a|

[a]

Japanese Spanish

|e, |

[]

Assimilate to the Assimilate to the phonetically closest phonetically closest

segmentsegment

Catalan

|e, |

[e,]

- acquired early & bottom up- acquired early & bottom up-non plastic in L2-non plastic in L2

II. PhonotacticsII. Phonotactics

• A perceptual effect? A perceptual effect? (Polivanov, 1974; Sapir, 1925) • Vowel Vowel degeminationdegemination in French in French

no contrast between long vs. short vowel

loanwords:

[tokjo] [tokjo]

[kjoto] [kjoto]

What counts as a segment?What counts as a segment?The influence of phonotacticsThe influence of phonotactics

• Vowel Vowel epenthesisepenthesis in Japanese in Japanese legal syllables: V, CV, VN, CVN

illegal syllables: *CVC, *CCV, ...

loanwords:

“Sphinx” [sufikusu]

“Christmas” [kurisutomasu]

insert /u/ after coda insert /u/ after coda consonant, or inside consonant, or inside onset cluster onset cluster (insert (insert /o/ after dental stop)/o/ after dental stop)

Dupoux, Kakehi, Hirose, Pallier, & Mehler (1997)

Conditions:Conditions:clustercluster: : ebuzo-ebzoebuzo-ebzovowel lengthvowel length: : ebuzo-ebuebuzo-ebuzozo

-50

0

50Cluster - Vowel score (%)Cluster - Vowel score (%)

French JapaneseFrench Japanese

A B BA B

time

AS1 S2 S3

Response:Response: A or B A or B

female voicefemale voice male voicemale voice

% u detection% u detection

0

20

40

60

80

100

Other No 0ms 14ms 29ms 44ms 58ms Full

Japanese

French

Vowel detection taskVowel detection task

[ebzo][ebzo] [ebuzo][ebuzo][ebazo][ebazo]

Speeded ABX taskSpeeded ABX task

Illusory vowels?Illusory vowels?






Lemma retrieval

LemmaLemma

Central ExecutiveCentral ExecutiveSegment IdentificationSegment Identification


ResponseResponse

*epenthesis**epenthesis*






Lemma retrieval

LemmaLemma

Central ExecutiveCentral ExecutiveSegment IdentificationSegment Identification


Orthographic codeOrthographic code

Phoneme-grapheme

ResponseResponse







Lemma retrieval

LemmaLemma

Central ExecutiveCentral Executivemulti-talker ABXmulti-talker ABX


ResponseResponse










Lemma retrieval

LemmaLemmaModeling the TaskModeling the Task


ResponseResponse

Articulatory loopArticulatory loop




LemmaLemma






Lemma retrieval

LemmaLemmaModeling the TaskModeling the Task


ResponseResponse


Prelexical effect?Prelexical effect?

Dupoux, Pallier, Kakehi & Mehler (in press)

LEXICON

mikado sokudo

[sokdo]

[mikdo]

0 50 100

% u detection % word decision

0 50 100

Transcription task Lexical decision

Prelexical effect!Prelexical effect!

Dupoux, Pallier, Kakehi & Mehler (in press)

LEXICON

mikado sokudo

/sokudo/

/mikudo/

0 50 100

% u detection % word decision

0 50 100

|mikdo| |sokdo|

Transcription task Lexical decision

Vowel epenthesis






Lemma retrieval

LemmaLemma

Central ExecutiveCentral ExecutiveLexical decisionLexical decision


ResponseResponse



Dehaene-Lambertz, Dupoux & Gout (2000)

0 25 50 75 100Contro

l

Devia

nt

Distr

Ebuzo …EbizoEbzo … Ebizo

Ebuzo … EbzoEbzo … Ebuzo

Ebuzo …EbuzoEbzo … Ebzo

Japanese French

|ebuzo| vs |

ebzo|

_

+

+

p

.001 .01 .05 .001.01.05

Time600 ms

S1BA

S2BA

S3BA

S4BA

S5AA

Deviant

Control

6 female voices male voice

400

800

200

+v

-400

Mismatch NegativityMismatch Negativity

How early? How early? High density ERPsHigh density ERPs

Where in the brain ? Where in the brain ? an fMRI studyan fMRI study

TR = 3.3 sec

A A A B B

time

Aacquisition S1 S2 S3

Deviant

Control

Conditions:Conditions: French French (N=7)(N=7) Japanese Japanese (N=7)(N=7)

ebuzo-ebzo phonological acousticebuzo-ebuzo acoustic phonological

Stimuli:Stimuli:20 items, same talker,no phonetic variability

AAX task:AAX task:

020406080

100

Phonological Acoustic

%e

rro

r

400

500

600

700

800

RT

(m

s)

Behavioral Results:Behavioral Results:

Jacquemot, Pallier, Dehaene, Lebihan and Dupoux (submitted)

Difference detection circuitDifference detection circuit

Phonological - AcousticPhonological - Acoustic

p<.005

PhonologicalPhonological AcousticAcoustic

p<.001

Jacquemot, Pallier, Dehaene, Lebihan and Dupoux (submitted)

Phonetic effectsPhonetic effects

0%10%20%30%40%50%60%70%80%90%

100%

pe

rce

nt

/i/

Brazilian Portuguese

European Portuguese

• Japanese: Japanese: |ebno|[ebuno], but |edno|[edono]; |du|[dzu]

• Brazilian Portuguese:Brazilian Portuguese: CV, CLV, CVs, CVn, CVr

• European Portuguese:European Portuguese: idem, but unstressed vowel deletion– complex phonetic syllables

If phonetic syllables matter, vowel epenthesis in BP, not EPIf phonetic syllables matter, vowel epenthesis in BP, not EP

[ebzo][ebzo] [ebizo][ebizo][ebazo][ebazo]Work in progress with Parlato & Frota

Plasticity: L2 acquisitionPlasticity: L2 acquisition• French-Japanese fluent bilinguals (N=7)

– native Japanese speakers

– late learners of French

– between 2 and 7 years in France

Nakamura & Dupoux (work in progress) Nakamura & Dupoux (work in progress)

sequence repetition

0

20

40

60

80

100

ebiza vs eboza ebza vs ebuza

contrast

pe

rce

nt

err

or

lexical decision (french stimuli)

0

10

20

30

40

50

60

policier plicier boulanger blanger

V insertion V deletion

err

or

rate



Surface form

Underlying form


Phonetic decoding Phonetic decoding (II)(II)

[e.bu.zo, en.zo]

|ebzo, enzo|

Japanese

Assimilate to the Assimilate to the phonetically closest legal phonetically closest legal

formform

Brazilian Portuguese

|ebzo|

[e.bi.zo]

Segments

Phonotactics - acquired early & bottom up- acquired early & bottom up-non plastic in L2-non plastic in L2

III. SuprasegmentalsIII. Suprasegmentals

Another dimension of phonetic decoding: Another dimension of phonetic decoding: suprasegmentalssuprasegmentals

Suprasegmentals: tones, stress, Suprasegmentals: tones, stress, pitch accentpitch accent

FrenchFrench SpanishSpanish

(Dupoux, Pallier, Sebastian, Mehler, 1997)(Dupoux, Pallier, Sebastian, Mehler, 1997)

– Stress ‘ deafness ’ Stress ‘ deafness ’ • ABX discrimination taskABX discrimination task

– Difficulties for English hearers with Chinese tones (Wang et al. 2000)

– Tokyo dialect versus non-accented Mito & Kumamoto dialect (Otake & Cutler, 1999).

• significantly less sensitivity to pitch accent in making lexical judgments

• more reliance on guessing (based on vocabulary statistics)

– Duration ‘ deafness’: (Dupoux et al. 1997)• |to:kjoo| [tokjo]• |kjo:to| [kjoto]

Perception of stress

Peperkamp, Vendelin & Dupoux (in preparation)

0

10

20

30

40

50

60

70

80

90

100

French Finnish Hungarian Spanish

perc

ent e

rror

phoneme

stress

• Sequence repetition:– AABAB answer 11212

– conditions:

phoneme: [muku - munu]

stress: [nmi - numi]

• Predictable stress:French, Finnish, Hungarian

• Contrastive stressSpanish

0 100110

250

freq

uenc

y (H

z)

relative duration (%)

mípa

mipá

[mipa][mipa]


Dupoux, Peperkamp & Sebastián-Gallés (2001)

6 tokens per item

0

10

20

30

40

50

60

French Spanish

6 tokens per item

0

10

20

30

40

50

60

70

French Spanish

% e

rro

r

phoneme

stress

1 token per item

0

5

10

15

20

25

30

French Spanish

1 token per item

0

10

20

30

40

50

60

70

French Spanish

% e

rro

r

phoneme

stress

Sequence repetitionSequence repetition



Surface form

Underlying form


Plasticity: L2 acquisition

Peperkamp, Sebastián-Gallés & Dupoux (in preparation)

• Sequence repetition– AABA answer 1121

• Late bilinguals (N=20):– native French

– learned Spanish after age 11

0

10

20

30

40

50

60

70

80

90

French French bilinguals Spanish

0

10

20

30

40

50

60

70

80

90

French Frenchbilinguals

Spanish

% e

rro

r

phoneme

stress



Surface form

Underlying form


Phonetic decoding Phonetic decoding (III)(III)

|bébe, bebé|

[bebe]

French

Assimilate to the Assimilate to the phonetically closest legal phonetically closest legal

surface formsurface form

Spanish

[bébe, bebé]

|bébe, bebé|

Suprasegmentals

Phonotactics

Segments

- acquired early & bottom up- acquired early & bottom up-non plastic in L2-non plastic in L2

IV. Consequences for IV. Consequences for PhonologyPhonology


Phonological Encoding

Surface form

Underlying form


Consequences for PhonologyConsequences for Phonology(or Why Loanword Phonology Does Not Exist)(or Why Loanword Phonology Does Not Exist)

French

‘walkman’/wkman/

Korean

‘baby’/bebi/

Engl. ‘pepsi’/ppsi/

White Hmong

‘sphinx’/sfinks/

Japanese

|wkman||pejbi| |pesi| |su.fi.ku.su|

[wkman][pejbi]

[pe(p)si] [su.fi.ku.su]

Apply phonology or Apply phonology or loanword phonology loanword phonology

(hidden rankings)(hidden rankings)

Problem #1: Problem #1: where does the where does the underlying form come from?underlying form come from?

• historical loanwords– used by monolingual speakers– no direct psychological reality of input-output analysis– diachronic interpretation

• on-line adaptations (Shinohara 1997, Kenstowicz & Sohn 2001)

– foreign words that are borrowed ‘here-and-now’

– for illegal forms, unfaithful perception, even in bilinguals

Original underlying form not available

Original underlying form not available

Note: adaptation of legal forms

Loanword adaptations of legal forms does not motivate a special ‘loanword phonology’ component (rules or hidden rankings)

• legal forms– Korean: [sinema] < Engl. cinema

(cf. native [kámani] ‘rice bag’)

– over-application of a default pattern in the language

– similar to overgeneralization processes with native forms (child phonology, language change)

Problem #2: The role of phoneticsProblem #2: The role of phonetics

• Adaptation of [, ]– European French: [s, z] vs. Canadian French: [t, d]– Hindi: [t, d]

• Adaptation of consonant clusters

– Japanese: insertion of [u]

– Brazilian Portuguese: insertion of [i]

– White Hmong: deletion

Why would a phonetic distance metrics matter?

Choice of adaptation is not necessarily driven by phonological markedness in the borrowing language:

Problem #3: LearnabilityProblem #3: Learnability

/ppsi/ NOCODA FILL PARSE

pep.si *!pe.p .si *!

pe<p>.si *

/ppsi/ NOCODA PARSE FILL

pep.si *! pe.p .si *!

pe<p>.si *

White Hmong

Japanese, Brazilian Portuguese

No universal ranking of Fill and Parse

No independent evidence for one ranking or the other in a given language

Solution: illegal formsSolution: illegal forms











[su.fi.ku.su]

|sfinks|‘sphinx’

/su.fi.ku.su/

- Perceptual assimilation- Perceptual assimilation- Phonetically based- Phonetically based- Learnable bottom-up- Learnable bottom-up

[su.fi.ku.su]

|su.fi.ku.su|

Solution: legal formsSolution: legal forms











- Overapplication of - Overapplication of common patterncommon pattern- No special loanword - No special loanword componentcomponent

[sinéma]

/sínema/

|sinéma||sínema|

[sínema]

Predictions• Adaptations of illegal forms

– can involve processes that do not otherwise occur in the language

– can only be accounted for in terms of phonetic distance to legal forms

• choice between epenthesis and deletion depends upon the presence vs. absence of phonetically reduced vowels

• non-adaptation occurs only with forms that are relatively distant from the closest legal form

• Adaptations of legal forms– involve default phonological rules that are otherwise present in the

language

clusion• Phonology heavily impacts perception

– perception is not faithful– phonetically-based perceptual assimilation– for illegal segments, phonotactics, suprasegmentals– learnable bottom-up during 1st year of life– not plastic (still exists in late bilinguals)

• Perception heavily impacts phonology– no loanword phonology (phonetics in perception + phonology in production)

– no child phonology (see Peperkamp, this afternoon)

– language change (Peperkamp, submitted)

CON

Documents

The Phonetic Filter Hypothesis: How phonology impacts speech perception (and vice versa) Emmanuel Dupoux Ecole des Hautes Etudes en Sciences Sociales,