Upload
others
View
15
Download
0
Embed Size (px)
Citation preview
HAL Id: tel-01076418https://tel.archives-ouvertes.fr/tel-01076418
Submitted on 22 Oct 2014
HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.
Model-based clustering for categorical and mixed datasets
Matthieu Marbac-Lourdelle
To cite this version:Matthieu Marbac-Lourdelle. Model-based clustering for categorical and mixed data sets. Statistics[math.ST]. université lille 1, 2014. English. tel-01076418
♥♠ér♦ ♦rr
❯♥rsté ♦rt♦r P P♥é
ès
♣rés♥té ♣♦r ♦t♥t♦♥
♣ô♠ ♦t♦rt
♥rsté
♣été té♠tqs
♦ès ♠é♥ ♣♦r sst♦♥♥♦♥ s♣rsé ♦♥♥és qtts t
♠①ts
♣r
tt r♦r
♦t♥ s♣t♠r ♥t r② ♦♠♣♦sé
♠trs rs Pr♦ssr ❯♥rst② ♦ t♥s ♣♣♦rtr♥ r♥ Pr♦ssr ❯♥rsté ♦♥t♣r ♣♣♦rtrs ① rtr r ♥r ② ①♠♥tr♦s ❲r Pr♦ssr ❯♥rsté ①♠♥trrst♦♣ r♥ Pr♦ssr ❯♥rsté rtr❱♥♥t ❱♥ Pr♦ssr ❯♥rsté rtr
ès ♣ré♣ré é♣rt♠♥t té♠tqs♦rt♦r P P♥é ❯ ❯♥rsté ❱♥ sq ❳
♠r♠♥ts
rst ♦ s♥r② ♦ t♦ t♥ ♠trs rs ♥ ♥ r♥♦r ♣t♥ t♦ r♣♦rt ♦♥ ts tss s s s ① ♥ ♦s ❲r♦r tr ♥ ♣rt♣t♦♥ s ①♠♥rs ♥ t P ♥s
rss é♠♥t ♣r♦♦♥s r♠r♠♥ts à ♠s ① rtrs tèsrst♦♣ r♥ t ❱♥♥t ❱♥ ♣♦r r ♥r♠♥t ♦rs s tr♦s ♥♥és ❱♦s ③ s str ♥♦♠r① ♦♥ss t♦t ♥ ♠ ss♥t♥ rté q♥t ① ①s rrs q ♣ ①♣♦rr q ♦rt♠♥t♣♣réé ♦s r♠r ♠♦r ♦♥♥é ♦♣♣♦rt♥té ♠♥tr à rr♦rs ♠♦♥ st ♠str t ♦s êtr é♠♥és ♣♦r ♠ tr♦r ♥ ♥♥♠♥t tès
r♠r r♥ q ♠ ♦♥♥é ♦ût à rr ♦rs ♥♦s ré♥♦♥s q s s♦♥t éés ss ♣rtèr♠♥t r♦♥♥ss♥t ♥rs r♥ r♠♥♥ q ♠ ♥♦ré ♣s ♠♦♥ ♠str t q t♦♦rs s♣r♥r t♠♣s ♠ ♣r♦r ♣ré① ♦♥ss
rss é♠♥t ♠s r♠r♠♥ts à ♥s♠ éq♣ ♦ù ♠♦♥♥tért♦♥ été té râ à ♥r♥ ♥
♦rs s tr♦s ♥♥és tès ♣sr é♦rr ♠♦♥ ♥s♥♠♥t r♥ ♠r rst♥ Pr t ♥ qs
r♠r ♣rs♦♥♥ ♦rt♦r P♥é ♥ ♣rtr ♥ r♥t② éè♥ réérq ♥qs r♥ç♦s t♦♠t t ♠r r♦ ♠s
P♥♥t tt tès ♣sr é♥r ♥♦♠rss ♣rs♦♥♥s♣r♠ sqs r Pr♠t ①♥r ♥♠♥ Prr♦s t P♦rt t ♥ s ♠s ♦♥t été ♣rtèr♠♥t ♠é♠♦rs râ à ♥t♦♥ ♠ t Prr r à ♦ t s ♥ ♣s ♠s ♥t ♣♦r ss♦♥ss ♥ ♣rtr ♣♥♥t ♠ rét♦♥
r♠r s ♠♠rs ♠ ♠ qs ♠♥t ♦ ♥♦♥ s ♥♥s ♣♦rr s♦t♥ t ♣r q ♥ ss sr s ♦♥t t♦s rt♥ ttr ♠ tèsrss ♥ ♠♥t♦♥ s♣é ♣♦r é♦ q ♠ s♣♣♦rt q♦t♥
♥♥ ♣rés♥t ♠s ①ss à ① q ♦♥ts♠♥t ♦é ♠♥t♦♥♥rt q s♣èr ♥ ♠♥ t♥r♦♥s ♣s rr
♦♥t♥ts
♠r♠♥ts
♦r♦r
♥ rt♦♥s ♥ ♥♦tt♦♥s
str ♥②ss stt ♦ t rt r ♦ t str♥ ♣♣r♦s ♥rts ♦♥ ♥t ♠①tr ♠♦s Pr♠tr st♠t♦♥ ♦ st♦♥ ♦♥s♦♥
♦s str♥ ♦r t♦r t
str ♥②ss ♦ t♦r t sts stt ♦ t rt ♥ ♦ str ♥②ss ♦r t♦r t ♦♠tr ♣♣r♦s ♦♥r ♠①tr ♠♦s ①trs ♦ trs t t♥t ss ♠♦ ♦♥s♦♥
♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s ♥tr♦t♦♥ ①tr ♦ ♥trss ♥♣♥♥t ♦s Prs♠♦♥♦s ♦ strt♦♥ ①♠♠ ♦♦ st♠t♦♥ ♦rt♠ ♦ st♦♥ ♦rt♠ ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ t♦ r t sts ♦♥s♦♥
♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s ♥tr♦t♦♥ ①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s
♦♥t♥ts
①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠ ♦ st♦♥ tr♦♣♦st♥s s♠♣r ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ t♦ r t sts ♦♥s♦♥
♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s strt ♦♦s
♦♥s♦♥ ♦ Prt
♦s str♥ ♦r ♠① t
str ♥②ss ♦ ♠① t sts stt ♦ t rt ♥ ♦ str ♥②ss ♦r ♠① t r ♦ s♠♣ ♠t♦s t♦ str ♠① t ①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s ❯♥r♥ ss♥ ♠①tr ♠♦ ♦♥s♦♥
♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s ♥tr♦t♦♥ ①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠ ♦ st♦♥ ♦rt♠ ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ t♦ r t sts ♦♥s♦♥
♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t ♥tr♦t♦♥ ①tr ♠♦ ♦ ss♥ ♦♣s ②s♥ ♥r♥ tr♦♣♦st♥s s♠♣r ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ tr r t sts ♦♥s♦♥
♦♥s♦♥ ♦ Prt
♥r ♦♥s♦♥ ♥ ♣rs♣ts
♣♣♥① ♦ Prt ♥r ♥tt② ♦ t ♠①tr ♦ t t♦ ①tr♠ ♣♥♥②
strt♦♥s
♦♥t♥ts
♥r ♥tt② ♦ t ♠①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s
♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt ♦♦ ♦ t ♠①tr♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s
♣♣♥① ♦ Prt ♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st str
t♦♥s ♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s
♦r♣②
♦r♦r
♥ t ♥ t♦ str
t qst♦♥ s ♦♠ ♥rs♥② s② t♥s t♦ t ♥rs ♣r♦r♠♥ ♦ ♦♠♣t♥ r♦r ♣rtt♦♥rs r ♥ t sts r ♥rs♥② ♥♦r♠t♦♥r t s♦ ♥rs♥② strs ♥ t ♥♦r♠t♦♥♦♥t♥ ♥ t t sts ♥ rt② ♥tt♥ ♦r t♦ ♠♥ rs♦♥s tq♥tt② ♦ t ♥ tr ♦♠♣①t② s sttst ♠t♦s r ♠♥t♦r② t♦♥②③ s t sts
str♥ s ♥ ♣♣r♦ rs t ♣r♦♠ s ② t r q♥tt②♦ t ♥ ts ♠ s t♦ r♦♣ t ♥s ♥t♦ s♣ sss st ♣r♦s ♠♥♥ s♠♠r② ♦ t t st tr♦♦t rtrst♥s ♦ t ♥ ♦r ♦t str♥ s ♥tr ♦r ♥st♥t ♥ ♦r s ♥t♦ t♦ sss ♣♥ts ♥ ♥♠s t ♥♠s rs♣t ♥t♦ ♥rtrt ♥ rtrt t rtrt r ss ♥t♦ sss♠♠♠s ss rs ♠♣♥s r♣ts
♣r♦st ♠t♦s ♣r♠t t♦ ♣r♦r♠ t str ♥②ss ♥ r♦r♦s♦♥t①t ♠♦♥ ts ♠t♦s t ♥t ♠①trs ♦ ♣r♠tr strt♦♥s s♠♠r③ t t ② t ♣r♠trs ♦ ss ♦r♦r ♥ ts ♦♥t①t tss ♣r♦st t♦♦s r t♦ ♥sr t t qst♦♥s ♦ str♥②ss t ♦ ♦ t ♥♠r ♦ sss t ♦r♣② s ♣r♦ ♦t♦♥t♥♦s t sts ♥♦t s♦rt ♥ t t r ♠♦r ♦♠♣① ♥ ts♦♥t①t t ♠ ♦ ts ♠♥sr♣t s t♦ st② ①st♥ ♣r♦st ♠t♦s ♥t♦ ♣r♦♣♦s ♥ ♦♥s t♦ str ♦♠♣① t sts
t♦ ♦ts ♦ ts ♦r
❲ ♦s ♦♥ t♦ stt♦♥s r t t sts r ♦♠♣① t s r ♥s r sr ② t♦r rs ♥ t s r t② r sr② ♠① rs r♥t ♥s ♦ rs s t♦ t♠ts r st
♠♦s str♥ ♦r t♦r t sts ♠♦s str♥ ♦r ♠① t sts
t♦r rs r t t♦ str s♥ t② t sttst♥♥ t ♠♥② ♦♠♥t♦r ♥s s t② ♥rss ♥ t rs r ♣♥♥t ♥ t s♠ ss ♥ t ♠♦s rqr r ♥♠r ♦♣r♠trs ♥ ♦rr t♦ t ♥t♦ ♦♥t t ♥trss ♣♥♥s s t ss ♣♣r♦ ss♠s t ♦♥t♦♥ ♥♣♥♥ t♥ rs ♦r
♦r♦r
ts ♣♣r♦ s s ♥ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ s ♦t♥② tr♥t ♣♣r♦s ♥ ♣r♦♣♦s t tr ♥srs st② ♥♦♠♣t♥ t ♦♠♥t♦r ♣r♦♠ ♦ t ♠♦ st♦♥ s ♥♦t ②s s♦ ♦r♦r ts ♠t♦s ♥ sr r♦♠ ♥stt② ♦r r♦♠ ♦ ♥tr♣rtt② ♥ts ♦♥t①t ♦r ♦♥trt♦♥ ♦♥ssts ♥ t♦ ♣rs♠♦♥♦s ♠①tr ♠♦s ♦ t♦ str t♦r t ♣rs♥t♥ ♥trss ♣♥♥s ♠♥ ♦ ts ♠♦s s t♦ r♦♣ t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ②stt♥ s♣ strt♦♥s ♦r ts ♦s ♦t ♠♦s ♦♥sr t ♥trss♣♥♥s t♥ t rs ♦t ♠♦s ♣r♦ ♣r♠trs t♦ s♠♠r③ t t ♣r♦♣♦s r♦r♦s ♣♣r♦ t♦ ♣r♦r♠ t ♠♦ st♦♥ ♥ srr♥② s③t♦♥ ♦ t ♣r♠trs ♥ ♦ t t♦rt str♥ s ♠♦tt ② t t tt t♦r t r s② sss t② r ♥♠r♦s t rs t♦ ♦sr ♥trss ♦rrt t ♥rss
st② ♦ t str ♥②ss ♦ ♠① t sts s t s♦♥ ♦t ♦ts ♦r s ♣r♦♠ s ♠♦tt ② t t tt t rr♥t t sts r ♦t♥♦♠♣♦s t r♥t ♥s ♦ t ss ♣♣r♦ ss t♦r ♥②③rs♠t♦s ♥tr♣rtt♦♥ ♦ s ♠t♦ s ♦t♥ ♦♠♣① s♥ t ♣r♠trsr ♥♦t rt t♦ t rs ♥ tr ♥t s♣ tr ss ♣♣r♦s♦♥sst ♥ ♣♣②♥ s♣ ♠①tr ♠♦ ♦♥ ts t ♥ s t♦t ♦ ss strt♦♥s ♦r ♠① rs ❲ ♣r♦♣♦s t♦ ♠①tr ♠♦s t♦ ts ♣ rst ♦♥ s ss s♥ t s ♥ ①t♥s♦♥ ♦ ♥♦♥♠t♦ t♦ str t♦r t sts ♥ t ♠♦ ♦♠♥s ss♥ strt♦♥s ♥ ♥r ♦st rrss♦♥s s ts ♠♦ ♥②③s t sts t♦♥t♥♦s ♥ t♦r rs s♦♥ ♠♦ s t ♠♥ ♦♥trt♦♥ ♦ts tss s♥ t ♦s t♦ ♥②③ t sts t ♥② ♥ ♦ rs ♠tt♥ ♠t strt♦♥ ♥t♦♥ s ♠♦ s ♥ s ♠①tr ♦ ss♥ ♦♣s ts ♣rsr♥ ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥ ♦r rs ♦ ♦♠♣♦♥♥t rtr♠♦r ts ♣♣r♦ ♠♦③s t ♥trss♣♥♥s ♥② ♥♦t tt s③t♦♥ t♦♦ s s ②♣r♦t ♦ts ♠♦
r♥③t♦♥ ♦ t ♠♥sr♣t
♠♥sr♣t s ♥t♦ t♦ ♠♥ ♣rts ♦rrs♣♦♥♥ t♦ t t♦ ♥s♦ t ♦ ♥trst ♦r ♣rs② t s ♦r♥③ s ♦♦s
♣tr s r ♦r ♦ t ♠♥ str♥ ♠t♦s ♥ ♥rr♠♦r t ♦ss ♦♥ t r♥t s♣ts rt t♦ ♥t ♠①tr ♠♦s t ♥tr♦s t ♥r ♥♦t♦♥s ♥ ♦rt♠s s ♥ t ♦♦♥♣trs
Prt ♦s str♥ ♦r t♦r t
♣tr ♦♥ssts ♥ t stt ♦ t rt ♦ t ♠t♦s ♣r♦r♠♥ tstr ♥②ss ♦ t♦r t sts
♦r♦r
♣tr ♣rs♥ts ♦r rst ♦♥trt♦♥ t♦ t t♦r t ♥②ssr♠♦r ♣r♦♣♦s ♠♦ r♦♣s t rs ♥t♦ ♦♥t♦♥②♥♣♥♥t ♦s s♣ strt♦♥ ♦ t ♦s ♠♦③s t♥trss ♣♥♥s ♥ ♣r♦s s♣ ♦♥t s♠♠r③♥t str♥t ♦ ts ♣♥♥s ts rsts r ♣rt ♦ t rt ♦s str♥ ♦r ♦♥t♦♥② ♦rrt t♦r t❬❱❪
♣tr ♣rs♥ts ♦r s♦♥ ♦♥trt♦♥ t♦ t t♦r t ♥②ss r♠♦r s ♠♦ ♦♥ssts ♥ ♠①tr ♠♦ r♦♣st rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ ♦♦s ♣rs♠♦♥♦s ♠t♥♦♠ strt♦♥ r t r ♣r♠trs♦rrs♣♦♥ t♦ ts ♠♦s ts rsts r ♣rt ♦ t rt ♥t♠①tr ♠♦ ♦ ♦♥t♦♥ ♣♥♥s ♠♦s t♦ str t♦rt ❬❱❪
♣tr strts ♦t ♣s ♣r♦r♠ t ♥r♥ ♦♦t ♣r♦♣♦s ♠♦s s ♣tr ♥ s♦ s s tt♦r ♦♦t ♣s s♥ t ♣r♦s ♣rs♥tt♦♥ ♦ tr ♠♥ ♥t♦♥s ♥♠♥② sr♣ts ♦♥ t♦ ♣r♦r♠ t str ♥②ss
Prt ♦s str♥ ♦r ♠① t
♣tr ♦♥ssts ♥ t stt ♦ t rt ♦ t ♠t♦s ♣r♦r♠♥ tstr ♥②ss ♦ ♠① t sts
♣tr ♣rs♥ts ♦r rst ♦♥trt♦♥ t♦ t str ♥②ss r♠♦r ♦ ♠① t sts t ♦♥t♥♦s ♥ t♦r rs ♠♦ s r r♦♠ t ♠t t♥t ss ♠♦ ♦♣ t♦ str t♦r t sts ♦r ts ♠♦ t ♦♠♣♦♥♥t strt♦♥s ♦t ♦♥t♥♦s rs r ss♥ ♥ t♦s ♦ t t♦r rs ♦♥t♦♥② ♦♥ t ♦♥t♥♦s ♦♥s r ♥r ♦st rrss♦♥s ♠♦ st♦♥ ♥ t ♣r♠tr st♠t♦♥ r s♠t♥♦s② ♣r♦r♠ ② ♠ ♦rt♠
♣tr ♣rs♥ts t ♠♥ ♦♥trt♦♥ ♦ ts tss t ♦♥ssts ♥ ♠①tr ♠♦ ♦ ss♥ ♦♣s s t ♠♦ ♣r♦r♠s t str ♥②ss ♦ t sts ♦♠♣♦s ♦ ♥② ♥ ♦ rs ♠tt♥ ♠t strt♦♥ ♥t♦♥ ts rsts r ♣rt ♦ t rt♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t ❬❱❪
♥ rt♦♥s ♥ ♥♦tt♦♥s
♥ rt♦♥s
♥r
♠♣ ♠①♠♠ ♣♦str♦r♠♣ ♠①♠♠ ♣♦str♦r st♠t♠ ♠①♠♠ ♦♦ st♠t ♠t strt♦♥ ♥t♦♥♣ ♣r♦t② strt♦♥ ♥t♦♥
♦rt♠s
♠ ①♣tt♦♥①♠③t♦♥ ♦rt♠♠ ♥r③ ①♣tt♦♥①♠③t♦♥ ♦rt♠
♠♠ r♦ ♥ ♦♥t r♦s♠ t♦st ①♣tt♦♥①♠③t♦♥ ♦rt♠
♥♦r♠t♦♥ rtr
♥♦r♠t♦♥ rtr♦♥ ②s♥ ♥♦r♠t♦♥ rtr♦♥ ♥trt ♦♠♣t♦♦
♥ ♥♦tt♦♥s
♥♦tt♦♥s ♥ ♥ t t ♦♦♥ rs
rs r ♥♦t t r ttrs t ♣r♠trs r ♥♦t t r ttrs
♠t♠♥s♦♥ ♦ts r ♥♦t ② ♦ s②♠♦s t ♥♠♥s♦♥ ♦ts r ♥♦t ② t♥ s②♠♦s
♥ rt♦♥s ♥ ♥♦tt♦♥s
❱rs ♥ ♦srt♦♥s
X i st ♦ t e r♥♦♠ rs rt t♦ ♥ ixi ♦sr s ♦ X i
x′i tr♥s♣♦s ♦ xi
x
i sst ♦ xi ♦♠♣♦s ♦ t c ♦♥t♥♦s rsx
i sst ♦ xi ♦♠♣♦s ♦ t d srt rsmj ♥♠r ♦ ♠♦ts ♦ r jZi r♥♦♠ r ♦ t ss ♠♠rs♣ ♦ t ♥ izi ♦sr s ♦ Zi
yi s♦♥ t♥t r rt t♦ ♥ i rqr① n s♠♣ ① = (x1, . . . ,xn)③ n s♠♣ ③ = (z1, . . . , zn)② n s♠♣ ② = (y1, . . . ,yn)
Pr♠trs
θ ♦ ♣r♠trs ♦ t ♠①trπ t♦r ♦ ♣r♦♣♦rt♦♥sαk ♣r♠trs rt t♦ ♦♠♣♦♥♥t kΓk ♠tr① ♦ t ♦rrt♦♥ rt t♦ ♦♠♣♦♥♥t kν ♥♠r ♦ ♣r♠trs
♠♣♦rt♥t ♥trs
c ♥♠r ♦ ♦♥t♥♦s rsd ♥♠r ♦ t♦r rse ♥♠r ♦ rs c+ d = eg ♥♠r ♦ sssn s③ ♦ t s♠♣nk s③ ♦ ss k ♦♠♣t ♦♥ t ③③② ♣rtt♦♥♥k s③ ♦ ss k ♦♠♣t ♦♥ t r ♣rtt♦♥
ss strt♦♥s
Dg(.) rt strt♦♥ ♦ s③ gG(.) ♠♠ strt♦♥
Nc(µ,Σ) crt ss♥ strt♦♥ t ♠♥ µ ♥ ♦r♥ ♠tr① Σ
Mg(.) ♠t♥♦♠ strt♦♥ ♦ s③ gP(.) P♦ss♦♥ strt♦♥
♥ rt♦♥s ♥ ♥♦tt♦♥s
ss t♦♦s
p(.; .) ♣r♦t② strt♦♥ ♥t♦♥P (.; .) ♠t strt♦♥ ♥t♦♥
φc(.;µ,Σ) ♣ ♦ Nc(µ,Σ)Φc(.;µ,Σ) ♦ Nc(µ,Σ)
Φ1(.) ♣ ♦ N1(0, 1)KL(f1, f2) r r♥ r♦♠ f1 t♦ f2 f2 rr♥
p(①;θ) ♦srt ♦♦L(θ;①) ♦srt ♦♦♦
p(①, ③;θ) ♦♠♣tt ♦♦L(θ;①, ③) ♦♠♣tt ♦♦♦
tik(θ) ♣r♦t② tt xi s r♥ ② ♦♠♣♦♥♥t k
♣tr
str ♥②ss stt ♦ t rt
♠♥ ♣r♣♦s ♦ ts ♣tr s t♦ r t trtr ♦t str ♥②ss ♦t tt ♦r ♠ s ♥♦tt♦ ①st ts ♣r♥♣② ♦s ♦♥ t ♠♦s ♣♣r♦s ♥ ♦rr t♦ ♥ t r♥t ♥♦t♦♥s♦♣ ♥ ts ♠♥sr♣trst② ♣rs♥t r♥t ♣♣r♦s ♦♠tr ♥♣r♦st t♦ str t t ♦♥② rt rq♥tst ♥ t ②s♥ ♣♣r♦s s t♦ ♥r t ♥t ♠①tr ♠♦s ♥② ♣rs♥t s♦♠rtr ♣r♦r♠♥ t ♠♦ st♦♥ ♥ ♣r♦st♦♥t①t♦ t♦② ①♠♣s strt t r♥t ♥♦t♦♥s ♥ t♦rt♠s tr♦ ts ♣tr ♥ ♦♥t♥♦s ss♥ t s t sst ♦♥
♦♦ st♦r② ② ♥ t♦ t♥s ♠st ♥ ♦t ♦ t
rrs ♦♥ ①♣r♥♦♥ t♥ ♦rt t
r ♦ t str♥ ♣♣r♦s
str♥ ♥
♦②s ♣rtt♦♥rs r ♦t♥ ♥ ♦♠♣① t sts tt ♥♦t ♥ts ♠♥sr♣t ② x = (x1, . . . ,xn) sr♥ n ♥s xi = (x1i , . . . , x
ei ) ② e
rs s ♦♠♣①t② s ♥r② ♥♦ ② t r ♥♠r ♦ ♥s♦r♠♥ t ♣rtt♦♥rs ♥r ♠ ♥♦r♠t♦♥s rtr♠♦r ts♦♠♣①t② ♥ ♥rs ② t sr♣t♦rs t♦ t ♥♠r ♦ rs ♦r t♦tr ♥tr ♦r ♥st♥ t♦r ♦r ♠① rs
str♥ s ♥r ♥sr t♦ ts ♣r♦♠ ♥rs♥② ♠rs tt ♦♠♣tr ♦♣♠♥t ♥ ts t♥q s♠♠r③s t t ② r♦♣♥
♣tr str ♥②ss stt ♦ t rt
t ♥s ♥t♦ g sss ♦r♥ t♦ ♦t ♦♦♥ ♣r♥♣s t ss ♦♠♦♥t② r♦♣♥ s♠r ♥s ♥t♦ t s♠ ss ♥ t ss s♣rt②t♦ ♥s rs♥ r♦♠ t♦ r♥t sss r str♦♥② r♥t ♥ t ①t ♥t♦♥ ♦ ss s s♣ t♦ t str♥ ♠t♦ st ② t♣rtt♦♥r t s ②s s ♦♥ ts t♦ ♣r♥♣s
♦r♥ t♦ ts ♣r♥♣s t str♥ ♠t♦s tr② t♦ tr♠♥ t t♥tt♦r z = (z1, . . . , zn) r t t♦r zi = (zi1, . . . , zig) ♥ts t ss ♠♠rs♣ ♦ t ♥ xi ② s♥ ♦♠♣t s♥t ♦♥ zik = 1 xi s t ♥t♦ ss k ♥ zik = 0 ♦trs ♦t tt sss t♦ ♥tr♣rt ♦r t s♣st ♦ t ♦♠♥ r t t ♦♠ r♦♠ ♥ str♥ ♠t♦ ♣r♦s ♥ ♥t s♠♠r② ♦ t t ♦♥② ts rst♥sss r ♠♥♥
ss ♠♠rs♣s ♦ t ♥s t♦ st♠t t t ♥♠r♦ sss s ♥r② ♥♥♦♥ s ♥ ♥t str♥ ♠t♦ ♣r♦s t♦♦st♦ ♣ t ♣rtt♦♥r ♦r t st♦♥ ♦ t ♥♠r ♦ sss str♥s tr ♠♥ ♦s t♦ st♠t t ♣rtt♦♥ t♦ ♣r♦ ♠♥♥ sss ♥t♦ st t♦♠t② t ♥♠r ♦ sss
♦r t r t sts n ♥ e r ♣rtt♦♥rs ♥ s♠t♥♦s② strt ♥s ♥ t rs ♦ ♣rtt♦♥s ♥ ts sr ♦♥ ♠♦♥t ♥s ♥ ♦♥ ♠♦♥ t rs s ♣♣r♦ s ♥♠ ♦str♥❬ ❪ t t s ♥♦t ♦♣ ♥ ts tss r ♦♥② st② t str♥♣r♦♠
♠t♦s ♣r♦r♠♥ t str ♥②ss r ♥t♦ t♦ r ♠st ♦♠tr ♠t♦s s ♦♥ s♦♠ st♥s t♥ ♥s ♥ t ♣r♦st ♠t♦s ♠♦③♥ t t ♥rt♦♥ ♥ ts st♦♥ ♦t ♣♣r♦sr t ♥ ♥r r♠♦r rtss t② r strt ♦♥ rt♦♥t♥♦s t st ♣rs♥t ♦ s♥ t ♦s t♦ s② s③ t rsts♦ s♣ stts ♦ t rt rt t♦ ♠♦r ♦♠♣① stt♦♥s t♦r ♥♠① t sts r ♥ tr ♥ t ♥tr♦t♦♥s ♦ Prt ♥ Prt
t t st ❬❪ s ♦♥ t ♣ ♠ss st st ♦♥t♥s t t♥ t♠ t♥ r♣t♦♥s ♥ t rt♦♥♦ t r♣t♦♥s ♦r t t ②sr ♥ ❨♦st♦♥ t♦♥ Pr❲②♦♠♥ ❯ s♣② ② r ♠ s t♦ ♣r♦ ♠♥♥ s♠♠r② ♦ t t st x = (x1, . . . ,xn) r ♥ xi ∈ R
2s ♥ ts ①♠♣ n = 272 ♥ e = 2
t t st t ♣rs♥tt♦♥
r ♦ t str♥ ♣♣r♦s
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0
5060
7080
90
eruptions
wai
ting
r t t st
♦♠tr ♣♣r♦s
♥rts
♥ ♦♠tr ♣♣r♦s r♦♣ t st ♦ t str♥ ♠t♦ss ♦♥ t st♥ ♠sr♠♥t t♥ ♥s ♥ ♦rr t♦ ss♥ t ♠♦sts♠r ♥s ♥t♦ t s♠ ss ② ①rt♥ t ss ♦♠♦♥t② ♦♥♣t ♥ ♥trt ♦t ♦ t ♦♦♥ t♦ ♥s rs♥ r♦♠t s♠ ss t♦ ♠♦r s♠r t♥ t♦ ♥s ♥ r♥t ss♠♠rs♣ s ♠ ♥ ①♣rss ② t ♦♦♥ ♠t♠t rt♦♥♦r st♥ D(., .) ♥ ♦r (i0, i1, i2, i3) s tt zi0 = zi1 ♥ zi2 6= zi3 :
D(xi0 ,xi1) ≤ D(xi2 ,xi3).
Pr ♣r♦♠ ♥ s rt♦♥ s t♦ sts ② t qr♣ts ts ♦t ♥r② s t♦ ♠♣t② st s♦t♦♥ ♥♠r ♦ ♦♥str♥ts rtr♠♦r t s ♥♦t r③ t♦ ♣r♦r♠ ♥ ①st ♣♣r♦ ♦♠♣ts t ♦t rtr♦♥ ♦r t ♣♦ss ♣rtt♦♥s ♥ ♦r s♠♣♦ s③ n = 40 tt str ♥ g = 3 sss t ♥♠r ♦ t ♣♦ss ♣rtt♦♥ss r♦② q t♦ 2.1018 s ♦♠♣tr ♣r♦r♠♥ 109 ♣rtt♦♥s ♣r s♦♥♥s 64000 ②rs t♦ t t ♣♦ssts
♦ rtr♦♥ ♥ ♣rt t s s t♦ ♦♣t♠③ ♦ rtr♦♥ rt♥t ss ♦♠♦♥t② r♥t rtr ♦t♥ s ♦♥ rst s ♥s♦ ♣r♦♣♦s s t r♥♥♥ ①♠♣ s rtr r s② ♦♣t♠③ ② ♥♦rt♠ ♦♥ ♥ ①st ♣♣r♦ s ♥trt
trtr ♦ ts st♦♥ rst② ♣rs♥t t tr ♠♦st ♦♠♠♦♥ rtr♦ ♥trst ♥ t rs r ♦♥t♥♦s ♦♥② t t ♠♥s
♣tr str ♥②ss stt ♦ t rt
♦rt♠ s t ♠♦st ss ♦♠tr ♣♣r♦ t♦ str ♥s sr ② ♦♥t♥♦s rs ❬r❪ ♥② ts ♦rt♠ s strt ♦♥ tt t st
t s ♥tr♦ t ♠tr① ♦ t ♦ s♠♣ ♦r♥ ♥♦t ② T
♥ ♥ s
T =1
n
n∑
i=1
(xi − x)(xi − x)′,
r x = 1n
∑ni=1 xi s t ♠♥ t♦r ♦ t ♦ s♠♣ s ♠tr①
♥ rtt♥ s s♠ ♦ t♦ ♠trs
T = W +B,
r t ♥trss ♦r♥ ♠tr① W ♥ r t ♥trss ♦r♥ ♠tr① B r ♥ ②
W =1
n
g∑
k=1
n∑
i=1
(xi − xk)(xi − xk)′ ♥ B =
1
n
g∑
k=1
♥k(xk − x)(xk − x)′,
xk = 1
♥k
∑ni=1 zikxi s t ♠♥ t♦r ♥ ♥k =
∑ni=1 zik t s③ ♦ ss
k s ♠trs rtr③ ♦t ♦♥strt♦♥s ♣r♥♣s ♦ t sss♥ sss r ♦♠♦♥♦s t♥ t st♥s t♥ t ♥s ss♥ t♦ ss ♥ ts ♥tr r s♠ s♦ W s s♠ sss r s♣rt t♥ t ♥trs ♦ t sss r ♠t② t♥② s♦ B s rs t ♣rtt♦♥r ♥ str t t t st ② ♦♣t♠③♥ r♥t rtr rt t♦ ts ♠trs ♠♦♥ t♠ t ♠♦st ssr t ♦♦♥ min tr(W ) min det(W ) ♦r max tr(BW−1) ❲rr t♦ t ♦♦ str ♥②ss ② rtt ♥ s ♥ t ❬❪ ♦r ♠♦r ts
t t st ♣t♠③ rtr ♦r ♦♥t♥♦s rs
♠♥s ♦rt♠
♥ ♦② ♣r♦♣♦s t ♠♥s ♦rt♠ r♦♥ ♥ t t♦ ♣s t ❬♦❪ ss♦t t♦ st♥ D(., .) ts ♦rt♠ ♠s t♠♥♠③♥ t ♦♦♥ ♥rt
I(z,θ;x) =n∑
i=1
g∑
k=1
zikD2(xi,µk),
r θ = (µ1, . . . ,µk) ♥ r µk s t ♥tr ♦ t ss k trt♥ r♦♠♥ ♥t ♦ t ss ♥trs t ♠♥s ♦rt♠ tr♥ts t♥ t♦st♣s t ss♥♠♥t ♦ ♥ t♦ t ss ♠♥♠③♥ t st♥ t♥♠ ♥ t ss ♥tr ♥ t ♦♠♣tt♦♥ ♦ t ss ♥trs
r ♦ t str♥ ♣♣r♦s
trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s ♦♦s ss ♠♠rs♣ z
[r] = argminz
I(z,θ[r];x)
z[r]ik =
1 k = argmink′
D2(xi,µ[r]k′ )
0 ♦trs
♥tr♦ st♠t♦♥ θ[r+1] = argminθ
I(z[r],θ;x)
µ[r+1]k =
1
♥[r]k
n∑
i=1
z[r]ik xi,
r ♥[r]k =
∑ni=1 z
[r]ik
♦rt♠ ♠♥s ♦rt♠
s ts ♦rt♠ ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ I(z,θ;x) t s ♠♥t♦r②t♦ ♣r♦r♠ r♥t ♥t③t♦♥s ♥ t♦ ♣ t ♦♣ (z,θ) ♠♥♠③♥ t♦t ♥rt
♠r ♠♥s ♦rt♠ ♥ ♦♣t♠③ rtr♦♥ t ♠♥s ♦rt♠ strs ♦♥t♥♦s t ② s♥ t ♥ st♥ t♥ t ♦♣t♠③st rtr♦♥ min tr(W )
①t♥s♦♥s ♦ t ♠♥s ♦rt♠ ♦♠ ♣♣r♦s tt♠♣t t♦ rt rs ♦ t ♠♥s ♦rt♠ ♦r ♥st♥ t ♠♥s ♦rt♠❬❱❪ ①t♥s t ss ♦♥ ② r♥♦♠③ s♥ t♥q ♠♣r♦♥ ts♣ ♥ t r② ♦ t ♠♥s
♦ ♠♥② sss st♦♥ ♦ t ♥♠r ♦ sss ♥ ♥♦t rt②♣r♦r♠ ② t ♥rt rtr♦♥ ♥ ♥ s♥ ts ttr s rs♥ tt ♥♠r ♦ sss g ♦r t ♦t rtr♦♥ rs ♣t ♥g ♥rss ♥ ♥ ts ♣t s r t sss r ♥♦ ♠♦r♦♠♦♥♦s t ss ♦r♣♣♥ ♥rss rst rtr♦♥ ♦♥ssts ♥st♥ t rst ♥♠r ♦ sss ♦ ts ♣t t t s r tt ts ♥♦ rtr♦♥ s ♥♦t r② r♦r♦s ♥ ♣rt t rtr♦♥ ♥ ♥♣ ♥s♦♠ ♣ts r ♦sr tr rtr r s ♦r ♥st♥ ❬r❪t t② r s ♦♥ rst ♣♣r♦
♣tr str ♥②ss stt ♦ t rt
❲ s t ♠♥s ♦rt♠ t♦ str t t t st ♦r♥t♦ r r♥ t ♦t♦♥ ♦ t ♥rt ♦r r♥t ♥♠rs ♦sss r♦♠ ♦♥ t♦ t ♥ st t♦ sss ♣rtt♦♥ ♥t ss ♥trs r s♣② ② r t t sts s♦ s♠♠r③ ② t♦ ♣r♦s ♦ r♣t♦♥s t r♣t♦♥s t s♦rtt♥ t♠ ♥ rt♦♥ ♥tr t (2.09, 54.75) r s♣② t rs ♥ t r♣t♦♥s t rr t♥ t♠ ♥ rt♦♥ ♥tr t(4.30, 80.28) r s♣② t r tr♥s
t t st ♠♥s ♦rt♠ ♣♣r♦
1 2 3 4 5 6 7 8number of classes
iner
tia0
1000
020
000
3000
040
000
5000
0
♥rt
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0
5060
7080
90
eruptions
wai
ting
ttr ♣♦t ♥ ss ♥trs
r t♣ts ♦ t t str ♥②ss ♣r♦r♠ ② ♠♥s ♦rt♠ sttr ♣♦t ♥ts t ♣rtt♦♥ ② t ♦♦rs ♥ t t♥ s②♠♦s t ss ♥trs r r♣rs♥t ② t ♦ s②♠♦s ♥ t ♦♦r ♦ tr ss
♠ts ♦ t ♦♠tr ♣♣r♦s
r♥t t♦rs ♣♦♥t ♦t t rs ♥r♥t t♦ t ♦♠tr ♣♣r♦ss ♦r ♥st♥ t ♦♦ t ♥②ss ♣tr ② ♦rt ❬♦❪ ♦st tr ♠♥ rs r ♣rs♥t r
♦ st♦♥ ♣r♦r♠ ② rst ♣♣r♦s ♥ t ♦♠tr ♣♣r♦s r ♣♦ss ♥sr t♦ t str♥ ♥ ♠♥② t♦rt♣r♦♠s ♠② rs ❬♦❪ ② ♥r② st t ♥♠r ♦ sss ② s♥ rst ♣♣r♦s t s♦♣ ♦ t rtr♦♥ s rtr♠♦r ♦t♦s ♦ t ♠tr ♥ t rtr♦♥ s r ♠♣♦rt♥t s♣ts ♦ ts ♠t♦s s♥ t② ♥♦ ♠♥② ♥ ss♠♣t♦♥s r ♥r② ♥♦r ② t♣rtt♦♥rs ♦r tr ♠♣t s r s t② ♥♦ r♥t ♣rtt♦♥s
r ♦ t str♥ ♣♣r♦s
①t♥s♦♥ ♦ t ♦♥s♦♥s t♦ t ♦ ♣♦♣t♦♥ t ♦♥s♦♥s ♦ str ♥②ss s ♦♥ s♠♣ t♦ ①t♥ t♦ t ♦ ♣♦♣t♦♥t s ♠♥t♦r② t♦ ♥rst♥ s♦ t♦ ♠♦③ t t ♥rt♦♥ ①t♥s♦♥♦ t ♦♥s♦♥s ♦t♥ ② ♦♠tr ♠t♦ r ♥♦t ♦ ♥ s st ♣r♦st r♠♦r s s♦ ♠♥t♦r②
♥ t ♠ss♥ s ♦♠tr ♣♣r♦s ♥♥♦t rt② ♠♥t sts t ♠ss♥ s ♥ t② t♦ ♣r♦r♠ ♥ rtrr② ♠♣tt♦♥ ♦r t② t♦ ♥♦r ♥s t ♠ss♥ s t ♣r♦st♣♣r♦s r t♦ r♦r♦s② ♠♥ s t
♥s t♥ ♦♠tr ♥ ♥rt ♣♣r♦s
♥② ♦♠tr ♣♣r♦s ♥ ♥tr♣rt s ♣r♦st ♦♥s r♥tr ♣r♦st ♥ ss♠♣t♦♥s s♦♠ ①♠♣s r ♥ ♥ ts tss ♥♦rr t♦ ♣r♦st t♦♦s ♥ t♦ r t ss♠♣t♦♥s ♠ ② t str♥♠t♦s ♦♣ ts tss ♥ ♣r♦st r♠♦r
♥rt ♣♣r♦s
♥rts
♥ ♠①tr ♠♦s r ♥tr t♦♦s t♦ str t t ② ♣♣r♦♥ tr strt♦♥s s tr ♣r♦st r♠♦r ①♣♥s t t ♥rt♦♥ ♥ ts ♦♥t①t t ♥♦t♦♥ ♦ ss ♦♠♦♥t② s ♥ ② t ♦♦♥ t ♥s ♦ t s♠ ss rs r♦♠ t s♠ ♣r♦t② strt♦♥ ts strt♦♥ s ♥r② ss♠ t♦ ♥♠♦ ts ss♠♣t♦♥ ♥ r① ♦r ♥st♥ t ♦♠♣♦♥♥t strt♦♥ ♥ ts ♠①tr ♦ ♣r♠trstrt♦♥s t♦ ♥rs t ♠♦ ①t② ❬+❪
t♥t r ♥ ss ♠♠rs♣ ss ♠♠rs♣ ♦ t ♥ i s qtt r♥♦♠ r ♥ g ♠♦ts ♥ ♥♦t ② Zi =(Zi1, . . . , Zig) ② s♥ s♥t ♦♥ s t ss ♠♠rs♣ ♦♦s ♠t♥♦♠ strt♦♥
Zi ∼ Mg(π1, . . . , πg),
r πk ♥♦ts t ♣r♦♣♦rt♦♥ ♦ ss k s♦ ♥tr♣rt s t ♣r♦t② ♣r♦r tt ♥ ♥ rss r♦♠ ss k ss ♣r♦♣♦rt♦♥ πk rs♣ts♦t ♦♦♥ ♦♥str♥ts 0 < πk ≤ 1 ♥
∑gk=1 πk = 1 ♦t tt t str♥
♥ s t♦ st♠t t ♦ t r③t♦♥ zi ♦ t t♥t r Zi
♦♥t♦♥② ♦♥ t ♦sr t xi
sr rs ss k s rtr③ ② t strt♦♥ ♦ t ertr♥♦♠ r X i = (X1
i , . . . , Xei ) ♥ ♦♥ t s♣ X ♦♥t♦♥② ♦♥ t
r③t♦♥ zi ♦ t r♥♦♠ r Zi s strt♦♥ s ♥♦t ② pk(xi)r k s s tt zik = 1 ♥
X i|Zi = zi ∼ pk:zik=1(xi).
♣tr str ♥②ss stt ♦ t rt
strt♦♥ ♦ ♦t ♦sr ♥ t♥t rs ② s♥ ts ♣r♦t②♦♠♣♦st♦♥ P (X i,Zi) = P (Zi)P (X i|Zi) t ♣r♦t② strt♦♥ ♥t♦♥♣ ♦ (xi, zi) ♥♦t ② t ♥r ♥♦tt♦♥ p(.) s ♥ s ♦♦s
p(xi, zi) =
g∏
k=1
(πkpk(xi))zik .
♥ ts ♠♦ s s t♦ str t s zi r ♦♥sr s ♠ss♥ ss ♦t♥ ♦t ♥t♦♥s ♦ t ♥t ♠①tr ♠♦ ♥ ts ♥rt♠♦ ② s♠♠♥ t ♣r♦s qt♦♥ ♦r t ♣♦ss s ♦ Zi
♥t♦♥ ♥t ♠①tr ♠♦ ♥t ♠①tr ♠♦ t g ♦♠♣♦♥♥ts ♥s t ♠r♥ strt♦♥ ♦ t r♥♦♠ r X i ts ♣ s rtt♥s
p(xi) =
g∑
k=1
πkpk(xi).
♥rt ♠♦ s♠♣♥ r♦♠ t ♠①tr ♠♦ ♥ ② s♣r♦r♠ ② t ♦♦♥ ♥rt ♠♦ ♥t♦ t♦ st♣s
t♣ t ss ♠♠rs♣ s♠♣♥ Zi ∼ Mg(π1, . . . , πg t♣ t ♦♥t♦♥ t s♠♣♥ X i|Zi = zi ∼ pk:zik=1(xi)
sst♦♥ r
③③② ♥ r ♣rtt♦♥ ❲♥ t t strt♦♥ p(xi) s ♥♦♥ t ♥t♦♥ ♦ Zi|X i = xi s strt♦rr
Zi|X i = xi ∼ Mg(ti1, . . . , tig),
r tik s t ♦♥t♦♥ ♣r♦t② tt xi s r♥ r♦♠ ♦♠♣♦♥♥t k s♥ ②
tik =P (Zik = 1,X i = xi)
P (X i = xi)=πkpk(xi)
p(xi).
❱t♦r ti = (ti1, . . . , tig) s♦ ♥s ③③② ♣rtt♦♥ ♥ s t♦ ♦♠♣tt rs ss♦t t♦ t r ♣rtt♦♥ zi
rr♦r rs ♥ sst♦♥ r r♦♠ ts ③③② ♣rtt♦♥ ♥ ♥ tsst♦♥ rr♦r e(.) ss♦t t♦ (zi, ti) ②
e(zi, ti) = 1−g∑
k=1
(tik)zik .
♠①♠♠ ♣♦str♦r r ♠♣ ♠♥♠③s t sst♦♥ rr♦r ② ss♥♥♥ ♥ ♥t♦ t ss ♥ t rst ♣r♦t② s t ♥s tsst♦♥ r r(.) s ♦♦s
∀xi ∈ X , r(xi) = k ∀k′ tik ≥ tik′ .
t♦♥ ♦ t rs ♦ t sst♦♥ rr♦r s rt ♥t ♦ t ♣r♦st ♠t♦s s♥ t ♦♠tr ♦♥s ♥♥♦t q♥t② t rr♦r rs ss♦tt♦ tr sst♦♥ r
♥rts ♦♥ ♥t ♠①tr ♠♦s
♥rts ♦♥ ♥t ♠①tr ♠♦s
♥ s ♠♦s ss♠ tt t ♦sr ♥s r ♥♣♥♥t②r♥ r♦♠ t s♠ strt♦♥ ❲ ♥♦ q② sr t s♠♣r♠tr♠①tr ♠♦s ♠ ss♠♣t♦♥s ♦♥ t ♦♠♣♦♥♥t strt♦♥s ♥ sr t ♣r♠tr ♠①tr ♠♦s ss♠ tt t ♦♠♣♦♥♥tstrt♦♥s r ♣r♠tr ♦♥s ♦t tt t ♠♦ sr♣t♦♥ ♣rs♥t rs s ♦♥ ♥t ♠①tr ♠♦s ② ♥ ♥ P ❬P❪ ♥ tstss ♦s ♦♥ t ♣r♠tr ♠①tr ♠♦s s♥ t② ♣r♠t ♥ sr♥tr♣rtt♦♥ ♦ t sss
♠♣r♠tr ♠①tr ♠♦s
♦♥str♥ts ♦♥ t ♦♠♣♦♥♥t strt♦♥s s♠♣r♠tr ♣♣r♦s ♦ ♥♦t ss♠ tt t ♦♠♣♦♥♥ts ♦♦ ♣r♠tr strt♦♥s ♦r ♦r rs♦♥s ♦ ♥tt② s♦♠ ♦♥str♥ts t♦ ♠♣♦s ♦r t ♦♠♣♦♥♥ts ♦r ♥st♥ strt♦♥s t♦ ♦♥ t♦ t ♠② ♦ ♥♠♦strt♦♥s ♦r s②♠♠tr strt♦♥s ❬❲❪
♥r♥ st♠t♦♥ ♦ t ♦♠♣♦♥♥t strt♦♥s ♥ ♣r♦r♠ ②♦rt♠s ♥s♣r r♦♠ t ♠ ♦rt♠ ❬❪ ss r♥ ♣♣r♦s❬❪ ♣ ♠①t♦♦s ❬❪ ♦s s t♦ str t t ② s♥ s♠♣r♠tr ♠①tr ♠♦
♦♥♥tt② rs s♠♣r♠tr ♠①tr ♠♦s r r② ①s♦ t② ♥ s② t t t strt♦♥ ♦r ts ①t② ♥♦s ♥♠♣♦rt♥t rs ♦ ♥♦♥♥tt② ♥ r r♥ ♦ t st♠t ♠♦rtr♠♦r t ss ♥tr♣rtt♦♥ ♥ t s♥ t ♦♠♣♦♥♥ts ♥ ♥♦t s♠♠r③ ② ♣r♠trs s ♥ t ♣r♠tr ♠①tr ♠♦s ♦♥ ts tss ♦♥② st② t ♣r♠tr ♠①tr ♠♦s ♦r t ♥r♣r♦♣rts r ♥♦ ♦♣
♣r♠tr ♠①tr ♠♦s
♥rts
♥ s ♠♦s ♠ t s♣♣♠♥tr② ss♠♣t♦♥ tt ♦♠♣♦♥♥t ♦♦s ♣r♠tr strt♦♥ s♦ pk(xi) = p(xi;αk) r αk r♦♣s t♣r♠trs ♦ ♦♠♣♦♥♥t k ♥s r s♦ r♥ ② ♣r♠tr strt♦♥ p(xi) = p(xi;θ) r θ = (π,α) ♥♦ts t ♦ ♣r♠tr rπ = (π1, . . . , πg) s t t♦r ♦ t ss ♣r♦♣♦rt♦♥s ♥ r α = (α1, . . . ,αg)r♦♣s t ♣r♠trs ♦ t ♦♠♣♦♥♥ts
♥t♦♥ ♥t ♣r♠tr ♠①tr ♠♦ ♣ ♦ t ♥t ♣r♠tr♠①tr ♠♦s t g ♦♠♣♦♥♥ts s ♥ ②
p(xi;θ) =
g∑
k=1
πkp(xi;αk).
♣tr str ♥②ss stt ♦ t rt
♥tr♣rtt♦♥ t ♣r♠trs s ♠♦s r ♠♦r ♠♥♥ t♥ ts♠♣r♠tr ♣♣r♦s s♥ ss ♥ s♠♠r③ ② ts ♣r♦♣♦rt♦♥ πk♥ t ♣r♠trs ♦ ts strt♦♥αk ♣r♦ts ♦ t ss ♠♠rs♣stik r s♦ ♣r♠tr③ ② θ s♦ t② r ♥♦ ♥♦t ② tik(θ) t
tik(θ) =πkp(xi;αk)
∑gk′=1 πk′p(xi;αk′)
.
♦♠♣♦♥♥ts tr ♦ t♥ r♥ ♥ ♥♠r s ♣♣r♦ s♥♦t r② rstrt s♥ ♠①tr ♦ ♣r♠tr strt♦♥s ♥ ♣♣r♦ ♥②strt♦♥ t ♥② ♣rs♦♥ t strt♦♥s ♦♠♣♦♥♥t strt♦♥s ♥♣♣r♦ strt♦♥ t s♠ s♣♣♦rt st ② ♥rs♥ t ♥♠r ♦♦♠♣♦♥♥ts ♥ t s♠♣ s③ s t ♦♦♥ ①♠♣ s ♠①tr ♦st♥r strt♦♥s ♥ ♠♦③ ② ♦♠♣① strt♦♥s ♦r t♠①tr s ♠♦r ♠♥♥ ♥ t ♥♠r ♦ sss st②s s♠ rtr♠♦rs t ♠①tr ♠♦ s st♠t ♦♥ ♥t s♠♣ t ♣rtt♦♥r srs t♠♦ ♣r♦r♠♥ t st tr ♦ t♥ ts s t t tr ♠♦ ♥ts r♥ s ② t tt♦♥s ♦ t s♠♣♥ r♦r t s ♠♣♦rt♥ttt t ♦♠♣♦♥♥ts ♦♦ st♥r strt♦♥s r ♣t t♦ t t
①♠♣ ♥st② st♠t♦♥ ♥ Pr③♥♦s♥tt st♠t♦r t x t♦ t s♠♣ ♦ s③ n r ♥ xi ∈ R s ♥♣♥♥t② r♥ ② tstrt♦♥ rtr③ ② ts ♣ f(x) Pr③♥♦s♥tt st♠t♦r ❬Pr❪s ♥ s
p(y;θ) =n∑
i=1
1
nhK
(
y − xih
)
.
♦t tt ♥s ♥t ♠①tr ♠♦ t n ♦♠♣♦♥♥ts r ♣r♦♣♦rt♦♥ s q t♦ 1/(nh) ♥ r t strt♦♥ ♦ ♦♠♣♦♥♥t i s ♣r♠tr③② ♦♥ xi ♦r i = 1, . . . , n ❯♥r s♦♠ rrt② ♦♥t♦♥s ♦♥ t ♥t♦♥ K(.)♥ ♥r s♦♠ rt♦♥s t♥ t s♠♣ s③ n ♥ t ♥t h t ♣p(y;θ) ♦♥rs t♦ t tr ♣ f(x) s ♦r ♥st♥ ❬❩❪ ♦r ♠♦r tsr strts t Pr③♥♦s♥tt st♠t♦r ♣♣r♦♥ ♣ ② s♥t ♥♦r♠ r♥ ♦r t ss♥ ♦♥
K♥♦r♠(y) =
1/2 |y| < 10 ♦trs
♥ Kss♥(y) =1√2πe−
12y2 .
①tr ♥ ♥ ♦ rs ♣r♠tr ♠①tr ♠♦s ♥ str♥s ② ♣♣r♦♥ t strt♦♥ ♦ t rs ♥ tr ♥t s♣♦s② t strt♦♥s ♦ t ♦♠♣♦♥♥ts t♦ rs♣t t ♥tr ♦ trs s t ♠①tr ♠♦s r s t♦ ♥②③ r♥t ♥s ♦ t sts♦r ♥st♥ t ♠①tr ♦ P♦ss♦♥ strt♦♥s ❬❪ ♥ str ♥tr t t ♠①trs ♦ t♥t strt♦♥s ❬P❪ ♥ str t ♦♥t♥♦s ♦♥st t ♠①tr ♠♦s r s♦ s t♦ str ♥t♦rs ❬ ❲❪ r♥t ❬❪ ♦r ♥t♦♥ t ❬P❪ s ♦ t♦r t s ♦♣ ♥Prt Prt s ♦s ♦♥ t str♥ ♦ ♠① t ♦r t ♠♦st♦♠♠♦♥ ♠①tr ♠♦ s t ss♥ ♦♥ tt t ♥♦
♥rts ♦♥ ♥t ♠①tr ♠♦s
0 5 10 15
0.00
0.05
0.10
0.15
0.20
0.25
0.30
n = 3
0 5 10 15
0.00
0.05
0.10
0.15
0.20
0.25
0.30
n = 25
0 5 10 15
0.00
0.05
0.10
0.15
0.20
0.25
0.30
n = 100
0 5 10 15
0.00
0.05
0.10
0.15
0.20
0.25
0.30
n = 1000
r ♣ ♦ t tr strt♦♥ ♦tt r ♥ ts st♠ts♦t♥ ② t ♥♦r♠ r♥ t♥ r ♥ ② t ss♥ r♥ ♦r r r t ♥t s h = lnn
ss♥ ♠①tr ♠♦
♥ ss♥ ♠①tr ♠♦ s ♥tr♦ s♠t♥♦s② t♦ t♣r♠tr ♠①tr ♠♦ ♥ ♦rr t♦ str t Prs♦♥s r t st ❬P❪t s ♣♦r t♦♦ t♦ str ♦♥t♥♦s t ♦s sss s t♦ t♦ ♠♥rs♦♥s ♥ t ♦♥ ♥ ts ♣t ♥t♦♥ ♦ ss s ♥ ♦r♥ tt ♥tr ♥t♦♥ ♦ ss ♥ t ♦tr ♥ ts ♦♠♣tt♦♥ trtt②♣r♠ts ♥ s② ♥r♥ ss♥ ♠①tr ♠♦ ss♠s tt r♥♦♠r X i|Zik = 1 s ♥ ert ss♥ r ♦s t ♠♥ t♦r s♥♦t ② µk ♥ ♦s t ♦r♥ ♠tr① s ♥♦t ② Σk s♦
X i|Zik = 1 ∼ Ne(µk,Σk).
s ♦t♥ t ♦♦♥ ♥t♦♥ ♦ t ss♥ ♠①tr ♠♦
♥t♦♥ ss♥ ♠①tr ♠♦ t xi ∈ Re t ♦♥t♥♦s r
rs♥ r♦♠ ss♥ ♠①tr ♠♦ t g ♦♠♣♦♥♥ts ts ♣ s rtt♥ s
♣tr str ♥②ss stt ♦ t rt
♦♦s
p(xi;θ) =
g∑
k=1
πkp(xi;αk) t p(xi;αk) = φe(xi;µk,Σk),
r φe(xi;µk,Σk) =1
(2π)e/2|Σk|1/2exp
(
−12(xi − µk)
′Σ
−1k (xi − µk)
)
♥ rαk =
(µk,Σk)
ss ♥tr♣rtt♦♥ ss♥ ♠①tr ♠♦ ♣r♦s s♠♠r② ♦ ss tr♦♦t ts ♥tr ♣♦st♦♥ µk ♥ ts s♣rs♦♥ ♠tr① Σk rt♥ ts♣♥♥s t♥ t ♣rs ♦ rs
Prs♠♦♥♦s ♠♦s ❲♥ t s♠♣s r s♠ t ♥♦r♠t♦♥ ♦t t♥trss ♣♥♥s t♥ rs s ♥♦t ♣rs♥t ♥ t t ♥ ♥rt sr♥ tr ♦ ♠② ttr ♦♥str♥ts ♦♥ t ♣r♠tr s♣ r rst♥ ♠♦s r ♣rs♠♦♥♦s ♠♦s ♦r ♥st♥ s ♦♥t s♣tr ♦♠♣♦st♦♥ ♦ Σk ♣r♦♣♦s ♥ ❬❪ ♦rt♥ ♣rs♠♦♥♦s ♠♦sr t ❬❪ s ts ♠♦s r s♥st t♦ t ♥t ♦ ♠sr♠♥t ♦ trs ♥ ♠② ♦ ss♥ ♠①tr ♠♦ ♥♠ rt s ♣r♦♣♦s ② r♥ ♥ ♦r♠ ❬❪ ♥② ♥♦t tt t s♣tr ♦♠♣♦st♦♥♦ Σk ♥ s t♦ str ♠♥s♦♥ t ❬❪
♦trs ♥② s♦trs ♣r♦r♠♥ t st♠t♦♥ ♦ t ss♥ ♠①tr♠♦s r r ♠♣t ♥ t s♦♥ ♦ ts ♠♦s s s♠♦♥ t♠ ♦♥ ♥ t t tr ♦♦rs st ❬❪ ①♠♦ ❬+❪♥ ①t♦♦ ❬❨❪
s ♦t rs ♦ t t st r ♦♥t♥♦s st♠t ♦♠♣♦♥♥t ss♥ ♠①tr ♠♦ st♦r♠s ♦ ♦t rs r s♣② ♥ r ♥ t st♠t ♠r♥ ♣ ♦ ♦t ♦♠♣♦♥♥ts r s♣r♠♣♦s ♠♦ s♠♠r③s t t st s ♦♦s
♠♦rt② ss π1 = 0.64 s t ss ♦ t str♦♥ r♣t♦♥ss♥ µk = (4.29, 80.00)
♠♥♦rt② ss π2 = 0.36 s t ss ♦ t r♣t♦♥ss♥ µk = (2.04, 54.51)
ss ♦ t str♦♥ r♣t♦♥s s ♠♦r s♣rs t♥ t ss ♦ t r♣t♦♥s s♥ t r♥ ♦ ♦t rs r rr rs♣t②(0.15, 40.90) ♥ (0.10, 57.68) ♥ t rs r ♣♦st② ♦rrt ♥ ♦t sss tr ♣♥♥② str♥t s rr ♥ t ♠♥♦rt②ss t♥ ♥ t ♠♦rt② ♦♥ ♥ t ♦♥t ♦ ♦rrt♦♥ s qt♦ 0.50 ♥ t ss ♦ t r♣t♦♥s t s q t♦ 0.23 ♥ tss ♦ t str♦♥ r♣t♦♥s
t t st ss♥ ♠①tr ♠♦ str♥
♥rts ♦♥ ♥t ♠①tr ♠♦s
Den
sity
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0
0.0
0.2
0.4
0.6
0.8
r♣t♦♥s
Den
sity
40 50 60 70 80 90
0.00
0.01
0.02
0.03
0.04
0.05
❲t♥
r st♦r♠s ♥ ♠r♥ ♥sts ♦ t ♦♠♣♦♥♥t ♠①tr ♠♦♦r t t st
❲ r♠r tt t s♠♠r② ♣r♦ ② t ss♥ ♠①tr ♠♦ s♠♦r ♣rs t♥ t s♠r② ♦t♥ ② t ♠♥s ♦rt♠ ♥ t ♠♥s ♦rt♠ ♦s ♥♦t ♦♥sr t ss ♣r♦♣♦rt♦♥s s♦ t♠♣t② ss♠s tt ♦t ♥s ♦ r♣t♦♥s r q♣r♦ rtr♠♦r t ss♥ ♠①tr ♠♦ ♣r♦s ♥ ♥②ss ♦ t ♥trss♣♥♥s s s♣② ② r rs t sttr ♣♦t ♦ t♣rtt♦♥ ♥ t ♣ss ♦ q♣r♦t②
t t st ♦♠♣rs♦♥ t♥ ♦t str♥ rsts
①tr ♠♦ t ♦♥t♦♥ ♥♣♥♥ ss♠♣
t♦♥
♥ ♦♥t♦♥ ♥♣♥♥ ♠♦ ♠ s♦ ♥♦♥ s ♥ ②s♦r t♥t ss ♠♦ s ♠①tr ♠♦ ss♠♥ t ♦♥t♦♥ ♥♣♥♥t♥ rs ♦ t ♦♥t♦♥ ♣r♦t② ♦ t ert r♥♦♠ rX i = (X1
i , . . . , Xei ) s rtt♥ s
P (X i|Zik = 1) =d∏
j=1
P (Xji |Zik = 1).
♦s② ts ss♠♣t♦♥ s r t♥ t ♦ ♥♣♥♥ ss♠♣t♦♥♥ t ♣♥♥② t♥ t rs s ♠♦③ ② t strtr ♥ sss♦ t strt♦♥
♥t♦♥ ♠ ♠♠♦ s ♠①tr ♠♦ ss♠♥ t ♦♥t♦♥♥♣♥♥ t♥ t rs s t ♣ ♦ t ♥ xi rs♥ r♦♠
♣tr str ♥②ss stt ♦ t rt
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0
5060
7080
90
eruptions
wai
ting
r t♣ts ♦ t t str ♥②ss ♣r♦r♠ ② ♦♠♣♦♥♥tss♥ ♠①tr ♠♦ sttr ♣♦t ♥ts t ♣rtt♦♥ ② t ♦♦rs ♥t t♥ s②♠♦s t ♥trss ♣♥♥s r ♣t ② t ♣ss ♦q♣r♦t② ♦ t ss♥ ♦♠♣♦♥♥ts t ♥s ♦♥♥ t♦ t ss♦ t str♦♥ r♣t♦♥s r s♣② t r tr♥s ♥ t♦s ♦♥♥ t♦ tss ♦ t r♣t♦♥s r s♣② t rs
t ♠ ♠♦ s rtt♥ s
p(xi;θ) =
g∑
k=1
πk
d∏
j=1
p(xji ;αkj),
r αkj ♥♦ts t ♠r♥ ♣r♠trs rt t♦ r j ♦r ♦♠♣♦♥♥t k
①♠♣ ♦rrt t r♥ ② ♠ r s♣②s s♠♣ r♥② t ♠ ♠♦ t ♦r rt ss♥ ♦♠♣♦♥♥ts ♥ ts ①♠♣ t sstrt♦rr tt ♦t rs r ♥♦t ♥♣♥♥t
♥♥ rsts ♥ ♣rt ♠ ♠♦ ♦t♥s ♦♦ rsts ♥ ♣rts♥ t rqrs ♣r♠trs s sss ② ♥ ♥ ❨ ❬❨❪s t ♥ r③ ♦♦ tr ♦ t♥ t s ♥ t r♥ ts s♣rst②s rt ♥t ♦r t s♠ t sts s♥ t ♥♦r♠t♦♥ ♦ t ♥trss♣♥♥② s ♥r② ♥♦t ♣rs♥t sss ♦ t ♠ ♠♦ s s♦ ①♣♥② ts ♠♥♥ s♣t ♥ ♥ t ♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥tsr ss ♦r ♥st♥ ♥ t② ♦♥ t♦ t ①♣♦♥♥t ♠② ss♥ s♠♠r③ ② t ♣r♠trs ♦ ts ♠r♥s
s ♦ t ♥trss ♦rrt t ❲♥ t ♦♥t♦♥ ♥♣♥♥ss♠♣t♦♥ s ♦t t ♠ ♠♦ srs r♦♠ sr ss ♥ s s t ss ♥♠r s ♥♦♥ t♥ t ♣rtt♦♥ ♦♠s s t ss♥♠r s ♥♥♦♥ t♥ t ♠ ♠♦ ♦rst♠ts t t♦ ttr t t t s♦r ♥st♥ t ♣♣t♦♥ ♣rs♥t ♥ ❬❱❪
♥rts ♦♥ ♥t ♠①tr ♠♦s
0 2 4 6 8 10
02
46
8
X1
X2
Zi1 = 1Zi2 = 1Zi3 = 1Zi4 = 1
r rt s♠♣ rs♥ r♦♠ t ♠ ♠♦ t ♦ ♣♥♥②t♥ rs p(xi;θ) =
∑4k=1
14N2(2k, I)
①♠♣ s ♣rtt♦♥ t t ♦♠♣♦♥♥ts ♦♠♦sst ss♥♠①tr ♠♦ ♦s t ♣ ♦ xi ∈ R
2 s ♥♦t ② f(xi) = 13φ2(xi;µ1,Σ) +
23φ2(xi;µ2,Σ) r Σ =
(
1 ρρ 1
)
♥ ρ 6= 0 ♦♣t♠ sst♦♥ rs
♠♥♠③♥ t ②s rr♦r s
r②s(xi) : zik =
1 (xi − µk)′Σ−1(xi − µk) < (xi − µℓ)
′Σ−1(xi − µℓ) t ℓ = 2− k0 ♦trs
t t ♠ ♠♦ ♥ t s♠ ♦♥♠♥s♦♥ ♠r♥ strt♦♥s t♥ t♠♦ ♥ ② f(xi) ♣ ♦ ts ♠ ♠♦ s rtt♥ s
p(xi;θ) =1
3φ2(xi;µ1,Γ) +
2
3φ2(xi;µ2,Γ) t Γ =
(
1 00 1
)
.
sst♦♥ rs ss♦t t♦ ts ♠♦ s
r♠(xi) : zik =
1 (xi − µk)′(xi − µk) < (xi − µℓ)
′(xi − µℓ) t ℓ = 2− k0 ♦trs
s t ♣rtt♦♥ st♠t ② t ♠ ♠♦ s s s♥ t st Ω r♦♣st ♥s r t sst♦♥ rs sr s ♥♦t ♠sr qs t♦ ③r♦
Ω = xi ∈ R2 : r②s(xi) 6= r♠(xi).
♥tt② ♦ t ♠①tr ♠♦s
ss♥t ♦♥t♦♥ sss ♣r♦ ② str ♥②ss r ♥tr♣rttr♦♦t t ♣r♠trs ♦ t ♦♠♣♦♥♥ts t s s♦ r tt t ♣r♠trsr ♥q ♦r ① strt♦♥ s t♦ ♠♦s ♥ t s♠ strt♦♥♠st t s♠ ♣r♠trs s♦ rr t♦ t ♥tt② ♦ t ♠♦
♣tr str ♥②ss stt ♦ t rt
♥t♦♥ ♥tt② t t♦ ♠①tr ♠♦s ♥ t s♠ ♥tr♦r t ♦♠♣♦♥♥ts ♥ rs♣t② ♣r♠tr③ ② θ ♥ θ′ t♥ t ♠♦s ♥t
∀xi ∈ X p(xi;θ) = p(xi;θ′) ⇔ θ = θ′.
Pr♦♠ t♦ t r♥ ♦ t ♦♠♣♦♥♥ts ♦s② t ♠①tr♠♦s r ♥♦t strt② ♥t t ♦♥② ♣ t♦ s♣♣♥ s♥ t sss♥ r s strt ② t ♦♦♥ ①♠♣ ♦r ts s ♦ ♥♦♥♥tt② s ♥♦t sr r s♥ t ♥tr♣rtt♦♥ ♦ t ♣rtt♦♥ st②s♥t
①♠♣ ♥ ♦ t ♦♠♣♦♥♥ts t t ♦♠♣♦♥♥t ♠①tr ♠♦s ♣r♠tr③ ② θ = (π1, π2,α1,α2) ♥ θ′ = (π2, π1,α2,α1) t♥ ♦t ♣r♠tr sts ♥ t s♠ strt♦♥
∀xi ∈ X , p(xi;θ) = π1p(xi;α1) + π2p(xi;α2)
= π2p(xi;α2) + π1p(xi;α1)
= p(xi;θ′).
♥ ♦ t ♦♠♣♦♥♥ts ♥ ♥r♥ ♦t tt ts ♦♥t♦♥ ♦ ♥♦♥♥tt② ♦s ♥♦t str t ♠ ♦rt♠ s ♥①t st♦♥ t ♥ ②s♥r♠♦r t ♥ ♥♦ t st♥ ♣♥♦♠♥♦♥ ❬t❪
❲② ♥t ♠①tr ♠♦s ♥ ♦rr t♦ ♦ t ♣r♦♠ t♦t ♦♠♣♦♥♥t r♥ t ♥♦t♦♥ ♦ ♥tt② s ♥tr♦ ♦r t♠①tr ♠♦s ❬❪
♥t♦♥ ❲ ♥tt② ♠①tr ♠♦ ♥ p(xi;θ) s ♣s ② ♥t ♥
∀xi ∈ X p(xi;θ) = p(xi;θ′) ⇔ θ ♥ θ′ r q♥t
♥② ♠①tr ♠♦s r ② ♥t ♠♦♥ t♠ ♦♥ ♥ t t ♥t♠①trs ♦ ss♥ strt♦♥s t ♥t ♠①trs ♦ ♠♠ strt♦♥s ♥t ♥t ♠①trs ♦ P♦ss♦♥ strt♦♥s r ♠♥ rts st② t ♥tt② ♦ t ♠①tr ♠♦s ❬ ❨❪ ♥ ♦rr t♦ ♥ ♦ trrs♦♥♥ ♣rs♥t t t♦r♠ ♦ r ❬❪ ♠♦♥strts t ♥tt② ♦ s♦♠ ♥rt ♠①tr ♠♦s ♦r ♥st♥ t ♥rtss♥ ♠①tr ♠♦
♦r♠ ♦♥t♦♥s ♦ ♥tt② ❬❪ t P = P ♠②♦ ♦♥♠♥s♦♥ ♠t strt♦♥ ♥t♦♥s t tr♥s♦r♠s ψ(t) ♥♦r t ∈ Sψ t ♦♠♥ ♦ ♥t♦♥ ♦ ψ s tt t ♠♣♣♥ M : P 7→ ψs ♥r ♥ ♦♥t♦♦♥ ♣♣♦s tt tr ①sts t♦t ♦rr♥ () ♦ P stt F1 ≺ F2 ♠♣s Sψ1 ⊆ Sψ2 t ①st♥ ♦ s♦♠ t1 ∈ Sψ1 t1 ♥♥♣♥♥t ♦ ψ2 s tt lim
t→t1ψ2(t)/ψ1(t) = 0 ♥ t ss ♦ ♥t
♠①trs ♦ P s ② ♥t
Pr♠tr st♠t♦♥
Pr♦♣♦st♦♥ ♥tt② ♦ t ♥rt ss♥ ♠①tr ♠♦ ❬❪ ss ♦ ♥t ♠①trs ♦ ♥rt ss♥ strt♦♥s s ② ♥t
Pr♦♦ t Φ1(.;µ, σ2) ♥♦t t ss♥ ♠t strt♦♥ ♥t♦♥ t
♠♥ µ ♥ r♥ σ2 > 0 ts tr ♣ tr♥s♦r♠ s ♥ ② ψ(t) =exp(σ2t2/2−µt) rr t ♠② ①♦r♣② ② Φ1(xi;µ1, σ
21) ≺ Φ1(xi;µ2, σ
22)
σ1 > σ2 ♦r σ1 = σ2 t µ1 < µ2 ♥ ♦r♠ ♣♣s t Sψ =(−∞,∞) ♥ t1 = +∞
♦r s♦♠ ♠①tr ♠♦s r ♥♦t ♥t t r ♦ ♥trst s♥ t②♣r♦ ♠♥♥ rsts ♥ ♣rt ♥ s♥ tr ♣r♠trs s♠ ♥t
♥r ♥tt② s t ♥tt② ♦♥t♦♥ ♦ t♦♦ str♥♥t ss rstrt ♦♥t♦♥ ♥♠ ♥r ♥tt② s ♥tr♦
♥t♦♥ ♥r ♥tt② ♠♦ s ♥r② ♥t ♥t ♣r♠tr s♣ r t ♠♦ s ♥♦t ♥t ♣ t♦ t ♦♠♣♦♥♥t r♥ s ♠sr q t♦ ③r♦
s ♦♥ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♠♥ ts ♥ ♦s ❬❪ ♥ s♥t ♦♥t♦♥ ♦ t ♥r ♥tt② ♦rt ♠①tr ♦ ♠t♥♦♠ strt♦♥s s ♠♦ s st ♥ Prt rts ♦♥ t ♣r♦♦ ♦ ts ♥r ♥tt② r ♥ ♦r s♦♠ ♠①tr♠♦s r ♥♦t ♥r② ♥t s♦ tr ♣r♠trs ♥♥♦t ♥tr♣rt sstrt ② t ♦♦♥ ①♠♣
①♠♣ ♦♥♥r ♥tt② ♦ t ♠①tr ♦ ♥♦r♠ strt♦♥st xi ∈ [a1; b2] r♥ ② t ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ♥♦r♠ strt♦♥s♦s t ♣ s
p(xi;θ) = πU [a1, b1] + (1− π)U [a2, b2],
r θ = (π, a1, b1, a2, b2) U [., .] ♥♦ts t ♣ ♦ ♥♦r♠ strt♦♥ ♥r a1 < b1 a2 < b2 a1 < a2 b1 < b2 ♥ a2 < b1 s ♠♦ s q♥t t♦t ♦♦♥ tr♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ♥♦r♠ strt♦♥s ♦s t ♣ s
p(xi;θ′) = ε1U [a1, a2] + ε2U [a2, b1] + (1− ε1 − ε2)U [b1, b2],
r θ′ = (ε1, ε2, a1, a2, b1, b2) ε1 = π a2−a1b1−a1
♥ ε2 = π b1−a2b1−a1
+ (1− π) b1−a2b2−a2
s
t strt♦♥ ♦ xi ♥ ♠♦③ t t♦ r♥t ♣r♠tr③t♦♥s θ ♥ θ′
Pr♠tr st♠t♦♥
trtr ♦ ts st♦♥ str♥ ♦ t st ② ♥t ♠①tr ♠♦rqrs t st♠t♦♥ ♦ t ♠♦ ♣r♠trs ♥ ts st♦♥ ♣rs♥t tt♦ ♠♦st ♣♦♣r st♠ts ♥ t ♠①tr ♠♦ ♦♥t①t t ♠①♠♠ ♦♦st♠t rtr ♥♦t ② ♠ ♥ t ♠①♠♠ ♣♦str♦r st♠t rtr♥♦t ② ♠♣ ♦r ♦t st♠ts ♣rs♥t t st♠t♦♥ ♦rt♠s ♥tr trs s♣ t♦ t ♠①tr ♠♦s
♣tr str ♥②ss stt ♦ t rt
♥♥♥ ①♠♣ ♥ ts st♦♥ ♥t♦♥s ♥ ♦rt♠s r ♥ ♥ ♥r③ s ♥ t② r strt ② t ♦♦♥ r♥♥♥ ①♠♣ ①trt r♦♠t rt ②s♥ ♦♥ ♥ ♥r♥ ♦♥ ①trs ♦ strt♦♥s ♦ r♥ ♥rs♥ ♥ P ♦rt ❬❪
t xi ∈ R ♥ t t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦♦s t ♣ s rtt♥ s
p(xi;θ) =2∑
k=1
πkφ1(xi;µk, σ2k),
r θ = (π1, µ1, σ21, π2, µ2, σ
22) ♥ r φ1(.;µk, σ
2k) ♥♦ts t ♣ ♦
t ♥rt ss♥ r N1(µk, σ2k)
♥♥♥ ①♠♣ ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr
①♠♠ ♦♦ st♠t♦♥
♥ ts st♦♥ ♥ t ♦♦ ♥t♦♥ ♥ t st♠t ♦ t ♠①♠♠♦♦ ♦s sr t ♠♥ ♣r♦♣rts ♥ ♥tr♦ t ♥♦t♦♥♦ ♦♠♣tt ♥ t ♦♦ ♥t♦♥ ss♦t t♦ t ♥♠ ♦♠♣tt♦♦ ♥t♦♥
①♠♠ ♦♦ st♠t
♥ ♦♦ ♥t♦♥ ♦s t ♦ ♥♦r♠t♦♥ ♦♥t♥ ♥ tt st ♦r s t s ♠♦r ♦♠♦rt t♦ ♦r t t ♦rt♠ ♦ ts♥t♦♥ ♣rs♥t t ♥t♦♥s ♦ ♦t ♥t♦♥s
♥t♦♥ ♦♦ ♥t♦♥ ♦r ♥ s♠♣ x ts ♥t♦♥ ♦♠♣t t t ♣♦♥t θ s ♥ ② p(x;θ) =
∏ni=1 p(xi;θ)
♥t♦♥ ♦♦♦ ♥t♦♥ ♦♦♦ ♥t♦♥ ♦♠♣tr♦♠ t s♠♣ x ♥ t ♦♥ t ♣♦♥t θ s ♥ ②
L(θ;x) =n∑
i=1
ln p(xi;θ).
♦r ♥ s♠♣ x ♦♠♣♦s t n ♥s xi ∈ R t♥ t ♦♦ ♥ t ♦♦♦ ♥t♦♥s r ♥ s
p(x;θ) =n∏
i=1
2∑
k=1
πkφ1(xi;µk, σ2k) ♥ L(θ;x) =
n∑
i=1
ln2∑
k=1
πkφ1(xi;µk, σ2k).
♥♥♥ ①♠♣ ♦♦ ♥ ♦♦♦ ♥t♦♥s
Pr♠tr st♠t♦♥
♥ rq♥tst r♠♦r ♥t t♦ ♥r ♦r♥ t♦ t ♥♦r♠t♦♥ ♥② t t ♦ ♥tr ♣♣r♦ s t♦ sr t ♠①♠♠ ♦♦ st♠t♠ ♥♦t ② θ
♥t♦♥ ♠ ♠①♠♠ ♦♦ st♠t s ♥ ②
θ = argmaxθ
L(θ;x).
s t ♦♦♦ ♥t♦♥ ♦r s♠r② t ♦♦ ♥t♦♥ s tr♥t ts ♦♥t♦♥ s ♥r② r ♦r t ♠①tr ♠♦s t♠ s ♦t♥ ② s♦♥ t qt♦♥s ♥♥ ♦ t r♥t ♥ ♥♦♥ ♣♦st ♥t ss♥ ♠tr①
∇L(θ;x) = 0.
Pr♦♣rts ♦ t ♠ ♠ s ♣♦♣r st♠t s♥ t s ♥r rstrt ♦♥t♦♥s ♦♦ ♣r♦♣rts
t s ♥q t ♣r♦t② t♥♥ t♦ s t s♠♣ s③ r♦s t♦ ♥♥t② t s ♦♥sst♥t t s s②♠♣t♦t② ♥s t s s②♠♣t♦t② ss♥ t s②♠♣t♦t② ♠♥♠③s t r r♥
ts ♦♥ t ♦♥t♦♥s ♥♦♥ ts ♣r♦♣rts r ♥ ♥ ♦r② ♦ P♦♥tst♠t♦♥ ♣tr ② ♠♥♥ ♥ s ❬❪ s t ♠ s♦♦ ♣r♦♣rts t s ♥tr t♦ st② ts ①st♥ ♥ t ①sts t ♠t♦s♣r♦r♠♥ ts st♠t♦♥
♥r② ①st♥ ♥ t ♥q♥ss ♦ t ♠ s ♥♦t r♥t ♦rt ♠①tr ♠♦s ♥ t ♦♦♦ ♥t♦♥ ♥ ♥♦t ♣♣r ♦♥s t r♥♥♥ ①♠♣ ♥ s s t ♦♦ ♥t♦♥ ♥ t♥ t♦ t♥♥t② s stt♦♥ ♥♠ ♥r② ❬❪ ♥♦s ♥♦♥sst♥t st♠t♦rs♥ s s t st♠t♦r r②♥ ♥ ♥♦♥ ♥t ♦♦♦ ss♦ sr
❲ ♥t t♦ t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ ♦♥ ts♠♣ x ss♠ tt µ1 = x1 ♦sr ♠♦ ♥r② s♥
limσ21 → 0σ22 > 0
L(θ;x) = ∞.
♥♥♥ ①♠♣ ♥r②
♥t♦♥ ♠ ♥ ♥♦♥ ♦♦♦ ❲♥ t ♦♦♦♥t♦♥ s ♥♦t ♣♣r ♦♥ t ♠ s ♥ s
θ = argmaxθ∈θ:L(θ;x)<+∞ ♥ ∇L(θ;x)=0
L(θ;x).
♣tr str ♥②ss stt ♦ t rt
♦t tt t ♦♦♦ ♥t♦♥ s ♥r② sr ♦ ♦♣t♠ ♥rs t t② ♥ ♥♥ t ♠ s ♣♥♦♠♥♦♥ s ♥♦ strt ♦♥t r♥♥♥ ①♠♣
❲ ♥rt s♠♣ ♦ s③ r♦♠ t ♦♠♣♦♥♥t ♥rt ♠①tr♠♦ ♦s t ♣r♠trs r ♥ s ♦♦s
π1 = 1/3, π2 = 2/3, µ1 = −1, µ2 = 3.5 ♥ σ21 = σ2
2 = 1.
❲ ss♠ tt t ♣r♦♣♦rt♦♥s ♥ t r♥s r ♥♦♥ ♥ ♥tt♦ st♠t t ♠♥s ② ♠①♠♠ ♦♦ r s♣②s t ♦♦♦ ♥t♦♥ ♦r♥ t♦ t s ♦♥ ♦t ♣r♠trs ♥ ♥♦sr tt ts ♥t♦♥ s t♦ ♦♣t♠ ♦ ♦♥ s ♦t r♦♥(−1, 3.5) t ♦ ♦♥ s ♦t r♦♥ (3.5,−1)
♥♥♥ ①♠♣ ♦♦♦ ♦♣t♠ t ♥♥♦♥ ♠♥s
−4 −2 0 2 4 6
−4
−2
02
4
µ1
µ 2
r ♦♦♦ s ♦r t s♠♣ ♦ s③ ♦r♥ t♦ t s♦ (µ1, µ2)
♦ ①♣t s♦t♦♥ ♦r t ♠①tr ♠♦s t sr ♦ t ♠ ♥♦st♦ s♦ qt♦♥s ♥ ♥♦ ♥②t s♦t♦♥ rt ♦♠♣tt♦♥ ♦ t♠ s ♥♦t s② s ♦ t s♣ ♦r♠ ♦ t ♦♦♦ ♥t♦♥ s♠ ♦♦rt♠s ♦ s♠s ♦ ♣ ♦ s♦♠ trt ♣r♦rs s♦ s t t♦♥♣s♦♥ ♦rt♠ t ♦r ♥st♥ ♥ ♠r ♦♣t♠③t♦♥t♦rt ♥ ♣rt s♣ts ② ♦♥♥♥s rt ♠r ♥ st③ ❬❪ ♦r ts ♠♣♠♥tt♦♥ s ♦t♥ ♦♠♣① s♥ t♥♦s t ♦♠♣tt♦♥ ♦ t rts ♦ t ♦♦
Pr♠tr st♠t♦♥
♠①tr ♠♦s ♥ ss♠♥t s ♦ t ♥♥t♦♥ ♦ t ♠♦rt♠ ♦s t ♠♣♠♥tt♦♥ s s♠♣ s♥ ♥♦ rt ♦ t ♦♦s ♥♦ s ts ♦rt♠ s s♣③ ♦r t ♠ss♥ t rst② ♥t ♥♦t♦♥ ♦ ♦♠♣tt ♦r t ♠①tr ♠♦ ♥ s♦♥② t t
sr t ♥ ♦♠♣tt ♦r s♠♣ x t ♥rt ♠♦s ss♠ tt t r♥ ♦ ♥ xi ♥♦s t ♣r♠♥r② s♠♣♥ ♦zi s Prr♣ ♥rt ♠♦ ♥ t♦♥ s t♦r z s ♥♦srt s ♦♥sr s ♠ss♥ s x s ♥♠ t ♦sr t t♦♣ (x, z) s ♥♠ t ♦♠♣tt ♥ t s♠ ② t ♦♦♦ ♥t♦♥ ♦♠♣t ♦♥ (x, z) s ♥♠ t ♦♠♣tt ♦♦♦ ♥t♦♥ ♥ t srtt♥ s ♦♦s
L(θ;x, z) =n∑
i=1
ln p(xi, zi;θ)
=n∑
i=1
ln
(
g∏
k=1
(πkp(xi;αk))zik
)
=n∑
i=1
g∑
k=1
zik ln (πkp(xi;αk)) ,
② strt♥ r♦♠ t rt♦♥ t♥ t ♣ p(xi;θ)p(zi|xi;θ) = p(xi, zi;θ) ♦♥♥ t ♦♦♥ rt♦♥
L(θ;x) = L(θ;x, z) + e(z,x;θ).
r e(z,x;θ) = −∑ni=1
∑gk=1 zik ln tik(θ) s L(θ;x) ≥ L(θ;x, z) s♥ tik(θ)
s ♣r♦t②
♦r t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ t ♦♠♣tt ♦♦♦ ♥t♦♥ ♦♠♣t ♦♥ θ ♦r t s♠♣ x ♥ t ♣rtt♦♥ z s ♥ s
L(θ;x, z) =2∑
k=1
nk lnπkσk
− 1
2
2∑
k=1
n∑
i=1
zik(xi − µk)
2
σ2k
− n ln√2π.
♥♥♥ ①♠♣ ♦♠♣tt ♦♦♦
♦rt♠s ♦r ♠①♠♠ ♦♦ st♠t♦♥
♠ ♦ t ♠①tr ♠♦s s ♥r② ♦t♥ ♥ ♠ ♦rt♠ sst♦♥ s ♦t t♦ t ♣rs♥tt♦♥ ♦ ts ♦rt♠ ♥ ♦ ts ①t♥s♦♥s
♣tr str ♥②ss stt ♦ t rt
♠ ♦rt♠
Prs♥tt♦♥ ♦ t ♠ ♦rt♠ ①♣tt♦♥①♠③t♦♥ ♦rt♠rtr ♥♦t ② ♠ s ♣r♦♣♦s ② ♠♣str r ♥ ♥ ♥ ❬❪ ts ♦♠♥s ♦ ♣♣t♦♥ r str t♥ t ♠①tr ♠♦s s♥ ts s♣③ ♥ t s ♦ ♠ss♥ s ♥ t ♦♥t①t ♦ t ♠①tr ♠♦st ss ♠♠rs♣s ♦ t ♥s r ♥tr♣rt s ♠ss♥ s ♦ttt ts ♦rt♠ ♦s t♦ str t st t ♠ss♥ s ② ♠♥② rstrt ss♠♣t♦♥s ♠♥ ♥t ♦ ts ♦rt♠ s ts s♠♣t② s♥ t ♦♣t♠③s t ♦♦ ♥t♦♥ t♦t ♦♠♣t♥ ts rtsrtr♠♦r s♥ ts ♠♣♠♥tt♦♥ ♥ ♣r③ t st②s ♥t ♥ tss ♦♥r♦♥t t r t sts s st♦♥ s st ♥ ♦r ♦ ts ♦rt♠t rr ♥♥ ♠♦r ts ♦ rr t♦ ♦rt♠ ♥ ①t♥s♦♥s ② ♥ ♥ rs♥♥ ❬❪
♥t♦♥ ♦ t ♠ ♦rt♠ ♠ ♦rt♠ s ♥ trt ♦♥ strt♥r♦♠ ♥ ♥t ♦ t ♣r♠tr tr♥ts t♥ t t♦ ♦♦♥ st♣s t ♦♠♣tt♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦ st♣ ♥ ts ♠①♠③t♦♥ ♠ st♣
trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ t Q(θ;θ[r]) r
Q(θ;θ[r]) = Eθ[r] [L(θ;x, z)] ,
st♣ st θ[r+1] s s
θ[r+1] = argmaxθ
Q(θ;θ[r]).
♦rt♠ ♠ ♦rt♠
t♦♣♣♥ rtr ♦ rtr r ♥r② s ♠♦st ♦♠♠♦♥ ♦♥ ♦♥ssts♥ st♦♣♣♥ t ♦rt♠ ♥ t ♥rs ♦ t ♦♦♦ s ♦r t♥ ttrs♦ ε ♦s♥ ② t sr s♦ ♥
L(θ[r+1];x)− L(θ[r];x) < ε.
s♦♥ ♦♥ ①s ♥ ♥ t ♥♠r ♦ trt♦♥s ♣r♦r♠ ② t ♦rt♠
♦s t ts ♥r♥t t♦ t ♠①tr strtr st♠t♦♥ ♦ t ♣r♠trs s ♦ ♦r ♠①tr ♠♦ t♦t ♦♥str♥t t♥sss t ♥r♥ ♦ s ♠♦ ♥ ♠ ♥ t ss ♠♠rs♣s ♦t ♥s r ♥♦♥ s ♠①tr ♠♦ ♦s t st♠t s trt ♥t sr♠♥♥t ♥②ss s♦ ♥ t s r ♥♦♥ ♥ ②s ♥rr♥ str♥ ♣r♦♠ ♦r ♥st♥ t ♠①tr ♠♦s ♦s t ♦♠♣♦♥♥t
Pr♠tr st♠t♦♥
strt♦♥s ♦♥ t♦ t ①♣♦♥♥t ♠② ♥ ♥ ♥♦ ♦♥str♥t t♦tr ♥ ①♣t② ♦♠♣t
❲ ♦♥sr t ss ss♥ ♠①tr ♠♦ ♦s t ♦♠♣♦♥♥tsr ♥ ② trt♦♥ [r] ♦ t ♠ ♦rt♠ s rtt♥ s
st♣ t ♦♥t♦♥ ♣r♦ts
tik(θ[r]) =
π[r]k p(xi;α
[r]k )
p(xi;θ[r])
.
st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦
π[r+1]k =
n[r]k
n, µ
[r+1]k =
1
n[r]k
n∑
i=1
tik(θ[r])xi,
σ2[r+1]k =
1
n[r]k
n∑
i=1
tik(θ[r])(xi − µ
[r+1]k )2,
r n[r]k =
∑ni=1 tik(θ
[r])
♥♥♥ ①♠♣ ♠ ♦rt♠
Pr♦♣rts ♦ t ♠ ♦rt♠ ❯♥r rstrt ss♠♣t♦♥s ❬❲❪ t♠ ♦rt♠ ♣r♦s sq♥ ♦ st♠ts θ[r] ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ t ♦♦♦ ♥t♦♥ s ♦♣t♠♠ ♦♥② ♣♥s ♦♥ t ♥t③t♦♥θ[0] ♥ t ♦♦ ♥t♦♥ ♥rss t trt♦♥ ♦ t ♠ ♦rt♠
∀[r], L(θ[r+1];x) ≥ L(θ[r];x).
s ♦rt♠ ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ t s ♠♥t♦r② t♦ ♣r♦r♠ srr♥t ♥t③t♦♥s ♥ ♦rr t♦ ♦♣ t♦ t t ♠ ♥ ♦tr r ♦ t♠ ♦rt♠ s ts s♣ ♦ ♦♥r♥ ♥ ts ♦rt♠ ♥ ♦♥r s♦②s♣② ♥ t sss r ♦r♣♣ s ♠♥② t♦rs ♥ ♥trst♥ t rt♦♥ ♦ t ♠ ♦rt♠ s ♦r ♥st♥ ❬❱ ❪ tr tsr♣t♦♥ ♦ t ♠ ♦rt♠ ♣♣ ♦♥ t ss♥ ♠①tr ♠♦s ♣rs♥ttr ♦ ts ①t♥s♦♥s r♥ ts rs
♣tr str ♥②ss stt ♦ t rt
❲ ♦♥sr t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ t♥♦♥ ♣r♦♣♦rt♦♥s ♥ r♥s ♥ s s t ♠ st♣ ♦♥② ♦♥ssts ♥ ♦♠♣t♥ µ[r+1]
k s♥ t ♦tr ♣r♠trs r ♥♦♥ r s♣②s t s ♦ t ♦♦ ♦♠♣t t trt♦♥s ♦ t♦r♥s ♦ t ♠ ♦rt♠ r♥ ♣r♥t t tr♥s s ♥t③ t(−1,−1/2) ♥ ♦♥rs t♦ t ♠ t r♥ ♣r♥t t sqrss ♥t③ t (3.5, 3) ♥ ♦♥rs t♦ ♦ ♠①♠♠ ♦ t ♦♦♥t♦♥
♥♥♥ ①♠♣ ①♠♣ ♥ ♠ ♦rt♠
−4 −2 0 2 4 6
−4
−2
02
4
µ1
µ 2
r ♦♦♦ s ss♦t t t♦ sq♥s ♦ ♣r♠trs ♣r♦♥ ② ♥ ♠ ♦rt♠
①t♥s♦♥s ♦ t ♦rt♠
♦rt♠ ♦♠t♠s t s♦t♦♥ ♦ t ♠ st♣ s ♥♦t ①♣t ♥ s s t ♥r③♠ ♠ ♦rt♠ ♥ s ♠ st♣ s s♦ r♣② ♠ ♦♥ ♦♥② rqrs t ♥rs ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦ s t trt♦♥ [r] t st♣ s ♥♥ t ♠st♣ tr♠♥s θ[r+1] s s
Q(θ[r+1];θ[r]) ≥ Q(θ[r];θ[r]).
♠ ♦rt♠ ♣s t ♠♦♥♦t♦♥ ♣r♦♣rt② ♦ t ♥rs ♦ t ♦♦♥t♦♥ ♦r trt♦♥ s ♥rt r♦♠ t ♠ ♦rt♠ ♦r ts♦rt♠ rqrs ♠♦r trt♦♥s t♥ t ♠ s♥ ts ♦♥r♥ s s♦r
Pr♠tr st♠t♦♥
♥ ♦rt♠s ♥ ♦rr t♦ ♦r♦♠ t tr ♠♥ rs♦ t ♠ ♦rt♠ str♦♥ ♣♥♥② t t ♥t③t♦♥ ♣♦♥t ♦ ♦♣t♠♠ ♦♥r♥ ♥ s♦ ♦♥r♥ t t♦st♠ s♠ ♦rt♠ s♣r♦♣♦s ❬+❪ ♦rt♠ ♥♦r♣♦rts st♦st st♣ s st♣ t♥t st♣ ♥ t ♠ st♣ rt ② t r♥♦♠ ♠♣tt♦♥ ♣r♥♣ sq♥ ♥rt ② t s♠ ♦rt♠ ♦♥rs t♦ ♥q stt♦♥r② strt♦♥♦s t♦ p(θ|①)
trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ t t ♦♥t♦♥ ♣r♦ts
tik(θ[r]) =
π[r]k p(xi;α
[r]k )
p(xi;θ[r])
.
st♣ s♠♣ t ss ♠♠rs♣ s s
z[r]i ∼ M(ti1(θ
[r]), . . . , tig(θ[r])).
st♣ st θ[r+1] s s
θ[r+1] = argmaxθ
L(θ;x, z[r]).
♦rt♠ s♠ ♦rt♠ ♦r t ♠①tr ♠♦s
♦rt♠ s st♦♣♣ tr ♥♠r ♦ trt♦♥s ♦s♥ ② t sr ♦ttt ♥♦tr rs♦♥ ♦ ts ♦rt♠ ♥♠ s♠ ♣r♦s ♥ ♠♦st sr②♦♥r♥ t♦ t ♥q stt♦♥r② strt♦♥ ❬❪ t s tr ♦ t♥ rs♦♥ s♠t ♥♥♥ ♠ ♦rt♠ ♥ t s♠ ♦rt♠ ♥ t st♣ ♥♥♥ s ♥tr♦ tr t s st♣ ♥ ♦rr t♦ r t ♠♣t ♦ tr♥♦♠ ♣rtrt♦♥s ♣r♦r♠ ② t s st♣ s rt♦♥ ♥rss t t♥♠r ♦ trt♦♥s s ♥ t s♠ strts t ♦rs t s♠ ♦rt♠t♥ t t♥s t♦ t ♠ ♦rt♠ ♥ t ♥♠r ♦ trt♦♥s ♥rss
♦rt♠ sst♦♥♠ ♠ ♦rt♠ ❬❪ s ♥r ♦rt♠ t♦ ♦♠♣t t st♠t ♥ t♦ ♥ t ♣rtt♦♥ ♥r t sst♦♥♣♣r♦ s t ♣r♦s t ♦♣ (θ, z) ♠①♠③♥ t ♦♠♣tt ♦♦♦
argmax(θ,z)
L(θ;x, z).
♥ t st♠t ♦ t ♠①♠♠ ♦♠♣tt s s ♥ ♥♦♥sst♥t❬♦❪ ts rsts ♥ ttr t♥ t♦s ♦ t ♠ ♥ t s♠♣ s③ ss♠ ♥ ♥ t sss r s♣rt rtr♠♦r t ♠ ♦rt♠♦♥rs str t♥ t ♠ ♦rt♠ ♠ ♦rt♠ ♥♦r♣♦rts sst♦♥ st♣ t♥ t st♣ ♥ t ♠ st♣ ♦r♥ t♦ t ♠♣ ♣r♥♣s ts ♦♥r♥ s♣ s ①♣t t♦ str t♥ t ♦♥r♥ s♣ ♦t ♠ ♦rt♠
♣tr str ♥②ss stt ♦ t rt
trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ s st♣ t Q(θ;θ[r]) r
tik(θ[r]) =
π[r]k p(xi;α
[r]k )
p(xi;θ[r])
.
st♣ ♠♥♠③ e(z,x;θ[r]) s♦
z[r]ik =
1 tik(θ[r]) ≥ tiℓ(θ
[r]) ∀ℓ = 1, . . . , g0 ♦trs
st♣ st θ[r+1] s s
θ[r+1] = argmaxθ
L(θ;x, z[r]).
♦rt♠ ♠ ♦rt♠ ♦r t ♠①tr ♠♦s
♠r ♠ ♦r t s♣r ss♥ ♠①tr ♠♦ ♥ ♠♥s ♦rt♠ ♠♥s ♦rt♠ s q♥t t♦ t ♠ ♦♥ ♥ t ♠♦ t♥ s t s♣r ss♥ ♠①tr ♠♦ t q ♣r♦♣♦rt♦♥s ❬❪
rs ♦ t ♠①♠♠ ♦♦ ♣♣r♦s ♥ t ①t♥s♦♥s♦ t ♠ ♦rt♠ r ts ♠♥ rs tr ♣r♦♠s st② ♥r♥t t♦ t♠①♠♠ ♦♦ ♣♣r♦ ♣♣ ♦♥ ♠①tr ♠♦s
rst ♦♥ s t t② t♦ ♥ t ♦ ♠①♠♠ ♦ t ♦♦♥t♦♥ ♦t tt ts ♣r♦♠ s ♠♦r ♣rs♥t ♥ t s♠♣s r s♠s♥ t ♦♦ ♥t♦♥ ♥ r② ♠♣②
s♦♥ ♦♥ s t♦ t ♣♣r♦♥ ♦ t ♦♦ ♥t♦♥ ♥ ts ♥t♦♥ s ♣♣r♦♥ ♦r t♦r t t ♥ ♣♣r♥♦♥ ♥ ♦tr stt♦♥s s ♦r ♥st♥ t tr♦sst ss♥♠①tr ♠♦ s t st♠t rtr♥ ② t ♦rt♠ ♥ ♦♥ t♥r② ② ♥ s s ts st♠t s s♦ s ♥ ♥♦♥sst♥t
tr ♦♥ s ♦t t rrt② ♦♥t♦♥s r ♦t♥ ♦t ♦rt s♠ t sts s t st♠t♦♥ ♥ ♥♦ ♥ ♦rtt♥
①♠♠ ♣♦str♦r st♠t♦♥
②s♥ r♠♦r ♥ t ②s♥ r♠♦r t ♣r♠tr θ s ss♠ t♦ ts r♥♦♠ r ♦s t ♣r♦r strt♦♥ s ♥♦t ② p(θ) sstrt♦♥ ♦♥t♥s t ♥♦r♠t♦♥ ♦♥ θ ♥ ② ♥ ①♣rt s t tr♠♣r♦r ♥ ♥tr♣rt s ♦r t♦ ♦sr t t r r r♥t ♣r♦rstrt♦♥s ♥ tr ♠♣t ♦♥ t ♥r♥ ♥ ♥♦t ♥ s♣② ♦rt s♠ t sts s strt♦♥s r ♣rs♥t ♥ ②s♥ ♦ ②P ♦rt ❬♦❪ ♥ r ts ♦♥ t ②s♥ r♠♦r t ♣r♦rstrt♦♥ ♦♥t♥s t ♥♦r♠t♦♥ ♥ ② ♥ ①♣rt t strt♦♥ r♦♠
Pr♠tr st♠t♦♥
♥r♥s r ♠ s t ♣♦str♦r ♦♥ ♣♦str♦r strt♦♥ ♦♥t♥s t♣r♦r ♥♦r♠t♦♥ ♥ ② t ①♣rt p(θ) ♥ ② t t (x) s t tr♠♣♦str♦r ♥ ♥tr♣rt s tr t♦ ♦sr t t
P♦str♦r strt♦♥ ♥ ♦♦ ♥t♦♥ ②s r ♥♦s ttt ♣♦str♦r strt♦♥ p(θ|x) s ♥ s
p(θ|x) = p(x|θ)p(θ)p(x)
.
♦t tt t ♥♦r♠t♦♥ ♥ ② t t s rt t♦ t ♦♦ ♥t♦♥p(x|θ) s t ♥t♦♥ ♦ t ♦♦ ♥t♦♥ s r s♥ ts ♥t♦♥♦♥t♥s t ♥♦r♠t♦♥s ♥ ② t t ♥ ♦t rq♥tst ♥ ②s♥r♠♦rs s ♥t♦♥ s s♦ ♦♠♠♦♥ s ♦r ♦t ♦♠♠♥ts ♥p(x) s ♦♥st♥t ♦r♥ t♦ θ t ♦♦♥ rt♦♥ s s ♥ p(x) s ♥♦t♦♠♣t
p(θ|x) ∝ p(x|θ)p(θ).
♥ ♥ts ♦ t ②s♥ ♣♣r♦ ②s♥ ♣♣r♦s ♦rt ♠①tr ♠♦ r t ♥ ♥t ①tr ♥ r♦ t♥ ♦s ② rürt♥ttr ❬❪ r♦♠ r ♦♥ ♥ ①trt t ♦r ♦♦♥ ♠♥qts
♣r♦r s s♠♦♦t t ♦♥ t ♣r♦♠s ♦ t ♥rt s♦t♦♥
s ♠t♦s t ♥t♦ ♦♥t t ♣r♠tr ♥rt♥t② ② st② ♥ s r rrt② ♦♥t♦♥s r ♦t s♠ t
st ♠①tr t s♠ ♦♠♣♦♥♥t ♣r♦♣♦rt♦♥s s♥ t② ♦ ♥♦t r② ♥s②♠♣t♦t ♥♦r♠t②
r ♠♣♠♥tt♦♥ s ♥♦t ♦♠♣① ♥ t ♦♠♣♦♥♥t strt♦♥s ♦♥ t♦ t ①♣♦♥♥t ♠② ♥ ♥ s s t ♦♥t ♣r♦rstrt♦♥s ♣r♦ ①♣t ♣♦str♦r strt♦♥s ❬♦❪
❲ ♦♥sr t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ t♥♦♥ ♣r♦♣♦rt♦♥s ♥ r♥s ♥ ts s θ = (µ1, µ2) ss♠♥♣♥♥ t♥ t ♣r♦r strt♦♥s ♥ s t r②s ♥♦♥♥♦r♠t ♦♥s s♦
p(θ) = p(µ1)p(µ2) t µ1 ∼ N (ξ, κ) ♥ µ2 ∼ N (ξ, κ),
r ξ ♥ κ r ②♣r♣r♠trs r s♣②s t s ♦ t♣r♦r ♥ ♦ t ♣♦str♦r strt♦♥s ♦♠♣t ♦♥ s♠♣ ♦ s③
♥♥♥ ①♠♣ ②s♥ r♠♦r
♣tr str ♥②ss stt ♦ t rt
−4 −2 0 2 4 6
−4
−2
02
4
µ1
µ 2
♣r♦r strt♦♥
−4 −2 0 2 4 6
−4
−2
02
4
µ1
µ 2
♣♦str♦r strt♦♥
r Pr♦r ♥ ♣♦str♦r strt♦♥s ♦r ♦♠♣♦♥♥t ♥rt ♠①tr♠♦ t ξ = 1 ♥ κ = 9
♠♦♦t t ♥ ♥rt s♦t♦♥ ②s♥ r♠♦r ♥ ♦s♦♠ ♥r② ♣r♦♠s ② s♥ ♣r♦r strt♦♥s ♣r♦s s♠♦♦tt s strt ♦
r♦♠ t s♠♣ x rs♥ r♦♠ t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ t ♥♦♥ ♣r♦♣♦rt♦♥s t µ1 = 0 ♥ µ2 = x1 t ♠ st♦ ♥r t ♣r♠tr θ = (σ2
1, σ22) rq♥tst ② ♥ sr r♦♠
♥r② s♦t♦♥ s strt ♥ ♥♥♥ ①♠♣ ♥ ②s♥② ss ♥♣♥♥ ss♠♣t♦♥ t♥ ♣r♦r strt♦♥s ss♦t t ♦♥t ♣r♦r strt♦♥s ♥♦s tt
p(θ) = p(σ21)p(σ
22) r 1/σ2
1 ∼ G (c0, C0) ♥ 1/σ22 ∼ G (c0, C0) .
♥ t ♣♦str♦r strt♦♥ p(θ|x) =∑z∈Z p(θ|x, z)p(z|x) r Z =
1, 2n t♥ ts strt♦♥ s ♣♣r♦♥ ②
p(θ|x) ≤∑
z∈Z
p(θ|x, z) ♥ p(θ|x, z) = p(σ21|x, z)p(σ2
2|x, z).
♥ 1/σ2k|x, z ∼ G
(
c0 +♥k2, C0 +
∑
i:zik=1(xi−µk)
2
2
)
♥ s♥ t ♠♦
♦ G(α, β) s α−1β
♥ α ≤ 1 t♥
p(θ|x) < +∞ ∀θ.
♥♥♥ ①♠♣ ♠♦♦t t ♥ ♥rt s♦t♦♥
♦t tt t ♦♥t ♣r♦r strt♦♥s r ♦t♥ s s♥ t② s♥♥t②s♠♣② t ♥r♥ ♦r t ♦ ♦ t ②♣r♣r♠trs ♣r♠trs ♦t ♣r♦r strt♦♥ ♥ t ♥ t r②s ♥♦♥ ♥♦r♠t ♣r♦r s
Pr♠tr st♠t♦♥
♥♦t s♦t♦♥ s s♦ t♦ ① t ②♣r♣r♠trs ② s♥ ♥ ♠♣r②s♥ ♣♣r♦ tr♠♥s t ②♣r♣r♠trs ♦r♥ t♦ t ts ♦r ♥st♥ ❬❪ ♦r t ss♥ ♠①tr ♠♦
②s♥ ♥r♥ ②s♥ ♣♣r♦s ♦t♥ ♥ s♠t♦♥ ♠t♦s r♦ ♥ ♦♥t r♦ ♠♠ t♦ ♥rr ♦ tr ♦♣♠♥t s r♥ts t s rt t♦ t ♦♠♣tt♦♥ ♣♦r ♥ t ♣♦str♦r strt♦♥♦♥t♥s t ♦ ♥♦r♠t♦♥ ♦t θ ♥♦r♠t♦♥ ♥ ② t ①♣rt ♥ ② tt ♥② ♥r♥ ♦♥ θ r s ♦♥ ts strt♦♥ t s s♦ ♥tr t♦ ♦♣tt ♣♣r♦ s♠r t♦ t ♦♥ s ♥ t rq♥tst r♠♦r ♦ ♥t t♦♦t♥ t st♠t ♦ t ♠①♠♠ ♣♦str♦r st♠t ♠♣ ♥♦t ② θ
♥t♦♥ ①♠♠ ♣♦str♦r st♠t ♠①♠♠ ♣♦str♦rst♠t s ♥ ②
θ = argmaxθ
p(θ|x) = argmaxθ
p(x|θ)p(θ).
♠r ♥ t♥ t ♠ ♥ ♠♣ ♦t tt t ♣r♦r ♦♦s ♥♦r♠ strt♦♥ t♥ p(x|θ) ∝ p(θ|x) ♥ s s t ♠ s s♦ q t♦t ♠♣
s t ♠♣ s rs♦♥ st♠t t t ♥ t t♦ ♦t♥ t♥ s♦ r♣ ② t ♠♥ ♦r t ♠♥ ♦ t ♣♦str♦r strt♦♥ ♦tttr st♠ts r ♠♦r s② ♦t♥ ♠♠ ♦rt♠s ❲ ♥♦ t t♠♥ ♦rt♠s ♣r♦r♠♥ t ②s♥ ♥r♥ ♦♥ θ
♦rt♠s ♦r ♠①♠♠ ♣♦str♦r st♠t♦♥
♦rt♠ ♦r ②s♥ st♠t♦♥
♥ ♠ ♦rt♠ ♥ ♠♦ t♦ ♣r♦ t♠♣ ♦r t st♠t♦ t ♠①♠♠ ♣♥③ ♦♦ ❬r❪ s ♦t s ② s♥ t♦♦♥ rt♦♥ ♦t♥ ② ♣♣②♥ t ②s r ♥ ② s♥ t ♦rt♠♥t♦♥
argmaxθ
p(θ|x) = argmaxθ
L(θ;x) + ln p(θ).
♦ ♦t♥ t ♠♣ t ♠ st♣ ♦ t ♠ ♦rt♠ ♦♥ssts ♥ t ♠①♠③t♦♥♦ t ①♣tt♦♥ ♦ t ♦♠♣t t ♣♦str♦r strt♦♥ p(θ|①, ③) t trt♦♥[r] t ♠ st♣ tr♠♥s t ♣r♠tr θ[r+1] s s tt
θ[r+1] = argmaxθ
Q(θ;θ[r]) + ln p(θ).
♣tr str ♥②ss stt ♦ t rt
t ♦♠♣♦♥♥t ♥rt ss♥ ♠①tr ♠♦ ♦s ♦♥② t ♠♥sr ♥♥♦♥ ♣r♦r strt♦♥ ♦ t ♣r♠trs s ♥ ♥ s♦ t ♠ st♣ s rtt♥ s
µ[r+1]k =
σ2kξ + κ
∑ni=1 tik(θ
[r])xi
σ2k + κn
[r]k
.
♥♥♥ ①♠♣ ②s♥ st♠t♦♥ ♦ ss♥ ♠①tr
♦rt♠s ♥ ②s♥ st♠t♦♥
trtr ♦ ts st♦♥ ❲ ♥♦ ♣rs♥t s♦rt ♦r ♦ tr ♠♥ ♦rt♠s s t♦ ♥r t ♣r♠trs ♦ ♠①tr ♠♦ t s s♠♣rt tr♦♣♦sst♥s ♦rt♠ ♥ t tr♦♣♦st♥s s♠♣r rr ♥t♥ ♠♦r ts ♥ r♣♦rt ♦♥ ♦♥t r♦ ttst t♦s ②P ♦rt ❬❪ s ♦rt♠s r ♠♠ ♦♥s ♦s t r♦ ♥s t ♣♦str♦r strt♦♥ p(θ, z|x) s t stt♦♥r② strt♦♥ s t②s♠♣ sq♥ ♦ ♣r♠trs ♦r♥ t♦ tr ♣♦str♦r strt♦♥ s♥ ts♣♣r♦ ♦s s t♦ ♣r♦r♠ t ②s♥ ♥r♥
♠r ♠♦sts♦r♥ stts ♥♦♥ r♥t ♥t③t♦♥s ♦ t♦rt♠s ♥ t ♠♠ ♦rt♠s ♥ ♥ rr ♥ r♦ r♦♥ r ♥♦t t♦rt② s♥st t♦ t ♦ ♦♣t♠ tr ♦r s ♥♦t s♦♣rt ♥ ♣rt s ♦r ♥st♥ ❬❪ ♥ tr r tr♣♣♥ stts r ♠♦sts♦r♥ stts rqr♥ s♦ r ♥♠r ♦ trt♦♥s t♦ s♣r♦♠ t♠ tt t ♦rt♠ s ♥r② st♦♣♣ ♦r
s s♠♣r
♥ s s♠♣r s t ♠♦st ♣♦♣r ♣♣r♦ t♦ ♣r♦r♠ t②s♥ ♥r♥ ♦ ♠①tr ♠♦ s♥ t ss t t♥t strtr ♦ t ts ♦rt♠ s t ♦♥ ♦♥t♦♥ strt♦♥s r♦♠ t s s② t♦s♠♣
s s♠♣r ♥ ♠①tr ♠♦s s s♠♣r s ♥ trt ♦rt♠ ♦s ♦♥ trt♦♥ s s♣t ♥ t♦ ♠♥ st♣s ♦r t ♠①tr ♠♦ r♠♦r♥ ts ♦rt♠ tr♥t② s♠♣s t ss ♠♠rs♣s ♦♥t♦♥② ♦♥t ♣r♠trs ♥ ♦♥ t t ♥ t ♣r♠trs ♦♥t♦♥② ♦♥ t ss ♠♠rs♣s ♥ ♦♥ t t s ts stt♦♥r② strt♦♥ s p(θ, z|x) tr♦r tsq♥s ♦ t ♥rt ♣r♠trs r s♠♣ r♦♠ tr ♣♦str♦r strt♦♥p(θ|x)
Pr♠tr st♠t♦♥
s ♦rt♠ ♥ p(θ|①) s ♠r♥ stt♦♥r② strt♦♥ strtsr♦♠ ♥ ♥t θ[0] t♥ tr♥ts t♥ t♦ st♣s t trt♦♥[r] t ♣r♦r♠s t t♦ ♦♦♥ st♣s
z[r] ∼ z|θ[r],x
θ[r+1] ∼ θ|z[r],x.
♦rt♠ s s♠♣r ♦r t ♠①tr ♠♦s
♠♣♥ ♦ t ss ♠♠rs♣ ♥♣♥♥ t♥ ♥s ♦st♦ s② s♠♣ t t♦r z s♥ p(z|θ[r],x) =
∏ni=1 p(zi|θ[r],xi) ♥ z
[r]i
s s♠♣ r♦♠ t ♦♦♥ ♠t♥♦♠ strt♦♥
z[r]i |θ[r],xi ∼ M(ti1(θ
[r]), . . . , tig(θ[r])).
♠♣♥ ♦ t ♣r♠trs ❲♥ tr s ♥♦ ♦♥str♥t t♥ t ♣r♠trs ♦ r♥t sss t ♦♦♥ ♦♠♣♦st♦♥ s s t♦ s♠♣ θ[r+1]
p(θ[r+1]|z[r],x) = p(π[r+1]|z[r])g∏
k=1
p(α[r+1]k |z[r],x).
♦t tt π s ♥♣♥♥t ♦ t t ♦♥t♦♥② ♦♥ t ss ♠♠rs♣s s ♣r♦r ♦ π s t ♦♥t r②s ♥♦♥ ♥♦r♠t ♣r♦r ♥ s s t♣r♦r ♥ t ♣♦str♦r strt♦♥s ♦ t ss ♣r♦♣♦rt♦♥s r rs♣t② ♥②
π ∼ Dg
(
1
2, . . . ,
1
2
)
♥ π|z[r] ∼ Dg
(
1
2+ ♥[r]
1 , . . . ,1
2+ ♥[r]
g
)
,
r r♠♥ tt ♥[r]k =
∑ni=1 z
[r]ik ❲ ♥♦ strt ts ♦rt♠ t t
r♥♥♥ ①♠♣
♣tr str ♥②ss stt ♦ t rt
❲ ss♠ tt t ♣r♦r ♦ σ2k s G−1(c0, C0) ♥ tt t ♣r♦r ♦ µk
♦♥t♦♥② ♦♥ σ2k s N1(b0, B
−10 σ2
k) trt♦♥ [r] ♦ t s s♠♣r♥ p(θ|x) s stt♦♥r② strt♦♥ s rtt♥ s ♦♦s
∀i = 1, . . . , n z[r]i |xi,θ[r] ∼ M2(ti1(θ
[r]), ti2(θ[r]))
π[r+1]|z[r] ∼ D2
(
1
2+ n
[r]1 ,
1
2+ n
[r]2
)
∀k = 1, 2 µ[r+1]k |x, z[r], σ2[r]
k ∼ N1(b[r]k , B
[r]k )
∀k = 1, 2 σ2[r+1]k |x, z[r], µ[r+1]
k ∼ G−1(c[r]k , C
[r]k ),
r b[r]k =B0b0+
∑ni=1 z
[r]ik xi
B0+n[r]k
B[r]k =
σ2[r]k
B0+n[r]k
c[r]k = c0 +n[r]k +1
2♥ C
[r]k =
C0 +12(∑n
i=1 z[r]ik (xi − µ
[r+1]k ) + B0(µ
[r+1]k − b0)
2)
♥♥♥ ①♠♣ s s♠♣r
♠♣ s♠♣♥ ♦♥t♦♥ s t s s♠♣r s t♦ ♣r♦r♠ ♥♠r♦ trt♦♥s t s s♦t② ♥ssr② tt st♣ ♥♦s s♠ s♠♣♥ t♠ t ♦♥t♦♥ strt♦♥s ♦ z[r]
i ♥ π[r+1] r ①♣t t s♠♣♥ ♦α[r+1] ♥ ♠♦r ♦♠♣t ♦♥t ♣r♦r strt♦♥s r s♦ ♥r②s s♥ t② ♣r♦ ss ♣♦str♦r strt♦♥ s t s♠♣♥ ♦ α[r+1]
s s② ♥ tr s ♥♦ ♦♥str♥t t♥ t ♣r♠trs ♥ t s r ts♠t♦♥ ♦ p(α[r+1]
k |z[r],x) s t♦♦ ♠ t♠ ♦♥s♠♥ ♥♦tr ♣♣r♦ t♥t s s♠♣r s t♦ s
tr♦♣♦sst♥s ♦rt♠
♥ ♠ ♦ t tr♦♣♦sst♥s ♦rt♠ s t♦ s♠♣ sq♥♦ θ ♦r♥ t♦ ts ♣♦str♦r strt♦♥ p(θ|x) s ♦rt♠ rqrs ♥ ♥str♠♥t strt♦♥ ♥♦t ② q(.;θ) ♥ t rs♣t t♦ t ♦♠♥t♥♠sr ♦ t ♠♦ t trt♦♥ [r] t ♥str♠♥t strt♦♥ ♥rts ♥t θ⋆ ♦♥t♦♥② ♦♥ t rr♥t ♦ θ ♥ t ♥t s ♣tt ♣r♦t② λ[r] ♥ ②
λ[r] = ♠♥
p(θ⋆|x)q(θ[r];θ⋆)
p(θ[r]|x)q(θ⋆;θ[r]); 1
.
Pr♠tr st♠t♦♥
s ♦rt♠ s p(θ|①) s stt♦♥r② strt♦♥ trt♥ r♦♠ ♥♥t θ[0] ts trt♦♥ [r] s rtt♥ s
θ⋆ ∼ q(θ;θ[r])
θ[r+1] =
θ⋆ t ♣r♦t② λ[r]
θ[r] t ♣r♦t② 1− λ[r].
♦rt♠ tr♦♣♦sst♥s ♦rt♠
②r
♥ ❲♥ st♣ ♦ s s♠♣r s t t♦ ♣r♦r♠ t ②r♠♠ ♦rt♠s r ♦t♥ s ♠♦st ♣♦♣r ♣♣r♦ s t♦ s♠♣ sq♥♦ θ ♦r♥ t♦ tr♦♣♦st♥s s♠♣r ♥ ts ♣♣r♦ t tst♣s ♦ t s s♠♣r r r♣ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s♦rt♠ ♦r t stt♦♥r② strt♦♥ ♦ t r♦ ♥ st②s q t♦p(θ, z|x)
s ♦rt♠ ♣r♦r♠♥ t ♥r♥ ♦r t ♠①tr ♠♦s s p(θ|①)s ♠r♥ stt♦♥r② strt♦♥ trt♥ r♦♠ ♥ ♥t θ[0] tstrt♦♥ [r] s rtt♥ s
z[r] ∼ z|θ[r],x
θ⋆ ∼ q(θ;θ[r])
θ[r+1] =
θ⋆ t ♣r♦t② λ[r]
θ[r] t ♣r♦t② 1− λ[r],
r q(.;θ) s t ♥str♠♥t strt♦♥ ♦ t tr♦♣♦sst♥s st♣♥ r λ[r] s ts ♣t♥ ♣r♦t② ♥ ②
λ[r] = ♠♥
p(θ⋆|z[r],x)q(θ[r];θ⋆)
p(θ[r]|z[r],x)q(θ⋆;θ[r]); 1
.
♦rt♠ tr♦♣♦st♥s s♠♣r
❲ ♥♦ strt ts ♦rt♠ t t r♥♥♥ ①♠♣
♣tr str ♥②ss stt ♦ t rt
trt♦♥ [r] ♦ ts ♦rt♠ s rtt♥ s
z[r] ∼ z|θ[r],x
π[r+1] ∼ π|z[r]
α⋆ ∼ q(α;α[r])
α[r+1] =
α⋆ t ♣r♦t② λ[r]
α[r] t ♣r♦t② 1− λ[r],
r q(.; .) s t ♥str♠♥t strt♦♥ ♦ t tr♦♣♦sst♥s st♣♥ r λ[r] s ts ♣t♥ ♣r♦t② ♥ ②
λ[r] = ♠♥
p(α⋆|z[r],x)q(α[r];α⋆)
p(α[r]|z[r],x)q(α⋆;α[r]); 1
.
♥♥♥ ①♠♣ tr♦♣♦st♥s s♠♣r
♦ st♦♥
♥ t ♠♦ st♦♥ ♥
♥t♦♥ ♦ t s ♦♥sr t ♥r ♥t ♠①tr ♠♦
p(xi;θ) =
g∑
k=1
πkp(xi;αk),
θ ∈ Θ r t ♣r♠tr s♣ Θ s ♥ ② t ♥♠r ♦ ♦♠♣♦♥♥ts ♥t ♥tr ♦ ♦♠♣♦♥♥t ♠♦ m r♦♣s t st ♦ t strt♦♥s♥ ② s♦
m = p(xi;θ) : θ ∈ Θ.
♠ ♠♦ m ♥s t ♥tr ♦ t ♦♠♣♦♥♥t strt♦♥s ♥ t♥♠r ♦ ♦♠♣♦♥♥ts s t s ♥r② ♥♥♦♥ t ♠♦ s t♦ ♥rr♦r♥ t♦ t t s ♥ ∆ s t st ♦ t ♠♦s ♦♥sr ②t ♣rtt♦♥r ♥ t ♠ s t♦ ♥ t st ♠♦ ♠♦♥ ∆
♦♦♦ ♥t♦♥ ♥ ♠ ♠♦s ♦♦ ♥t♦♥ ♥r② ♦s t♦ st♠t t st ♠♦ ♦r♥ t♦ t t ♦r ts♣♣r♦ ♥ ♥♦t rt② ♣♣ ♥ t ♠①tr ♠♦ ♦♥t①t ♥ ♥ s s ♦t ♦ ♠♦s r ♠ ♦r ♥st♥ ss♥ ♠①tr ♠♦ ttr ♦♠♣♦♥♥ts ②s ♦t♥s st ♦♦ s t♥ t ss♥ ♠♦st t♦ ♦♠♣♦♥♥ts s t st ♠♦ s t ♠♦ ♠s t sttr ♦ t♥ ts qt② ♦ st♠♥t t♦ t t ♥ ② ts ♦♦ ♥ ts ♦♠♣①t② ♥♠r ♦ ♣r♠trs
♦ st♦♥
♥♦r♠t♦♥ rtr ❲ s ♥ t♦♥ tt rst rtr ts♦♣ ♦ t ♦♦ ♥t♦♥ ♥ s t♦ st t ♥♠r ♦ sss s♣②♦r t ♦♠tr str♥ ♠t♦s ts ♣♣r♦s ♥ s t♦ st t♠♦ ♦ ♣r♦st ♠t♦ t s ♠♦r ♦♥♥♥t t♦ s ♥♦r♠t♦♥ rtr ♣r♦♣♦s ② t ♣r♦st r♠♦r s rtr rtr t r♦r♦s②♣r♦r♠ t ♠♦ st♦♥ ♦r♥ t♦ ♥ ♦t ♦ t st♠♥t rtr ♦r ♥ ♦t ♦ sst♦♥ rtr♦♥ ♥r② ts rtrrqr t ♠ rt t♦ ♠♦ ♥ ∆ s♥ t② ♥ ♦t♥ rtt♥ s ♣♥③t♦♥ ♦ t ♦♦♦ ♥t♦♥
m = L(θ;x)− h(νm),
r θ s t ♠ ♦ t ♠♦ m r νm s t ♣r♠trs ♥♠r ♦ t♠♦ m ♥ r h(.) s ♥t♦♥ ♥ ② t rtr♦♥ ♦t tt qt② ♦ ♥t ♦r t ♥♦r♠t♦♥ rtr♦♥ s t ♦♥sst♥② ♥ ♠♥s♦♥ssr♥ ♦♦ s②♠♣t♦t ♦r
♥t♦♥ ♦♥sst♥② ♥ ♠♥s♦♥ ♦r rtr♦♥ rtr♦♥ s ♦♥sst♥t♥ ♠♥s♦♥ t sts t s♠♣st tr ♠♦ t ♣r♦t② ♦♥ ♥ ts♠♣ s③ t♥s t♦ t ♥♥t②
♥♦r♠t♦♥ rtr ♦r t t st♠♥t
s st♦♥ s ♦t t♦ t ♠♦ st♦♥ st②s t ♣r♦♠ s❬❪ ♣tr ♦r t ♠①tr ♠♦s s♣② t st♦♥ ♦ t ss ♥♠r ♣r♥♣② s♥ t ♠♦s r ♠ ♥ s♥ t ♥♦r♠t♦♥ rtrr ♦♥② s②♠♣t♦t② tr
rq♥tst rtr♦♥
♥ rq♥tst r♠♦r t ♠ s t♦ ♥ t ♠♦ ♠♥♠③♥ t r r♥ ❬❪ ♦ t tr strt♦♥ rt t♦ t st♠t♦♥
♥t♦♥ r r♥ t xi ∈ Re t r
r♥ ♦ t ♣ f(xi) rt t♦ t ♣ g(xi) s
(f, g) =
∫
xi∈X
f(xi) ln f(xi)dxi −∫
xi∈X
f(xi) ln g(xi)dxi.
t f(xi) t ♣ ♦ t tr ♠♦ ♥ t ♠♦ ♠♥♠③♥ s q♥tt♦ ♥ t ♠♦ ♠♥♠③♥ t tr♠ ♦♥ t♥ s ♦ t ♣r♦s qt♦♥s ♦r ♠♦ m t ♠ s t♦ ♦♠♣t
η(xi; f,m, θ) =
∫
xi∈X
f(xi) ln p(xi; θ)dxi,
r p(xi; θ) s t ♣ ♦ t ♠♦ m ♣r♠tr③ ♥ ts ♠ θ s t strt♦♥ f s ♥♥♦♥ s t ♥tr st♠t♦r ♦ η(xi; f,m, θ) ♥ ②
η(xi; f ,m, θ) =1
nL(θ;xi).
♣tr str ♥②ss stt ♦ t rt
♦r ts st♠t♦r srs r♦♠ t ♦♦♥ s
b = Ef
[
η(xi; f ,m, θ)− η(xi; f,m, θ)]
.
s t st ♠♦ ♠♦♥ ∆ ♠①♠③s t ♦rrt ♦♦♦
argmaxm∈∆
L(θ;x)− b.
❬❪ s♦ tt t ♦rrt tr♠ s s②♠♣t♦t② q t♦ t ♥♠r♦ ♣r♠trs
♥t♦♥ rtr♦♥ ♥♦r♠t♦♥ rtr♦♥ s♥ s
(m) = L(θ;x)− ν,
r θ s t ♠ ♦ t ♠♦ m ♥ ν ts ♥♠r ♦ ♣r♠trs
s t rtr♦♥ s ♥ st♠t♦r ♦ t ①♣tt♦♥ ♦ t ♠♥ ♦ t ♦♦♦ st② ♦ t rtr♦♥ ♣r♦♣rts ♥ ♦ ts ①t♥s♦♥ s ♥ ❬♦③❪ ♦r t ♦r ♦ t rtr♦♥ ♥ ♥♦♥sst♥t
Pr♦♣♦st♦♥ s ♥♦t ♦♥sst♥t s ♥♦t ♦♥sst♥t ♥ ♠♥s♦♥ ♥♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts r ♠
Pr♦♦ t ♠♦ m0 ♦s t ♠ ♦ ♠♥s♦♥ ν0 s ♥♦t ② θ0 ♥ t m1
♦s t ♠ ♦ ♠♥s♦♥ ν1 s ♥♦t ② θ1 s s m0 s t tr ♥♠r m0
♥ m1 r ♠ ♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts ♥ ν0 < ν1♥
2 ((m1)− (m0)) = 2(
L(x; θ1)− L(x; θ0))
− 2(ν1 − ν0)
D→ χ2ν1−ν0
− 2(ν1 − ν0).
s t rtr♦♥ s ♥♦t ♦♥sst♥t limn→∞
P ((m1) > (m0)) > 0)
s♥ P (χ2ν1−ν0
> 2(ν1 − ν0)) > 0 ♦t tt t ♠♦♥strt♦♥ ♥ ♥♦t ♣r♦r♠ t♦ st t ♥♠r ♦ sss ♥ ♥ s s t ♦♥r♥ t♦ t♦♦ rt♦ s ♥♥♦♥ s♥ t ♥♦s ②♦rs ♦♣♠♥t ♦♥ t ♦rr♦ t ♣r♠trs s♣ ♦r t s ♦t♥ ♥ ♦sr tt t rtr♦♥sts ♠♦r ♦♠♣t ♠♦s ♥ t tr ♦♥ s ♥ t st ♦ t ♠♦s❬❪
②s♥ rtr♦♥
t p(m) t ♣r♦r strt♦♥ ♦ m ∈ ∆ t ♣♦str♦r strt♦♥ ♦ ♥trsts ♥ ② s♥ t ②s r s ♦♦s
p(m|x) = p(x|m)p(m)
p(x).
♦ st♦♥
♥ ♥ ②s♥ ♣♦♥t ♦ t st ♠♦ m⋆ ♠①♠③s t ♣♦str♦rstrt♦♥
m⋆ = argmaxm∈∆
p(m|x) = argmaxm∈∆
p(x|m)p(m).
s t q♥tt② ♣r♦r♠♥ t ♠♦ st♦♥ s t ♥trt ♦♦ s♦♥♠ ♠r♥ ♦♦ ♦r ♥ ♥ ②
p(x|m) =
∫
θ∈Θ
p(x,θ|m)p(θ|m)dθ,
r t ♣r♠tr s♣ Θ ♣♥s ♦♥ m ts q♥tt② ♥ ♣♣r♦ ♠♥② ♠t♦s s t r ♦ ❬❲❪ t ♠♦st ss ♦♥ s t♦ st ♣♣r♦①♠t♦♥ ❬❪ ♣♣r♦①♠ts ln p(x|m) ② s♥ ♣♣♣r♦①♠t♦♥ ♥ ② r♣♥ t ♠♣ ② t ♠
♥t♦♥ rtr♦♥ ②s♥ ♥♦r♠t♦♥ rtr♦♥ s♥ s
(m) = L(θ;x)− ν
2lnn,
r θ s t ♠ ♦ t ♠♦ m ♥ r ν ♥♦ts ts ♥♠r ♦ ♣r♠trs
s rtr♦♥ ss♠s rrt② ♦♥t♦♥s ♦♥ t ♣ ♠② ♥♦t r② t ♠①tr ♠♦s rtr♠♦r ts ♣♣r♦①♠t♦♥ s ♦♥② s②♠♣t♦t② tr
Pr♦♣♦st♦♥ s ♦♥sst♥t rtr♦♥ s ♦♥sst♥t ♥ ♠♥s♦♥♥ ♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts r ♠
Pr♦♦ t ♠♦ m0 ♦s t ♠ ♦ ♠♥s♦♥ ν0 s ♥♦t ② θ0 ♥ t m1
♦s t ♠ ♦ ♠♥s♦♥ ν1 s ♥♦t ② θ1 s s m0 s t tr ♥♠r m0
♥ m1 r ♠ ♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts ♥ ν0 < ν1♥
2 ((m1)− (m0)) = 2(
L(x; θ1)− L(x; θ0))
− 2(ν1 − ν0) lnn
D→ χ2ν1−ν0
− 2(ν1 − ν0) lnn.
② s♥ t ♥♦tt♦♥ ∆ν = ν1− ν0 t ♦♦♥ rst s ♦t♥ ② ♣♣②♥ t②s ♥qt② ♥ n s r
P ((m1) > (m0)) ≤ P (|χ2∆ν
−∆ν | > ∆ν(−1 + lnn))
≤ 2∆ν
(∆ν(−1 + lnn))2n→∞→ 0,
s♥ E[χ2∆ν
] = ∆ν ♥ ❱r(χ2∆ν
) = 2∆ν s t rtr♦♥ s ♦♥sst♥t♥ ♠♥s♦♥ ♥ ♠♦s t t s♠ ♥♠r ♦ ♦♠♣♦♥♥ts r ♠♦t tt t ♠♦♥strt♦♥ ♥ ♥♦t ♣r♦r♠ t♦ st t ♥♠r ♦ sss♥ ♥ s s t ♦♥r♥ t♦ t ♦♦ rt♦ s ♥♥♦♥ s♥ t♥♦s ②♦rs ♦♣♠♥t ♦♥ t ♦rr ♦ t ♣r♠trs s♣ ♦r t rtr♦♥ s ♠♦r r♦st t♥ t ♦♥ t ♥ ♦rst♠ts t ♥♠r♦ ♦♠♣♦♥♥ts ♥ t tr ♠♦ s ♥♦t ♥ ∆ ❬❪
♦t tt ② s♥ ♦② ♦♥ ♣r♠tr③t♦♥ ❬r❪ s♦s tt t rtr♦♥ s ♦♥sst♥t st♠t♦r ♦ t ♦rrt ♥♠r ♦ ♦♠♣♦♥♥ts ♥ t strt♦♥
♣tr str ♥②ss stt ♦ t rt
rs ♠♣ ∆ s r ♦r t st♠t♦♥ ♦ t ♣r♠trs s ♦♠♣①t♥ t ①st ♣♣r♦ s ♥♦t ♦ ♥ ts ♣♣r♦ ♦♥ssts ♥ ♦♠♣tt♦♥ ♦ ♥ ♥♦r♠t♦♥ rtr♦♥ ♦r t ♠♦s s♦ t s t♠ ♦♥s♠♥rtr♠♦r t ♣rtt♦♥r ♦♥② ss t st♠t ss♦t t♦ t st ♠♦s t ♦tr st♠ts r ♥♦t s ♦r t t ♥②ss s r s♦ ② t ♣♣r♦ ♦ t rrs ♠♣ ❬r ❪ r t ♠♦ ♥t ♣r♠trs r s♠t♥♦s② st♠t ❯♥♦rt♥t② ts ♣♣r♦ ♥♦st ♦♠♣tt♦♥ ♦ t ♣r♦ts ♦ t ♠♦ tr♥st♦♥ ♥ ♦♠♣①♦r ts ♦t ♦♥ssts ♥ ♦♥ t st♠t♦♥ ♦ t ♣r♠trs ♦ t ♠♦s ♥ ∆ t ♥ ♦♠ ♠♥t♦r② ♥ t ♠♦ s♣ ♦♠s s Prt
♥♦r♠t♦♥ rtr♦♥ ♦r t ♣rtt♦♥ st♠♥t
♥ Pr♦①② t ♦♥sst♥② ♦ t ♥♦r♠t♦♥ rtr♦♥ ♥ r ♥ s t ♠♦s r r♦♥ t rtr♦♥ s②♠♣t♦t②♦rst♠ts t ♥♠r ♦ ♦♠♣♦♥♥ts ♦r♥ t♦ t ss s♣rt♦♥ ♦ r♥ ① ♥ ♦rt ❬❪ ♣r♦♣♦s t♦ ♥ sst♦♥♦t ♥ t ♥♦r♠t♦♥ rtr♦♥ ♥ s s t st ♠♦ ♠①♠③s t♥trt ♦♠♣tt ♦♦ s t t♦r z s ♥♥♦♥ t s r♣ ②ts ♠♣ ♥♦t ② z t t t ♠
♥t♦♥ ①t ♥trt ♦♠♣tt ♦♦ sss ♠♦ t sst♦♥ ♠ t s ♥ s
①(m) = ln p(x, z|m)
= ln
∫
θ∈Θ
p(x, z|m,θ)p(θ|m)dθ.
♦r ♥ t ♥tr ♥ ①♣t s ♦r ♥st♥ t st② ♥❬❪ ♦r t ♠ ♦ ♠t♥♦♠ strt♦♥s t s ♥♦t ♥r② t ss ♥ ♣♣r♦①♠t rs♦♥ ♦ ts rtr♦♥ s
♥t♦♥ ♥trt ♦♠♣tt ♦♦ ♥ ♣♣r♦①♠t ②
(m) = ln p(x, z|m, θ)− ν
2lnn.
♦♥s♦♥
♣♣t♦♥ ♦♥ r t st ♦ t ♥♦r♠t♦♥ rtr
❲ ♥ ①♠♣ ♦ t s♥ ♦ t ♥♦r♠t♦♥ rtr ♦♥ t t t st ♦r r♥t ♥♠rs ♦ sss t ♦rt♥ ♣rs♠♦♥♦sss♥ ♠①tr ♠♦s ❬❪ ♦♠♣♦s t st ♦ t ♦♥sr ♠♦s r s♣②s t s ♦ t ♥♦r♠t♦♥ rtr ♦r r♥t♥♠rs ♦ sss ♦♦♦ ♥t♦♥ s ♥rs♥ t t ss ♥♠r rtr♦♥ sts ♦r sss t rtr♦♥ sts tr sss♦r ♦t rtr stt t♦ st t ♥♠r ♦ sss s t♣rtt♦♥r s♥ rs♣t② ♥ s♦ ♥②s t ♣rtt♦♥ ♥tr sss rs♣t② t♦ sss ♥② t rtr♦♥ str♦♥②sts t ♣rtt♦♥ ♥ t♦ sss s ♣rtt♦♥ s♠s rst ♦r♥t♦ t sttr ♣♦t
t t st ♦ st♦♥ ② ♥♦r♠t♦♥ rtr
1 2 3 4 5 6
−12
50−
1200
−11
50
number of classes
crite
rion
valu
es
log−likeaicbicicl
♥♦r♠t♦♥ rtr
1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0
5060
7080
90
eruptions
wai
ting
ttr ♣♦t t t ♣rtt♦♥ ♥ t♦ sssst ②
r rtr♦♥ s ♦r t st ♦ t ♦rt♥ ss♥ ♠①tr ♠♦s♦r r♥t ♥♠r ♦ sss t♦ str t t t st
♦♥s♦♥
s ♦r♣ ♣tr s ♣rs♥t t ♥r r♠♦r ♦ t ♠①tr ♠♦s ♥ s ♥ strt ② t ♦♠♣♦♥♥t ss♥ ♠①tr ♠♦s t s ♣♦♥t♦t tt t ♥r ♠ ♠♦ ♦s t♦ s② str ♦♠♣① t t♦r ♦r ♠① t t t rs ♦ s ♥ t t r♥trss ♦rrt
♦t st♠t♦♥ ♠t♦s rq♥tst ♥ ②s♥ ♥ ♣rs♥t ♥ tstss ♦r t rq♥tst ♣♣r♦ t♦ ♥r t ♣r♠trs s♥ t ♦s ♥♦t
♣tr str ♥②ss stt ♦ t rt
rqr s♦♠ ♥♦r♠t♦♥ ♣r♦r ♦r t ♠ s ♥trt ♣r♦r♠t ♥r♥ ♥ t ②s♥ r♠♦r ♥ s s ♦r t ♦♥t ♣r♦rstrt♦♥s ♥ ♦rr t♦ s② st♠t t ♣r♠trs ②♣r♣r♠trs ♦t ♣r♦r strt♦♥s r st t♦ ② ♥♦r♠t
♠♦ st♦♥ s ♣r♦r♠ ② s♥ t rtr♦♥ s t ♠♦st ♦♠♠♦♥♥♦r♠t♦♥ rtr♦♥ ♦r t ♠①tr ♠♦s
♣r♣♦s ♦ t ♦♦♥ ♣trs s t♦ st② ♥ t♦ ♣r♦♣♦s ♥ ♠①tr♠♦s ♦♥ t♦ str ♦♠♣① t t♦t ss♠♥ t ♦♥t♦♥ ♥♣♥♥ t♥ t rs ❲ ♥♦ ♦s ♦♥ t t♦r t st str♥
Prt
♦s str♥ ♦r
t♦r t
s ♣rt ♦t t♦ t str ♥②ss ♦ t♦rt s s♣t ♥t♦ ♦r ♣trs rst ♦♥ ♣rs♥ts ♥ ♦r ♦ t str♥ ♣♣r♦s ♦t t♦ t t♦r t sts ❲ ♠♥②♦s ♦♥ tr ♠♥ ♠♦s ♣♣r♦s t ♦♥r ♠①tr ♠♦s t ♠①trs ♦ trs ♥ t ♠t t♥t ss ♠♦s s ♠♦s r strt♦♥ s♠ r t st s♦♥ ♥ t tr ♣trs ♣rs♥t ♦r ♦♥trt♦♥s t♦ ts r♠♦r ❲ ♣rs♥t t♦ ♥ ♠①tr♠♦s r ①t♥s♦♥s ♦ t ss t♥t ss♠♦ ♦r s ♠♦s t rs r r♦♣ ♥t♦♦♥t♦♥② ♥♣♥♥t ♦s s♣ strt♦♥s ♦ t ♦s ♠♦③ t ♥trss ♣♥♥ss rsts r ♣rt ♦ t♦ s♠tt rts st ♣tr s ♦t t♦ ♣r♦♣♦s ♠♦ ♦♠♣rs♦♥ strt ♦♥ t ①♠♣ ♦ t ♦r ♣tr s♦♥ ♣r♣♦s ♦ ts ♣tr s t♦ ♣rs♥t ♦r ♣s ♣r♦r♠♥ t ♥r♥ ♦ ♦t ♣r♦♣♦s♠♦s
♦s ♦♥② sr♠♥ ♦ tt t s t t t t trr♦s ♥ tr ♣s r t♥ ②♦trs ♦ r ♦♥♥ tt t
s t t ♦ t♦♥ t♥ ♦rt t
♦ ♦♥t♥ts
str ♥②ss ♦ t♦r t sts stt ♦ t rt
♥ ♦ str ♥②ss ♦r t♦r t
♦♠tr ♣♣r♦s
♦♥r ♠①tr ♠♦s
①trs ♦ trs
t t♥t ss ♠♦
♦♥s♦♥
♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
♥tr♦t♦♥
①tr ♦ ♥trss ♥♣♥♥t ♦s
Prs♠♦♥♦s ♦ strt♦♥
①♠♠ ♦♦ st♠t♦♥ ♦rt♠
♦ st♦♥ ♦rt♠
♠r ①♣r♠♥ts ♦♥ s♠t t sts
♥②ss ♦ t♦ r t sts
♦♥s♦♥
♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
♥tr♦t♦♥
①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s
①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠
♦ st♦♥ tr♦♣♦st♥s s♠♣r
♠r ①♣r♠♥ts ♦♥ s♠t t sts
♥②ss ♦ t♦ r t sts
♦♥s♦♥
♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s
strt
♦♦s
♦ ♦♥t♥ts
♦♥s♦♥ ♦ Prt
♣tr
str ♥②ss ♦ t♦r t
sts stt ♦ t rt
♣r♣♦s ♦ ts ♣tr s t♦ ♣rs♥t t ♠♥ ♣♣r♦s t♦ str t♦r t sts rst st♦♥ s ♦t t♦ t ♦♠tr ♠t♦s♥ ♠♦r ♣rs② t♦ t ♠♥s ♦♥s ♦tr st♦♥s sr t tr ♠♥ ♠♦s♠t♦s ♦r ♣rs② t s♦♥ st♦♥ ♦ss♦♥ t ♦♥r ♠①tr ♠♦s t s♦ ts tss t♥t ss ♠♦ s s♣ ♠♦♦ t ♦♥r ♠①tr ♦♥ ss♠♥ t ♦♥t♦♥♥♣♥♥ t♥ t rs s ♠♦ s ♦♥trst ♦r s s♥ t ♣r♦♣♦s ♠♦s ♥tr♦ ♥t t♦ ♦♦♥ ♣trs ♦♥sst ♥ t♦ ①t♥s♦♥s♦ ts ♠♦ tr st♦♥ s ♦t t♦ t ♣rs♥tt♦♥ ♦ t tr ♠①tr ♠♦s st st♦♥♣rs♥ts t ♠t t♥t ss ♠♦ss ♠♦s r strt ♦♥ r t st tr♦♦t ts ♣tr
②s ♦ s♦r t ②♦ s ②♦♦ r♥ t t ②♦ t♦ ♣
②♦r ♠♦t str♥st ♠♥② ♦rt♣♣② ♦ r♥s ♦♠r
♥ ♦ str ♥②ss ♦r t♦r t
♥tr♦t♦♥ t♦r rs r ♦t♥ ♣rs♥t ♥ t t sts s♥t② r s② ss t② ♥♦ ② s rs s ♦ rst②t s ♥♦t ♦♥♥♥t t♦ s③ t♦r t ♥ tr ♥t s♣ ♦ ts s t♦ ♦♥tr♥ ② ♥ s② ♥tr♣rtt♦♥ ♦ t ♣rtt♦♥ ♦♥② t
♣tr str ♥②ss ♦ t♦r t sts stt ♦ t rt
♦♠♥t♦r ♣r♦♠s r qt♦s ♥ s♦♠ ♥trss ♣♥♥s t♦ ♠♦③ ♥♠r ♦ ♣r♠trs ♥ ♥♠r ♦ ♠♦s ♥ ♦♠♣tt♦♥ s t ♣♣rs ♠♣♦rt♥t ♦r s tt t ♠♦s s t♦ str rs♣t tt♦ ♦♦♥ ♦ts
♦ r ♦ts s ♦♥ t ♠♦s ♣rs♥t ♥ ts ♦r♣♣tr ♣t t t ♦♥ t ♦♦♥ r ♦ts
♦s t♦ ♣r♦ ♠♥♥ ♣r♠trs t♦ ♦♥tr♥ t ♦ s③t♦♥
♦s t♦ t ♥t♦ ♦♥t t ♥trss ♣♥♥s ② ♠t♥ t♦♠♥t♦r ♣r♦♠s rt t♦ ♦t ♦ t ♥♠r ♦ ♣r♠trs ♥ ♦t ♠♦ st♦♥
t r♦♦t ts ♣rt ♦♥sr t drt t♦r ♦ t♦rrs ♥♦t ② xi = (x1
i , . . . ,xdi ) ♥ ♥ ♥ s♣ X t♦r
r xji = (xjhi ;h = 1, . . . ,mj) smj ♠♦ts ♥ ss ♦♠♣t s♥t♦♥ s s xjhi = 1 ♥ i ts ♠♦t② h ♦r r j ♥ xjhi = 0♦trs
trtr ♦ ts ♣tr t♦♥ ♦ss ♦♥ t ♦♠tr ♠t♦s ♣r♠tt♥ t♦ str t♦r t sts ♦tr st♦♥s r ♦t t♦ ♣r♦st♠t♦s ♥ t♦♥ ♣rs♥ts t ♦♥r ♠①tr ♠♦ s t rr♥ t♦ str t♦r t ♥ ♣r♦st r♠♦r t♦♥ ♣rs♥tst ♠①tr ♦ ♣♥♥② trs t♦♥ ♣rs♥ts t ♠t t♥t♠♦s
♥♥♥ ①♠♣ r♥ ts ♣tr t r♥t ♠t♦s ♦♥ t♦ strt♦r t sts r strt ♦♥ t ♥♠♥s ♥tstr② t ❬❪ s ss t♦r t st
s ♥r② t st ♣rs♥t ♥ ♦♥ssts ♥ t ♥♦ss ♥② ♥tsts ♦r ♣r♠♦rs ♥ ♠♦rs ♠ s t♦ rtr③t ♦r ♦ t ♥tsts ♦r♥ t♦ tr ♥♦ss
♥♥♥ ①♠♣ ♥tstr② t st
♦♠tr ♣♣r♦s
t♦s ♦♥ t ♥t s♣
♠♥s ♦rt♠ ♦r t♦r t
♥ ♠♥s ♦rt♠ ♦r t♦r t ♣r♦♣♦s ② ♠♦♥r♥② ❬❪ ss t ♦♠♣t s♥t ♦♥ ♦ t t♦r rs
♦♠tr ♣♣r♦s
♥tst ♥tst rq♥② rq♥②
♦r♣ r♦ss♥♦ss ♦ ♠♦rs ♥ ♣r♠♦rs ② ♥tsts ❬❪ t r ♥♦s s s♦♥ ♦r r♦s
♥ ♦r s ♣♣r♦ xjhi s ♦♥sr s ♥r② r st♥s t♦ str s t sqr ♦♥ tt ♦ t
sqr st♥ sqr st♥ ts ♥t♦ ♦♥t t t ♦ ♠♦t② ♥ t st♥ ♦♠♣t ② ♥ ♠♦r ♠♣♦rt♥ t♦ t rr♠♦ts t♥ t♦ t ♠♦st ♦♠♠♦♥ ♦♥s
♥t♦♥ sqr st♥ t ① = (x1, . . . ,xn) t♦ t s♠♣ ♦♠♣♦s ② n ♥s xi sr ② d t♦r rs sqr st♥ t♥ xi1 ♥ xi2 t 1 ≤ i1, i2 ≤ n s ♥ ②
Dχ2(xi1 ;xi2) =d∑
j=1
mj∑
h=1
(xjhi1 − xjhi2 )2
njh,
r njh =∑n
i=1 xjhi
♦♠♠♥ts ♠♥ r ♦ ts ♣♣r♦ s tt t t♦r ♦ t ss♠♥sr s t♥ ③r♦ ♥ ♦♥♦s ♥♦t ♥t t rtrsts ♦t sss ♦t♥ ♣rtt♦♥ s s♦ ② ♥tr♣rt
♠♦s ♦rt♠ ♦r t♦r t
♥ ♠♦s ♦rt♠ s ♥ ♣r♦♣♦s ② ❩ ♥ ❬❪ t♦♦ t ♣r♦♠ rt t♦ t str ♥②ss ♦ r t♦r t sts s
♣tr str ♥②ss ♦ t♦r t sts stt ♦ t rt
♦rt♠ ①t♥s t ♠♥s ♦♥ ② s♥ s♠♣ ♠t♥ ss♠rt② ♠sr♦r t♦r rs t trt♦♥ t ♣ts t ♠♦s ② rq♥②s ♠t♦ ♥ ♦rr t♦ ♠♥♠③ ♦st ♥t♦♥
ss♠rt② ♥ ♠♦s
♥t♦♥ t♥ ss♠rt② t t♦ drt t♦r rs xi1♥ xi2 ♠t♥ ss♠rt② ♦♥ts t ♠s♠ts t♥ ♦t xi1 ♥xi2 s ss♠rt② s ♥ ②
D1(xi1 ,xi2) =d∑
j=1
δ(xji1 ,xji2) t δ(xji1 ,x
ji2) =
1 xji1 6= xji2
0 xji1 = xji2.
♥t♦♥ ♦ ❬❪ ♠♦ ♦ t s♠♣ ① = (x1, . . . ,xn) s t♦rµ ∈ X s s µ = (µjh; j = 1, . . . , d;h = 1, . . . ,mj) ♠♥♠③s
D(①;µ) =n∑
i=1
D1(xi;µ).
♦t tt µ s ♥♦t ♥ssr② ♥ ♠♥t ♦ ① ♥ tt t s ♥♦t ♥ssr② ♥q
♣t♠③ rtr♦♥ ❲♥ t ss♠rt② ♥ ② ♥t♦♥ s st♥ t ♠♦s ♦rt♠ ♦♣t♠③s t ♦♦♥ rtr♦♥
I(③,θ;①) =n∑
i=1
g∑
k=1
d∑
j=1
δ(xji ,µjk),
r θ = (µ1, . . . ,µg) ♥ r µk s t ♠♦ ♦ ss k
♦rt♠
trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ ss ♠♠rs♣ z
[r] = argminz
I(z,θ[r];x)
z[r]ik =
1 k = argmink′
D1(xi,µ[r]k′ )
0 ♦trs
♥tr♦ st♠t♦♥ θ[r+1] = argminθ
I(z[r],θ;x)
µ[r+1]k = argmin
µk
n∑
i=1
z[r]ikD1(xi;µk).
♦rt♠ ♠♦s ♦rt♠
♦♠tr ♣♣r♦s
♦♠♠♥ts ♠♦s ♦rt♠ t ♠♥s ♦♥ ♦♥rs t♦ ♦♠♥♠♠ ♦ t ♥t♦♥ I(③,θ;①) t s s♦ ♠♥t♦r② t♦ ♣r♦r♠ r♥t ♥t③t♦♥s ♥ ♦rr t♦ ♦♣ t♦ t t ♦ ♠♥♠♠ ♦ ts ♥t♦♥ ♥② ♥♦ttt t ♥tr♦ st♠t♦♥ st♣ s tt ② t ♥t♦♥ ♦ D1(., .) ♥ts ♦♣t♠③t♦♥ s ♣r♦r♠ ♦♦r♥ts ② ♦♦r♥ts
♣♣r♦ s ♦♥ t rtr♦♥ s ♦♠♣t ♦r r♥t ♥♠rs♦ sss t♦ ♣rtt♦♥s ♦ ♦ ♥trst
♥tr♣rtt♦♥ rst ♦♥ s♣ts t t ♥t♦ t♦ sss ♦s t♠♦s r ♥ ② ♦t ♠♦st ♣rs♥t ♥♦ss ♥tsts ♠ ttt t♦♦t s s♦♥ ♥ ♥tsts ♠ tt t t♦♦t s s♦♥ ①♣t tst ♥tst s♦♥ ♦♥ s♣ts t t ♥t♦ tr sss t st t t♦ ♣r♦s ♠♦s t ♥♦ss r ♥tsts ♠ tt tt♦♦t s r♦s ①♣t t rst ♥tst s② t ♠♦s ♣♣r♦♦s ♥♦t ♦ t♦ r② ♥rst♥ ts t st
♥♥♥ ①♠♣ ♠♦s str♥
t♦s ♦♥ t t♦r s♣
♥ ❲♥ ♠♥② rs r ♦rrt t② ♣r♦ s♦♠ r♥♥t ♥♦r♠t♦♥ s t ♥ ♥t t♦ ♣r♦r♠ st♦♥ ♦ rs ♦r rt♦♥♦ t s♣ ♠♥s♦♥ ❲♥ t rs r t♦r t♣ ♦rrs♣♦♥♥ ♥②ss ♥ s ♥ ♦rr t♦ r t s♣ ♠♥s♦♥ ♥ts ♠t♦ ♣r♦s ♥♠r ♦♦r♥ts ♦r ♥ r♦r t s♣♦ss t♦ s ss ♦♠tr ♣♣r♦ t♦ str ♥♠r t t ♠♥s ♦rt♠ ❲ ♣rs♥t t ♠t♦ ♦ ♥ ❲ ♦♥ ♥ ❨ ♥❬❪ ♦♠♥s ♥ ♠♥s ♦rt♠ ♥ ♥ r♠♦r
♦tt♦♥s ❲ r♠♥ tt t s♠♣ ① = (xi; i = 1, . . . , n) s ♦♠♣♦s t♥s sr ② d t♦r rs s s♥t ♦♥ tf ♥♦t♥ t ♠tr① ♦ s③ n× d r d ≤ mj ♦rrs♣♦♥s t♦ t d♠♥s♦♥r♣rs♥tt♦♥ ♦ t d t♦r rs t wj t ♠tr① ♦ ts ♦ s③mj × d ❲ ♦♥sr ③ s t ♠tr① ♦ s③ n× g r t r♦s ♦rrs♣♦♥ t♦ t♥s ♥ t ♦♠♥ t♦ t ss ❲ ♥♦t ② θ t ♠tr① ♦ t ♥tr♦s ♦ t str ♥ t t♦r s♣
♣t♠③ rtr♦♥ ♠ s t♦ ♦♠♥ ♥ ♠♥s ♦rt♠ s♦t ♣r♦♠ s q♥t t♦ t ♠♥♠③t♦♥ ♦ t ♦♦♥ rtr♦♥
Iα1,α2(③,θ;①) = α1
d∑
j=1
SS(f − ①jwj) + α2SS(f − ③θ),
♣tr str ♥②ss ♦ t♦r t sts stt ♦ t rt
r α1 > 0 α2 > 0 α1 + α2 = 1 r ①j s t ♠tr① r t ♠♥t (i, h)s q t♦ ♦♥ ♥ i ts ♠♦t② h ♦r r j ♥ s q t♦ ③r♦♦trs ♥ r SS(f) = tr(f ′f)
♠r ♥ t ♦♣ α1, α2) ❲♥ α1 = 1 t rtr♦♥ ♥ ② st st♥r ♦♥ ♦r ❲♥ α2 = 1 t rtr♦♥ ♥ ② s q♥tt♦ t st♥r ♦♥ ♦r t ♠♥s ♦rt♠ s ♦r ♦trs s ♦ (α1, α2)ts rtr♦♥ ♣r♦r♠s tr♦ t♥ t ♥ t ♠♥s ♦ts
♦rt♠ st♠t♦♥ ♦ (f ,wj, ③,θ) s ♣r♦r♠ ② t tr♥t♥st sqrs ♦rt♠ ♣r♦♣♦s ② ❬❪ s ♦rt♠ ♦♥rs t♦ ♦ ♠♥♠♠ ♦ t ♥t♦♥ Iα1,α2(③,θ;①) ♦ sr r♥t ♥t③t♦♥s ♦ts ♦rt♠ t♦ ♦♥ ♥ ♦rr t♦ ♦t♥ t st♠t♦rs ♠♥♠③♥ tsrtr♦♥
trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ ❲t ♠tr① ♥ ♥tr♦s ♦♣t♠③t♦♥
wj[r] = (①j′①j)−1①j
′f ♥ θ[r] = (③[r]
′③[r])−1③[r]
′f .
t♦r s♣ ♦♣t♠③t♦♥
f [r+1] = argmaxf
tr
(
f ′
[
α1
d∑
j=1
①j(①j′①j)−1①j
′+ α2③
[r](③[r]′③[r])−1③[r]
′
]
f
)
.
Prtt♦♥ ♦♣t♠③t♦♥
③[r+1] = argmin③
SS(f [r+1] − ③θ[r]).
♦rt♠ ♦rt♠ ♠♥♠③♥ Iα1,α2(③,θ;①) ❬❪
♦♠♠♥ts ♥ t♦♥ t♦ t ss ♠ts ♦ t ♦♠tr ♣♣r♦s tr♣r♦♠s r rs ② ts ♠t♦
rst ♣r♦♠ s ♦t t ♣r♠trs (α1, α2) r ① ② tsr ♥ tr s ♥♦ r ♥t② tr♠♥s t♠ tr♠♣ts ♦♥ t ♣rtt♦♥ r s♥♥t
s♦♥ ♣r♦♠ s ♦t t s③ ♦ t t♦r s♣ ♥ t s♣♠♥s♦♥ d s rtrr② ① ② t sr t rs ♦ ♦♦s♥ ♥♦r♠t♦♥
st ♣r♦♠ s ♦t t ss ♥tr♣rtt♦♥ s ♦♠♣① ♥t sss r s♠♠r③ ② t ♥tr♦s r ♥♦t ♥ ♥ t♥t s♣ t ♥ t t♦r s♣ rt ② ♦♠♥t♦♥s ♦ t ♦r♥rs
♦♥r ♠①tr ♠♦s
♦♥r ♠①tr ♠♦s
♥ ❲ s♥ ♥ ♣tr tt t ss♥ ♠♦ ♥ ss ♦♠♣♦♥♥t strt♦♥ ♥ t rs r ♥♠r ♥ t s♠ ② t♦♥r ♠♦ s t♦r t ♥②ss ② rst ❬r❪ s ♥tr②s s ♦♠♣♦♥♥t strt♦♥ ♥ t rs r t♦r ♦r t♦♠♣t ♦♥r ♠♦ st♠ts t ♣r♦t② ♦ t ♠♦t② r♦ss♥s ts s♦ ♠♥t♦r② t♦ ♠♣♦s ♦♥str♥ts ♦♥ ts ♠♦ t♦ t ♦♥r ♠①tr♠♦
trtr ♦ ts st♦♥ ss ♣♣r♦ ss♠s ♦♥t♦♥ ♥♣♥♥ t♥ t rs s ♠♦ s ♥♠ t♥t ss ♠♦ ♦r ♥ ②s❬♦♦❪ ♥ s ♥ t ♥ t♦♥ tr ♦♥r ♠①tr ♠♦s r① t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ r ♣rs♥t ♥ t♦♥
t♥t ss ♠♦
♦ ♣rs♥tt♦♥
♥ s ♠①tr ♠♦ ss♠s tt t rs r ♥♣♥♥t ♦♥t♦♥② ♦♥ ss ts ts ♦♠♣♦♥♥ts ♦♦ ♣r♦t ♦ ♠t♥♦♠ strt♦♥s
♥t♦♥ t♥t ss ♠♦ t xi t drt t♦r rs♥ s♥t ♦♥ xi rs r♦♠ t t♥t ss ♠♦ t g ♦♠♣♦♥♥tst♥ ts ♣ s rtt♥ s ♦♦s
p(xi;θ) =
g∑
k=1
πkp(xi;αk) t p(xi;αk) =d∏
j=1
mj∏
h=1
(αjhk )xjhi ,
r θ = (π,α) r π s ♥ ♦♥ t s♠♣① ♦ s③ g rα = (α1, . . . ,αg)
♥ r αk = (α1k, . . . ,α
dk) s s tt αj
k = (αjhk ;h = 1, . . . ,mj) s ♥ ♦♥t s♠♣① ♦ s③ mj ♦t tt αjhk ♥♦ts t ♣r♦t② tt ♥ ♥rs♥ r♦♠ ♦♠♣♦♥♥t k ts ♠♦t② h ♦r r j
s♣t ts s♠♣t② t t♥t ss ♠♦ s t♦ ♦♦ rsts ♥ ♣rt❬❨❪ ♦r r♥t rs t ♦r s♥s ❬+❪ ♦r ♥ ♠♥❬+❪
♦ ♥tt② ♥r ♥tt② ♦ t t♥t ss ♠♦ s♣r♦ ② ♠♥ ts ♥ ♦s ❬❪ ❲ ♥♦ ♣rs♥t trt♦r♠ rr ♥trst ② ts ♣r♦♦ ♥ rr t♦ t rt ❬❪
♦r♠ ♥r ♥tt② ♦ t t♥t ss ♠♦ ❬❪ t t♠♦ ♥ ② ♥t♦♥ t d ≥ 3 ♣♣♦s tr ①sts tr♣rtt♦♥ ♦t st S = 1, . . . , d ♥t♦ tr s♦♥t ♥♦♥♠♣t② ssts S1 S2 S3 s tt κb =
∏
j∈Sbmj t♥
min(g, κ1) + min(g, κ2) + min(g, κ3) ≥ 2g + 2.
♣tr str ♥②ss ♦ t♦r t sts stt ♦ t rt
♥ ♠♦ ♣r♠trs r ♥r② ♥t ♣ t♦ s♣♣♥ ♦r♦r t stt♠♥t r♠♥s ♥ t ♠①♥ ♣r♦♣♦rt♦♥s π r ① ♥♣♦st
♥s t t ♦♠tr ♣♣r♦s s s♦♥ ♥ ❬♦❪ t ♦♠tr♣♣r♦ ♦♦♥ ♦r t ♣rtt♦♥ ♥t♦ g sss ♠①♠③♥ t ♥♦r♠t♦♥ rtr♦♥♦r t χ2 rtr♦♥ s ♣♣r♦①♠t② q♥t t♦ ss♠ tt ♥s r r♥② t♥t ss ♠♦
Pr♠tr st♠t♦♥
♥r♥ ♦ t t♥t ss ♠♦ ♥ ♣r♦r♠ ♥ rq♥tst ♦r ♥ ②s♥ r♠♦r
♥ rq♥tst ♣♦♥t ♦ t st♠t♦♥ ♦ t ♠ ♥ ♣r♦r♠ ♥ ♠ ♦rt♠ ♣rs♥t ♦ ♦r ② ts ①t♥s♦♥s ♦t tt t ♦♦♥t♦♥ s ♣♣r♦♥ s♦ tr s ♥♦ ♥r② ♣r♦♠
♥ ②s♥ r♠♦r t st♠t♦♥ ♥ ♣r♦r♠ ② s s♠♣r♦t tt ② ♦♦s♥ t r②s ♥♦♥ ♥♦r♠t ♦♥t ♣r♦rs t ♣♦str♦rstrt♦♥s r ①♣t ♥ ♥ ①t ♥♦r♠t♦♥ rtr♦♥ ♥ ♦♠♣t
❲ ♥♦ t ♦t rq♥tst ♥ ②s♥ ♣♣r♦s
rq♥tst r♠♦r ♠ ♥ s② ♦t♥ ② t ♦♦♥ ♠
♦rt♠
trt♥ r♦♠ t ♥t ♦ θ[0] trt♦♥ [r] ♦ t ♠ ♦rt♠ srtt♥ s
st♣ t ♦♥t♦♥ ♣r♦ts
tik(θ[r]) =
π[r]k p(xi;α
[r]k )
p(xi;θ[r])
.
st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦
π[r+1]k =
n[r]k
n♥ αjh[r+1]
k =
∑ni=1 tik(θ
[r])xjhi
n[r]k
,
r n[r]k =
∑ni=1 tik(θ
[r])
♦rt♠ ♦rt♠ ♦r t t♥t ss ♠♦
②s♥ r♠♦r ss ss♠♣t♦♥ ♦ t ♥♣♥♥ t♥ t♣r♦r strt♦♥s ♦ t ss ♣r♦♣♦rt♦♥s π ♥ ♦ t ss ♣r♠trs αj
k ♥♦s
♦♥r ♠①tr ♠♦s
tt
p(θ) = p(π)
g∏
k=1
d∏
j=1
p(αjk).
s t r②s ♥♦♥ ♥♦r♠t ♣r♦r ♦r ♠t♥♦♠ strt♦♥ s ♦♥trt ♦♥ t ♣r♦r strt♦♥ s rtt♥ s ♦♦s
p(π) = Dg
(
1
2, . . . ,
1
2
)
♥ p(αjk) = Dmj
(
1
2, . . . ,
1
2
)
.
♥r♥ s s♦ ♠ ② t ♦♦♥ s s♠♣r ♥rts sq♥ ♦ ♣r♠trs r♦♠ tr ♣♦str♦r strt♦♥s ♦t tt ts ♦rt♠s s② ♣r♦r♠ s♥ ♦♥t ♣r♦r strt♦♥s ♥♦ ①♣t ♣♦str♦r strt♦♥s
trt♥ r♦♠ t ♥t ♦ θ[0] trt♦♥ [r] ♦ t s s♠♣r♥ p(θ, ③|①) s stt♦♥r② strt♦♥ s rtt♥ s
∀i = 1, . . . , n z[r]i |xi,θ[r] ∼ Mg
(
ti1(θ[r]), . . . , tig(θ
[r]))
π[r+1]|z[r] ∼ Dg
(
1
2+ n
[r]1 , . . . ,
1
2+ n[r]
g
)
∀(k, j) αj[r+1]k |x, z[r] ∼ Dmj
(
1
2+ n
j1[r]k , . . . ,
1
2+ n
jmj [r]k
)
,
r ♥[r]k =
∑ni=1 z
[r]ik ♥ ♥jh[r]k =
∑ni=1 z
[r]ik x
jhi
♦rt♠ s s♠♣r ♦r t t♥t ss ♠♦
①t rtr♦♥ ② s♥ t ♣r♦♣rts ♦ t ♦♥t ♣r♦r strt♦♥s❬❪ ♣r♦♣♦s ♥ ①t rs♦♥ ♦ t rtr♦♥ ♦r t t♥t ss ♠♦♥ t ♥trt ♦♠♣tt ♦♦ ♦ ts ♠♦ ♥ ②
p(①, ③) =
∫
θ∈Θ
p(①, ③;θ)p(θ)dθ,
s ①♣t ② s♥ t ♣r♦r strt♦♥s ♥ ♥ ♦r ♥② ♦♣ (①, ③)t ♥trt ♦♠♣tt ♦♦ s q t♦
p(①, ③) =Γ(g
2)
Γ(12)g
∏gk=1 Γ(♥k +
12)
Γ(n+ g2)
g∏
k=1
d∏
j=1
Γ(mj
2)
Γ(12)mj
∏mj
h=1 Γ(♥jhk + 1
2)
Γ(♥k +♠j
2)
.
❱t♦r ③ s r♣ ♥ t ♦ qt♦♥ ② ts ♠①♠♠ ♦♦ st♠t ③② s♥ t ♠♣ r ♥ t ①t rtr♦♥ s ♥ s ♦♦s
① = ln p(①, ③).
♣tr str ♥②ss ♦ t♦r t sts stt ♦ t rt
♥ ❬❪ t t♦rs ♣r♦♣♦s t♦ s ♥ ♦rr t♦ ♦♠♣t t ♦♠♣tt ♦♦ ② s♥ ♠♣♦rt♥ s♠♣♥ ♣♣r♦ ② ♥r♥ ② tr♥♠r ①♣r♠♥ts tt t ①t rtr♦♥ ♦t♣r♦r♠s t ss s②♠♣t♦t♥♦r♠t♦♥ rtr ♥ s ♥ t ①t rtr r t② t♦ ♦r
♣♣r♦ ♠ r st♠t ♦r r♥t ♥♠rs ♦ sss ♥t rtr♦♥ s s t♦ st t st ♥♠r ♦ sss
♥tr♣rtt♦♥ st ♠♦ s t t♥t ss ♠♦ t tr ♦♠♣♦♥♥ts st♠t sss ♥ ♥tr♣rt s ♦♦s
♠♦rt② ss π1 = 0.72 r♦♣s t tt ♥♦s s s♦♥t str♦♥ ♣r♦t② ② t ♥tsts s ♣r♦t② s♣♣r t♥ ♦r t rst ♦r ♥tsts ♥ q t♦ ♦r tst ♦♥
s♦♥ ss π2 = 0.20 r♦♣s t tt ♠ s s♦♥ ②t rst ♦r ♥tsts t ♠♦r ♥rtt t♥ ♥ t ♣r♦s ss♣r♦t② t♥ ♥ t② r ♠ s r♦s② t st ♥tst t ♣r♦t②
tr ss π3 = 0.08 r♦♣s t tt ♠♥② r sr♦s s♣② ② t t ♥tst
♥♥♥ ①♠♣ t♥t ss ♠♦ str♥
Prs♠♦♥♦s rs♦♥s ♦ t t♥t ss ♠♦
♥♠r ♦ ♣r♠trs rqr ② t t♥t ss ♠♦ s q t♦
(g − 1) + gd∑
j=1
(mj − 1).
s ts ♥♠r s ♥r② str♦♥② s♠r t♥ t ♥♠r ♦ ♣r♠trsrqr ② t ♦♥r ♠♦ s q t♦
∏dj=1mj
♦r ttr sr♥ tr ♦ ♥ ♦t♥ ② r♥ t ♥♠r♦ ♣r♠trs ♦r t t♥t ss ♠♦ s ♣rs♠♦♥♦s rs♦♥s ♦ tt♥t ss ♠♦ s ♥tr♦ ② ① ♥ ♦rt ❬❪ ♦r ♥r②rs t♥ ts ♠♦s s ①t♥ t♦ t t♦r rs ❬♦❪ ♦♥str♥ts ♦♥ t ♣r♠tr s♣ rqr t ♥tr♦t♦♥ ♦ ♥ ♠♦♣r♠tr③t♦♥ ❲t ts ♥ ♣r♠tr③t♦♥ t ♠t♥♦♠ strt♦♥ ♦r j ♦r ♦♠♣♦♥♥t k s tr♠♥ ② ts ♥tr a
jk ♥♦t♥ t ♠♦rt②
♠♦t② ♥ ts s♣rs♦♥ ♣r♠tr εjk
♥t♦♥ tr♥t ♣r♠tr③t♦♥ ♦ t ♣rs♠♦♥♦s t♥t ss ♠♦ t♥t ss ♠♦ ♥ ♣r♠tr③ s ♦♦s
p(xi;θ) =n∑
k=1
πk
d∏
j=1
(
(1− εjk)ajhk (εjk/(mj − 1))1−a
jhk
)xjhi,
♦♥r ♠①tr ♠♦s
s t 0 < εjk < 1 t ♣rs♠♦♥♦s ♠♦ ss♠s tt ♦♥ ♠♦ ♦rrs♣♦♥♥ t♦ t ♠♦st ② ♠♦t② s rtrst ♦r ♠t♥♦♠ tr♠♥♥ ♣r♦t② ♠ss s ♥♦r♠② s♣r ♠♦♥ t ♦tr ♠♦ts s♠♦ rqrs (g−1)+gd ♣r♠trs ♦tr ♣rs♠♦♥♦s ♠♦s r ♦t♥② ss♠♥ t qt② ♦ εjk t♥ t ss ♦r t♥ t rs ♦r t♥t ss ♥ t rs
♠ts ♦ t t♥t ss ♠♦
t♥t ss ♠♦ ♠② sr r♦♠ sr ss ♥ t t r ♥trss ♦rrt ♦r ♥st♥ ♥ ♣♣t♦♥ ♣rs♥t ♥ ❬❱❪ s♦s tt t♥tss ♠♦ r♠t② ♦rst♠ts t ♥♠r ♦ sss ♥ t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ s ♦t ❲ ♥♦ ♣rs♥t tr tr♥t ♠①tr♠♦s r①♥ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♦t tt t rr st ♥♠r ♦ rs t r s t rs t♦ ♦sr ♦♥t♦♥② ♦rrtrs ♥ t st ♥ ♦♥sq♥t② t r s t rs t♦ ♥♦ s ss② s♥ t t♥t ss ♠♦
♦♥r ♠①tr ♠♦s t ♥trss ♣♥♥
s
♥ ♦♥r ♠♦s ❬r❪ ♣r♣♦s s t♦ ♠♦③ t ♥ ♦♣r♦t② ② st♥ ♥trt♦♥s t♥ rs s t ♦♥r ♠①tr♠♦ s ♥ s ♦r ♦♥ t♠ ❬r ❪ t♦ str t♦r t stt ♥trss ♣♥♥s ♦t tt s♦♠ ♦♥str♥ts t♦ ♠♣♦s ♦♥ ♦♥r ♠♦ ♥ ♦rr t♦ ♦t♥ t ♠♦ ♥tt②
♣tr str ♥②ss ♦ t♦r t sts stt ♦ t rt
♣♣r♦ s♣♥ ♥ ♥♠♥ ❬❪ ♣♣② ♦♥r♠①tr ♠♦ t♦ t t t ♦t tt t♦rs st♠t sr ♠♦s① ② ♥ ♦s t st ♦♥ ♦♥srs ♠①tr t ♦r♦♠♣♦♥♥ts
♥tr♣rtt♦♥ rst t♦ ♦♠♣♦♥♥ts t ♥t♦ ♦♥t t ♥trt♦♥s t♥ t ♥tsts ♥ st t♦ ♦♠♣♦♥♥ts r s♣s♥ tr ♦ ♦♥② ♦♥ ♠♦t② ♥trt♦♥ ♥ t ♥♦ss rrs♣t② r♦s ♥ s♦♥
♦♠♠♥ts ♦t tt ts ss♠♣t♦♥s r rqr ② t t♦rs t♦ tr rst ♥tr ♥ ts ♠♦ ts t t ttr t♥ t♠ ♠♦ ♥ t ♦tr ♥ ts ♥tr♣rtt♦♥ ♥s t ♥②ss ♦ ♦rsss s♦ t t s♠♠r② s ♠♦r ♦♠♣① ♥② ♦ rt③t ♥ ♦ t t♦ s♣ sss ♠♦♥ ♦♥② ♦♥ ♠♦t② r♦ss♥♥ ts sss ♣♣rs s rt② ♥ ♦rr t♦ ♠♦③ t♦♥t♦♥ ♣♥♥s
♥♥♥ ①♠♣ ♦♥r ♠①tr ♠♦ str♥ ❬❪
♦ st♦♥ ② ♦♥sr♥ ♥trss ♣♥♥② ♦ ♦rr ♦♥ t t♦rs♦ ❬❪ ♦t♥ ♦♦ rsts ♦r t str♥ ♦ r♦r♣ r♦ss♥♦stss t♦rs ♣r♦r♠ t ♠♦ st♦♥ ② s♥ ♦rr ♠t♦ tr♠♥s t ♥trss ♥trt♦♥ ♦r ♥♦t tt ts ♣♣r♦ s s♦♣t♠♥ ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ t ♥♦r♠t♦♥ rtr♦♥ s ② t ♣rtt♦♥r ♠♦ ♣rs♥t ♥ ❬❱❪ ♦♥srs t ♥trt♦♥s ♦ ♦rr t♦t r♥ ♥ t ♣♣t♦♥ tr r ♦♥② ♥trt♦♥s ♦ ♦rr ♦♥ rst♠t s ♦r t ♣r♦s rt t♦rs t♦ tr♠♥ ② ♥ t♥trss ♥trt♦♥s ♠♦ st♦♥ ♦r t ♦♥r ♠①tr ♠♦s s ♦♠♣① ♣r♦♠ s♥ t ♥♠r ♦ ♠♦s ♦♠s t t ♥♠r ♦rs
♦♦ ♠♥② ♣r♠trs ♥♠r ♦ ♣r♠trs rqr ② t ♦♥r♠①tr ♠♦ ♥rss t t ♥♠r ♦ ♠♦ts ♥ t t ♦♥sr♦rr ♦ ♥trt♦♥s s ts ♠♦ ♥ t t t t t ♠② ♥ t♦♦♠♥② ♣r♠trs ♦ tr s ♥ ♦rtt♥ rs ♥ t ♥tr♣rtt♦♥ ♦♠srr rtr♠♦r t ♣r♠trs ♥ ♣♦♦r② ♠♥♥ tr r t♦♦♥♠r♦s
♦♥s♦♥ ♦♥r ♠①tr ♠♦ s ♣♦r t♦♦ t♦ str t♦r t ♦r t s ♠♣♦rt♥t t♦ ♠♣♦s ♦♥str♥ts ♦♥ t ♣r♠trs s♣♥ ♦rr t♦ ♣r♦ ♠♥♥ ♠♦ ♦t ♠①tr ♠♦s ♣rs♥t ♥ ♣tr ♥ ♣tr ♥ ♥tr♣rt s ♦♥r ♠①tr ♠♦s t s♣
①trs ♦ trs
♦♥str♥ts ♦♥tr♦ t ♥♠r ♦ ♣r♠trs ♥ ♣r♦ ♠♥♥sss ♦t ♠♦s r ♥ t ♥ ♥t ♣♣r♦ t♦ ♣r♦r♠ t ♠♦st♦♥ ♥ ②s♥ r♠♦r
①trs ♦ trs
♣♥♥ trs
♥ s ♣♣r♦ ♣r♦♣♦s ② ♦ ♥ ❬❪ ♦♥ssts ♥♣♣r♦①♠t♥ srt ♠trt ♣r♦t② strt♦♥ t ♣♥♥ trs t ♣r♦t ♦ s♦♥♦rr strt♦♥s
♥t♦♥ P ♦ ♣♥♥ tr strt♦♥ t t tr T = E, V r E = 1, . . . , d ♥ V = (j, j′) : j ∈ E ♥ j′ ∈ E \ j r xi ss♠♣ r♦♠ ♣♥♥ tr strt♦♥ ♥ ② T t♥ ts ♣ s rtt♥s ♦♦s
p(xi;α) =
∏
(j,j′)∈V p(xji ,x
j′
i ;βjj′)
∏dj=1 p(x
ji ;α
j)vj−1,
r α = (αj,βjj′
; j = 1, . . . , d; j′ s s (j, j′) ∈ V ) r vj ♥♦ts t r♥ ♦ t ♥♦r ♦ j ♣ ♦ ♦♠♣♦♥♥t k s ♥ ②
p(xji ;αj) =
mj∏
h=1
(αjh)xjhi ♥ p(xji ,x
j′
i ;βjj′) =
mj∏
h=1
mj′∏
h′=1
(βjj′h′h′)x
jhi xj
′h′
i ,
t αj = (αjh;h = 1, . . . ,mj) ♥ βjj′
= (βjj′hh′ ;h = 1, . . . ,mj;h
′ = 1, . . . ,mj′) ♣r♠tr αjh ♥♦ts t ♣r♦t② tt r j ts ♠♦t② h ♥ t♣r♠tr βjj
′hh′ ♥♦ts t ♣r♦t② tt t ♦♣ ♦ rs (j, j′) tst ♦♣ ♦ ♠♦ts (h, h′)
st♠t♦♥ s s♦♥ ♥ ❬❪ t ♠①♠♠ ♦♦ st♠t ♥ rt②♦t♥ ② s♥ t rs ♦rt♠ st♠ts t tr ♦ ♠♥♠ ♥t❬r❪ ♦ t r♥ t t♥ t t♦ r♥♦♠ rs Xj ♥Xj′ s ♥ ② t ♠t ♥♦r♠t♦♥ ♥ s
I(Xj,Xj′) =
mj∑
h=1
mj′∑
h′=1
p(Xjh = 1, Xj′h′ = 1) lnp(Xjh = 1, Xj′h′ = 1)
p(Xjh = 1)p(Xj′h′ = 1).
r♦♠ ts ♥t♦♥ t ♠♣r ♠t ♥♦r♠t♦♥ s ♦r s♠♣ ①
♥t♦♥ ♠♣r ♠t ♥♦r♠t♦♥ ♠♣r ♠t ♥♦r♠t♦♥ t♥ ①j = (xji ; i = 1, . . . , n) ♥ ①j
′= (xj
′
i ; i = 1, . . . , n) ♦♠♣t r♦♠ ts♠♣ ① s ♥ s
I(①j,①j′
) =
mj∑
h=1
mj′∑
h′=1
f(xjh, xj′h′) ln
f(xjh, xj′h′)
f(xjh)f(xj′h′),
r f(xjh, xj′h′) = 1
n
∑ni=1 x
jhi x
j′h′
i ♥ f(xjh) = 1n
∑ni=1 x
jhi
♣tr str ♥②ss ♦ t♦r t sts stt ♦ t rt
♦♠♣t I(①j,①j′
i ) ∀(j, j′) ♥① t d(d − 1)/2 r♥s ♦r♥ t♦ tr t ♦ t
t bℓ s rtr t♥ ♦r q t♦ t t bℓ′ ♥r j < j′
t b1 ♥ b2
♦r ℓ = 3 t♦ d(d−1)/2 t r♥ bℓ t ♦s ♥♦t ♦r♠ ②t t st ♣r♦s② st
♦rt♠ st♠t♦♥ ♦ t ♣♥♥ tr
♠r ❯♥q s♦t♦♥ t ts r r♥t t♥ t s♦t♦♥♦ ♦rt♠ s ♥q
r ♠①tr ♠♦
♥ s ♣♣r♦ ♣r♦♣♦s ② ♥ ♦r♥ ❬❪ ♥r③s t ♣r♦st trs t♦ t ♠①tr ♠♦ r♠♦r t♦rs ss♠tt ♦♠♣♦♥♥t ♦♦s strt♦♥ ♣r ♣♥♥ tr ♥ ♥
st♠t♦♥ ♥ rq♥tst r♠♦r t ♠ s s② ♦t♥ ② ♥ ♠
♦rt♠ ♠ st♣ ♠①♠③s t ①♣tt♦♥ ♦ t ♦♠♣t ♦♦ ②s♥ ♦rt♠ r t ♠♣r ♠t ♥♦r♠t♦♥ s ♦♠♣t ♦r♥t♦ t ♦♥t♦♥ ♣r♦ts ♦ t ss ♠♠rs♣s ♥ ②s♥ r♠♦rt ♠♣ s s♦ ♦t♥ ② s♣ ♠ ♦rt♠ ♠①♠③♥ t ♣♦str♦rstrt♦♥
♣♣r♦ ❲ str t t st t ♠①tr ♠♦s ♦ ♣♥♥②trs t r♥t ♥♠rs ♦ sss ♥ s t rtr♦♥ t♦ stt st ♦♥
♥tr♣rtt♦♥ st ♠♦ s t ♦♠♣♦♥♥t ♦♥ ♦t tt ts♠♦ rqrs t st♠t♦♥ ♦ ♣r♠trs t t♥t ss ♠♦rqrs ♦♥② ♣r♠trs ts rtr♦♥ s ttr t♥ t♦♠♣♦♥♥t t♥t ss ♠♦ rs♣t② ♥ ts ♦rst s ♥♦t ttr s♥ t tr♦♠♣♦♥♥t t♥t ss ♠♦ ♦t♥s rtr♦♥ ♦ t rtr♦♥ s r rt② ♦st ♣rtt♦♥s r r♥t s s♦♥ ② t ♦♥s♦♥ ♠trs ♣rs♥t ♥
♥♥♥ ①♠♣ r ♠①tr ♠♦ str♥
t t♥t ss ♠♦
tr tr
tr trtr tr tr
♦♥s♦♥ ♠trs t♥ t ♣rtt♦♥ ♦t♥ ② t ♦♠♣♦♥♥t♠①tr ♦ trs ♥ t ♣rtt♦♥ ♦t♥ ② t ♦♠♣♦♥♥t t♥t ss♠♦ t tr♦♠♣♦♥♥t t♥t ss ♠♦
♦♥s♦♥ ♠♥ ♣r♦♠ ♦ ts ♠♦s s tt t② rqr t♦♦ ♦t♥ ♥♥trt ♥♠r ♦ ♣r♠trs rtr♠♦r t tr strtr s ♦t♥ ♥st♥ t t st s tt t ♥ t♥ t tr strtr ♥ r②r♥t s t ♥tr♣rtt♦♥ s ♦♥ ts strtr ♥ rr♥t ♥②♥♦t tt t ♠①trs ♦ trs r ♠♥♥ ♣r♥♣② ♥ t tr strtr①♣♥s s♦♠ s rt♦♥s♣s
t t♥t ss ♠♦
♥ s t♦ ♦♥sr t♦ t♥t rs rst ♦♥ s t♦r♥ s rt t♦ t ss ♠♠rs♣ s♦♥ ♦♥ s ♦♥t♥♦s ♥rt♦r ♠trt ♥ ♠♦③s t ♥trss ♣♥♥s
r ♦ ts ♠t♦s ❲♥ ♦rts r t ♦♥t♦♥♣♥♥s t♥ t t♦r ♦♥s ♥ ♠♦ ② ♦st ♥t♦♥❬♦r ❲❪ ② ss♠♥ tt ts ♦rts r ♥♦sr t ♠tt♥t ss ♠♦ ❬❱r ❱r❪ ♥tr② ♥♦r♣♦rts t ♥trss ♣♥♥ss ♠♦ s ♦♥♥t♦♥s t t ♣♣r♦ ♦ ❨ ♥ ♥ t♥r ❬❪ r t ♥trss ♣♥♥s r ♠♦ ② t♥t ♦♥t♥♦sr t ♣r♦t ♥t♦♥ ②r ♠♦ ❬t❪ ♥ ♦r ss t♦r ♥②ss ♠♦ s tt t♦ tr t♦r rs ♦r t♦ t♦s t♦r rs ♥ ♣♥♥s s ♠♦r ♥r ♣♣r♦ ♥t② ♦♥♥ r♣② ❬❪ ♣r♦♣♦s t ♠①tr ♠♦ ♦ t♥t trts ♥②③rs ss♠s tt t strt♦♥ ♦ t t♦r rs ♣♥s ♦♥ ♦t t♦r t♥t r t ss ♥ ♠♥② ♦♥t♥♦s t♥t trts rs ♥r♥ s s♦ t ♣♦♥t s s♦ rt♦♥ ♣♣r♦ ts ♠♦s ♦♥sr t ♥trss ♣♥♥s tr ♠♥ r s ttts ♣♥♥s t♦ ♥tr♣rt ♠♦♥ rt♦♥s t t♥t rs ♣rt♥♥t ♥tr♣rtt♦♥ ♥ t
♦s ♦♥ t ❬❪ ♣♣r♦ ♠t t♥t ss ♠♦ ♣r♦♣♦s♥ ❬❪ s ♥tr♦ t♦ ♥②③ ♥r② t sts t ①♣♥s t ♥trss♣♥♥s ② ♦t ♥t♦♥
♥t♦♥ ❬❪ ♠t t♥t ss ♠♦ ♣ ♦ ♦♠♣♦♥♥t k
♣tr str ♥②ss ♦ t♦r t sts stt ♦ t rt
s rtt♥ s
p(xi;αk) =
∫
R
d∏
j=1
Φ(akj + bkjt)xj1i (1− Φ(akj + bkjt))
1−xj1i dΦ(t),
r Φ(.) ♥♦ts t ♠t strt♦♥ ♥t♦♥ ♦ st♥r ♥♦r♠ r
♥ ♣rt ts ♣ s ♣♣r♦①♠t ② s♥ t ssr♠t qrtr ♠ s ♦t♥ ② s♥ ♥ ♠ ♦rt♠ ❲ ♥♦ ♣rs♥t t rst ♦ ts♠♦ ♦r t r♥♥♥ ①♠♣
♣♣r♦ s t st s♣ ♠♦ t ♦r sss ♣r♦♣♦s ♥❬❪ s♠s rt t t♦rs ♦ ❬❪ ♣rr t♦ s t r♥♦♠ts ♠♦s ♥ t♥t ss ♥②ss t t♦ sss ② ss♠ tt♦♥t♦♥ ♣♥♥s ♥ ♠♦③ ② s♥ ♦♥t♥♦s t♥tr rs ♠♦♥ t ♥s ♦r♥ t♦ t t♦rst t♥t ♦♥t♥♦s r ♥ rt t ♥♥ ♦ t ♦♥t♦♥ ♦♠s
♥tr♣rtt♦♥ ♦r♥ t♦ t t♦rs ♦♥ ss r♣rs♥ts t s♦♥tt ♥ t ♦tr r♣rs♥ts t r♦s ♦♥s r♥♦♠ t r♣rs♥ts t ♣t♥t s♣ ♥r♦r rtrsts ♦ t ①r② ♠sr ♠♦ ♦s ♥♦t rqr t t♦ t♦♥ rt sss str ♥tr♣rtt♦♥ s sr ♥ t s ♥♦t s② t♦ t t str♥t♦ t ♥trss ♣♥♥s
♥♥♥ ①♠♣ str♥ t t r♥♦♠ ts ♠♦
♦♥s♦♥ ② ♥ t♦ s ♦ t♥t rs t ♠t t♥t ss♠♦s ♣r♠ts t♦ ♦♥sr t ♥trss ♣♥♥s ♦r t s ♥♦t s② t♦rtr③ ts ♣♥♥s s♥ tr s ♥♦♥ ♣r♠tr rt♥ t str♥t♦ ts ♣♥♥s
♦♥s♦♥
ss t♥t ss ♠♦ s ♦t♥ s ♥ t s♠♣ s③ s r s ts ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ s ♦t r♥t ♠t♦s ♦t♦ str t t ② t♥ ♥t♦ ♦♥t t ♥trss ♣♥♥s ♦rtr s ♥♦t ♥② ♠♦ ♣r♦s ♦♥ ♦♥t t♦ rtr③ t str♥t ♦ts ♣♥♥s
♥ ts ♦r ♥♦t s♣♦♥ ♦t t ♠①tr ♦ t♦r ♥②③rs s♥ ♥t t♦ ♦r t ♠♦s r s② ♥tr♣rt ♦ ♦s ♦♥♠♦s str t ♥s ② ♠♦♥ t strt♦♥ ♦ t rs♥ tr ♥t s♣
♦♥s♦♥
♦♥r ♠①tr ♠♦s s♠s t♦ t ♠♦st ♥r ♦♥ ❲ s♦ ♣r♦♣♦s♥ t t♦ ♦♦♥ ♣trs t♦ ♠①tr ♠♦s s♣ ♦♥str♥ts t♦ts ♥r ♠♦ s ♦t ♣r♦♣♦s ♠♦s ♦ t♦ s♠♠r③ t ♥trss♣♥♥s t ♣r♠trs
♣tr
♦s str♥ t ♦s
♦ ①tr♠ strt♦♥s
s ♣tr ♥tr♦s ♥ ①t♥s♦♥ ♦ t t♥tss ♠♦ s ♠♦ r♦♣s t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s s♣ strt♦♥♦ t ♦s ♠♦③s t ♥trss ♣♥♥s ♥♣r♦s ♦♥ ♦♥t s♠♠r③♥ t str♥t ♦ts ♣♥♥s ♠①♠♠ ♦♦ ♥r♥ s ♣r♦r♠ ② ♠♦rt♠ t ♦♠♥t♦r ♣r♦♠s ♦ t ♠♦st♦♥ r ♦ ② ♠♠ ♦rt♠♠r ①♣r♠♥ts ♦♥ s♠t ♥ r t sts♥r♥ t ♠♥ rtrsts ♦ ts ♥ ♠①tr♠♦
♥ ♥r s♦s ♣r♦♠ t♦trt♥ t♥ ♠♦r
♦r r♥r
♥tr♦t♦♥
❲ ♣r♦♣♦s t♦ ①t♥ t ss t♥t ss ♠♦ ♦r t♦r t ② ♥ ♠①tr ♠♦ r①s t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ t♥t rs ❲ rr t♦ t ♣r♦♣♦s ♠♦ s t ♠①tr ♦ ①tr♠ ♣♥♥②strt♦♥s ♣r ♦s ♥♦t ② ♠
♠♠♦ r♦♣s t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♥t ss ♠♥ ♥trss ♣♥♥s r ts ♥r♥ ② t r♣rtt♦♥♦ t rs ♥t♦ ts ♦s s ♣♣r♦ ♦♥ ♠♦♥ ♦ t ♠♥ ♦♥t♦♥ ♥trt♦♥s s rst ♣r♦♣♦s ② ♦r♥s♥ ♥ ♥t ❬ ❪♥ ♦rr t♦ str t sts t ♦♥t♥♦s ♥ t♦r rs ♦r t ♠♠♦ ♦ ♦♦s ♣rtr ♣♥♥② strt♦♥ ♦♥ssts ♥ ♦♠♣♦♥♥t ♠①tr ♦ t ♥♣♥♥ ♥ t ♠①♠ ♣♥♥② strt♦♥ ♦r♥ t♦ t r♠rs ❱ rtr♦♥ s s♣ strt♦♥ ♦ t ♦s
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
♣r♦s ♦♥ ♣r♠tr s♠♠r③♥ t str♥t ♦ t ♦♥t♦♥ ♣♥♥s ♦t rs s r ♣r♠tr s t ♣r♦♣♦rt♦♥ ♦ t ♠①♠♠ ♣♥♥②strt♦♥ rtr♠♦r t ♥tr ♦ t ♦♥t♦♥ ♣♥♥s s r♥ ♦t② t rt♦♥ ♥ ② t ♠①♠♠ ♣♥♥② strt♦♥ s t ♠♦♣ts t t ♦♥ t ♠♥ ♦♥t♦♥ ♣♥♥s ♥ tr str♥ts
♣r♦♣♦s ♠♦ ♥ ♥tr♣rt s t♦ ♣rs♠♦♥♦s rs♦♥ ♦ ♦♥r ♠①tr ♠♦ ♥ ts ♥ts r♦♠ ts ♥tr♣rtt ♣♦r rst ♥s t ♦♥sr ♥trt♦♥s ② r♦♣♥ ♥ t s♠ ♦ t rs r ♦♥t♦♥② ♣♥♥t str♥t ♦ ts ♣♥♥② s rt② t ♣r♦♣♦rt♦♥ ♦ t strt♦♥ ♦ ♠①♠♠ ♣♥♥② ♦♠♣r t♦ tt ♦t ♥♣♥♥ strt♦♥ s♦♥ ♦ s♣rst② s ♥ ② t s♠rt♦♥ ♦ t ♣r♠trs ♦ t ♠①♠♠ ♣♥♥② strt♦♥ ♦ t ♦s ♦r ♦♥r ♠①tr ♠♦s t st♦♥ ♦ t ♣rt♥♥t ♥trt♦♥s s ♦♠♥t♦r ♣r♦♠ r♦r ♣r♦♣♦s t♦ ♣r♦r♠ t ♠♦ st♦♥ ♠♠ ♦rt♠ ♥ ♦rr t♦ ♦ t ♥♠rt♦♥ ♦ t ♠♦s sts ♥r ♣♣r♦ ♦ s♦ st t ♥trt♦♥s ♦ ♠♦r ♥r ♦♥r♠①tr ♠♦
trtr ♦ ts ♣tr s ♣tr s ♦r♥③ s ♦♦s t♦♥ ♣rs♥ts t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs t♦♥ ♣rs♥ts t ♥ ♠①tr ♠♦ t♥ ♥t♦ ♦♥t t ♥trss ♣♥♥s t♦♥ s ♦t t♦ t st♠t♦♥ ♦ t ♣r♠trs ② ♠①♠③t♦♥♦ t ♦♦ ♥ t s r t ss ♥♠r ♥ t ♦s ♦ rsr s♣♣♦s t♦ ♥♦♥ t♦♥ ♣rs♥ts ♠♠ ♦rt♠ ♦♥ ♦♠♥t♦r ts ♥r♥t t♦ ♦ st♦♥ t♦♥ ♣rs♥ts rsts ♦♥s♠t t t♦♥ strts t ♠ ♠♦ ♦♥ t♦ r str♥ ♥s ♦♥s♦♥ s ♥ ♥ t♦♥ ♦t tt tt♦r ♦ t ♣strt ♣r♦r♠♥ t ♠♦ st♦♥ ♥ t st♠t♦♥ ♦ t ♣r♠trs♦ ♠ s ♥ ♥ ♣tr ts rsts r ♣rt ♦ t rt ♦sstr♥ ♦r ♦♥t♦♥② ♦rrt t♦r t ❬❱❪
①tr ♦ ♥trss ♥♣♥♥t ♦s
♥ ♠①tr ♠♦ ♦ ♥trss ♥♣♥♥t ♦s ♦♥srs tt♦♥t♦♥② ♦♥ ss k rs r r♦♣ ♥t♦ k ♥♣♥♥t ♦s ♥ ♦ ♦♦s s♣ strt♦♥
♣rtt♦♥ ♦ t rs ♣r ss r♣rtt♦♥ ♦ t rs ♥t♦♦s tr♠♥s ♣rtt♦♥ σk = (σk1, . . . ,σkk) ♦ 1, . . . , d ♥ k s♦♥t ♥♦♥♠♣t② ssts r σkb r♣rs♥ts t sst b ♦ rs ♥ t ♣rtt♦♥ σks ♣rtt♦♥ ♥s t t♦r ♦ t♦r rs xkb
i = xσkbi = (x
kbji ; j =
1, . . . , dkb) s t sst ♦ xi ss♦t t♦ σkb ♥tr dkb = r(σkb)
♣ strt s ♦♥ ♦r st t t ♦♦♥ r tt♣sr♦rr♣r♦t♦rr♦♣❴
①tr ♦ ♥trss ♥♣♥♥t ♦s
s t ♥♠r ♦ rs t t♦ ♦ b ♦ ♦♠♣♦♥♥t k t♦r xkbji =
(xkbjhi ;h = 1, . . . ,m
kbj ) ♦rrs♣♦♥s t♦ r j ♦ ♦ b ♦r ♦♠♣♦♥♥t k ♥
ss ♦♠♣t s♥t ♦♥ r mkbj s t ♥♠r ♦ ♠♦ts ♦r t
r xkbji s xkbjhi = 1 ♥ i ts ♠♦t② h ♦r r x
kbji
♥ xkbjhi = 0 ♦trs
♠r r♥t ♥trss ♣♥♥s r♥t rs r♣rtt♦♥s ♥♦s r ♦ ♦r ♦♠♣♦♥♥t ♥ t② r r♦♣ ♥t♦ σ = (σ1, . . . ,σg)
♥t♦♥ ①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rst xi t♦ t drt t♦r r rs♥ r♦♠ ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs ♦s t ♣rtt♦♥ s ♥♦t ② σ ♥t ♣r♠trs ② θ ♥ ts ♣ s rtt♥ s ♦♦s
p(① i;σ,θ) =
g∑
k=1
πkp(① i;σk,αk) t p(① i;σk,αk) =
k∏
b=1
p(①kbi ;αkb),
r αk = (αk1, . . . ,αkk) ♥ r p(① kbi ;αkb) s t ♣ ♦ t ♦ b ♦ t
♦♠♣♦♥♥t k ♣r♠tr③ ② αkb
①♠♣ ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦rs t xi = (x1
i , . . . ,x5i ) t t♦r ♦ t♦r rs ♦♦♥
t ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♣rtt♦♥♦ t rs ♦ ts ♠♦ s σ = (σ1,σ2) t σ1 = (1, 2, 3, 4, 5) ♥σ2 = (1, 5, 2, 4, 3) r strts t ♥trss ♣♥♥s t♥♥t♦ ♦♥t ② t ♠♦ ♥ ♥ts tt t ♥trss ♦rrt♦♥ s♥t ♥ ♥ts tt ts ♦rrt♦♥ s t♥ ♥t♦ ♦♥t
X5
X4
X3
X2
X1
X1
X2
X3
X4
X5
x11
x12
ss
X3
X4
X2
X5
X1
X1
X5
X2
X4
X3
x21
x22
x23
ss
r ♥trss ♣♥♥s t♥ ♥t♦ ♦♥t ② t ♦♠♣♦♥♥t♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs t σ1 =(1, 2, 3, 4, 5) ♥ σ2 = (1, 5, 2, 4, 3)
♦t tt t ss t♥t ss ♠♦ t ♦♥t♦♥ ♥♣♥♥ ♦ r♣rs♥t ② t s ♦ t ♦♥ ♥ ♦♥ t ttr
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
t ♠♦s ♣♣r♦ ♣r ♦♥t♦♥② ♥♣♥♥t ♦s s r② ♥r s♥ ♥② strt♦♥ ♥ ♦s♥ ♦r ♦ strt♦♥ p(① kb
i ;αkb) ♠①tr ♠♦ ② ♦♥t♦♥ ♥♣♥♥t ♦s s ♣rs♠♦♥♦s rs♦♥♦ t ♦♥r ♠①tr ♠♦ ♥ t strt♦♥ ♦ ♦ tr♠♥s ♥trt♦♥s r ♦♥sr ♦t tt t ♦rr ♦ ts ♥trt♦♥s s tr♠♥ ② t ♥♠r ♦ t rs ♥t♦ t ♦ ♥② t ♥trt♦♥st♥ rs ♦ r♥t ♦s ③r♦ ♥ t♦s t♥ rs ♦ ts♠ ♦ ♥ ♠♦③ ② t s♣ strt♦♥ ♦ t ♦ ♠t♥s ♦ ts ♠♦ r k = d ♦r ss s q♥t t♦ t t♥t ss ♠♦
♥r ♥tt② ♥r ♥tt② ♦ t ♠①tr ♠♦s ♦r t♦r t ♥ t② ♣r♦ ♦r ② ♥ s♦♠ ♦♥str♥ts ♦♥t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♥ s ♦r♠ ❬❪s t ♥r ♥tt② ♦ t ♠♦ s ♦t♥ ② s♥ ts ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ t♥ ♦s ♥r t♦ s♥t ♦♥t♦♥s
♦r♦r② ♥r ♥tt② ♦ t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs q t♥ sss σ1 =, . . . ,= σg t 1 ≥ 3
♥ tt t ♦ strt♦♥s p(①kbi ;αkb) r ♥t ♥ υb rs ♦
r♦♠ t♥ s♣♣♦s tr ①sts tr♣rtt♦♥ ♦ t st S = σ11, . . . ,σ11 ♥t♦tr s♦♥t ♥♦♥♠♣t② ssts S1 S2 ♥ S3 s tt κu =
∏
j∈Suυj t♥
min(g, κ1) + min(g, κ2) + min(g, κ3) ≥ 2g + 2.
♥ ♠♦ ♣r♠trs r ♥r② ♥t ♣ t♦ s♣♣♥
Pr♦♦ r s t♦♥ r♦♠ xkbi t♦ x
kbi r x
kbi s t♦r r
♥ υb ♠♦ts r xkbi ♦♦s t t♥t ss ♠♦ s♦ ts ♥t
t② s ♥ ② ♦r♠ s ♦♥ t♦ t ♥r ♥tt②♦ t ♠♦ r♥ x
kbi
♦r♦r② ♥r ♥tt② ♦ t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ rs tr ①sts tr♣rtt♦♥ ♦ σk qs ♦r k = 1, . . . , g ♥t♦ tr s♦♥t ♥♦♥♠♣t② ssts S1 S2 S3
∀k ∈ 1, . . . , g, ∀σkb ∈ σk, ∃u ∈ 1, 2, 3 s σkb ∈ Su,
♥ tt t ♦ strt♦♥s p(①kbi ;αkb) r ♥t ♥ υb rs
♦ r♦♠ s tt κu =∏
j∈Suυj t♥
min(g, κ1) + min(g, κ2) + min(g, κ3) ≥ 2g + 2.
♥ ♠♦ ♣r♠trs r ♥r② ♥t ♣ t♦ s♣♣♥
Pr♦♦ ♣r♦♦ s s♠r t♥ t ♣r♦♦ ♦ ♦r♦r② ♦t tt t ①st♥♦ xkb
i s ssr ② t qt② ♦ t tr♣rtt♦♥ ♦ t σk t♥ sss
Prs♠♦♥♦s ♦ strt♦♥
Prs♠♦♥♦s ♦ strt♦♥
♥ ♠ s t♦ ♥ ♣rs♠♦♥♦s strt♦♥ ♦r ♦ ttts ♥t♦ ♦♥t t ♣♥♥② t♥ rs rtr♠♦r t ♣r♠trs♦ t strt♦♥ ♥s ♦ ♠st ♠♥♥ ♦r t ♣rtt♦♥r ♥ ts♦♥t①t ♣r♦♣♦s t♦ ♠♦③ t strt♦♥ ♦ ♦ ② ♠①tr ♦ tt♦ ①tr♠ strt♦♥s ♦r♥ t♦ t r♠rs ❱ rtr♦♥ ♦♠♣t ♦♥ t ♦♣s ♦ rs ♠♦ rsts ♥ ♦♠♣♦♥♥t ♠①tr t♥ ♥♥♣♥♥ strt♦♥ ♥ ♠①♠♠ ♣♥♥② strt♦♥ ♥ s② ♥tr♣rt ② t sr
♠①♠♠ ♣♥♥② strt♦♥ s ♥tr♦ rst t♥ t ♠①tr♠♦ ♦ ①tr♠ ♣♥♥② strt♦♥s ♣r ♦s ♠ s s♦♥② t
♠r rr rs ❲t♦t ♦ss ♦ ♥rt② t rs r ♦♥sr s ♦rr ② rs♥ ♥♠r ♦ ♠♦ts ♥ ♦
∀(k, b) mkbj ≥ m
kbj+1 r j = 1, . . . , dkb − 1.
①♠♠ ♣♥♥② strt♦♥
♥ ♠①♠♠ ♣♥♥② strt♦♥ s ♥ s t ♦♣♣♦st strt♦♥ ♦ ♥♣♥♥ ♦r♥ t♦ t r♠rs ❱ rtr♦♥ ♦♠♣t ♦♥ t♦♣s ♦ rs ♥ t ♥♣♥♥ strt♦♥ ♠♥♠③s ts rtr♦♥ t ♠①♠♠ ♣♥♥② strt♦♥ ♠①♠③s t ❯♥r ts strt♦♥t ♠♦t② ♥♦ ♦ ♦♥ r ♣r♦s t ♠①♠♠ ♥♦r♠t♦♥ ♦♥ t ssq♥t rs
♠r ♦♥r♣r♦ ♥t♦♥ ♥ ♦t tt t s ♥♦♥r♣r♦ ♥t♦♥ ♥ t♥ rs ♥ x
kbi rss r♦♠ ts strt♦♥ t
♥♦ ♦ t r ♥ t rst ♥♠r ♦ ♠♦ts tr♠♥s ①t② t ♦trs t t rrs ♦s ♥♦t ♥ssr② ♣♣②
♠r ss srt♦♥s s strt♦♥ ♥s sss srt♦♥s r♦♠ t s♣ ♦ xkbji t♦ t s♣ ♦ xkbj+1
i t j = 1, . . . , dkb−1 rtt t rs r ♦rr ② rs♥ ♥♠r ♦ ♠♦ts ♥ ♦ ♥t t s r♣r♦ ♥t♦♥ ♥ ♦♥② ♥ mkb
j = mkbj+1
Pr♠tr③t♦♥ ♥ t rst r tr♠♥s t ♦tr ♦♥s ts strt♦♥ s ♥ ② ♣r♦t t♥ t ♠t♥♦♠ strt♦♥ ♦ t rstr ♣r♠tr③ ② t ♦♥t♥♦s t♦r
τ kb = (τhkb;h = 1, . . . ,mkb1 ) t τhkb ≥ 0 ♥
mkb1∑
h=1
τhkb = 1,
♥ t ♣r♦t t♥ t ♦♥t♦♥ strt♦♥s ♥ s s♣ ♠t♥♦♠strt♦♥s ♦ ♦♥t♦♥② ♦♥ x
kb1hi = 1 t♥ ♦r j = 2, . . . , dkb x
kbji
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
♦♦s ♠t♥♦♠ strt♦♥ ♣r♠tr③ ② t srt t♦r
δhjkb = (δhjh
′
kb ;h′ = 1, . . . ,mkbj ) t δhjh
′
kb ∈ 0, 1,m
kbj∑
h′=1
δhjh′
kb = 1 ♥m
kb1∑
h=1
δhjh′
kb ≥ 1.
♦t tt t ♦ ♦♥str♥ts ♥ t sss srt♦♥s ② ♥♦t♥ δkb =(δhjkb ;h = 1, . . . ,m
kb1 ; j = 2, . . . , dkb) t strt♦♥ ♦ ♠①♠♠ ♣♥♥②
♥ ♥♦ ♥
♥t♦♥ ①♠♠ ♣♥♥② strt♦♥ t xkbi t dkbrt
t♦r r ♦♦♥ t ♠①♠♠ ♣♥♥② strt♦♥ ♦s t srt ♣r♠trs r ♥♦t ② δkb ♥ ♦s t ♦♥t♥♦s ♦♥s r ♥♦t ②τ kb ♥ ts ♣ s rtt♥ s ♦♦s
p(xkbi ; τ kb, δkb) = p(x
kb1i ; τ kb)
dkb∏
j=2
p(xkbji |xkb1
i ; δhjkbh=1,...,mkb1
)
=
mkb1∏
h=1
(
τhkb
dkb∏
j=2
mkbj∏
h′=1
(δhjh′
kb )xkbjh′
i
)xkb1hi
.
①♠♣ rt ♥ trrt ♠①♠♠ ♣♥♥② strt♦♥s tt ♠①tr ♠♦ ♦s t ♦s ♦ rs ♦r t rst ♦♠♣♦♥♥t r ♥② σ1 = (1, 2, 3, 4, 5) strt♦♥s ♦ t ♦s r ♠①♠♠ ♣♥♥②♦♥s ♦s t ♣r♠trs r t ♦♦♥
δ11111 = δ21211 = δ31311 = δ41311 = δ1j112 = δ2j212 = 1,
τ 11 = (0.1, 0.3, 0.2, 0.4) ♥ τ 12 = (0.5, 0.5).
r s♣②s t ♣r♦ts ♦ t ♦♥t strt♦♥s ② t r ♦ r♦①s ♦t tt δkb ♥s t ♦t♦♥s r t ♣r♦ts r ♥♦♥ ③r♦ ♦t♦♥ ♦ r ♦①s ♥ τ kb ♥s t ♣r♦ts ♦ ts ♥♦♥ ③r♦ s r ♦t r ♦①s
♥tt② s♥t ♦♥t♦♥ ♦ ♥tt② s t♦ ♠♣♦s τhkb > 0 ♦r h = 1, . . . ,m
kb1 s strt♦♥ s r② ♠t ♥trst s t s s♦
♥rst tt t ♥ ♠♦st ♥r s ♦♥ ♥♦ ♣rs♥t ♦ t♦ s t ♥ ♠♦r ♥t ②
♦ strt♦♥ ♠①tr ♦ t♦ ①tr♠ str
t♦♥s
♥ ❲ ss♠ tt t ♦s ♦♠♣♦s ② t st t♦ rs ♦♦ ♦♠♣♦♥♥ts ♠①tr t♥ ♥ ♥♣♥♥ strt♦♥ ♥ ♠①♠♠ ♣♥♥② strt♦♥ t ♦ ♦♠♣♦s ② ♦♥ r ♦♦ ♠t♥♦♠strt♦♥
Prs♠♦♥♦s ♦ strt♦♥
τ11
1
τ11
2
τ11
3τ11
4
x1111
x1112
x1113
x1114
x1123
x1122
x1121
rst ♦ ♦ ss ♦♥ ♦ ♦ ss
r ♦ ①♠♣s ♦ ♦ strt♦♥s ♦♦♥ ♠①♠♠ ♣♥♥②strt♦♥ r m11
1 = 4 m112 = 3 ♥ m12
1 = m122 = m
123 = 2
♥t♦♥ ♦s str♥ ♦ ♦s ♦ ①tr♠ strt♦♥s drt t♦r r xi s ♥rt ② ♠ ♠♦ t s r♥ ② ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦s t ♣ s rtt♥ s♦♦s
p(① i;σ,θ) =
g∑
k=1
πk
k∏
b=1
p(①kbi ;αkb).
♦r♦r t ♣ ♦ ♦ b ♦r ♦♠♣♦♥♥t k s rtt♥ s
p(①kbi ;αkb) =
(1− ρkb)p(①kbi ; ξkb) + ρkbp(①
kbi ; τ kb, δkb) dkb > 1
p(①kbi ; ξkb) ♦trs
r p(① kbi ; τ kb, δkb) s t ♣ ♦ t ♠①♠♠ ♣♥♥② strt♦♥ ♥
② ♥ r p(① kbi ; ξkb) s t ♣ ♦ t ♥♣♥♥ strt♦♥ ♥
② p(① kbi ; ξkb) =
∏dkb
j=1
∏mkbj
h=1 (ξjhkb )xkbjhi ♣r♠tr αkb = (ρkb, ξkb, τ kb, δkb)
r♦♣s t ♣r♠trs ♦ ♦ b ♦r ♦♠♣♦♥♥t k ♥② t r ρkb ∈ [0, 1] st ♣r♦♣♦rt♦♥ ♦ t ♠①♠♠ ♣♥♥② strt♦♥
♠r ♦ ♣r♠trs ♠ ♠♦ rqrs tt t♦♥ ♣r♠trs♦♠♣r t t ♠ ♠♦ ♥ ♦r ♦ t t st t♦ rst ♥♠r ♦ t♦♥ ♣r♠trs ♣♥s ♦♥② ♦♥ t ♥♠r ♦ ♠♦ts ♦t rst r ♦ t ♦ ♥ ♥♦t ♦♥ t ♥♠r ♦ rs ♥t♦ t ♦ ♥♠r ♦ ♣r♠trs ♦ ♠ ♥♦t ② ν♠ s s♦ ♥ ②
ν♠ = (g − 1) + gd∑
j=1
(mj − 1) +∑
(k,b)|dkb>1
mkb1 .
♥ t♦♥ t ♠ ♠♦ s s② ♥tr♣rt s ①♣♥ ♥ t ♥①t♣rr♣ ♦t tt t ♠t♥ s r ρkb = 0 ♥s t ♦ strt♦♥② t ♥♣♥♥ ♦♥ ♥ ts ♣rtr s t ♣r♠trs ♦ t ♠①♠♠♣♥♥② strt♦♥ r ♥♦ ♦♥r ♥
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
♥♥ ♦ strt♦♥ ❯♥r ts strt♦♥ t ♣r♦♣♦rt♦♥ ♦ t♠①♠♠ ♣♥♥② strt♦♥ rts t t♦♥ r♦♠ ♥♣♥♥ ♥rt ss♠♣t♦♥ tt t tr♥t strt♦♥ s t ♠①♠♠ ♣♥♥② strt♦♥ ♣r♠tr ρkb s ♥ ♥t♦r ♦ t ♥trr ♣♥♥②♦ t ♦ t s ♥♦t r ♣rs ♣♥♥② ♠♦♥ rs t ♣♥♥② t♥ rs ♦ t ♦ rtr♠♦r t st②s ♦♥ ♥ t♥♠r ♦ rs s rr t♥ t♦ t r♠rs ❱ s ♥♦♥ ♣♣r♦♥♥ ts s ♥trr ♣♥♥s r ♥ ② δkb str♥t ♦ts ♣♥♥s s ①♣♥ ② τ kb ♥ ts t♦r s t t ♦ t♦rr♣rs♥t ♠♦t② r♦ss♥s ♦♠♣r t t ♥♣♥♥ strt♦♥
Prs♦♠♥♦s ♦♥r ♠①tr ♠♦ ❲ ♥tr♣rt t ♠ ♠♦ s t♦ ♣rs♠♦♥♦s rs♦♥ ♦ t ♦♥r ♠①tr ♠♦ rst ♦♥ s♥ ② t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s tr♠♥♥ t ♦♥t♦♥♥trt♦♥s t♦ ♠♦ s♣ ♦ strt♦♥ s s♦♥ ♦ ♣rs♠♦♥② s♥ ♠♦♥ t ♥trt♦♥s ♦ ② ♦♠♣♦♥♥t ♦♥② t♦s♦rrs♣♦♥♥ t♦ t ♠①♠♠ ♣♥♥② strt♦♥ r ♠♦ t ♦tr♦♥s r ♦♥sr s ♥
♥tt② ♣r♦♣♦s strt♦♥ s ♥t ♥r t ♦♥t♦♥ ttt ♦ s ♦♠♣♦s ② t st tr rs dkb > 2 ♦r tt t ♠♦t②♥♠r ♦ t st r ♦ t ♦ s rtr t♥ t♦ mkb
2 > 2 s rst s♠♦♥strt ♥ ♣♣♥① ❲ r♠♥ tt t ♣r♠tr ρkb s ♥ ♥t♦r♦♥ t♦ ♠sr t ♣♥♥② t♥ rs ♥♦t ♠t t♦ ♣♥♥②t♥ ♦♣s ♦ rs ♦r dkb = 2 ♥ m
kb2 = 2 t♥ t ♦
strt♦♥ s ♥♦t ♥t s♦ ♥ ♦♥str♥t s ♥ ♦rr t♦ t♠♦st ♠♥♥ ♣r♠trs t ♦s♥ ♦ ρkb s t rst ♠①♠③♥t ♦♦♦ s t♦♥ ♦♥str♥t ♦s ♥♦t s② t ♥t♦♥ ♦ ρkbs ♥ ♥t♦r ♦ t ♣♥♥② str♥t t♥ t rs ♦ t s♠ ♦rtr♠♦r ts ♦♥str♥t s ♥tr s♥ ♦s t t st ♣♥♥sr ♥t ♦t tt ρkb s♠s t♦ ♦rrt t t r♠rs ❱ s strt② t ♦♦♥ ①♠♣
①♠♣ r♠rs ❱ ♥ ρkb t♦ ♠srs ♦ t ♣♥♥② r ♣rs♥ts t ♥ t♥ t r♠rs ❱ ♥ ρkb ♦♥ s♠t rt ♥r②rs ♦r s s♦ ♥ ♦sr ♥ ♠♥② ♦tr stt♦♥s
①♠♠ ♦♦ st♠t♦♥
♦rt♠
♠ t ① = (① 1, . . . ①n) t s♠♣ ♦♠♣♦s t n ♥♣♥♥t ♥ ♥t② strt ♥s ss♠ t♦ rs r♦♠ t ♠ ♠♦ r♦♠ tss♠♣ t ♠ s t♦ st♠t t ♠ ♦r ① ♠♦ m ♥ ② (g,σ)
①♠♠ ♦♦ st♠t♦♥ ♦rt♠
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Cramer’s V
ρkb
r ♦t♦♥ ♦ ρkb ♦♠♣t t t ♥tt② ♦♥str♥t ♦r♥t♦ t r♠rs ❱ ♦r t♦ ♥r② rs
♦♠♥t♦r ♣r♦♠ ❲ s♥ ♥ t♦♥ tt t ♥r♥ ♦r ♠①tr ♠♦ ♥ ♣r♦r♠ ♥ ♠ ♦rt♠ ♦r ♦♥ ♦ ts ①t♥s♦♥s t♠①♠③t♦♥ ♦ t ♦♠♣tt ♦♦♦ s s② ♦r t s ♥♦t t s♦r t ♠ ♠♦ s♥ t st♠t♦♥ ♦ t srt ♣r♠trs ♦ t ♠①♠♠♣♥♥② strt♦♥ s ♦♠♥t♦r ♣r♦♠ ♥ S(a, b) s t ♥♠r♦ ♣♦ss srt♦♥s r♦♠ st ♦ r♥ a ♥t♦ st ♦ r♥ b t♥ δkb s♥ ♥ t srt s♣ ♦ ♠♥s♦♥
∏dkb−1j=1 S(m
kbj ,m
kbj+1 ) ♦ ♥ ①st
♥♠rt♦♥ ♦r st♠t♥ t srt ♣r♠trs s ♥r② ♠♣♦ss ♥ ♦ ♦♥t♥s rs t ♠♥② ♠♦ts ♥♦r ♠♥② rs
①♠♣ ♦♠♥t♦r ♣r♦♠ ♥♦ ② t srt ♣r♠trs ♦ t tr rs ♥ mkb = (5, 4, 3) ♠♣s 51 840 ♣♦ssts ♦r δkb
st♠t♦♥ ♠♣ ♣r♠trs r st♠t ♠ ♦rt♠ ♦♥t ss ♣r♦♠ ♥♦ ② t ♥♥♦♥ ss ♠♠rs♣ t ts ♠ st♣t ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦ s ♥♣♥♥t② ♣r♦r♠ ♦♥ t ♣r♠trs ♦ ♦ s t t ♠ st♣ ♦ trt♦♥[r] t ♦♠♥t♦r ♣r♦♠ ♦ t srt ♣r♠tr st♠t♦♥ ♦r ♦ b ♦♦♠♣♦♥♥t k s ♦r♠ ② tr♦♣♦sst♥s ♦rt♠ ♦s t stt♦♥r②strt♦♥ s ♦s t♦ p(δkb|①kb, ③[r]) ♣r♦♣♦s strt♦♥ ♦ ts ♦rt♠r♥♦♠② s♠♣s t ♥t δ⋆kb t ♥t (ρ⋆kb, ξ
⋆kb, τ
⋆kb) s tr♠♥
st② ♦♠♣t ♥ ♦rr t♦ ♠①♠③ p(ρ⋆kb, ξ⋆kb, τ
⋆kb, δ
⋆kb|①kb, ③[r]) ♦t tt t
♦♥t♥♦s ♣r♠trs (ρ⋆kb, ξ⋆kb, τ
⋆kb) r ♦♥t♦♥② ♦t♥ ② ♥ ♠ ♦rt♠
② ♥tr♦♥ s♦♥ t♥t r ♥♦t♥ t ♠♠rs♣ ♦ t ♣♥♥②strt♦♥s ♦ t ♦ ♥♣♥♥ ♦r ♠①♠♠ ♣♥♥② strt♦♥
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
♦ ♦rt♠
♥ ♥r♥ ♦ ♣r♦r♠ ♥ ♠ ♦rt♠ ♦r♦♠♥t ♣r♦♠ ♦ t ss ♠♠rs♣ ♦r s t ♦♣t♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦ ♦♥ t srt ♣r♠trs s ♣r♦r♠ st♦st ♦rt♠ ♥ ♦♥② ssr t ♥rs ♦ t ①♣tt♦♥ ♦t ♦♠♣tt ♦♦♦ ♥ ♥♦t ts ♠①♠③t♦♥ ♦ t ♥r♥ s ♣r♦r♠ t ♦♦♥ ♠ ♦rt♠
trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ s st♣ t ♦♥t♦♥ ♣r♦ts
tik(θ[r]) =
π[r]k p(xi;σk,α
[r]k )
p(xi;σ,θ[r])
.
st♣ ♥rs ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦
π[r+1]k =
n[r]k
n♥ α
[r+1]kb s s Lkb(α
[r+1]kb ;①, t[r]) ≥ Lkb(α
[r]kb ;①, t
[r]),
r Lkb(αkb;①, t[r]) =
∑ni=1 tik(θ
[r]) ln p(xkbi ;αkb) t t[r] = (tik(θ[r]); i =
1, . . . , n; k = 1, . . . , g) ♥ n[r]k =
∑ni=1 tik(θ
[r])
♦rt♠ ♠ ♦rt♠ t♦ ♦t♥ t ♠ ♠♦ ♠
s ♦rt♠ s st♦♣♣ tr rmax trt♦♥s ♦♣t♠③t♦♥ ♦♥ αkb s♥♣♥♥t② ♣r♦r♠ ♦r (k, b) t t ♠ st♣ ② t ♦♦♥ tr♦♣♦sst♥s ♦rt♠
♦s ♦♥ t st♣ ♦ t ♦rt♠
♥ tr♦♣♦sst♥s ♦rt♠ s ♥♣♥♥t② ①t ♦r (k, b) ♥ ♦rr t♦ ♣r♦r♠ t ♠ st♣ ♦ ♦rt♠ ♦r ① (k, b) ts♦rt♠ s stt♦♥r② strt♦♥ ♦s t♦ p(αkb|①, t[r]) ♥ t s ♣r♦r♠t trt♦♥ [r] ♦ t ♦ ♠ ♦rt♠ t s♠♣s sq♥ ♦ t ♦♣r♠trs (α[r,0]
kb , . . . ,α[r,smax]kb ) r smax s t ♥♠r ♦ trt♦♥s ① ② t
sr s t ♦rt♠ ♠s t ♥♥ t ♠①♠③♥ t ①♣tt♦♥ ♦ t♦♠♣tt ♦♦♦ ♣t
α[r+1]kb = argmax
s=1,...,smax
Lkb(α[r,s]kb ;①, t[r]).
❲ ♥♦ t t tr♦♣♦sst♥s ♦rt♠ t♥ t ts ♥str♠♥t strt♦♥ q(.;α[r,s]
kb )
①♠♠ ♦♦ st♠t♦♥ ♦rt♠
trt♥ r♦♠ t ♥t α[r,0]kb = α
[r]kb ts trt♦♥ [s] s rtt♥ s
α⋆kb ∼ q(αkb;α
[r,s]kb )
α[r,s+1]kb =
α⋆kb t ♣r♦t② λ[r,s]
α[r,s]kb t ♣r♦t② 1− λ[r,s].
♦rt♠ tr♦♣♦sst♥s ♦rt♠
♦s ♦♥ t ♣r♦♣♦s strt♦♥ ♥str♠♥t strt♦♥ q(αkb;α[r,s]kb )
s♠♣s t ♥t α⋆kb ♥ t♦ st♣s rst② t ♥♦r♠② s♠♣s t ♥t
δ⋆kb ♠♦♥ t ♥♦r♦♦ ♦ δ[r,s]kb ♥♦t ② ∆(δ
[r,s]kb ) s ♥♦r♦♦ s
♥ s t st ♦ t ♣r♠trs r t ♠♦st t♦ srt♦♥s r r♥t r♦♠t♦s ♦ δ[r,s]
kb ♦♥② t ♦♠♣ts t ♦♥t♥♦s ♣r♠trs ♦♥t♦♥② ♦♥∆(δ
[r,s]kb ) s s
(ρ⋆kb, ξ⋆kb, τ
⋆kb) = argmax
ρkb,ξkb,τkb
Lkb(ρkb, ξkb, τ kb, δ⋆kb;①, t
[r]).
♦t tt t ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦st②s ♥♦t strt♦rr ♥ ♥ t srt ♣r♠trs r ♥♦♥ ♦r② r♠r♥ tt t ♦ strt♦♥ s ts ♠①tr ♥tr♦ s♦♥t♥t r ♥t♥ t ♦ strt♦♥ ♠♠rs♣ ♥♣♥♥ ♦r ♠①♠♠ ♣♥♥② strt♦♥ s t ♦♥t♥♦s ♣r♠trs ♥ ② t♣r♦s qt♦♥ r ♦t♥ ② ♥ ♠ ♦rt♠ t ♥ t ♥①t st♦♥
①♠♣ ♦r♦♦ ♦ t srt ♣r♠tr r stts t♠♥ts ♦ ∆(δkb) t dkb = 2 mkb1 = 3 mkb2 = 2 ♥ t δ121kb = δ221kb =δ322kb = 1 ♥ δh2h
′
kb = 0 ♦trs
♦s ♦♥ t ♣t♥ ♣r♦t② ♥ ♦rr t♦ ♦♠♣t t ♥t♦♥ ♦t tr♦♣♦sst♥s ♦rt♠ ♣rs t ♣t♥ ♣r♦t② s♥ ②
λ[r,s] = min
p(①kb, t[r];α⋆kb)
p(①kb, t[r];α[r,s]kb )
|∆(δ⋆kb)||∆(δ
[r,s]kb )|
; 1
,
|∆(δ[r,s]kb )| ♥♦t♥ t r♥ ♦ ∆(δ
[r,s]kb )
♠r ①st ♣♣r♦ s st♦st ♦♥ ❲♥ t s♣ ♦ ♣♦ssδkb s s♠ ♦r ①♠♣ ♥ t ♦ r♦♣s s♠ ♥♠r ♦ ♥r② rs♥ ①st ♣♣r♦ ♦t♥s t s♠ rsts s t ♣r♦♣♦s ♦rt♠ t ss♦♠♣tt♦♥ t♠ s t rt♥ ♣♣r♦ ①st ♦r st♦st ♣♥s♦♥ t ♥♠r ♦ rs ♥ ♠♦ts
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
xkb11
xkb12
xkb13
xkb22
xkb21
xkb11
xkb12
xkb13
xkb22
xkb21
xkb11
xkb12
xkb13
xkb22
xkb21
xkb11
xkb12
xkb13
xkb22
xkb21
xkb11
xkb12
xkb13
xkb22
xkb21
r ♦r t r♦ h′ ♥ t ♦♠♥ h ♥ts tt δh2h′
kb = 1♥ t tt δh2h
′
kb = 0 δkb r t ♠♥ts ♦ ∆(δkb)
tr♠♥t♦♥ ♦ (ρ⋆kb, ξ⋆kb, τ
⋆kb) ② t ♣r♦♣♦s str
t♦♥
s♦♥ t♥t r t rst t♥t t♦r ③ ♥ts t ss ♠♠rs♣ s♦♥ t♥t t♦r ♥♦ts t ♦ strt♦♥ ♠♠rs♣ t s♥♦t ② ② = (y
kbi ; i = 1, . . . , n; k = 1, . . . , g; b = 1, . . . ,k) r ykbi = 1
①kbi rss r♦♠ t ♠①♠♠ ♣♥♥② strt♦♥ ♦r ♦ b ♦ ss k ♥
ykbi = 0 ① kb
i rss r♦♠ t ♥♣♥♥ strt♦♥ ♦r ♦ b ♦ ss k
♦♠♣tt ♦♦♦ ♦ ♠①tr ♠♦ strt♦♥♦rrs♣♦♥s t♦ t ♠r♥ strt♦♥ ♦ t r♥♦♠ r ❳ ♦t♥ r♦♠t tr♣t strt♦♥ ♦ t r♥♦♠ rs (❳,❨,❩) ♥ t ♦s r♥♣♥♥t ♦♥t♦♥② ♦♥ ❩ t ♦♠♣tt ♦♦♦ ♦t ♥ ❨♥ ❩ s ♥ s
L(θ;x,②, ③) =
g∑
k=1
♥k ln πk +g∑
k=1
k∑
b=1
Lkb(αkb;x,②, ③),
①♠♠ ♦♦ st♠t♦♥ ♦rt♠
r Lkb(αkb;x,②, ③) ♥♦ts t ♦♠♣tt ♦♦♦ ♦ ♦ b ♦r♦♠♣♦♥♥t k ♥ ②
Lkb(αkb;x,②, ③) =
n∑
i=1
zik
(
(1−ykbi ) ln
(
(1−ρkb)p(①kbi ; ξkb)
)
+ykbi ln
(
ρkbp(①kbi ; τ kb, δkb)
)
)
.
♦♥t♦♥ st♠t♦♥ ♦ t ♦♥t♥♦s ♣r♠trs t trt♦♥ [s] ♦♦rt♠ ♣r♦r♠ t trt♦♥ [r] ♦ ♦rt♠ t srt ♥t♣r♠tr δ⋆kb s s♠♣ ♥ t ♦♥t♥♦s ♥t ♣r♠trs r ♥s ♦♦s
(ρ⋆kb, ξ⋆kb, τ
⋆kb) = argmax
ρkb,ξkb,τkb
Lkb(ρkb, ξkb, τ kb, δ⋆kb;①, t
[r]).
♦ ♦♥t♦♥② ♦♥ (δ⋆kb,①, t[r]) t ♦♥t♥♦s ♣r♠trs r ♦t♥ ② t
♦♦♥ ♠ ♦rt♠
trt♥ r♦♠ ♥ ♥t (ρ[0]kb ,α
[0]kb , τ
[0]kb) trt♦♥ [ℓ] s rtt♥ s
st♣ t t ♦♥t♦♥ ①♣tt♦♥ ♦ ykbi
ui(α[ℓ]kb) =
ρ[ℓ]kbp(①
kbi ; τ
[ℓ]kb, δ
⋆kb)
(1− ρ[ℓ]kb)p(①
kbi ; ξ
[ℓ]kb) + ρ
[ℓ]kbp(①
kbi ; τ
[ℓ]kb, δ
⋆kb),
st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦
ρ[ℓ+1]kb =
n[ℓ]kb
n[r]k
, ξjh[ℓ+1]kb =
njh[ℓ]kb
n[r]k − n
[ℓ]kb
♥ τh[ℓ+1]kb =
nh[ℓ]kb
n[ℓ]kb
,
t n[ℓ]kb =
∑ni=1 tik(θ
[r])ui(α[ℓ]kb) n
h[ℓ]kb =
∑ni=1 tik(θ
[r])ui(α[ℓ]kb)x
kb1hi
♥ njh[ℓ]kb =∑n
i=1 tik(θ[r])(1− ui(α
[ℓ]kb))x
kbjhi .
♦rt♠ ♠ ♦rt♠ t♦ ♦t♥ (ρ⋆kb, ξ⋆kb, τ
⋆kb)
♦♥tr ♥ ♦♣t♠♠ r♥ ♦r ①♣r♠♥ts ♠♣r② ♥♦t tt t ♦♦♦ ♥t♦♥ ♦ t ♠①tr t♥ t ♥♣♥♥ ♥ t♠①♠♠ ♣♥♥② strt♦♥s ♥q ♦♣t♠♠ ❲ ♦♥tr tt ts♥t♦♥ s ♥ ♥q ♠①♠♠
♠r ♥ ♠ ♦rt♠ t srt ♣r♠trs r ♥♦♥ ♥ ts♣ s r δkb r ♥♦♥ ♦r (k, b) t ♦♥t♥♦s ♣r♠trs ♦ st♠t ♥q ♠ ♦rt♠ t trt♦♥ [r] ♦ ts ♦rt♠ t st♣♦ ♦♠♣t ♦t ①♣tt♦♥s ♦ ③[r] ♥ ②[r] t ♠ st♣ ♦ st♠t t ♦♥t♥♦s ♣r♠trs ♠①♠③♥ t ①♣tt♦♥ ♦ t ♦♠♣tt♦♦♦
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
♦ st♦♥ ♦rt♠
♠ ♠ s t♦ st t ♠♦ ♥ ② (g, σ) ttr t t ts ♥ ②s♥ r♠♦r t ♠ s t♦ ♥ t ♠♦ ♥ t rst♣♦str♦r ♣r♦t②
Pr♦r strt♦♥s ❲ ♦♥sr tt p(g) = 1gmax
g ≤ gmax ♥ 0 ♦trsr gmax s t ♠①♠♠ ♥♠r ♦ sss ♦ ② t sr ♥ ss♠tt p(σ|g) ♦♦s ♥♦r♠ strt♦♥
P♦str♦r strt♦♥s st ♠♦ ♠①♠③s ts ♣♦str♦r strt♦♥♦r♥ t♦ t ♣r♦r strt♦♥s t s ♥ s
(g, σ) = r♠①g
[
r♠①σ
p(①|g,σ)]
r p(①|g,σ) ∝∫
θ∈Θ
p(①|θ, g,σ)p(θ)dθ.
♦ ♥ (g, σ) ♠♠ ♦rt♠ s s ♦r st♠t♥ r♠①σ p(①|g,σ) ♦r ♦ g ∈ 1, . . . , gmax s ♠t♦ ♠ts t ♦♠♥t♦r ♣r♦♠♥♦ ② t tt♦♥ ♦ t ♦ strtr ♦ rs s♥ t ♣r♦s r♥♦♠ ♠♦♥ t σ ♦ ♥trst
♠r ♥ t rrs ♠♣ rrs ♠♣ ♠t♦ ♦ s❬❪ ♦r ts ♣♣r♦ s rr② ♣r♦r♠ t ♠① ♣r♠trs ♦♥t♥♦s ♥ srt ♥ ♥ s s t s t t♦ ♥ ♠♣♣♥t♥ t ♣r♠trs s♣ ♦ t♦ ♠♦s ♦ ♣r♦♣♦s t♦ s ♥ sr ♠♠♦rt♠ ♥ p(σ|①, g) s stt♦♥r② strt♦♥
①♣♦rt♦♥ ♦ t s♣ ♦ t ♠♦s ②
♦rt♠
♥ s ♦rt♠ tr♥ts t♥ t♦ st♣s t ♥rt♦♥ ♦ ♥♦r♦♦ ♦♥t♦♥② ♦♥ t rr♥t ♠♦ ② ♣r♦♣♦s strt♦♥ ♥ t ♥rt♦♥ ♦ ♥ ♠♦ ♦♥♥ t♦ ts ♥♦r♦♦ ♦r♥ t♦ ts ♣♦str♦r♣r♦t②
♦ st♦♥ ♦rt♠
s ♠♠ ♦rt♠ s p(σ|①, g) s stt♦♥r② strt♦♥ trt♥r♦♠ ♥ ♥t ♦ t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s σ[0] tstrt♦♥ [q] s rtt♥ s
♦r♦♦ st♣ s♠♣♥ ♦ st♦st ♥♦r♦♦ Σ[q]
Σ[q] ∼ q(Σ;σ[q]).
♦ st♣ s♠♣♥ ♦ t r♣rtt♦♥ ♦ t rs ♥t♦ ♦sσ[q+1]
σ[q+1] = p(σ|①, g,Σ[q]) r p(σ|①, g,Σ[q]) ∝
p(①|g,σ) σ ∈ Σ[q]
0 ♦trs
♦rt♠ ♦rt♠ t♦ ①♣♦r t ♠♦s
❲ ♥♦ t ♦t st♣s ♦ t ♦ ♠♠ ♦rt♠
ts ♦ t ♦r♦♦ st♣ tr♠♥st ♥♦r♦♦ ♦ σ[q] ♦ ♥ s t st ♦ ♠♦s r t ♠♦st ♦♥ r s t ♦r ♦♥♦♠♣♦♥♥t ♥ ♥♦tr ♦ ♣♦sst② t♦ ♥ ♦
σ : ∃!(k, b, j) j ∈ σ[q]kb ♥ j /∈ σkb
∪
σ[q]
.
♦r s ts tr♠♥st ♥♦r♦♦ ♥ r② r ♦r ♣r♦♣♦sstrt♦♥ ♦s r♥ t t♦ st♦st ♥♦r♦♦ Σ[q] ② ♠t♥ t♥♠r ♦ (k, b) r σkb ♦ r♥t t♦ σ
[q]kb s t s♠♣♥ ♦r♥
t♦ q(.;σ[q]) s ♣r♦r♠ ② t tr ♦♦♥ st♣s ♦♠♣♦♥♥t s♠♣♥
k[q] ∼ U [1, . . . , g].
♥ ♦ s♠♣♥
b[q]from ∼ U [1, . . . , B[q]
k[q]].
rr♥ ♦ s♠♣♥
b[q]to = b[q], B[q]
k[q]+ 1 r b[q] ∼ U [1, . . . , B[q]
k[q] \ b[q]from].
st♦st ♥♦r♦♦ Σ[q] s t♥ ♥ s
Σ[q] =
σ : ∃!(k, b, j) j ∈ σ[q]kb , j /∈ σkb ♥ j ∈ σkb′ t k = k[q], b = b
[q]r♦♠, b
′ ∈ b[q]t♦
∪
σ[q]
.
❲ ♥♦t ② σ[q+ε(e)] t ♠♥ts ♦ Σ[q] r ε(e) = e
|Σ[q]|+1♥ e = 1, . . . , |Σ[q]|
①♠♣ ♥♦r♦♦ Σ[q] r s♦s ♥ strt♦♥ ♦ ts ♥
t♦♥ ♦ t ♥♦r♦♦ Σ[q] ♥ σ[q]k = (1, 2, 3, 4)
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
X4
X3
X2
X1
X1
X2
X3
X4
X4
X3
X2
X1
X1
X2
X3
X4
X4
X3
X2
X1
X1
X2
X3
X4
X4
X3
X2
X1
X1
X2
X3
X4
X4
X3
X2
X1
X1
X2
X3
X4
X4
X3
X2
X1
X1
X2
X3
X4
X4
X3
X2
X1
X1
X2
X3
X4
X4
X3
X2
X1
X1
X2
X3
X4
r ①♠♣ ♦ t s♣♣♦rt ♦ Σ[q] ♥ t s ♦ ♦r rs trs ♦ t jt r♦ ♥ ♦ t j
′t ♦♠♥ r ♥ t s♠ ♦ t♥ t (j, j′) s ♣♥t ♥ s s ♣♥t ♥ t ♦trs ♠♥ts♦ Σ[q] b[q]from = 1 ♠♥ts ♦ Σ[q] b[q]from = 2
ts ♦ t Pttr♥ st♣ t t ♥rt♦♥ ♣ttr♥ st♣ t ♦rt♠ ♥st ♦ p(①|g,σ) ∀σ ∈ Σ[q] t♦ ♠♣♠♥t ♦rt♠ ② s♥ t ♣♣r♦①♠t♦♥ ts ♣r♦t② s ♣♣r♦①♠t ②
ln p(①|g,σ) ≃ L(θ;①, g,σ)− ν♠2
log(n),
θ ♥ t ♠①♠♠ ♦♦ st♠t♦r ♦t♥ ② t ♠ ♦rt♠ ♣r♦s② sr ♥ t♦♥ s t trt♦♥ [q] ♦r e = 1, . . . , |Σ[q]| tst♠t♦r θ
[q+ε(e)]ss♦t t♦ t ♠♥t σ[q+ε(e)] s ♦♠♣t ② ♦rt♠
♥t③t♦♥ ❲tr t ♥t st ♦r σ[0] t ♦rt♠ ♦♥rst♦ t s♠ stt♦♥r② strt♦♥ ♦r ts ♦♥r♥ ♥ r② s♦♥ t ♥t③t♦♥ s ♣♦♦r ♥ ♦s ♦♥sst ♥ t ♠♦st ♦rrt rs rr s♥♥t sst♦♥ s ♣♣ ♦♥ t ♠tr① ♦ r♠rs❱ st♥s ♦♥ t ♦♣s ♦ rs ❲ st ♦r σ[0]
k t ♣rtt♦♥ ♣r♦② t ♠♥♠③s t ♥♠r ♦ ♦s ♥ ①s t ♦s♦♥sst♥ ♦ ♠♦r t♥ ♦r rs ♦t tt t ♥♠r ♦ t rs t♥t♦ ♦ s ♠t t♦ ♦r ♦r t ♥t③t♦♥ s r② ♦s ♥♠♦r t♥ ♦r rs r ♦sr r♥ ♦r ①♣r♠♥ts ♦s② t♠♠ ♦rt♠ ♥ t♥ ♦t ts ♥t ♦♥str♥t ♥ssr②
t♦♣♣♥ rtr♦♥ ♦rt♠ s st♦♣♣ ♥ qmax sss trt♦♥s ♥♦t s♦r ttr ♠♦
♠r ①♣r♠♥ts ♦♥ s♠t t sts
♦♥sq♥s ♦ t ♠♦ st♦♥ ♦♥ t ♦
rt♠
♥ t trt♦♥ [q] ♦ t ♠♠ ♦rt♠ ♣r♦r♠♥ t ♠♦ st♦♥
♦rt♠ t ♠ ♦rt♠ ♦rt♠ st♠ts θ[q+ε(e)]
ss♦t t♦ t ♠♦ σ[q+ε(e)] ♦r e = 1, . . . , |Σ[q]| ♥ ts ♠♦s r ♦s t♦
σ[q] tr ♠①♠♠ ♦♦ st♠ts s♦ ♦s t♦ θ[q]
Pr♠trs ♦ t ♥♦♥♠♦ ♦s ♠ ♦rt♠ ♥t③t♦♥ s
s♦ ♦♥ ② t ♦ θ[q]
♦r t ♥♦♥ ♠♦ ♦s s ♥ s s
σ[q+ε(e)]kb = σ
[q]kb θ
[q+ε(e)][0]kb = θ
[q]
kb
Pr♠trs ♦ t ♠♦ ♦s ♦r t ♦tr ♦s t ♦♥t♥♦s ♣r♠trs r r♥♦♠② s♠♣ ♥ ♦rr t♦ ♦ t ♦♠♥t♦r ♣r♦♠s s sq♥t ♠t♦ t♦ ♥t③ δ[q+ε(e)][0]
kb srt♦♥s r♦♠ xkb1i t♦ x
kbji
r s♠♣ ♦r♥ t♦ ① ♥ t ♦♥t♥♦s ♣r♠trs ♣r♦s② s♠♣(ρ
[q+ε(e)][0]kb ,α
[q+ε(e)][0]kb , τ
[q+ε(e)][0]kb ) ♦r j = 2, . . . , dkb s ♦♦s
δ.j[q+ε(e)][0]kb ∝
n∏
i=1
p(xkb1i , x
kbji ; ρ
[q+ε(e)][0]kb ,α
1[q+ε(e)][0]kb ,α
j[q+ε(e)][0]kb , τ
[q+ε(e)][0]kb , δ.jkb)
z[q]ik ,
r δ
.j[q+ε(e)]kb = (δ
hj[q+ε(e)]kb ;h = 1, . . . ,m
kb1 ) ♥ r z[q]ik = E
[
Zik|xi,θ[q]]
♠r ♦t t ♥♠r ♦ trt♦♥s ♦ t ♦rt♠ rmax ss ♥ t♦♥ t ♦rt♠ s st♦♣♣ tr ① ♥♠r ♦ trt♦♥srmax t ♦rt♠ s st♦♣♣ ♦r ts ♦♥r♥ t ♣r♦♣♦s ♥t③t♦♥♠ts t ♣r♦♠s ♥ t ♠♦ s ♣♦str♦r ♣r♦t② t st② ♥ t ♥♦r♦♦ Σ[q] r♥ s♦♠ sss trt♦♥s s♦ ts ♦♦♦ ♥rs s ts ♦rt♠s r ♥tr♦ t ♥♠r ♦ trt♦♥s ♦ ♦rt♠ t ♠♦st ♥tr♥ ♦rt♠ s s♠ ❲♥ t st ♠♦ s st② ♦rt♠ ts ttr st② ♥ ts ♠♦ r♥ ♠♥② trt♦♥s s♦ ttr♦♣♦sst♥s ♦rt♠ ♥ t ♠ ♦rt♠ ♦rt♠ r♣r♦r♠ ♦ts ♦ t♠s s t s ♥♦t ♥ssr② t♦ r ♥♠r ♦ trt♦♥ss st♦♣♣♥ rtr♦♥
♠r ①♣r♠♥ts ♦♥ s♠t t sts
♣rs♥ts t st♠♥t ♣r♠trs s s ♦r t s♠t♦♥s
♦rt♠s ♠♠ ♠ tr♦♣♦sst♥s ♠
rtr qmax = 20× d rmax = 10 smax = 1 tmax = 5
❱s ♦ t r♥t st♦♣♣♥ rtr
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
t② ♦ t ♦rt♠ ♦r t δkb st♠t♦♥
♠ ♥ ts st♦♥ strt t ♣r♦r♠♥ ♦ t tr♦♣♦sst♥s ♦rt♠ st♠t♥ δkb s t♦♥ ♥ t r♥ ♦ ts ♥t③t♦♥ ♥② ♥ ts ♦rt♠ s ♥tr♦ ♥ t ♠♠ ♦rt♠ ♥ ♥ t ♠♦rt♠ rs♣t② st♠t t ♠♦ ♥ t ♣r♠trs ♥ t t♦♦♥r q② t s s♦♥ ♥ t ♦♦♥ s♠t♦♥s tt t ♦rt♠ st②sr♥t ♣ t♦ s① ♠♦ts ♣r r ♥ ♣ t♦ s① rs ♣r ♦ s♦♥t♦♥s ♦ ♥ ♠♦st stt♦♥s
①♣r♠♥t ♦♥t♦♥s ♠♣s ♦ s③ sr ② rs ♥ ts♠ ♥♠r ♦ ♠♦ts r ♥rt ② ♠①tr t♥ ♥ ♥♣♥♥strt♦♥ ♥ ♠①♠♠ ♣♥♥② strt♦♥ ♣r♠trs r s♦ st♠t ② t tr♦♣♦sst♥s ♦rt♠ sr ♥ t♦♥ s♥ ♦♥②♦♥ ss s ♥rt ♥t③t♦♥s ♦ t srt ♣r♠trs r ♣r♦r♠♦r♥ t♦ qt♦♥ t zi1 = 1 ♦r i = 1, . . . , 200
sts r s♦s t ♦①♣♦ts ♦ t ♥♠rs ♦ trt♦♥s rqr ②t tr♦♣♦sst♥s ♦rt♠ ♥ ♦rr t♦ ♥ t tr ♥s t♥ ♠♦ts ♠①♠③♥ t ♦♦ ♦r♥ t♦ ts s♠t♦♥s ♦♥ ♦srs ttt rsts ♦ ts ♦rt♠ r ♦♦ t♥s t♦ ts ♥t③t♦♥ ♦s s♥♥t② r♥ t ♥♠r ♦ trt♦♥s ♥ ♥ ♦rr t♦ ♥ t ♠①♠♠♦♦ st♠t♦rs
t② ♦ t ♦rt♠ ♦r ♠♦ st♦♥
♠ ♥ ♦rr t♦ strt t ♥② ♦ t ♦rt♠ ♦r t ♠♦ st♦♥♥ s♦ t ♥ st♠t♦♥ ♣r♦ss ♥t t♦ st② t ♦t♦♥ ♦ tr r♥ ♦r♥ t♦ t ♥♠r ♦ rs ♥ t♦ t s③ ♦t t st
①♣r♠♥t ♦♥t♦♥s ♥ ♠♥② stt♦♥s s♠♣s r ♥rt ♦r♥ t♦ t ♠ ♠♦ t t♦ ♦♠♣♦♥♥ts ♦t tt t ♣r♠tr u s♥tr♦ ♦r ♦♥tr♦♥ t ♦r♣♣♥ ♦ sss ♥ t s ♦s t♦ ♦♥ t♥t sss r s♦t② ♦r♣♣ s ♣r♠tr ① t rr♦r rt t♦ ♦r st stt♦♥
σkb = (d/b, 1 + d/b) ρkb = 0.6(1− u) τ kb = (0.60, 0.20, 0.20),
δh2h′
1b = 1 h = h′ δ1221b = δ2231b = δ3211b = 1 αj1b = (0.20, 0.20, 0.60),
α12b = α1
1b(1− u)+ (0.075, 0.850, 0.075)u ♥ α22b = α2
1b(1− u)+ (0.850, 0.075, 0.075)u.
♥ t t ♦rt♠ s st♦♣♣ s s♦♦♥ s t ♥s srt st♠t ♥♦♥ ♦♦r t♥ ♦r q t♦ t ♦♦ ♦t♥ t t tr srt ♣r♠trs s ♦r ts♠t♦♥
♠r ①♣r♠♥ts ♦♥ s♠t t sts
r ♦①♣♦ts ♦ t ♥♠r ♦ trt♦♥s rqr ② t tr♦♣♦sst♥s ♦
rt♠ ♥ ♦rr t♦ ♥ t st ♥s t♥ ♠♦ts ♦r♥ t♦ t ♥♠r ♦ ♠♦ts
♥ tsts r s♠t t ♣r♦♣♦rt♦♥ ♦ ♠①♠♠ ♣♥♥② strt♦♥ q t♦
r rs t t ♣r♦♣♦s ♥t③t♦♥ r ♠♦ts ♣r rs t t
♣r♦♣♦s ♥t③t♦♥ r rs t r♥♦♠ ♥t③t♦♥ r ♠♦ts ♣r
rs t r♥♦♠ ♥t③t♦♥
sts s♦s t ♠♥ ♥ t st♥r t♦♥ ♦ t r r♥ t♥ t ♣r♠trs s ♦r t t st ♥rt♦♥ ♥ tst♠t ♣r♠trs ♦r♥ t♦ t ♥♠r ♦ rs ❲♥ n ♥rss tr r♥ ♦♥rs t♦ ③r♦ t ♦♥r♠s t ♦♦ ♦r ♦ t♣r♦♣♦s ♦rt♠
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
d \ n
♠♥ st♥r t♦♥ ♦ t r r♥
♥②ss ♦ t♦ r t sts
♦♥tr♣t ♠t♦ ♦
t s t st s sst ♦ t t♦♥ ♥♦♥s ♦♥tr♣tPr♥ r② ❬❪ t s ♦♠♣♦s t ♠rr ♦♠♥ ♦ rtr ♥♦t ♣r♥♥t ♦r ♦ ♥♦t ♥♦ t② r t t t♠ ♦ ♥tr ♦r♥♣r♦♠ s t♦ ♣rt t rr♥t ♦♥tr♣t ♠t♦ ♦ ♥♦ s ♦♥tr♠♠t♦s ♦r s♦rt tr♠ ♠t♦s ♦ ♦♠♥ s ♦♥ r ♠♦r♣ ♥ s♦♦♦♥♦♠ rtrsts ♦♠♥ s sr ② ♥♥ rs ♥♠r ♦r♥ r ♦r♥ ♥ ♠♦r s ❲ ♥ ss ♥ ♠♦r s t♦♥ ❲ ♦ s♥st♦♥ ♦ s♥s ♦♣t♦♥ st♥r ♦ ♥ ♥① ♦ s r♦♥ ❲ ♦♥s♠♦r s♠ s ♥♦ ♦r♥ ❲❲♦ ②s ♦r ♥♦ ♥ ♠ ①♣♦sr ♦♦ ♦r♥♦t ♦♦ ♦r t ♥②ss t ♦♥tr♣t ♠t♦ s s ♥ ♥ ♦rr t♦♦r ♥ str♥ ♦♥t①t
♦ st♦♥ ♣rs♥ts t s ♦ t rtr♦♥ ♦r t ♠♥ t ♠ ♠♦s ❯♥t ♦r sss t rsts ♦ t ♠ ♠♦ r ttrt♥ t♠ ♦ t ♠ ♠♦ st♦♥ ♦ ss ♥♠r s ttr ♦r t ♠♠♦ s♥ t sts t tr ♥♠r ♦ sss t ♠ ♠♦ ♦rst♠tst
g ♠ ♠
❱s ♦ t rtr♦♥ ♦t♥ ② ♦t ♠♦s t r♥t♥♠rs ♦ sss st s ♦r♥ t♦ t rtr♦♥ r ♥ ♦
♦ ♥tr♣rtt♦♥ r s♠♠r③ t rsts ♦ t st♠♠♦♦r♥ t♦ t rtr♦♥ t ♦s t♦ sr t sss ② tr ♠♥ trs ♣r♦♣♦rt♦♥s ♥trss ♦rrt♦♥s ♥ ♦r♥ts t st♠t sss rr♣rs♥t t rs♣t t♦ tr ♣r♦♣♦rt♦♥ ♥ rs♥ ♦rr ♦t tt tr♦rrs♣♦♥♥ r ♣♥s ♦♥ tr ♣r♦♣♦rt♦♥ ♠t ♣r♦♣♦rt♦♥s r
♥②ss ♦ t♦ r t sts
♥t ♦♥ t t s ♥ sss tr ♥t♦♥s r ♥ rst ♦♥ st ♥trrs ♦rrt♦♥s ρkb ♦r t ♦s ♦ t ss ♦rr ② trstr♥t ♦ ♦rrt♦♥ ♥ rs♥ ♦rr s♦♥ ♦♥ s t ♥trrs♦rrt♦♥s τ kb ♦r ♦ r♥ ♦r♥ t♦ t str♥t ♦ tr ♣♥♥s ♥ rs♥ ♦rr tr s t rs r♣rtt♦♥ ♣r ♦s ♥ts tt t r s ss♥ t♦ t ♦ ♥ t ♥tstt ♦♥t♦♥② ♦♥ ts ss t r s ♥♣♥♥t ♦ t rs ♦ ts♦ ♦r ①♠♣ ts r s♦s tt t rst ss s ♣r♦♣♦rt♦♥ ♦ 0.49♥ tt t rs r s♣t ♥t♦ tr ♦s
ss ②♦♥ ♠s ♥r ts ss ♣r♦♣♦rt♦♥ s q t♦ r r t♦ ♣♥♥②
♦s ♥ ♦♥ ♦ ♦ ♥♣♥♥ ♦ ♥ ts ss t ♦♠♥ ♥ tr r♥ ♥♠r r
♦rrt ρkb t ♣rs♥ ♦ ♦t ①tr♠ stt♦♥s ②♦♥ ♦♠♥t♦t ♥ ♦ ♦♠♥ t ♦ts ♦ r♥ ①♣♥ ② ♦t δkb♥ τ kb
♦ t t♦♥ ♦ ♦t ♠♠rs ♦ ♦♣ r ♦s δkb♥ t♦♥ s ♠♦st ♣rs♥t τ kb
♦ t ♣rt ♦ s♠ s ♥r ♦♣ ♠♠rs ♦s♥ t♦r② t♦ ♥ tr ♥ tr ♥ ♥① st②s ♦ αkb
ss ♦ ♥ ♥♦t ♣rt♥ s♠ ♥r ts ss ♣r♦♣♦rt♦♥ s q t♦ r r t♦ ♣♥♥②
♦s ♥ ♦♥ ♦ ♦ ♥♣♥♥ ♦ tr s str♦♥ ♦rrt♦♥ t♥ t ♥ ♦ t s♥s
♦♣t♦♥ ♥ t s r♦♥ ρkb ♥ ts ss t ♦♠♥ ♣rt♥s♠ ♥r② s♥ t t ♦♣t♦♥s δkb ♥ τ kb
♦ ts ♦ s♦s ♥ t♥ t ♥♠r ♦ r♥ ♥ t ♦ t ♦♠♥ ♦r r t ♦♠♥ t ♠♦r r♥ t② δkb
♦ ♥ ts ss ♦t ♠♠rs ♦ t ♦♣ ♦♥ stsαkb
ss ♣♦♦r ♥ r ♠s ♥r ts ss ♣r♦♣♦rt♦♥ s q t♦ r s ♦♥ ♦ ♦
♥♣♥♥ ♦ ts s ss r t ♥♠r ♦ r♥ s r② % ♦
♦♠♥ t st r♥ t ♦♥ssts ♠♦st② ♦ rtr ♦ ♦♠♥t ♦ s ♦ t♦♥ s s tr s♥s ② ♦r ♥r♦♣s ♥ ♣rt ♦ s♠ s ♥r ♦♥ ♥ ts t♦r② ♥s ♥♦t ①♣♦s t♦ t ♠ αkb
♦♥s♦♥ t s ♥♦t tt t ♠ ♠♦ s ♠♦r r♥t ♦r ts t st♥ t ♥♠r ♦ sss s ♠t ♥ t② r ♥tr♣rt ♥ t♦♥ tss♠♣t♦♥ ♦ ♦♥t♦♥ ♥♣♥♥ t♥ rs s♠s t♦♦ str♥♥t ♦rs♦♠ ♦♣s ♦ r rt♦♥s♣ t♥ ♥ ♥♠r ♦ r♥ rt♦♥st♥ t t♦♥ ♦ ♦t ♠♠rs ♦ ♦♣ ♥ ♦♥tr② r sts②st♠ s ♣rs♥t
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
1 0.75 0.5 0.25 00 0.25 0.5 0.75 1
ρkb τkb σkb
0
0.49
0.86
1
Class 1
Chi
WAg
WEd
HEd
HOc
Liv
WRe
WWo
Med
Chi
WAg
WEd
HEd
HOc
Liv
WRe
WWo
Med
Chi
WAg
WEd
HEd
HOc
Liv
WRe
WWo
Med
Class 2
Chi
WAg
WEd
HEd
HOc
Liv
WRe
WWo
Med
Chi
WAg
WEd
HEd
HOc
Liv
WRe
WWo
Med
Chi
WAg
WEd
HEd
HOc
Liv
WRe
WWo
Med
Class 3
Chi
WAg
WEd
HEd
HOc
Liv
WRe
WWo
Med
r ♠♠r② ♦ t st ♠ ♠♦ ♦r♥ t♦ t rtr♦♥ ♦rt ♦♥tr♣t ♠t♦ ♦ t st
s str♥
t ♥s s♦♥ ♦♠♣♥② s ♦t ♥♦r♠t♦♥ r♦♠ tr♥ rrs ♥ ♦rr t♦ str s st s r sr② ♥♥ rs ♦ ♦r ♣tt ♦r s♥ ♣t ♦r ♦ t ♠♦trst ♦r t ♥ s♦ ♥ t rt trt♠♥t ♥st ♦♠♣t rs♣rt♦r② ss ♥ rr ♠s s♥t♦♥ s ♠s♠♣t②♥ ♠♣ ♠♦tr ♣r♥t trt♠♥t ♥st rs♣rt♦r② ss ♥rr
♥♦r♠t♦♥ rtr s♣②s t rtr♦♥ s ♥ t ♥♠r♦ ♣r♠trs rqr ② t ♠ ♥ t ♠ ♠♦s rtr♠♦r t ♦♠♣t♥ t♠ ♥ ♠♥ts ♦t♥ t ♣r♦ss♦r ♥t ♦r t♦ st♠tt ♠ ♠♦ ② strt♥ ♠♠ ♥s t st♦♣♣♥ rtr♦♥ ♦ qmax = 180 t ♠ ♠♦ ♥s s t t ♣ ①♠♦ ❬+❪
♦r t ♠ ♠♦ t rtr♦♥ sts ♥♠r ♦ sss s♥ tst t sss ♥tr♣rtt♦♥ ♦ t strs s s♦ t ♥ ♥ ss♠ tt t qt② ♦ t st♠t s r② ♣♦♦r r ♣s t♥tr♣rtt♦♥ ♦r t ♠ ♠♦ t ♦♠♣♦♥♥ts st ♠♦ ♦r♥ t♦t rtr♦♥ ts ♥tr♣rtt♦♥ s t s♠ s t ♥tr♣rtt♦♥ ♦ r ♦r ①♠♣ ts r s♦s tt t rst ss s ♣r♦♣♦rt♦♥ ♦ 0.29 ♥ t s♦♠♣♦s ♦ ♦r ♦s ♠♦st ♦rrt ♦ ♦ t rst ss s ρkb ≃ 0.80
♥②ss ♦ t♦ r t sts
g ♠
ν♠
♠ ν♠
t♠ ♠♥
sts ♦r t ♠ ♥ t ♠ ♠♦s ♦r♥ t♦ r♥t ♥♠rs♦ sss ♦r ♦t ♠♦s rst r♦ ♦rrs♣♦♥s t♦ t rtr♦♥ s ♥ ts♦♥ r♦ ♥ts t ♥♠r ♦ ♦♥t♥♦s ♣r♠trs st rsts ♦r♥t♦ t rtr♦♥ r ♥ ♦ ♦♠♣t♥ t♠ ♦r t ♠ ♠♦ st♠t♦♥ s♥ ♥ ♠♥ts
♥ t str♥t ♦ t st ♠♦ts ♥ s ♦s t♦ 0.85 s ♦ ♦♥ssts♥ t rs ♥
1 0.75 0.5 0.25 00 0.25 0.5 0.75 1
ρkb τkb σkb
0
0.29
0.56
0.76
0.9
1
Class 1
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Class 2
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Class 3
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Class 4
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Class 5
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
Apt
TOC
TRC
TDC
Iso
Dis
Emp
TRM
TDM
r ♠♠r② ♦ t st ♠ ♠♦ ♦r♥ t♦ rtr♦♥ ♦r ts t st
♥tr♣rtt♦♥ ♦ ss r s ♥♦ ♣♦ss ♥tr♣rtt♦♥ ♦ ss ♥♦ttt t ♦trs sss r s♦ ♠♥♥ s ts ♥ ❬❱❪
♥r ts ss s ♣r♦♣♦rt♦♥ q t♦ ♥ ♦♥ssts ♦ tr♦s ♦ ♣♥♥② ♥ ♦♥ ♦ ♦ ♥♣♥♥
♦ tr s str♦♥ ♦rrt♦♥ ρ11 t♥ t rs rrtrt♠♥t ♦ t ♥ ♠♦tr ♣r♥t trt♠♥t ♥st rs♣rt♦r②
♣tr ♦s str♥ t ♦s ♦ ①tr♠ strt♦♥s
ss s♣② t♥ t ♠♦t② ♥♦ trt♠♥t ♥st t rr ♥ t s♥ ♦ ♣r♥t trt♠♥t ♥st rs♣rt♦r② ss ♦ts ♠♦tr τ 11 ♥ δ11
♦ tr s str♦♥ ♦rrt♦♥ ρ12 t♥ t rs trt♠♥t ♥st rs♣rt♦r② ♥ss ♦ t ♥ ♠♦tr ♣r♥t trt♠♥t ♥st rr s♣② t♥ t ♠♦t② ♣r♥t trt♠♥t♥st rs♣rt♦r② ♥ss ♦ t ♥ t ♣rs♥ ♦ rr ♣r♥ttrt♠♥t ♦ ts ♠♦tr τ 12 ♥ δ12
♦ tr ①sts ♥♦tr str♦♥ ♥ t♥ t ♦r ♦ t♠♦tr t ♠♣t②♥ ♦ t ♠ ♥ ts s♥t♦♥ τ 13 ♥ δ13
♦ ts ♦ s rtr③ ② ♥ s♥ ♦ ♣r♥t trt♠♥t♥st ♦♠♣t ♥ ♦♥t♥s 50% ♦ t s ♥t ② ts ♥ss α14
♦♥s♦♥
② s♥ t ♦ ①t♥s♦♥ ♦ t ♠ ♠♦ ♥ ♠①tr ♠♦ t♠ ♠♦ s ♥ ♣r♦♣♦s t♦ str t♦r t ② t♥ ♥t♦ ♦♥tt ♥trss ♣♥♥② ♦ strt♦♥ ♦ t ♠ ♠♦ s ♥ s ♠①tr t♥ ♥ ♥♣♥♥ strt♦♥ ♥ ♠①♠♠ ♣♥♥② strt♦♥ s s♣ strt♦♥ st②s ♣rs♠♦♥♦s ♥ ♦s r♥t s ♦♥tr♣rtt♦♥ rst s ♥ ② t ♦s ♦ rs r♥ ♦t t♦♥t♦♥ ♣♥♥s t♥ rs ♥ ② t ♣r♦♣♦rt♦♥s ♦ t ♠①♠♠♣♥♥② strt♦♥s rtr③ t str♥t ♦ ts ♣♥♥s s♦♥ s ♠♦r ♣rs s♥ t ♣r♠trs ♦ t ♦ strt♦♥ rt t♥s t♥ ♠♦ts ♥ tr str♥ts ♠ ♠♦ s ♥ ♦♠♣rt♦ t t♥t ss ♠♦ ♦♥ t♦ r t sts
♣r♠tr ♥ t ♠♦ r s♠t♥♦s② st♠t ♠♠ ♦rt♠ s ♦rt♠ ♦s t♦ r t ♦♠♥t♦r ♣r♦♠s ♦ t ♦strtr tt♦♥ ♥ t ♥s t♥ ♠♦ts sr ♦r t st♠t♦♥ ♦t ♠①♠♠ ♣♥♥② strt♦♥ rsts r ♦♦ ♥ t ♥♠r ♦♠♦ts s s♠ ♦r r ♦r ♠♦r t♥ s① ♠♦ts t tt♦♥♦ ♦tr ♥s ♠ts s♦♠ ♣rsst♥t ts ♦ t ♦rt♠ ♥ s♦ ♥ts s ♣r♦♣♦s ♣♣r♦ t♦ st♠t t ♦ strtr s ♥♦t ♣t ♦rt sts t ♦ts ♦ rs
♠♥ r ♦ ts ♦rt♠ s ts ♥ t♦ ♦♠♣t t ♠ ss♦tt♦ ♥t ♠♦ s st♠t♦♥ s t♠ ♦♥s♠♥ ♥ ♦♥② t ♠
ss♦t t♦ t st ♠♦ s ♥tr♣rt s ♣r♦♣♦s ♥ t ♥①t ♣tr ♥ ♠①tr ♠♦ ♦♥ ts r s♥ ts ♥trt ♦♠♣tt♦♦ s ①♣t s ♣r♦♣rts ♦s t ♥s t♦ s t ♠ t♦ ♣r♦r♠t ♠♦ st♦♥
♥② t ♣r♦♣♦s ♠♦ ♥ s② ①t♥ t♦ t s ♦ ♦r♥ t♦r ts s♦♠ t♦♥ ♦♥str♥ts ♦♥ t ♣♥♥② strtr ♦ strt♦♥ ♦ ♠①♠♠ ♣♥♥② ♥ t♦ ♦t tt ts ♦♥str♥ts s♦♠t t ♦♠♥t♦r rsr ♦ t ♣♥♥② strtrs
♣tr
♦s str♥ t
♦♥t♦♥ ♣♥♥② ♠♦s
s ♣tr ♣rs♥ts ♦r s♦♥ ♦♥trt♦♥ t♦ t♠♦s r♠♦r ♣r♠tt♥ t♦ str t♦r t s ♦♥trt♦♥ ♦♥ssts ♥ ♠①tr ♠♦ r♦♣s t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦ ♦♦s ♣rs♠♦♥♦s ♠t♥♦♠ strt♦♥ r t r ♣r♠trs ♦rrs♣♦♥ t♦ ts ♠♦s ♥r♥ s s② ♣r♦r♠ ♥ ♠ ♦rt♠ t ♥ ♦ t ♠♦ st♦♥ s tt② ♥ ♥t ♣♣r♦①♠t♦♥ ♦ t ♥trt ♦♠♣tt ♦♦♠r ①♣r♠♥ts ♦♥ s♠t ♥ r t sts♥r♥ t ♠♥ rtrsts ♦ ts ♥ ♠①tr♠♦
r♠ ♣♦ rrst ts ♠♥ ts ♥ ♠ts
③③s rs t♥ r♦
♦ r♠ P♦
♥tr♦t♦♥
♥ ts ♣tr ♣rs♥t s♣rs ♠①tr ♠♦ r①s t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ ♥ ♦rr t♦ ♦r♦♠ t ss s ② t t♥t ss♠♦ ♥ ♦r♦♠s t ♠♥ r ♦ t ♠ ♠♦ ♥ t♠♦ st♦♥ ♥ s② ♥ ♥t② ♣r♦r♠ ② ♦♥ t ♦♠♥t♦r♣r♦♠s s st♣ ♦s ♥♦t rqr t ♠ s♥ t ♥trt ♦♠♣tt♦♦ ♥ ♣rs② ♣♣r♦ rst② t ♠♦ st♦♥ ♥ ♣r♦r♠② ♠♠ ♦rt♠ r t ♠ s ♥♦t rqr ♦♥② t ♣r♠trs r♦♥② st♠t ♦r t st ♠♦ s♥♥t② ♠ts t ♦♠♣tt♦♥ t♠
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
s ♥ ♠♦ ♥♠ ♦♥t♦♥ ♦s ♦ rrr ♥ ts rt ②♠♠ r♦♣s t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♦r ♦♥sr♥ t♠♥ ♦♥t♦♥ ♣♥♥s ♦r♦r t s♣ strt♦♥ ♦ t ♦ s ♠t♥♦♠ strt♦♥ ♣r ♠♦s s strt♦♥ ss♠s tt ♠♦t②r♦ss♥s ♥♠ ♠♦s r rtrst ♥ tt t ♦tr ♦♥s ♦♦ ♥♦r♠strt♦♥ s t ss♦t ♠t♥♦♠ strt♦♥ s ♣rs♠♦♥♦s s♥ts r ♣r♠trs r ♠t t♦ t ♣r♠trs ♦ t ♠♦s
s s♠♣ ♠①tr ♠♦ ♠♠ s ♦♦ ♥r ♥ t ♦♥ ♥ t♠♠ ♠♦ ♥s t ♠①tr ♠♦ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♠ s♥ t ♦s ♠♥② ss tr♦ ♠♦③♥ ♦ t ♠♥ ♦♥t♦♥♣♥♥s ♥ t ♦tr ♥ t ♥s t ♠①tr ♠♦s r①♥ tsss♠♣t♦♥ s♥ ts ① strt♦♥ ♦ t ♦ rqrs ♣r♠trs ♦ttt s t ♠ ♠♦ t ♠♠ ♠♦ ♥ ♥tr♣rt s ♣rs♠♦♥♦srs♦♥ ♦ t ♦♥r ♠①tr ♠♦ ♥ t r♣rtt♦♥ ♦ t rs♥t♦ ♦s ♥s t ♦♥sr ♥trt♦♥s t strt♦♥ ♣r ♠♦s ♥t♦♦s ♥s s♣ strt♦♥ ♦r ♥trt♦♥ rtr♠♦r rst♥sss r ♠♥♥ s♥ t ♥trss ♣♥♥s r r♦t ♦t ② t♦♦♠♣♠♥tr② s t ♦ r ♥trt♦♥ ♥ t ss♦t ♠♦♥trt♦♥ tr♦ ♦t♦♥s ♥ ♣r♦ts ♦t tt t ♠♠ ♠♦ s ♦♠♣r♥s ♣♣r♦ s♥ t ♥s t ♠ ♠♦ ♥ ♣rt ♦ ts ♣rs♠♦♥♦s rs♦♥s ♣rs♥t ♥ t♦♥
♦r ① ♠♦ ♥♠r ♦ sss r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♥♥♠rs ♦ ♠♦s t ♠①♠♠ ♦♦ st♠t s ♦t♥ ♥ ♠ ♦rt♠ ♠♦ s st tr♦♣♦st♥s ♦rt♠ ♥ ts♦rt♠ s s s♠♣r ♥rts ♥ r♣rtt♦♥ ♦ t rs ♥t♦♦s ♥ ♥ ♥♠r ♦ ♠♦s ② ♦♥ tr♦♣♦sst♥s st♣ t s ♣r♦r♠♦r ① ♥♠r ♦ sss ♥ ♦s ♦♠♥t♦r ♣r♦♠s ♥♦ ② t st♦♥ ♦ t ♦s ♦ rs ♥ ② t st♠t♦♥ ♦ t ♥♠rs ♦ ♠♦ss ♦rt♠ s s ♦♥ t t tt t ♥trt ♦♠♣tt ♦♦ rqr ♦r t ♣t♥ ♣r♦t② ♦♠♣tt♦♥ ♦ t tr♦♣♦sst♥s ♥st s s♠♣r s ss ♥ ♥♦♥ ♠♦s tr♦ ② ♥♦r♠t ♦♥t ♣r♦r ♥② ts ♣♣r♦ s t♦ ♠♥ ♥ts t ♣r♠ts t♦ rt s ♦ t ♣♣r♦ t s ♠♥t♦♥ tt t ♦rst♠t♦♥ ♦ t♥♠r ♦ ♠♦s ② ts ♣♣r♦ s strt r♥ ♦r ♥♠r ①♣r♠♥tsrtr♠♦r t ♦s s t♦ ♣r♦r♠ ♥ ♥t ♠♦ st♦♥ ♥ rs♦♥♦♠♣tt♦♥ t♠ s♥ t ♣r♠trs r ♦♥② st♠t ♦r t ♥q st♠♦ s ts ♣♣r♦ s ♣♦ss ♥sr t♦ t ♦♠♥t♦r ♠♦ st♦♥♣r♦♠ s ♥♦♥ t♦ r ♥ ♦r ♦♥r ♠①tr ♠♦
trtr ♦ ts ♣tr s ♣♣r s ♦r♥③ s ♦♦s t♦♥ ♣rs♥ts t ♦♥t♦♥ ♦s ♦ t♦♥ s ♦t t♦ ♠①♠♠ ♦♦ st♠t♦♥ ♥ ♠ ♦rt♠ t♦♥ ♣rs♥ts t tr♦♣♦st♥s s♠♣r ♣r♦r♠♥ t ♠♦ st♦♥ tr♦ t ♥trt ♦♠♣tt♦♦ ♥ t♦♥ s♦ tt t ♣r♦♣♦s ♣♣r♦ ♦r ♦♠♣t♥ t ♥trt ♦♠♣tt ♦♦ rs t ss ♦ t ♣♣r♦ ♦r
①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s
♦r ♥♠r② ♠♣s③ t ♦♦ ♦r ♦ t tr♦♣♦st♥ss♠♣r ♥ t ①t② ♦ t ♠♠ ♠♦ ♦♥ s♠t t t♦♥ ♣rs♥tst♦ str ♥②ss ♦ ♦♦ t sts ♣r♦r♠ ② t ♣ ♦♦s ♦♥s♦♥ s r♥ ♥ tr ①t♥s♦♥s r sss ♥ t♦♥ ts rsts r ♣rt ♦ t rt ♥t ♠①tr ♠♦ ♦ ♦♥t♦♥ ♣♥♥s ♠♦st♦ str t♦r t ❬❱❪
①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r
♠♦s
♥ ♣r♦♣♦s ♠♦ rrr s ♦♥t♦♥ ♦s ♦ ♠♠ss♠s tt t rs ♥♣♥♥t② r♦♠ ♠①tr ♦ g ♦♠♣♦♥♥ts ♦ ♦♥t♦♥② ♥♣♥♥t ♦s r t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s s qt♥ sss ♦ ♦♦s ♠t♥♦♠ strt♦♥ ♣r ♠♦s s ♠t♥♦♠ strt♦♥ ♥ r ♣r♠trs ♦rrs♣♦♥♥ t♦ t ♠♦s♦ t strt♦♥ ♦r ♣rs② t ♠♦s r ♥ s t ♦t♦♥s ♦ trst ♣r♦ts t ♦tr ♣r♠trs r q
♦♥t♦♥② ♥♣♥♥t ♦s q t♥ sss
r♣rtt♦♥ ♦ t rs s ss♠ t♦ q t♥ sss s♦ s t ♥♦tt♦♥s ♦ t ♠①tr ♠♦ ♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♥♥ t♦♥ ② ♦♠tt♥ t ♥①t♦♥ ♦♥ k
♣rtt♦♥ ♦ t rs qs t♥ sss r♣rtt♦♥ ♦ td t♦r rs xi = (①1i , . . . , ①
di ) ♥t♦ ♦s tr♠♥s ♣rtt♦♥ σ =
(σ1, . . . ,σ) ♦ 1, . . . , d ♥ s♦♥t ♥♦♥♠♣t② ssts s ♣rtt♦♥ ♥s♥ ♥rt t♦r rs xb
i = xσbi = (x
bhi ;h = 1, . . . ,mb) ♦t♥
② t ♦♥t♥t♦♥ ♦ t sst ♦ xi ss♦t t♦ σb r mb =∏
j∈σbmj
s t ♥♠r ♦ t ♠♦t② r♦ss♥s ♥t♦ ♦ b r xbi ss
s♥t ♦♥ s♥ xbhi = 1 ♥ i ts ♠♦t② h ♦r t ♥t♦r r t ♠♦t② r♦ss♥ h ♦ t ♥t rs t t♦t ♦ b ♥ xbhi = 0 ♦trs
r♣t ♥ ♠♦ s♣ ♠♦ s ♥ ② t ♥♠r ♦ ♦♠♣♦♥♥ts t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♥ t ♥♠r ♦ ♠♦s ♦r ♠t♥♦♠ strt♦♥ ♦ t s ♥ ② t tr♣t ω = (g,σ, ℓ) rℓ = (ℓ1, . . . , ℓg) r♦♣s t ♥♠rs ♦ ♠♦s t ℓk = (ℓk1, . . . , ℓk) ♥ rℓkb s t ♥♠r ♦ ♠♦s ♦ xb
i ♦r ss k (t 0 < ℓkb < mb)
♥t♦♥ ①tr ♦ ♦♥t♦♥② ♥♣♥♥t ♦s q t♥ sss t♦r r xi s r♥ ② ♠♠ ♠♦ ♥ ② ω ♥ ♣r♠tr③
♦♥♦ t tt♣sr♦rr♣r♦t♦rr♦♣❴
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
② θ ts ♣ s rtt♥ s
p(xi;θ,ω) =
g∑
k=1
πkp(xi;αk,σ, ℓk) t p(xi;αk,σ, ℓk) =∏
b=1
p(xbi ;αkb, ℓkb),
r θ = (π,α) ♥♦ts t ♦ ♠①tr ♣r♠trs r π = (π1, . . . , πg)s t t♦r ♦ ss ♣r♦♣♦rt♦♥s t 0 < πk ≤ 1 ♥
∑gk=1 πk = 1 ♥ r
α = (α1, . . . ,αg) s t t♦r r♦♣s t ♣r♠trs ♦ t ♠t♥♦♠strt♦♥s ♣r ♠♦s t αk = (αk1, . . . ,αk)
t♥♦♠ strt♦♥ ♣r ♠♦s
❲ ♥♦ s♣② t ♠t♥♦♠ strt♦♥ ♣r ♠♦s ♦ t s ♥tr♦ ts♣r♠tr s♣ ♦r t♦ ♥ ts ♣
♥t♦♥ Pr♠tr s♣ ♦ ♠t♥♦♠ strt♦♥ ♣r ♠♦s tαkb = (αhkb;h = 1, . . . ,mb) t t♦r ♦ s③ mb ♥ t τkb t ♠♣♣♥r♦♠ 1, . . . ,mb t♦ 1, . . . ,mb ♦rr♥ t ♠♥ts ♦ αkb ② rs♥ s αkb ♥♦ts t ♣r♠trs ♦ t ♠t♥♦♠ strt♦♥ ♣r ℓkb ♠♦st♥ t s ♥ ♥ t ♦♥str♥ s♠♣① S(ℓkb,mb) ♥ s ♦♦s
S(ℓkb,mb) =
αkb : 0 ≤ αhkb ≤ 1,mb∑
h=1
αhkb = 1, α(ℓkj+1)
kb = . . . = α(mb)kb
.
r s t s♦rtr ♥♦tt♦♥ α(h)kb = α
τkj(h)
kb s♦ α(h)kb ≥ α
(h+1)kb (1 ≤ h < mb)
♥t♦♥ t♥♦♠ strt♦♥ ♣r ♠♦s ♥rt t♦rt x
bi s mb ♠♦ts ♥ ♦♦s ♠trt strt♦♥ ♣r ℓkb ♠♦s
ts ♣ s s♦ rtt♥ s
p(xbi ;αkb, ℓkb) =
mb∏
h=1
(
αhkb)x
bhi ,
r αkb = (αhkb;h = 1, . . . ,mb) ∈ S(ℓkb,mb) ♥ αhkb s t ♣r♦t② tt
♥ i ts ♠♦t② h ♦ t ♦♥t♥t t♦r r xbi
①tr ♠♦ ♦ ♦♥t♦♥ ♠♦s
♥t♦♥ ①tr ♠♦ ♦ ♦♥t♦♥ ♠♦s t♦r r xis r♥ ② ♠♠ ♠♦ ♥ ② ω ♥ ♣r♠tr③ ② θ ts ♣ s ♥ ②
p(xi;θ,ω) =
g∑
k=1
πk
∏
b=1
mb∏
h=1
(
αhkb)x
bhi .
♠r ♠ ♠♦ s ♥ ♥ t ♠ ♠♦ s♥ t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ t♥ t ♥t rs s ♥ ② ♣tt♥ ♦♥r ♣r ♦ s♦ d = ♥ σ = (1, . . . , ) ♥ ② ①♥ t ♥♠r ♦♠♦s s t ♥♠r ♦ ♠♦ts ♦ t rs ♠♥s ♦♥ ℓkj = mj − 1
①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s
Pr♦♣rts ♦ t ♠①tr ♠♦ ♣r ♦♥t♦♥ ♠♦s
♦ s ♦ ♥tr♣rtt♦♥ ♠♠ ♠♦ s t♦ s ♦ ♥tr♣rtt♦♥rst② t ♥trss ♣♥♥s ♦ rs q t♥ sss r ♠♣s③ ② t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♥ ② σ ♦♥② t♥trss ♥ ♥tr♦ ♣♥♥s ♦ ♠♦ts ♣♦ss② r♥t t♥sss r s♠♠r③ ② t ♠♦s ♦t♦♥s ♥ ♣r♦ts
♦ ♦♠♣t tr♠s s♦rtr s♠♠r② ♦r strt♦♥ s s♦ ② s♥ t ♦♦♥ ♦♠♣t tr♠s κkb ♥ ρkb ♥ ♦♥ [0, 1] ②
κkb =ℓkb
mb − 1♥ ρkb =
ℓkb∑
h=1
α(h)kb .
② rt rs♣t② t ♦♠♣①t② ♥ t str♥t ♦ t ♥trss ♥ ♥tr♦ ♣♥♥s ♦r ♥st♥ t s♠r s κkb ♥ t rr s ρkb t ♠♦r♠ss ♥ rtrst ♠♦t② r♦ss♥s s t strt♦♥ ♥ t♠♦s r ♥tr♣rt s ♥ ♦r♦♥trt♦♥ t t ♥♦r♠ strt♦♥ ♠♦♥ t ♠♦t② r♦ss♥s
♥t② ♦t tt t r♣rtt♦♥ ♦ t rs r♥ts t ♠♦♥r ♥tt② s♥ t s q t♥ sss ♥ t ts ♦♥str♥tt rsts ♦ ❬❪ ♥ ♣♣ t♦ ♣r♦ t ♥r ♥tt② ♦ t♠♠ ♠♦ ts r ♥ ♥ ♣♣♥① s♣t t ♦♥str♥t t♦ ♦t s♠ r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ♦r t sss t ♠♦ st②s① s ♦ t s♣ ♦ strt♦♥
♠r ♦ ♣r♠trs ♠♥ ♦ t ♦r♠r ♣rs♠♦♥♦s rs♦♥s ♦ t♠♠♦ ♦♥t♦♥ ♥♣♥♥ ♦ ♣r♦♣♦s ♥ ❬❪ s t♦ ♦♥sr ♦♥②♦♥ ♠♦ ♦r ♠t♥♦♠ strt♦♥ ♦ t ♥t r s t♦♥ r♥t ♦♥str♥ts ♦ qt② r t♥ t♥ t rs ♥♦r sss♥ t ♠♥② ♦ ts ♠♦s r ♥ ♥t♦ t ♠♦ ♠② ♦ ♠♠ ② ♣tt♥ = d ♥ ℓkb = 1 ♥ t♦♥ t ♠♠ ♠♦s ♥ ν♠♠ ♣r♠trs ♥ ②
ν♠♠ = (g − 1) +
g∑
k=1
∑
b=1
ℓkb.
s ♠♦ ♦ t ♠♠ ♠② ♥ rqr ss ♣r♠trs t♥ ♠ ♠♦t ν♠ = (g−1)+g×∑
b=1(mb−1) ♣r♠trst♦ t ts ♥t♦ ♦♥t
t ♦♥t♦♥ ♣♥♥s
♣r♠tr③t♦♥ ♦ t ♦ strt♦♥
♥ ♣rs♠♦♥♦s rs♦♥s ♦ t ♠ ♠♦ ♥tr♦ ♥ ❬❪r ♠♥♥ s♥ ♠t♥♦♠ strt♦♥ s ①♣rss t t♦ t②♣s ♦♣r♠trs srt ♦♥ tr♠♥s t ♦t♦♥ ♦ t ♠♦ ♦ t strt♦♥♥ ♦♥t♥♦s ♦♥ s ts ♣r♦t② s t♦♥ ② s♥ t s♠
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
♣r♦♣♦s ♥ ♣r♠tr③t♦♥ ♦ t ♦ strt♦♥ ♥♦t ② (δkb,akb)s ♣r♠tr③t♦♥ tts t ♥tr♣rtt♦♥ ♥ t rt♥ ♦ t ♣r♦r ♥♣♦str♦r strt♦♥s rt t♦ t ♦ ♣r♠trs s t♦♥
♣r♠tr③t♦♥ srt ♣r♠tr δkb = δhkb;h = 1, . . . , ℓkb tr♠♥s t ♠♦ ♦t♦♥s s♥ δhkb ♥ts t ♠♦t② r♦ss♥ r t ♠♦h s ♦t t δhkb ∈ 1, . . . ,mb ♥ δhkb 6= δh
′
kb h 6= h′ ♦♥t♥♦s♣r♠tr akb = (ahkb;h = 1, . . . , ℓkb + 1) tr♠♥s t ♣r♦t② ♠ss ♦ t ℓkb♠♦s ② ts rst ℓkb ♠♥ts ahkb t h = 1, . . . , ℓkb) ♥ t ♣r♦t② ♠ss♦ t ♥♦♥♠♦ ② ts st ♠♥t aℓkb+1
kb ♣r♠tr akb s ♥ ♦♥ t♦♦♥ tr♥t s♠♣①
St(mb) =
akb : 0 ≤ ahkb ≤ 1, ∀h ≤ ℓkb + 1 ♥ ahkb ≥aℓkb+1kb
mb − ℓkb, ∀h ≤ ℓkb
.
♣r♠tr αkb ♥ t ♦♣ (δkb,akb) r rt ②
αhkb =
ah′
kb ∃h′ s tt δh′
kb = haℓkb+1
kb
mb−ℓkb♦trs
①♠♠ ♦♦ st♠t♦♥ ♥
♦rt♠
♠ t ① = (x1, . . . ,xn) t s♠♣ ♦♠♣♦s t n ♥♣♥♥t ♥ ♥t② strt ♥s ss♠♥ t♦ r♥ ② t ♠♠ ♠♦ r♦♠ tss♠♣ t ♠ s t♦ st♠t t ♠ ♦r ① ♠♦ ♥ ② ω
❲♥ ω s ♥♦♥ t ♠♠ ♠♦ ♥ ♥tr♣rt s ♠ ♠♦ ♣♣ ♦♥t ♦♥t♥t rs x
bi r ♦♥str♥ts r t♥ ♣r♠trs
s t ♠ ♥ s② ♦t♥ ② t ♦♦♥ ♠ ♦rt♠
♦ st♦♥ tr♦♣♦st♥s s♠♣r
trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ ♦♥t♦♥ ♣r♦ts ♦♠♣tt♦♥
tik(θ[r]) =
π[r]k p(xi;α
[r]k ,σ, ℓk)
∑gk′=1 π
[r]k′ p(xi;α
[r]k′ ,σ, ℓk′)
.
st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦
π[r+1]k =
n[r]k
n♥ α(h)[r+1]
kb =
n(h)[r]kb
n[r]k
(1 ≤ h ≤ ℓkb)
1−∑ℓkj
h′=1α(h′)[r+1]kb
mb−ℓkb♦trs
② s♥ t ♥♦tt♦♥s n[r]
k =∑n
i=1 tik(θ[r]) ♥ nh[r+1]
kb =∑n
i=1 tik(θ[r])x
bhi
♦rt♠ ♠ ♦rt♠ t♦ ♦t♥ t ♠ ♦ ♠♠ ♠♦
♠r ♥ t ♥t♦♥ τkb ♦t tt t t ♠ st♣ ♦ trt♦♥ [r] t♥t♦♥ τkb s r♥ s t rs♥ ♦rr♥ ♥t♦♥ ♦ t nh[r+1]
kb ♥ ♦ss t♦ ♥ n(h)[r+1]
kb t n(h)[r+1]kb ≥ n
(h+1)[r+1]kb
♦ st♦♥ tr♦♣♦st♥s
s♠♣r
Pr♦r strt♦♥s ❲ ss♠ tt p(g) = 1gmax
♦r g = 1, . . . , gmax ♥ ttp(σ) r♠♥ tt g ♥ σ r ♥♣♥♥t ♥ p(ℓ|g,σ) ♦♦ ♥♦r♠ strt♦♥s
♠ ♠ s t♦ ♦t♥ t ♠♦ ω = (g, σ, ℓ) s t rst ♣♦str♦r♣r♦t②
ω = argmaxg,σ,ℓ
p(①|g,σ, ℓ) = argmaxg,σ,ℓ
p(g,σ, ℓ|①).
t gmax ♠♦s ♥♦t ② ω(g) = (g,σ(g), ℓ(g)) ♦r g = 1, . . . , gmax r
(σ(g), ℓ(g)) = argmaxσ,ℓ
p(①|g,σ, ℓ) = argmaxσ,ℓ
p(σ, ℓ|①, g).
st ♠♦ s s♦ ♥ s
ω = argmaxg
p(ω(g)|①),
♥ s ♦♥ ② ♣♣②♥ t ♣♣r♦①♠t♦♥ ♠♦♥ t♦s gmax st ♠♦s
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
♥ ♥ ①st sr strt② s ♥♦t ♦ ♦r t♦ ♦rrt rs♦♥srst② t ♥♠r ♦ ♦♣s (σ, ℓ) ♥ ①ss② ♥ s♦♥② tst♠t♦♥ ♦ t ♠ ♦r ♦ t♠ s ♥ ♥♥ssr② st ♦ ♦♠♣tt♦♥t♠ tr♦♣♦st♥s s♠♣r strt② ♦r♦♠s ts t♦ rst t s♠ t♠ s ♥♦ sr
♦r ① ♦ g t ♦♣ (σ(g), ℓ(g)) s st♠t ② t ♦♦♥ tr♦♣♦st♥s s♠♣r ❬❪ ♥ p(σ, ℓ|g,①) s stt♦♥r② strt♦♥
s ♦rt♠ s p(σ, ℓ|g,①) s ♠r♥ stt♦♥r② strt♦♥ trt♥r♦♠ ♥ ♥t (σ[0], ℓ[0]) trt♦♥ [s] s rtt♥ s
θ[s+1] ∼ θ|ω[s],①, ③[s]
③[s+1] ∼ ③|ω[s],①,θ[s+1]
(σ[s+1], ℓ[s+1]) ∼ σ, ℓ|ω[s],①, ③[s+1],
r ω[s] = (g,σ[s], ℓ[s])
♦rt♠ tr♦♣♦st♥s s♠♣r t♦ ♦t♥ ω(g)
♠r ♥ t ♠♦ s♠♣♥ rt s♠♣♥ r♦♠ s ts st♣ s s♦ ♣r♦r♠ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠♦s t stt♦♥r② strt♦♥ s p(σ, ℓ|g,①, ③[r+1]) ♦r ts r ♥ ♥t♦♥
♠♣♥ ♦ t ss ♠♠rs♣s s t ♦sr t r ♥♣♥♥tt ♦♥t♦♥ strt♦♥ ♦ ③ s ss ♥ s rtt♥ s
p(③|ω,①,θ) =n∏
i=1
p(zi|ω,xi,θ) t p(zi|ω,xi,θ) =g∏
k=1
(tik(θ))zik .
♥ ts st♦♥ rst② t t ♦♥t♦♥ strt♦♥s s♠♣♥ t ♣r♠trs ♥♦t ② ♥str♠♥t ♠♥ts ② s♥ t ♦ ♣r♠tr③t♦♥♥ ♥ t♦♥ ♥ s♦♥② t t s♠♣♥ ♦ (σ, ℓ) ♦♥sr st ♥trst ♠♥ts
♠♣♥ ♦ t ♥str♠♥t ♠♥ts
❲ ♥♦ t t s♠♣♥ r♦♠ ♥ ② p(θ|ω[s],①, ③[s])
Pr♦r ss♠♣t♦♥ ❲ ss♠ t ♣r♦r ♥♣♥♥ t♥ t ss ♣r♦♣♦rt♦♥s ♥ t ♣r♠trs ♦ t ♦ strt♦♥s ♦ t ♣r♦r ♦ t ♦♣r♠tr s rtt♥ s ♦♦s
p(θ|ω) = p(π|ω)
g∏
k=1
mb∏
b=1
p(αkb|ω).
♦ st♦♥ tr♦♣♦st♥s s♠♣r
♦t tt ts ♣r♦♣rt② ♦ ♦♥t♦♥ ♥♣♥♥ s ♣t ② t strt♦♥ ♦ θ♦♥t♦♥② ♦♥ (ω,①, ③) s♥
p(θ|ω,①, ③) = p(π|ω,①, ③)g∏
k=1
mb∏
b=1
p(αkb|ω,①, ③).
Pr♦r ♥ ♣♦str♦r strt♦♥s ♦ π r②s ♥♦♥ ♥♦r♠t ♣r♦rstrt♦♥ ♦r ♠t♥♦♠ s ♦♥t rt strt♦♥ ❬♦❪ ♦t ♣r♦r ♥ t ♣♦str♦r strt♦♥s ♦ π ❬❪ r rs♣t② ♥ ②
π|ω ∼ Dg
(1
2, . . . ,
1
2
)
♥ π|ω,①, ③ ∼ Dg
(1
2+ ♥1, . . . ,
1
2+ ♥g
)
,
r ♥k =∑n
i=1 zik
Pr♦r strt♦♥ ♦ αkb ❲ ♥♦ s t ♣r♠tr③t♦♥ ♦ t ♦ strt♦♥ (δkb,akb) ♥ ♥ t♦♥ ❲ ss♠ t ♥♣♥♥ t♥t ♣r♦r ♦ δkb ♥ ♦ akb s♦
p(αkb|ω) = p(δkb|ω)p(akb|ω).
❲ s ♥♦r♠ strt♦♥ ♠♦♥ t ♠♦ ♦t♦♥s ♥ ♦♥t tr♥t rt strt♦♥ s ♣r♦r ♦ akb s♦
p(δkb|ω) =
(
mb
ℓkb
)−1
♥ akb|ω ∼ Dtℓkb+1
(
γ1kb, . . . , γℓkb+1kb ;mb
)
,
r t γhkb r t ♣r♠trs ♦ t tr♥t rt strt♦♥ s♦ ttakb|ω ∈ St(mb) ❲ ♥♦ ① γhkb = 1 stt♦♥ s ♥ ♥ ♣♣♥① ♣r♦♣♦s ♣r♦r s s♦ ② ♥♦r♠t s♥ t s ♥ ♥♦r♠ strt♦♥
P♦str♦r strt♦♥ ♦ αkb ♣♦str♦r strt♦♥ ♦ αkb s rtt♥ s
p(αkb|ω,①, ③) = p(δkb|ω,①, ③)p(akb|ω, δkb,①, ③).
strt♦♥ ♦ δkb|ω,①, ③ s ♠t♥♦♠ ♦♥ t t♦♦ ♠♥② s t♦ ♦♠♣t t δkb = δhkb;h = 1, . . . , ℓkb t st ♦♥t♥♥ t ♥s ♦ tℓkb rst s ♦ ♥hkb =
∑ni=1 zikx
bhi ♦rr
∀h ∈ 1, . . . , ℓkb − 1, ♥δhkbkb ≥ ♥
δh+1kbkb .
❲ ss♠ tt t r♥ t♥ t ♠♦ ♣r♦ts ♥ t ♥♦♥♠♦♣r♦ts r s♥♥t ♦ ♥ ♣♣r♦①♠t t ♦♥t♦♥ strt♦♥
p(akb|ω) ∝∏ℓkb+1
h=1(ahkb)
γh
kb−1
1
ah
kb≥
aℓkb+1
kb
mb−ℓkb
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
♦ δkb ② r ♥ δkb s ♣♣r♦①♠t♦♥ s str♥t♥ ② t st ♦♥r♥ s♣ ♦ t srt ♣r♠trs ❬❪ ♦♥r♥♥ ♥♦ akb s ts ♣r♦r s♦♥t ts ♦♥t♦♥ strt♦♥ s ①♣t② ♥ s
akb|ω, δkb,①, ③ ∼ Dtℓkb+1
(
1 + ♥(1)kb , . . . , 1 + ♥
(ℓkj)
kb , 1 + ♥ℓkbkb ;mb)
,
r ♥(h)kb s t ht rr ♦ t st ♥hkb;h = 1, . . . ,mb ♥ ♥ℓkbkb =
♥k −∑ℓkb
h=1 ♥(h)kb
♠♣♥ ♦ ♥ ♠♦ (σ[s+1], ℓ[s+1])
♥ s♠♣♥ ♦ ω[s+1] = (g,σ[s+1], ℓ[s+1]) ♦r♥ t♦ s ♣r♦r♠ ② ♦♥ trt♦♥ ♦ t ♦♦♥ ♠♠ ♦rt♠ ♦s t stt♦♥r② strt♦♥ s p(σ, ℓ|g,①, ③[r+1]) s ♦rt♠ s ♥ t♦ st♣s rst② ts♠♣s ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ ♥ r♣rtt♦♥ ♦t rs ♥t♦ ♦s ♥ t ♠♦ ♥♠r ♦ t ♠♦ ♦s ♥♦t rs♣t② ② σ[s+1] ♥ ℓ[s+1/2] ♦♥② t s♠♣s t ♠♦ ♥♠r ♦ ♦② ♦♥ ♠♠ trt♦♥ s t s♠♣♥ ♦ ω[s+1] s ♦♠♣♦s ♥t♦ t t♦♦♦♥ st♣s
s ♦rt♠ s p(σ, ℓ|g,①, ③[s+1]) s stt♦♥r② strt♦♥ t ttrt♦♥ [s] ♦ ♦rt♠ t s♠♣♥ ♦ ω[s+1] s ♣r♦r♠ ♦r♥t♦ ♦t ♦♦♥ st♣s
(σ[s+1], ℓ[s+1/2]) ∼ σ, ℓ|ω[s],①, ③[s+1]
ℓ[s+1] ∼ ℓ|ω[s+1/2],①, ③[s+1],
r ω[s+1/2] = (g,σ[s+1], ℓ[s+1/2])
♦rt♠ ♦rt♠
tr♦♣♦sst♥s ♦rt♠ t♦ s♠♣ ω[s+1/2]
s♠♣♥ ♦ ω[s+1/2] s ♣r♦r♠ ② ♦♥ trt♦♥ ♦ t tr♦♣♦sst♥s♦rt♠ ♥t♦ t♦ st♣s rst② t ♥str♠♥t strt♦♥ q(.;ω[s])♥rts ♥t ω⋆ = (g,σ⋆, ℓ⋆) ♦♥② ω[s+1] s s♠♣ ♦r♥ t♦ t♣t♥ ♣r♦t②
♥str♠♥t strt♦♥ ♥str♠♥t strt♦♥ q(.;ω[s]) s♠♣s ω⋆
♥ t♦ st♣s rst st♣ ♥s t ♦ tt♦♥ ♦ ♦♥ r ♥ ♣rtσ⋆ s ♥♦r♠② s♠♣ ♥ V (σ[s]) = σ : ∃!b s b ∈ σ
[s]j ♥ b /∈ σj s♦♥
st♣ ♥♦r♠② s♠♣s t ♠♦ ♥♠rs ♠♦♥ ts ♣♦ss s ♦r t♠♦ ♦s ℓ⋆kj = ℓ
[s]kj ♦r ♥♦♥♠♦ ♦s j s tt σ[s]
j = σ⋆j
♦ st♦♥ tr♦♣♦st♥s s♠♣r
♣t♥ ♣r♦t② ♣t♥ ♣r♦t② λ[s] s ♥ ②
λ[s] = min
p(①, ③[s+1]|ω⋆)
p(①, ③[s+1]|ω[s])
q(ω[s];ω⋆)
q(ω⋆;ω[s]); 1
.
♦♠♣tt♦♥ ♦ λ[s] ♥♦s t♦ ♦♠♣t t ♥trt ♦♠♣tt ♦♦ ❲ ♥♦ sr ♦ t♦ s♦ ts ♣r♦♠ t♦t s♥ t s
♣♣r♦①♠t♦♥ ♦r s♥ t♦♦ ♠ t♠ ♦♥s♠♥ ♠♠ ♠t♦s s♠♣♥♦ ω[s+1/2] s s♦ ♣r♦r♠ ② t ♦♦♥ tr♦♣♦sst♥s ♦rt♠
s ♦rt♠ s p(σ, ℓ|g,①, ③[s+1]) s stt♦♥r② strt♦♥ trt♥r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s
ω⋆ ∼ q(ω;ω[s])
ω[s+1/2] =
ω⋆ t ♣r♦t② λ[s]
ω[s] t ♣r♦t② 1− λ[s].
♦rt♠ tr♦♣♦sst♥s ♦rt♠
♦rt♠ t♦ s♠♣ ℓ[s+1]
s st♣ ♦s s t♦ ♥rs ♦r rs t ♠♦ ♥♠r ♦ ♦ ②♦♥ t trt♦♥ ♦ ℓ[s+1]
kb s s♠♣ r♦♠ p(ℓkb|ω[s+1/2],①, ③[s+1]) ♥ ②
p(ℓkb|ω[s+1/2],①, ③[s+1]) ∝
p(①b|③[s+1], ℓkb) |ℓkb − ℓ[s+1/2]kb | < 2
♥ ℓkb /∈ 0,mb.0 ♦trs
r ①b = (xbi ; i = 1, . . . , n) s ts ♦rt♠ rqrs t ♦♠♣tt♦♥ ♦
p(①b|③, ℓkb) ♥ ②
p(①b|③, ℓkb) =∫
S(ℓkb,mb)
mb∏
h=1
(αhkb)♥hkbdαkb
♥ tt t ♥♦
♥trt ♦♠♣tt ♦♦
♥trt ♦♠♣tt ♦♦ s ♥ s
p(①, ③|ω) = p(③|ω)
g∏
k=1
∏
b=1
p(①b|③, ℓkb).
♦t tt t q♥tts p(①, ③|ω) ♥ p(①j|③, ℓkb) r rs♣t② rqr t♦ ♦♠♣t t ♣t♥ ♣r♦t② ♦ t tr♦♣♦sst♥s ♦rt♠ ♥ ② ♥ t♦ s♠♣ t ♥♠r ♦ ♠♦s r♦♠ t ♥ t ②
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
♣♣r♦①♠t♦♥s ♦r ♥st♥ t ♥trt ♦♠♣tt ♦♦ s♣♣r♦①♠t ②
ln p(①, ③|ω) = ln p(①, ③|θ⋆,ω)− ν
2lnn+O(1),
r θ⋆ s t ♠①♠♠ ♦♠♣tt ♦♦ st♠t ♦r ts ♥ ♦♣♣r♦①♠t♦♥ s ♦♥② s②♠♣t♦t② tr ♥ ♥ ♦rst♠t t ♠♦ ♥♠rss t♦♥ s ③|ω ♦♦s ♥♦r♠ strt♦♥ ♠♦♥ t ♣♦ss♣rtt♦♥s ♣r♦♣♦s t♦ ♦♠♣t p(①b|③, ℓkb) t♦ ♦t♥ p(①, ③|ω) s ♦♠♣tt♦♥ s ♥♦t s② s♥ αkb s ♥ ♦♥ S(ℓkb;mb) ♥ ♥♦t ♦♥ t ♦ s♠♣①♦ s③ ℓkb ①♣t ♥ ℓkb = mb − 1 ♥ s s ♥ s t ♣♣r♦ ♦t ♠ ♠♦ ❬❪ ♥ ①♣t ♦r♠ s ♥ ♥ t ♦♦♥ ♣r♦♣♦st♦♥② ♣r♦r♠♥ ♥ ①t ♦♠♣tt♦♥ ♦ t ♥tr ♦r t ♦♥t♥♦s ♣r♠trs♥ ♥ ♣♣r♦①♠t♦♥ ♦♥ t srt ♦♥s ♦r t ♣r♦♦ s ♥ ♣♣♥①
Pr♦♣♦st♦♥ ♥trt ♦♠♣tt ♦♦ s ♣♣r♦①♠t ② ♥t♥ t s♠ ♦r t srt ♣r♠trs ♦ t ♠♦s ♦t♦♥s ♥ ② ♣r♦r♠♥ t ①t ♦♠♣tt♦♥ ♦♥ t ♦♥t♥♦s ♣r♠trs s♦
p(xb|z, ℓkb) ≈(
1
mb − ℓkb
)nℓkbkb
ℓkb∏
h=1
Bi(
1mb−h+1
; n(h)kb + 1; nhkb + 1
)
mb − h,
r Bi(x; a, b) = B(1; a, b)−B(x; a, b) ♥ r B(x; a, b) s t ♥♦♠♣t t♥t♦♥ ♥ ② B(x; a, b) =
∫ x
0wa(1− w)bdw
r♦♠ t ♣r♦s ①♣rss♦♥ t s strt♦rr t♦ ♦t♥ p(①, ③|ω)
♠r ①♣r♠♥ts ♦♥ s♠t t sts
♥trt ♦♠♣tt ♦♦ ♦♠♣rs♦♥ ♦ ♦t
♣♣r♦s
♠ r♥ ts ①♣r♠♥t t t ss ♦ t rtr♦♥ ♦r tst♦♥ ♦ t ♥♠r ♦ ♠♦s ♥ t ♥ ♥ ② t ♣r♦♣♦s ♦♠♣tt♦♥♦ t ♥trt ♦♠♣tt ♦♦
t ♥rt♦♥ ❲ ♥t t♦ ♦♠♣r ♦t ♣♣r♦s ♦r t st♦♥ ♦ t♥♠r ♦ ♠♦s ♦ s♠t s♠♣s ♦♠♣♦s t n ♥s rs♥r♦♠ ♠t♥♦♠ strt♦♥ ♣r ♠♦s Ms(r, r, r,
1−3rs−3
, . . . , 1−3rs−3
) t s ♠♦ts ♥ tr ♠♦s ♥ ♣r♦t② r ♦r r♥t s③s ♦ s♠♣ 105
s♠♣s r ♥rt t r♥t s ♦ (r, s)
sts r s ♦♠♣rs♦♥ t♥ t ♣r♦♣♦s ♣♣r♦ ♥ t ♣♣r♦①♠t♦♥ ♦r t st♦♥ ♦ t ♥♠r ♦ ♠♦s ♣r♦♣♦srtr♦♥ ♦t♥s ttr rsts t♥ t rtr♦♥ ♥ t ♦r st stt♦♥s♦r t r s♠♣ s③s rtr♠♦r t ♦s t♦ ♥r ♦rst♠ts t ♥♠r
♠r ①♣r♠♥ts ♦♥ s♠t t sts
0 100 200 300 400 500
0.0
0.2
0.4
0.6
0.8
1.0
sample size
prob
abili
ty
0 100 200 300 400 500
0.0
0.2
0.4
0.6
0.8
1.0
sample size
prob
abili
ty
0 100 200 300 400 500
0.0
0.2
0.4
0.6
0.8
1.0
sample size
prob
abili
ty
0 100 200 300 400 500
0.0
0.2
0.4
0.6
0.8
1.0
sample size
prob
abili
ty
r Pr♦t② tt t rtr♦♥ r♣rs♥t ♥ ♥ ♥s ♥t ♣r♦♣♦s ♣♣r♦ r♣rs♥t ♥ ♦ r ♥s st t tr ♥♠r ♦♠♦s r♣rs♥t ♥ ♣♥ ♥ ♥ ♦rst♠t t r♣rs♥t ♥ ♦tt ♥ r s r s r s r s
♦ ♠♦s ♥② ts rt② s s♠r t♥ t rtr♦♥ ♦♥ ❲ ♥tr ♥♦♥t♦ ♠♦r s♣ ♦♠♠♥ts
♥ s ♠♦s r ♣r♦t② ♠ss ♥ t② r s② tts♥ tr r ♠♦ts s ♦t rtr t s♠ ♦r s♥t② ♥ t tr ♥♠r ♦ ♠♦s t ♣r♦t② ♦s t♦ ♦♥ ♥ ♦r s♠s♠♣s
❲♥ t ♠♦ ♣r♦ts rs s t s ♠♦r t t♦ ♥t②t♠ ♥ s s t rtr♦♥ ttr ♥s t tr ♥♠r ♦ ♠♦st♥ t ♣r♦♣♦s ♣♣r♦ ♦r t s♠ s♠♣s s③ ♦r t♥ ♦rt rtr♦♥ s ♠♦rt rs t♦ ♦rst♠t t ♥♠r ♦ ♠♦s
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
t ♣r♦♣♦s ♣♣r♦ ♥rst♠ts ts ♥♠r ♥ t s r♦♥ ❲♥ ts♠♣ s③ s rr t♥ t ♣r♦♣♦s ♣♣r♦ ♦t♥s ttr rsts ♥t ♥s tr ♥♠r ♦ ♠♦s ♠♦st ②s t rtr♦♥ ♣s ♥♦rst♠t♦♥ rs
t ♥♠r ♦ ♠♦ts ♥rss s t♥ t ♣r♦♠ ♦♠s rr♥ t ♣r♦♣♦s ♣♣r♦ s♦ s♦s ts ♥trst s♥ t rtr♦♥ s str♦♥②s ♥ s s rtr♦♥ ♣s ts s ♥ ♦r r t st t ♣r♦♣♦s ♣♣r♦ ♠♦st ②s ♥s t tr ♥♠r ♦ ♠♦s ♥ ts♠♣ s③ s rr t♥
♥② ♥♦t tt ♥ t ♠♦r ♦♠♣① stt♦♥s ♥ s ♣r♦t② ♠ss ♦r t ♠♦s ♥ r ♥♠r ♦ ♠♦ts t ♣r♦♣♦s ♣♣r♦♥rst♠ts t ♥♠r ♦ ♠♦s ♥ t s♠♣ s③ s s♠ t♥ ♦♥rst♦ t tr ♠♦ s ♥ t s♠♣ s③ ♥rss ♦t tt ♥ s st s ♦ t rtr♦♥ st②s s♥♥t ♥ ♦r r t st
s ♦♥ ts ①♣r♠♥t t ♣r♦♣♦s rtr♦♥ ♣♣rs s t ♠♦st r♥ts♥ ts s②♠♣t♦t ♦r s ttr t♥ t s②♠♣t♦t ♦r ♦ t
rtr♦♥ ♥ t ♥r ♦rst♠ts t ♥♠r ♦ ♠♦s ♥ ts rt② ss♠r t♥ t rt② ♦ t rtr♦♥
♠t♦♥ t s♣ ♠♦
♠ r♥ ts ①♣r♠♥t t t ♦♦ ♦r ♦ t ♦rt♠s♠ ♦rt♠ ♥ tr♦♣♦st♥s s♠♣r ♦r ♣r♦r♠♥ t st♠t♦♥♦ t ♠ ♥ t ♠♦ st♦♥ ♦ t r ♥rt ♦r♥ t♦ ♠♠
♠♦ t♥ t ♠♦ ♥ t ♠ r st♠t qt② ♦ t st♠t♦♥s tr♠♥ ② t r r♥ ❲ s♦ tt ts q♥tt② ♦♥rs t♦ ③r♦ ♥ t s♠♣ s③ ♥rss ♦ ♦♥ t♦ t ♦♦ ♦r♦ ♦t ♦rt♠s
t ♥rt♦♥ t st ♦ s① rs t tr ♠♦ts s ♥rt♦r♥ t♦ ♦♠♣♦♥♥t ♠♠ ♠♦ t t ♦♦♥ ♣r♠trs
σ = (1, 2, 3, 4, 5, 6), ℓkj = 2, π = (0.5, 0.5), αkj = (0.4, 0.4, 0.2/7).
♠♦s r ♦t t r♥t ♠♦t② r♦ss♥s ♦r ♦t sss
sts ♦r r♥t s ♦ n = (50, 100, 200, 400, 800) s♠♣s r ♥rt r r♥ s ♦♠♣t t♥ t tr ♥ tst♠t ♣r♠trs ♣rs♥ts t ♠♥ ♦ ts r♥
s t r r♥ ♦♥rs t♦ ③r♦ ♥ t s♠♣ s③ ♥rss ♠ tt t st♠t strt♦♥ ♦♥rs t♦ t tr ♦♥ s ♦♥ t♦ t ♦♦ ♦r ♦ t st♠t♦♥ ♦rt♠
♠t♦♥ t ♠ss♣ ♠♦
♠ r♥ ts ①♣r♠♥t ♥r♥ tt t ①t② ♦ t ♠♠ ♠♦♦s t t♦ ♣ ♦♦ rsts ♥ t ♠♦ s ♠ss♣ s s♠t
♠r ①♣r♠♥ts ♦♥ s♠t t sts
n ♠♥
s
♥ ♥ st♥r t♦♥ ♦ t r r♥ ♦♠♣t t♥ t tr ♣r♠trs ♦ t s♣ ♠♦ ♥ t ♠①♠♠ ♦♦ st♠ts ss♦t t♦ t ♠♦ st ② t tr♦♣♦st♥s♦rt♠ ♦r r♥t s♠♣ s③s
s♠♣s ♦r♥ t♦ ♦♠♣♦♥♥t ♠①tr ♠♦ r t ♥trss ♣♥♥s r r♥t ♦r ♦t ♦♠♣♦♥♥ts t♥♥ ♣r♠tr ♦s s t♦ ♠♦②t str♥t ♦ t ♥trss ♣♥♥s ♥ t ss ♦r♣♣♥ rsts♦ t ♠♠ ♠♦ r ♦♠♣r t♦ t♦s ♦ t ♠ ♠♦
t ♥rt♦♥ t st ♦ s③ s s♠♣ r♦♠ t ♦♦♥ ♦♠♣♦♥♥t♠①tr ♠♦ ♦ ♠♥s♦♥ s①
p(x;θ) = 0.53∏
h=1
p(x2h−1,x2h;θ) + 0.5 p(x1;θ)p(x6;θ)2∏
h=1
p(x2h,x2h+1;θ),
t p(xj,xj+1;θ) = p(xj;θ)(
λ1xj=xj+1 + (1− λ)p(xj+1;θ))
♥ t p(xj;θ) =∑3
h=1(1/3)xjh s ♥ λ = 0 t s♠♣ s ♥rt ② ♥♦r♠ strt♦♥
♥ sss r ♦♥s rr s t t♥♥ ♣r♠tr λ t rr r t♥trss ♣♥♥s ♥ t ss s♣rt♦♥ ♦t tt ♠♠ s ♥♦t t tr♠♦ s♥ t ♦♥t♦♥② ♦rrt rs r ♥♦t t s♠ ♥ ♦t sss
sts ♦r r♥t s ♦ λ = (0.2, 0.4, 0.6, 0.8) s♠♣s r ♥rt r r♥ ss♦t t♦ t ♠♦ t t st ♥♠r ♦sss st ② t rtr♦♥ ♠♦♥ g = 1, ..., 4 s ♦♠♣t ♣rs♥ts t rsts ♦t♥ ② t ♠♠ ♥ t ♠ ♠♦s
λ ♠♠ ♠
r r♥ ♥ ♠♥ ♦ t ♥♠r ♦ sss ♦t♥② ♠♠ ♥ ♠
rr s λ t rr s t r r♥ ♦r ♦t ♠♦s♦r t ①t② ♦ t ♠♠ ♠♦ ♦s t♦ ♣ ♥ ♣t ♦ tr r♥ ts r♥ r♦s r♠t② str t t♠ ♠♦ rtr♠♦r ♥ t sss r s♣rt r ♦ λ t♠♠ ♠♦ ♥s ♠♦r ♦t♥ t tr ♥♠r ♦ sss t♥ t ♠ ♠♦
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
♥②ss ♦ t♦ r t sts
♦r ♦t ♣♣t♦♥s t st♠t♦♥ ♦ t ♠♠ ♠♦ s ♣r♦r♠ ② t ♣ ♦♦s ♦t t sts r ♥ ♦♦s ♦♣ ② t t♦rs
rs str♥
t ❲ st② ♦♦ t st sr♥ ♣♥s srs ② ♣♠ ♥ ①tr♥ ♠♦r♣♦♦ rtrsts ♣rs♥t ♥ ❬r❪s srs r ♥t♦ tr ss♣s r♦s rs r♠♥r rs ♥ srs rs
rs mj ♠♦ts♦r ♥♦♥ ♦♥t♥♦s
②r♦s ♥♦♥ r② ♣r♦♥♦♥s t ♥ t ♥ t
♦rr ♥♦♥ ♠♥②♥r ♠ ♠
Prs♥tt♦♥ ♦ t ♣♠ ♥ ①tr♥ ♠♦r♣♦♦ rssr♥ t ♣♥s
①♣r♠♥t stt♥s ss♣s ♠♠rs♣s ♦ t ♥s r ♥♦r g = 1, . . . , 6 t ♠ ♦ t ♠ ♠♦ s ♦t♥ ② ♥t③t♦♥s ♦ ♥ ♠♦rt♠ ♥s ♦ trt♦♥s r ♣r♦r♠ ♦r t ♠♦ st♦♥♦ t ♠♠ ♠♦ ♦♦ ② ♥t③t♦♥s ♦ ♠ ♦rt♠ t♦ ♥ t ♠
sts ♣rs♥ts t s ♦ t rtr♦♥ ♦r ♦t ♠♦s ♥r♥t ♥♠rs ♦ sss ♥ ♦t ♠♦s st t♦ ♦♠♣♦♥♥ts t s♦ t rtr♦♥ r ttr ♦r t ♠♠ ♠♦ t♥ ♦r t ♠ ♠♦ ♦r t♥♠rs ♦ sss s t ♠♠ ♠♦ ttr ts t t t♥ t ♠ ♠♦
g ♠♠ ♠
❱s ♦ t rtr♦♥ ♦r r♥t ♥♠rs ♦ sss ♦t♥ ②t ♠♠ ♥ t ♠ ♠♦s ♦ ♥ts t st s ♦ ts rtr♦♥
♦r♥ t♦ s♣②♥ t ♦♥s♦♥ ♠tr① t♥ t st♠t♣rtt♦♥s ♥ t ss♣s ♠ tt t rs r ♠♦r r♥t t♥t t♦ ♦tr ss♣s ♥ ♦t ♠♦s t t rs ♥ ss t st♠t ♣rtt♦♥s ② ♦t ♠♦s r s♠r r♠r tt t ♠♠ ♠♦ts ss ♦tr ss♣s ♥ ts ss t♥ t ♠ ♠♦
♥②ss ♦ t♦ r t sts
♠♠ ♠
ss ss ss ss r♦s
r♠♥r rs
♦♥s♦♥ ts t♥ t ss♣s ♥ st♠t ♣rtt♦♥ ♥t♦t♦ sss
r s♣②s t srs sttr♣♦t ♦♥ t rst ♦rrs♣♦♥♥ ♥②ss ♣♥ ♥ ♥ts t ss♣s ❲ ♥♦t tt t rs r ♥ t s♠♦t♦♥ ♦tt♦♠ t ♦r t rst ♣r♥♣ ♦rrs♣♦♥♥ ♠♣ ❲ s♣② t♣rtt♦♥ ♦rrs♣♦♥♥ t♦ t st ♠♦ t ♠♠ ♠♦ t t♦ ♦♠♣♦♥♥ts♥ r ♦t tt ♦r ♦t ♠♦s t rst ♣r♥♣ ♦rrs♣♦♥♥ ①s♦s t♦ ♥ sst♦♥ r
−1.0 −0.5 0.0 0.5 1.0 1.5
−1.
0−
0.5
0.0
0.5
1.0
1.5
2.0
First principal correspondence analysis
Sec
ond
corr
espo
nden
ce a
naly
sis
Dichrous Lherminieri Subalaris
−1.0 −0.5 0.0 0.5 1.0 1.5
−1.
0−
0.5
0.0
0.5
1.0
1.5
2.0
First principal correspondence analysis
Sec
ond
corr
espo
nden
ce a
naly
sis
Class 1 cmm Class 2 cmm
r rs ♦♥ t rst ♣r♥♣ ♦rrs♣♦♥♥ ♥②ss ♠♣ tt ss♣s ♥ t t st ♠♠ ♠♦ st♠t ♣rtt♦♥ ♦tr♥s ♥t t ♥s t ♥ ss ♦r t ♠♠ ♠♦ ♥ ♥ ss ♦r t ♠ ♠♦ ♥ ♥♦r♠ ♥♦s ♦♥ [0, 0.1] s ♦♥ ♦t ①s♦r ♥ ♥ ♦rr t♦ ♠♣r♦ s③t♦♥
❲ ♥♦ sr t st ♦♠♣♦♥♥t ♠♠ ♠♦ ♥ t st♠t♠♦ ss♠s ♦♥t♦♥ ♥♣♥♥ t♥ rs ts ♠♦ s ♦ ♥trsts ♦ ts s♣rst② ♥ t s ♠♦r ♣rs♠♦♥♦s t♥ t ♠ ♠♦ s♥ s♠ ♥♠r ♦ ♠♦s s st♠t s s♦♥ ② t s♠♠r② ♣r♦♣♦s ② κkj♥ ρkj ♥ ♥ ♥ ♣rs♥t ♥ s t rst rs rrtr③ ② ♠♦ts t ♣r♦t② s t rs r ♦♥t♦♥② ♥♣♥♥t κkj ♥ts t ♥♠r ♦ ♠♦ts ♥ ♣r♦t②♣♣r t♥ t ♥♦r♠ strt♦♥ ♦r ①♠♣ t ♠t♥♦♠ strt♦♥ ♦t r s s t♦ ♠♦s ♦r ♦t sss s♦ κkj = 2/3
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
♦r ②r♦s s ♦rr ♥rss ss
♠♠r② ♦ t ♠♠ ♠♦ t tr sss κkj s s♣② ♥ ♣♥♥ ρkj s s♣② ♥ ♣r♥tss
♠①♠♠ ♦♦ st♠ts ♦ t ♦♠♣♦♥♥t ♣r♠trs r ♣rs♥t② r sr ♦rrs♣♦♥s t♦ ♦ ♦ r ts ♥♦t♥ tt t st♠t ♠♦ ss♠s t ♦♥t♦♥ ♥♣♥♥ ♦r ♦ ♦ rs t ♠♦t② r♦ss♥s r ♦♥ ♠♦ s st♠t ♦r t st♦♥ ♦♠♣♦♥♥t r ♦s ♦r ts ♠♦t② r♦ss♥s s♣② tr ♠t♣r♦t② ♠sss ♦r ♦♠♣♦♥♥t t ♦♠♣♦♥♥t r ♥t ② r♥t♦♦rs s ♠♦t② r♦ss♥s r ♣rs♥t ② rs♥ ♦rr ♦ ♠t♣r♦t② ♠ss
♦t tt t ♠♦ ♦t♦♥s r sr♠♥t s♥ t ♠♦t② rs♣t s ♣r♦t② ♦ rs♣ ♦r ss t ♠♦t② trs♣ ♥ t s ♣r♦t② ♦ rs♣
0.0
0.2
0.4
1 3 2 4
collar.
0.0
0.4
0.8
1.2
3 2
eyebrows.
0.0
0.4
0.8
white black BLACK_white
sub−caudal.
0.0
0.5
1.0
1.5
none few
border.
0.0
0.4
0.8
male
gender.
r ss ♣r♠trs ♦ t ♦♠♣♦♥♥ts ♠♠ ♠♦ st♠t ♦♥ trs t ♦♦r rs♣t② t r② ♦♦r ♦rrs♣♦♥s t♦ t♣r♦t② ♠ss ♦ t ♠♦s ♦r ss rs♣t② t♦ ss
♥② t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ s♠s rst s♥ t ♦♥t♦♥ r♠rs ❱ ♠srs ♣rs♥t ♥ r s♠ ❲ s♦ ♣r♦r♠ ♦♦tstr♣ tst ♦ t ♦ ♥t② ♦ t r♠rs ❱ ② ♥rt♥ s♠♣s ❲ ♦t♥ ♣ ♦ s♦ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ st
♥②ss ♦ t♦ r t sts
ss
ss
tr① ♦ t r♠rs ❱ ♠srs ♦♠♣t ♦r♥ t♦ t st♠tsss
t ♥♠♠t♦♥s str♥
t ❲ ♥t t♦ str ♣t♥ts ❬❩❪ sr ② ♥r② rs♦rr♥ ♦ ♥s ♠r ♣♥ ♠ r♥ ♣s♥ Ps ♠trt♦♥♣♥s ♥ r♥♥ ♦ rtr r ♥ ② ♦♥ r ♥ tr ♠♦ts t♠♣rtr ♦ t ♣t♥t ♠ T < 37C 37C ≤ T < 38C ♥ 38C ≥ T ❲ ♥♦ tt s♦♠ ♣t♥ts ♦♥ ♦ t ♦♦♥ sss ♦ t r♥r② s②st♠♥♠♠t♦♥ ♦ r♥r② r ♥ ♣rts ♦ r♥ ♣s ♦r♥
①♣r♠♥t ♦♥t♦♥s ❲ s t s♠ ①♣r♠♥t ♦♥t♦♥s s t rsstr♥
sts ♣rs♥ts t s ♦ t rtr♦♥ ♦r ♦t ♠♦s ♥r♥t ♥♠rs ♦ sss ♦r ♥♠r ♦ sss t rtr♦♥ ♦t ♠♠ ♠♦ s ttr t♥ t♦s ♦ t ♠ ♠♦ rtr♠♦r t ♠♠ ♠♦sts tr sss t ♠ ♠♦ sts ♦r sss s ♣♥♦♠♥♦♥ ♥ t♦ t ♦t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♦ t ♠ ♠♦
g ♠♠ ♠
❱s ♦ t rtr♦♥ ♦r r♥t ♥♠rs ♦ sss ♥ ♦r t♠♠ ♥ t ♠ ♠♦s ♦ ♥ts t st s ♦ ts rtr♦♥
♦t tt t st♠t strt♦♥s ♦ t ♠ ♥ t ♠♠ ♠♦s r r♥t ♦t♥ ♣rtt♦♥ r s♦ r♥t s♣②s t ♦♥s♦♥♠trs t♥ t st ♠♠ ♠♦ ♥ t ♠ ♠♦s t tr ♥ ♦rsss s ♥s ♦♥sttt r♦♣ s s♣rt r♦♠ t♦tr ♥s ss ♦r t tr ♠♦s t ♦tr ♥s ss♠♠rs♣ tr♠♥ ② t st ♠♦
r s♣②s t ♥s ♦♥ t ♣r♥♣ ♦rrs♣♦♥♥ ♥②ss♠♣ r t st♠t sss r s♣rt
♠♠ ♠♦ t tr sss s t ♦♦♥ r♣rtt♦♥ ♦ t rs♥t♦ ♦s σ = (♠♣ Ps r, , ♠) s s♦♥ ② t s♠
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
♠♠
♠ ♠ ♠
♠♠
♠ ♠ ♠ ♠
♦♥s♦♥ ♠trs t♥ t st ♠♠ ♠♦ ♥ t ♠ ♠♦st tr ♥ ♦r sss
−0.5 0.0 0.5 1.0
−0.
8−
0.6
−0.
4−
0.2
0.0
0.2
0.4
0.6
Axe 1 of the multiple correspondance analysis
Axe
5 o
f the
mul
tiple
cor
resp
onda
nce
anal
ysis
r ♥s ♦♥ t ♣r♥♣ ♦rrs♣♦♥♥ ♥②ss ♠♣ t tst ♠♠ ♠♦ st♠t ♣rtt♦♥ ♥ ♥♦r♠ ♥♦s ♦♥ [0, 0.1] s ♦♥ ♦t ①s ♦r ♥ ♥ ♦rr t♦ ♠♣r♦ s③t♦♥ ♦♦rs♥ s②♠♦s ♥t t ss ♠♠rs♣
♠r② ρkj ♥ κkj s♣② ♥ t tr sss r ♦♥♥trt ♥ ♠♦t② r♦ss♥s ♦r t ♦ ♦♥ ♥ ♥ ♦♥ ♦t♦♥ t ♣r♦t② ♦s t♦♦♥ ♦r t t♦ ♦tr ♦s
♠♣ ♠ Ps rss ss ss
♠♠r② ♦ t ♠♠ ♠♦ t tr sss κkj s s♣② ♥♣♥ ♥ ρkj s s♣② ♥ ♣r♥tss
♦♦♥ ss ♥tr♣rtt♦♥ s s ♦♥ t ss ♣r♠trs s♣② ②r ♦t tt t rs r♥ ♣s♥ ♥ r♥♥ ♦ rtr r t♠♦st sr♠♥t ♦♥s
♠♦rt② ss r♦♣s ♥s ♥ ♥♦ ♥s ♥ ♥♦ ♠r♣♥
♦♥s♦♥
s♦♥ ss r♦♣s ♥s ♥ ♥♦ ♥s t ♠r ♣♥ tr ss r♦♣s ♥s ♥ ♥s ♥ ♠r ♣♥
rtr♠♦r ts ♥s s♦♠ r ♥ ♠trt♦♥ ♣♥
0.0
0.2
0.4
fiever fiever fiever fiever cold normal cold normalnormalnormal fiever normalyes yes no yes no no yes yes yes yes no yesno yes yes yes no no yes yes yes no no noyes no no yes no no yes no yes no no yes
Tem−Pus−Mic−Bur.
0.0
1.0
2.0
no yes
Nau.
0.0
1.0
2.0
yes no
Lum.
r st♠t ♣r♠trs ♦ t tr♦♠♣♦♥♥t ♠♠ ♠♦ s♣② ②t r♣♦t ♥t♦♥ ♦ t ♣ ♦♦s ♦♦r ♦rrs♣♦♥s t♦ ss r② ♦♦r ♦rrs♣♦♥s t♦ ss ♥ ♣ r② ♦♦r ♦rrs♣♦♥s t♦ ss
♦♥s♦♥
♥ ts ♣tr ♣rs♥t ♥ ♠①tr ♠♦ ♠♠ t♦ str t♦r t ts str♥t s t♦ r① t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♥t♦ st② ♣rs♠♦♥♦s s♠♠r② ♦ t strt♦♥ s ♥ ② κkj ♥ ρkj ss ♥ s♠♠r③ ② t ♠♦ ♦t♦♥s s s♦♥ ♦♥ t rs♣♣t♦♥ t ♠♠ ♠♦ ♥ ♦t♣r♦r♠ t ss t♥t ss ♠♦ ♥ t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ s tr t♥s t♦ ts s♣rst②
♦♠♥t♦r ♣r♦♠s ♥♦ ② t ♦ tt♦♥ ♥ ② t st♦♥♦ t ♥♠rs ♦ ♠♦s r ♦ ② tr♦♣♦st♥s ♦rt♠ s♦rt♠ ♥ s s t ♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt♦♦ ♥ ♥t② ♣♣r♦①♠t s ts ♣♣r♦ ♥ s t♦st t ♥trt♦♥s ♦ t ♦♥r ♠①tr ♠♦ ♣r ♦
♦r t ♠♦ s r② st♠t t t st s r ♥♠r ♦rs ♦♠ ♦♥str♥ts ♦♥ t ♦ rs r♣rtt♦♥ ♦ s♦ ♦r ♥st♥ t ♥♠r ♦ rs ♥t♦ ♦s ♦ ♠t t♦ tr rs♥♦tr s♦t♦♥ ♦ t♦ st♠t t ♠♦ ② ♦rrr strt②t t s ♥♦♥ tt ts ♠t♦s r s♦♣t♠
♣tr ♦s str♥ t ♦♥t♦♥ ♣♥♥② ♠♦s
♥② ♠♣♦s t qt② ♦ t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s♦r t sss s ♣r♦♣rt② ♦s s t♦ ♣r♦ t ♥r ♥tt② ♦t ♠♠ ♠♦ s ♦ ①t② s ♦♥tr♥ ② ① ♦ strt♦♥ ♦r ♦♥ ♦ tr② t♦ r① t ssqt② ♦ σ t t ♠♦♥♦♥tt② rs
♣tr
♦ ♦♠♣rs♦♥ ♣r♦r♠ ②
tr ♣s
s ♣tr ♠ s t♦ strt t ♣sstrt ♥ ♦♦s rs♣t② ♣r♦r♠t ♥r♥ ♦ t ♠ ♥ t ♠♠ ♠♦s♥ ♦rr t♦ ♠ ♠♦♥strt♦♥ ♦ ♦t ♣s s t♠ t♦ ♣r♦r♠ t str ♥②ss ♦ t r♥♥♥①♠♣ ♣rs♥t ② ♦ ♣tr ❲ r♠♥ tt ts t st s♣②s t t♦♥ s♦♥♦r r♦s ♦ ♥t ①r②s tt ♠② s♦ ♥♣♥t rs ♣r♦r♠ ② ♥tsts♦t tt ts ♣tr ♥ s♦ s s tt♦r ♦♦t ♣s ♥ t ♣r♦s ♣rs♥tt♦♥ ♦ tr♠♥ ♥t♦♥s ♥ ♠♥② sr♣ts ♦♥ t♦ ♣r♦r♠ tstr ♥②ss
♥♠s r q t s♦♠♥♠s r ♠♦r q t♥ ♦trs
♦r r ♥♠ r♠
strt
strt ♦r
Prs♥tt♦♥ ♣ strt ♣r♦r♠s t str♥ ♦ t♦rt ♦r♥ t♦ t ♠ ♠♦ ts ♠♥ ♥t♦♥s r ♠♣♠♥t ♥ ❲ r♠♥ tt ♥ ts ♠♦ rs r r♦♣ ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s ♥ ♦rr t♦ ♦♥sr t ♠♥ ♥trss ♦rrt♦♥s ♥trss♣♥♥② t♥ rs r♦♣ ♥s t s♠ ♦ s t♥ ♥t♦ ♦♥t② ♠①♥ t♦ ①tr♠ strt♦♥s r rs♣t② ♥♣♥♥ ♥ ♠①♠♠ ♣♥♥②
♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s
♦♥♦ strt ♣ s rr♥t② ♦♥ ♦r t t ♦♦♥ r tt♣sr♦rr♣r♦t♦rr♦♣❴ ♥stt♦♥ ♥t ♦♥ ♦ strt ♥ ♣r♦r♠ ② s♥ t ♦♦♥ sr♣ts
♥st ♦♠♠♥> ♥st♣sstrt
r♣♦stt♣♦r♣r♦t♦r
strt sr♣t strt ♥stt♦♥
strt ♦♥> rqrstrt
strt sr♣t strt ♦♥
st♠t♦♥ ♣r♠tr st♠t♦♥ ② ♠①♠♠ ♦♦ s ♣r♦r♠ ♠ ♦rt♠ s ♦rt♠ s ♦r t ♠♦ st♦♥ ♦s t♦♠♥t♦r ♣r♦♠s ♥ ② t ♦ strtr sr
♥ ♥t♦♥s
♥t♦♥s ♦♠♣♦s t strt ♣ ♥ ♥t♦♥ ♣r♦r♠s tstr ♥②ss ♦ t t ts t♥♥ ♣r♠trs ♥ s♣ ② t sr ②♥ s♣ ♥t♦♥ tr st ♥t♦♥s r ♠♣♠♥t ♥ ♦rr t♦r♥② ♣rs♥t t ♣r♠trs ② ♣r♦♥ ♥♠r ♦r r♣ s♠♠rs
str♥ ♥t♦♥ str ♥②ss ♥ ♣r♦r♠ t t ♥t♦♥strt t♥ ♦r r♠♥ts
> strtt ♥❴str
♠♦ strt② strt②tt
strt sr♣t str♥ ♥t♦♥
s ♥t♦♥ s t♦ ♠♥t♦r② r♠♥ts t r♠ t t♦ str ♦s t ♦♠♥s r ♥♦♥③r♦ ♥trs ♦r
t♦rs ♥ ♥tr t♦r ♥❴str s♣②♥ t ♥♠r ♦ sss
t s r t♥ ♦r t r♠♥ts ♠♦ ♥ strt② r♠♥t ♠♦ s t♦r ♥ t ♠♦t② ♥♠r ♦r r r♠♥t strt② s ♥ ♥st♥ ♦ t strt②t ss ♦♥
t♥s t st♠♥ts ♥♣ts ♣r♠trs rt t♦ t st♠t♦♥ ♦rt♠s
strt
♥t♦♥ strt rtr♥s ♥ ♥st♥ ♦ t stt ss ♦♥t♥s t ♦t♣ts
t♥♥ ♥t♦♥ st♠♥ts ♣r♠trs ♦ t st♠t♦♥ ♦rt♠s♦♥t♥ ♥ t strt②t ss ♥ s♣ ② t ♥t♦♥ strt②t
t♥ ♦r r♠♥ts
> strt②tt ♥❴♥t st♦♣❴rtr♦♥
♥♦t ♣rtt♦♥ ♣rtt♦♥tt
strt sr♣t t♥♥ ♥t♦♥
♥♣t t ♠tr① t s ♠♥t♦r② ♥ t tr ♦trs ♥♣t ♣r♠trs♦ t♦ t♥ t ♦rt♠s
r♠♥t ♥❴♥t sts t ♥♠r ♦ t♠s r ♠♠ ♥ sstrt ② t ♠♠ ♥s r ♥t③
r♠♥t st♦♣❴rtr♦♥ s t ♥tr ♦rrs♣♦♥♥ t♦ t ♥♠r♦ sss trt♦♥s ♦ t ♠♠ ♥ r ♥♦ ttr ♠♦ s ♥t♥ t ♦rt♠ s st♦♣♣ ② t t ts t s ♦ 20× d
r♠♥t ♣rtt♦♥ s t ♥t ♦ t r♣rtt♦♥ ♦ t rs♥t♦ ♦s σ[0] ② t t s q t♦ t ♣rtt♦♥ ♣r♦ ② t ♠♥♠③♥ t ♦ ♥♠r t♦t ♦ ♦♥sst♥ ♦ ♠♦r t♥♦r rs
r t♦♦ ♥t♦♥s strt ♣ s♦ ♣r♦s t♦♦ ♥t♦♥s s♠♠r② s♠♠r②❴♣♥♥s ♥ ♣♦t rs♣t② t♦ s♠♠r③ rsts t♦ ♣rs♥t t ♠♥ ♦♥t♦♥ ♣♥♥s ♥ t♦ s③ t ♣r♠trs
strt t♦ str t ♥tsts t st
str♥ t ♠ ❲ ♥♦ ♣rs♥t t rsts ♦ t ♠ ♠♦ ♦t♥② t ♣ strt ② s♥ t ♦♦♥ sr♣t
♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s
t st ♦♥> t♥tst
♥t♦♥ ♦ t t♥♥ ♣r♠trs ♦r t st♠t♦♥ ♦rt♠> st strt②t♥tst ♥❴♥t st♦♣❴rtr♦♥
st♠t♦♥ ♦ t ♦♠♣♦♥♥t ♠ ♠♦> rs strt♥tst ♠♦ r♣ strt②
st
strt sr♣t ♥tsts t st str♥
♦ st♦♥ rtr♦♥ sts t♦ sss t ♦ t ♠s tt t ♠ ♠♦ ttr ts t t t♥ t ♠♦ ♣rs♥t ♥❬❪ s♥ ts rtr♦♥ s rtr♦♥ s ♦r t ♠♥ t ♠ ♠♦s r s♣② ♥ ❲ ♥t t ♦♠♣t♥ t♠♥ s♦♥s ♦t♥ t ♣r♦ss♦r ♥t ♦r t♦ st♠t t ♠♠♦ r ♠♠ ♥s r strt t st♦♣♣♥ rs qmax = 100 t ♠ ♠♦ ♥s ss t♥ s t t ♣ ①♠♦ ❬+❪
g ♠ ♠
t♠ s
rtr♦♥ s ♦r t ♠ ♥ t ♠ ♠♦s ♦r♥ t♦r♥t ♥♠rs ♦ sss ♦r t ♥tstr② t st st s r ♥ ♦
❲ ♥♦t tt t ♠ ♠♦ ♦t♥s ttr s ♦r t rtr♦♥ t♥t ♠ ♠♦ ♥ g = 1, 2 ❲♥ t ♥♠r ♦ sss s rr g ≥ 3 t st♠ ♠♦ ss♠s t ♦♥t♦♥ ♥♣♥♥ t♥ rs
♦♠♣rs♦♥ ♦ t rsts rtr♦♥ sts t♦ sss ♦r t ♠♠♦ s rst s ♦r♥t t s♣tt♥ ♦ t tt t♥ t s♦♥ ♥t r♦s ♦♥s rtr♠♦r t t♦ ♠♥ rtrsts ♦ t ♦♥r ♠①tr♠♦ ♠♣♦s ♥ ❬❪ r t♦♠t② tt ② t ♠♦ ♠♣♦rt♥ ♦t t♦ ♠♦t② r♦ss♥s r t ♥tsts t s♠ ♥♦ss ♥ ♣♥♥② t♥ t ♥♦ss ♦ t ♥tsts ♥ s t st♠t♠♦ s ♦r♥t t t ♠♣♦s ♠♦ ♣rs♥t ♥ ❬❪ ♥♦ ♥♦r♠t♦♥s ♥ ♣r♦r
strt
st ♠♦ ♥tr♣rtt♦♥ ♠♦ ♥tr♣rtt♦♥ s tt ② t♦ t♦♦s♥t♦♥s ♥t♦♥ s♠♠r② ♣r♦s ♥r ♦r ♦ t ♠♦ ♥♦r♠t♦♥ rtr ♣r♦♣♦rt♦♥s ♦s ♦ rs ♥ ♥trss ♣♥♥sρkb
♥t♦♥ ♣r♦♥ ♠♦ ♦r> s♠♠r②rs
♠r ♦ sss ♦♦♦ Pr♦♣♦rt♦♥s ♦s r♣rtt♦♥ ♦ t rs ♦r t ss
❱rs ♦♦
♦s r♣rtt♦♥ ♦ t rs ♦r t ss ❱rs ♦
♦ ♦
strt sr♣t ♦ ♦r
♥t♦♥ s♠♠r②❴♣♥♥s ♦ss ♦♥ t ♥trss ♣♥♥② ♣r♠trs
♥t♦♥ ♣r♦♥ s♠♠r② ♦ t ♥trss ♣♥♥s> s♠♠r②❴♣♥♥srs
♦s r♣rtt♦♥ ♦ t rs ♦r t ss
♦ ♦♥t♥s t rs t ♦
s♦♥ s♦♥ s♦♥ s♦♥ s♦♥ r♦s r♦s r♦s r♦s r♦s♦s r♣rtt♦♥ ♦ t rs ♦r t ss
♦ ♦♥t♥s t rs t ♦
s♦♥ r♦s r♦s s♦♥
♦ ♦♥t♥s t rs t ♦
strt sr♣t ♠♠r② ♦ t ♥trss ♣♥♥s
♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s
♦r♥ t♦ t ♣r♦s ♦t♣ts t tt ♠♦ ♥ ♥tr♣rt s ♦♦s
♠♦rt② ss π1 = 0.86 ♠♥② trs t s♦♥ tt r s str♦♥ ♣♥♥② t♥ t ♥♦ss σ1 = (1, 2, 3, 4, 5) ♥ρ11 = 0.35 ♣♥♥② strtr ♦ t ♠①♠♠ ♣♥♥② strt♦♥ ♥ts ♥ ♦r ♦♥trt♦♥ ♦ ♦t ♠♦t② ♥trt♦♥s rt ♥tsts t s♠ ♥♦ss s♣② ♥ t② ♠ tt tt♦♦t s s♦♥ τ ❴s♦♥
11 = 0.95 ♥ τ ❴r♦s11 = 0.05
♠♥♦rt② ss π2 = 0.14 r♦♣s ♣r♥♣② t r♦s tt r s ♣♥♥② t♥ t ♥tsts ♥ ♣r♦s ♦♣♣♦st ♥♦ss t ♥♦ss ♦ t ♦tr ♦♥s r ♥♣♥♥t ♥ t ss σ2 =(3, 4, 1, 2, 5) ρ21 = 0.25 ♥ ρ22 = 0
st ♠♦ s③t♦♥ ♥② t ♥t♦♥ ♣♦t ♣r♦s r♣ s♠♠r② ♦ t ♣r♠trs
♥t♦♥ ♣r♦♥ t r♣ s♠♠r② ♥ ② r > ♣♦trs
strt sr♣t r♣ s♠♠r② ♦ t ♣r♠trs
♥ ♦r♥ts t st♠t sss r r♣rs♥t t rs♣t t♦ tr ♣r♦♣♦rt♦♥ ♥ rs♥ ♦rr ♦t tt tr ♦rrs♣♦♥♥ r ♣♥s ♦♥ tr♣r♦♣♦rt♦♥ ♠t ♣r♦♣♦rt♦♥s r ♥t ♦♥ t t s ♥ ssstr ♥t♦♥s r ♥ rst ♦♥ s t ♥trrs ♦rrt♦♥s ρkb♦r t ♦s ♦ t ss ♦rr ② tr str♥t ♦ ♦rrt♦♥ ♥ rs♥♦rr s♦♥ ♦♥ s t ♥trrs ♦rrt♦♥s τ kb ♦r ♦ r♥♦r♥ t♦ t str♥t ♦ tr ♣♥♥s ♥ rs♥ ♦rr tr ♦♥s t rs r♣rtt♦♥ ♣r ♦s ♥ts tt t r sss♥ ♥t♦ t ♦ ♥ t ♥ts tt ♦♥t♦♥② ♦♥ ts sst r s ♥♣♥♥t ♦ t rs ♦ ts ♦ ♦r ①♠♣ ts rs♦s tt t rst ss s ♣r♦♣♦rt♦♥ ♦ 0.86 ♥ tt t rs rss♥ ♥t♦ t s♠ ♦
♦♦s
♦♦s ♦r
Prs♥tt♦♥ ♣ ♦♦s ♣r♦r♠s t str♥ ♦ t♦rt ♦r♥ t♦ t ♠♠ ♠♦ ❲ r♠♥ tt ♥ ts ♠♦ rs rr♦♣ ♥t♦ ♦♥t♦♥② ♥♣♥♥t ♦s q t♥ sss ♥ tt ♦ ♦♦s ♠t♥♦♠ strt♦♥ ♣r ♠♦s t ♥t♦♥s ♦ ts ♣r ♠♣♠♥t ♥ ♦ t② s♦ ♠♣♠♥t ♥ ♥ ♦rr t♦ ♥rst ♦♠♣tt♦♥ s♣
♦♦s
1 0.75 0.5 0.25 00 0.25 0.5 0.75 1ρkb τkb
0
0.86
1
Class 1
de1
de2
de3
de4
de5
Class 2
de1
de2
de3
de4
de5
de1
de2
de3
de4
de5
r ♠♠r② ♦ t st ♠ ♦r♥ t♦ ♦r t ♥tsts t st
♦♥♦ ♦♦s ♣ s rr♥t② ♦♥ ♦r t t ♦♦♥ r tt♣sr♦rr♣r♦t♦rr♦♣❴ ♥stt♦♥ ♥t ♦♥ ♦ ♦♦s ♥ ♣r♦r♠ ② s♥ t ♦♦♥ sr♣ts
♥st ♦♠♠♥> ♥st♣s♦♦s
r♣♦stt♣♦r♣r♦t♦r
♦♦s sr♣t ♦♦s ♥stt♦♥
♦♦s ♦♥> rqr♦♦s
♦♦s sr♣t ♦♦s ♦♥
st♠t♦♥ ♣r♠tr st♠t♦♥ ② ♠①♠♠ ♦♦ s ♣r♦r♠ ②♥ ♦rt♠ s ♦rt♠ s s ♦r ♠♦ st♦♥ t♦ ♦ ♦♠♥t♦r ♣r♦♠s ♥♦ ② t ♦ strtr sr ♦t tt t s♦rt♠ ss ♥ ♥t ♣♣r♦①♠t♦♥ ♦ t ♥trt ♦♠♣tt ♦♦
♥ ♥t♦♥s
♦r ♥t♦♥s ♦♠♣♦s t ♦♦s ♣ ♦♥ ♥t♦♥ ♣r♦r♠s t str♥②ss ♥ tr ♥t♦♥s ♣ ♦r t rst ♥tr♣rtt♦♥
♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s
str♥ ♥t♦♥ str ♥②ss ♥ ♣r♦r♠ t t ♥t♦♥♦♦sstr t♥ s♥ r♠♥ts
> ♦♦sstr① s❴♥t s❴tr
s❴ ❴♥t ❴t♦
♦♦s sr♣t ♥tsts t st str♥
s ♥t♦♥ s t♦ ♠♥t♦r② r♠♥ts t r♠ ① t♦ ♥②③ ♦s ♦♠♥s r t♦rs ♥ ♥tr s stt♥ t ♥♠r ♦ sss
t s r t♥ ♦r t t♥♥ r♠♥ts ♥♠r ♦ ♠♠ ♥s ♣r♦r♠ t♦ st t st ♠♦ s st ② t
r♠♥t s❴♥t t s ♥♠r ♦ trt♦♥s ♦ t s s♠♣r s st ② t r♠♥t s❴tr
t s ♥♠r ♦ trt♦♥s ♦ t r♥♥ ♦ t s s♠♣r s st ② t
r♠♥t s❴ t s ♥♠r ♦ r♥t ♥t③t♦♥s ♦ t ♠ ♦rt♠ st♠t♥ t
♠ ♦r t st ♠♦ ♦r♥ t♦ t s s♠♣r s st ② t r♠♥t❴♥t t s
♠ ♦rt♠ s st♦♣♣ ♥ t ♥rs ♦ t ♦♦ s s♠rt♥ ❴t♦ t s
r t♦♦ ♥t♦♥s ♦♦s ♣ s♦ ♣r♦s t♦♦ ♥t♦♥s s♠♠r② r♣♦t ♥ ♣♦t rs♣t② s♠♠r② ♦ t ♠♦ r♣ s♠♠r② ♦ t ♣r♠trs ♥ sttr♣♦t ♦ t ♥s ♥ t♠t♣ ♦rrs♣♦♥♥ ♠♣
♦♦s t♦ str t ♥tsts t st
str♥ t ♠♠ ❲ ♥♦ s♣② t rsts ♦ t ♠♠ ♠♦ st♠tt t ♣ ♦♦s ② s♥ t ♦♦♥ sr♣t
> rs ♦♦sstr♥tst s❴♥t
s❴tr ❴♥t
♦♦s sr♣t ♥tsts t st str♥
♦ st♦♥ s ♦ t rtr♦♥ ♦t♥ ② t tr ♠♦s♠ ♠ ♥ ♠♠ r ♣rs♥t ♥
♠♦ tt♥ t st t t s t ♦♠♣♦♥♥t ♠ ♠♦ ♠♠♠♦ ts t t ttr t♥ t ♠ ♠♦ ♦r ♥♠r ♦ sss s♠r t♥
♦♦s
g ♠ ♠ ♠♠
rtr♦♥ s ♦r t ♠ t ♠ ♥ t ♠♠ ♠♦s ♦r♥ t♦ r♥t ♥♠rs ♦ sss ♦r t ♥tstr② t st st s r ♥♦
tr ♦t tt ♥ t ss ♥♠r s ♣♣r ♦r q t♦ tr ♦t ♠ ♥♠♠ ♠♦s r q♥t t♦ t ♠ ♠♦
♣♦ss rs♦♥ ①♣♥♥ t ♣♦♦r ♣r♦r♠♥ ♦ t ♠♠ ♠♦ ♦ ts ♦♥str♥t ♦ t qt② t♥ ss ♦ t r♣rtt♦♥ ♦ t rs ♥t♦♦s ❲ r♠♥ tt ts ss♠♣t♦♥ s ♥♦t ♠ ② t ♠ ♠♦
♥ ♦rr t♦ strt t t♦♦s ♥t♦♥s ♦ ♦♦s ♥♦ ♥②③ t ♦♠♣♦♥♥t ♠♠ ♠♦
♦♠♣♦♥♥t ♠♠ ♥tr♣rtt♦♥ ♠♦ ♥tr♣rtt♦♥ ♦ t ♠♠♠♦s tt ② tr t♦♦s ♥t♦♥ ♥t♦♥ s♠♠r② ♣r♦s ♥r♦r ♦ t ♠♦ ♥♦r♠t♦♥ rtr ♥♠rs ♦ ♠♦s τkb κkb
♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s
♥t♦♥ ♣r♦♥ ♠♦ ♦r> s♠♠r②rs
♠r ♦ rs ♠r ♦ ♥s ♠r ♦ ♠♦ts ss ♥♠r ♦♦♦
♦ ♥♠r
ss ss
♥①
ss ss ♣♣ ♥①
ss ss
♦♦s sr♣t ♥tsts t st str♥
♥t♦♥ r♣♦t ♣r♦s r♣ s♠♠r② ♦ t ♣r♠trs ♥ t ♣♦ts r♣♦t rt♥ t ♣r♦t② ♦ t ♠♦s ♣r ss ♦r ♦② ♦rr♥ t ♠♦t② r♦ss♥s ♦r♥ t♦ tr ♣♦str♦r ♣r♦t②
r♣♦t ♦ t ♣r♠trs ♣rs♥t ② r > r♣♦trs
♦♦s sr♣t ♥tsts t st str♥
♠♦rt② ss s♣② ♥ r② s ♠♥② ♦♠♣♦s t t s♦♥ ♥♦ss s♦♥ ss s♣② ♥ s ♦♠♣♦s t tt ♥♦s sr♦s ② s♦♠ ♥tsts s♣② t t ♦t tt t ♥tst ♠♥② ♥♦ss t tt s s♦♥ s♥ ts ♦rrs♣♦♥♥ r s ♠♦ ♥ ts ♦t♦♥♦r ♦t sss
♦♠♣♦♥♥t ♠♠ s③t♦♥ ♥t♦♥ ♣♦t ♣r♦s sttr♣♦t♦ t ♥s ② ♥t♥ tr ss ♠♠rs♣ ♦r♥ t♦ t ♠♣ r♥ ♦rrs♣♦♥♥ ♥②ss ♠♣ r t ①s r ♦s♥ ② t sr
♦♦s
0.0
0.4
0.8
sound sound carious sound sound carioussound carious carious carious sound soundsound sound carious carious carious sound
de1−de2−de3.
0.0
0.4
0.8
1.2
sound
de4.
0.0
0.4
0.8
carious sound
de5.
r ♠♠r② ♦ t ♠♠ ♣r♠trs
ttr♣♦t ♦ t ♥s ♣rs♥t ② r > ♣♦trs
♦♦s sr♣t ♥tsts t st str♥
♣tr ♦ ♦♠♣rs♦♥ ♣r♦r♠ ② tr ♣s
−0.5 0.0 0.5 1.0 1.5 2.0 2.5
−0.
50.
00.
51.
01.
5
Axe 1 of the multiple correspondence analysis
Axe
2 o
f the
mul
tiple
cor
resp
onde
nce
anal
ysis
r ttr♣♦t ♥ t rst ♦rrs♣♦♥♥ ♥②ss ♠♣ ♥s t t♦ t ♠♦rt② ss r s♣② ② r tr♥s t♦s t t♦t ♠♥♦rt② ss r s♣② ② rs ♥ ♥♦r♠ ♥♦s ♦♥ ❬ ❪s ♥ t♦♠t② ♦♥ ♦t ①s ♦r ♥ ♥ ♦rr t♦ ♠♣r♦s③t♦♥
♦♥s♦♥ ♦ Prt
❲ s♥ tt t ss t♥t ss ♠♦ ♦s t♦ t t s♠t sts t♥s t♦ ts s♣rst② ♥ ② ts ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♦r s t sts ♠♦r ♦♠♣① ♠♦s t♥ ♥t♦ ♦♥t t ♥trss♣♥♥s r rr♥t s♥ t ♥♦r♠t♦♥ ♦ ♦♥t♦♥ ♣♥♥② s ♥♦t♣rs♥t
❲♥ t ♥♠r ♦ ♥s s s♥t② r ♦r♥ t♦ t ♥♠r ♦rs t t♥t ss ♠♦ ♥ s ♥ ts ♦♥t♦♥ ♥♣♥♥ss♠♣t♦♥ s ♦t ❲ ♣rs♥t tr ♠♥ ♠♦s ♦ t ♦r♣②r①♥ ts ss♠♣t♦♥ t t② r ♥ t r♥t ts ♠♦ st♦♥ ♥stt② ♦r ♥ ♥tr♣rtt♦♥ ♦ sss s ♣r♦r♠ tr♦♦t♥♦tr t♥t r
❲ s♦ ♣r♦♣♦s t♦ ♠①tr ♠♦s r s♣ rs♦♥s ♦ t♦♥r ♠①tr ♠♦ ② ♦♥sr t ♠♥ ♥trss ♣♥♥s t♥st♦ ♦♠♣♦♥♥t strt♦♥ ♣r ♥♣♥♥t ♦s r ♠♥ str♥t s tt♦t ♠♦s ♥ s♠♠r③ ② ♠♥♥ ♣r♠trs ♥ t ♠♠♦ ♣r♦s ♦♥ ♦♥t ♥ ♦♥ ♣♥♥② rt♦♥s♣ ② ♦s ♦ rs t ♠♠ ♠♦ ♣r♦s rtrst ♠♦s ♥ t♦ ♥t♦rs♦ t ♣♥♥② str♥t ♣r ♦ ♦t tt ♦t ♠♦s ♥ ♦♥sr ♥trt♦♥s ♠♦♥ ♠♦r t♥ t♦ rs t s ♦♥s t ♥t♦ ♦♥t t♥trt♦♥s ♦ ♦rr ♦♥ ♦r t♦
s ♠♦s ♦ t ♦♥r ♠①tr ♠② ♦t ♣r♦♣♦s ♠♦s r ♥t ♦♠♣① ♥ ♦r t ♠♦ st♦♥ s t② s ♦ s♥ t♥♠r ♦ ♦♠♣t♥ ♠♦s ♥ rtr♠♦r t ♥♦r♠t♦♥ rtrr ♥r② s②♠♣t♦t s♦ t② r ♥ ♥ t ♥♠r ♦ ♠♦s s r♦r♥ t♦ t s♠♣ s③ ♦r ♥st♥ t rtr♦♥ s s t♦ st t♠♦ ♥♠r ❲ ♣r♦♣♦s ♠♠ ♦rt♠ ♣r♦r♠♥ r♥♦♠ ♠♦♥ t ♠♦s ♥ ♦rr t♦ r ts r ♦r t ♦♠♣tt♦♥t♠ ♥rss t t s③ ♦ t ♠♦ s♣ s ♣♥♦♠♥♦♥ s str♦♥♦st t♦ t ♥②ss ② t ♣r♦♣♦s ♠♦s ♦ t sts t r ♥♠r♦ rs s t ♠ ♠♦ s rsr ♦r t str ♥②ss ♦ t stst rs ♠♠ ♠♦ s ss ♦♠♣① ♥ ts ♠♦ st♦♥ ♦s ♥♦trqr ♥② ♣r♠tr st♠t ♦ t ♠♠ ♠♦ ♥ str ♠♦r ♦♠♣① tsts ♦r ♦r s t♦ s ts ♠♦s ♦♥ t sts ♥ t ♠♦st rs ♥ ♥ t ♥♠r ♦ rs s r tr r t♦♦ ♠♥② ♠♦s♥ ♦♠♣tt♦♥ ♦ t s ♦rt♠ rqrs t♦♦ ♠♥② trt♦♥s t♦ s♠♣♦r♥ t♦ ts stt♦♥r② strt♦♥ ♥ s s ♣r♠t ♣♣r♦ ♦♦♥ssts ♥ t st♠t♦♥ ♦ ♦♦ ♠♦ t ♥♦t t st ♦♥ s t ♠♦
♣tr
st♦♥ ♦ ♣r♦r♠ ② tr♠♥st t s♦♣t♠ ♣♣r♦ t♦rr ♠t♦
♥② t ♣♣r♦s ♣r♦r♠♥ t ♠♦ st♦♥ ♦t♥ rqr t♦ ♥rt ♣r♠trs ♦r ♥t ♠♦ t ♦♥② ♥tr♣rt ♦♥s r t♦srt t♦ t st ♠♦ r ♦ ♦♥ssts ♥ ♣r♦r♠♥ t ♠♦st♦♥ t♦t ♥♥ t ♣r♠trs ♦ t ♠♦ ♥ts ♥ tr t♦ ♥rt ♣r♠trs ♦♥② ♦r t st ♠♦ ♥ ♠♦s ♥ ts ♣r♦♣rt② s♠♣② t ♥ ♦ t ♠♦ st♦♥
Prt
♦s str♥ ♦r ♠①
t
s ♣rt ♦t t♦ t str ♥②ss ♦ ♠① ts s♣t ♥ tr ♣trs rst ♦♥ ♣rs♥ts ♥ ♦r ♦ t str♥ ♣♣r♦ s♣ t♦ t ♠① t sts ❲ ♠♥② ♦s♦♥ t t♦ ♠♥ ♠①tr ♠♦s r① t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ♥ ♥ ♣rt ♦t ♦ ♠trt strt♦♥s ♦r ♠① t s♦♥ ♣tr ♣rs♥ts ♠①tr ♠♦ t♦ str ♠① t sts t ♦♥t♥♦s ♥ t♦r rs s ♠♦ rs r♦♠ t ♠t t♥t ss♠♦ ♦♣ ♦r t t♦r t ♥②ss tr ♣tr ♣rs♥ts ♦♥ ♦ t ♠♥ ♦♥trt♦♥♦ ts tss t ♦♥ssts ♥ t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦s t♦ str t sts t ♥②♥ ♦ rs ♠tt♥ ♠t strt♦♥♥t♦♥ s rsts r ♣rt ♦ s♠tt rt
❯♥ ♦s ♥st ♣s ♦t♠♥ ① rs
♥ tr♦s r r
♦♥ ♠s sà ♣rtr ♦♠♥
étt ♦t♠té♣♥ r♦♦t ❱♦②s ♥
sr
♦ ♦♥t♥ts
str ♥②ss ♦ ♠① t sts stt ♦ t rt
♥ ♦ str ♥②ss ♦r ♠① t r ♦ s♠♣ ♠t♦s t♦ str ♠① t ①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s ❯♥r♥ ss♥ ♠①tr ♠♦ ♦♥s♦♥
♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s
♥tr♦t♦♥ ①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠ ♦ st♦♥ ♦rt♠ ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ t♦ r t sts ♦♥s♦♥
♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
♥tr♦t♦♥ ①tr ♠♦ ♦ ss♥ ♦♣s ②s♥ ♥r♥ tr♦♣♦st♥s s♠♣r ♠r ①♣r♠♥ts ♦♥ s♠t t sts ♥②ss ♦ tr r t sts ♦♥s♦♥
♦♥s♦♥ ♦ Prt
♣tr
str ♥②ss ♦ ♠① t sts
stt ♦ t rt
♣r♣♦s ♦ ts ♣tr s t♦ ♣rs♥t t ♠♥ ♣♣r♦s t♦ str ♠① t stsrst② ♠♣s③ t s♣t② ♦ t ♠① t♦r t str ♥②ss♦♥② ♣rs♥t s♦♠ ♥ ♣♣r♦s t♦ ♣r♦r♠t str ♥②ss ♦ s t♥② t t t♦ ♠♦st r♥t ♣♣r♦s t♦str ♠① t sts
é♥étq ♥ ♥♦èrs r♦♠♦s♦♠s ♥s t♠♦s♣èr
s t①s ♣♦r s ①st ♠♦♥ t♣s ♦♥t
♦r ésr ♥t ♥♦s ♣♦rtr
♥ ♦ str ♥②ss ♦r ♠① t
♥tr♦t♦♥ ♦②s ♠♥② t sts r ♦t♥ ♦♠♣♦s t ♠① rs r♥t ♥s ♦ rs ♥ t t st ♦ t s ss♥t t♦ t♦str s t sts t② ♥r♥t t♦ t str ♥②ss ♦ ♠① t♣r♦r♠ ② ♠①tr ♠♦s s t ♦ ♠trt strt♦♥ ♦r s t♥ t ss♥ rs♣t② t P♦ss♦♥ ♥ t ♠t♥♦♠ strt♦♥s r rr♥ t♦ str ♦♥t♥♦s rs♣t② ♥tr ♥ t♦r ttr s ♥♦ rr♥ strt♦♥ ♦r ♠① t ♦ t sst ♣♣r♦ ♦♥ssts♥ ss♠♥ t ♦♥t♦♥ ♥♣♥♥ t♥ t rs ♥ ♥ st♥ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦r ♦♠♣♦♥♥t ♦r ts♣♣r♦ ♥ t♦ ss ♦r♦r s♦♠ ♠♦s ♣♣r♦ t strt♦♥ ♦♥trss ♦rrt ♠① t t t② r ♥♦t s② ♠♥♥ ♥ t♦♥♠♥s♦♥ ♠r♥s ♦ ♦♠♣♦♥♥t ♦ ♥♦t ♦♦ ss strt♦♥s st ♠①tr ♠♦s r s t♦ str ts ♦t ♣♣rs t♦ r ♦r ss♥ t ♣r♦s ♠♥♥ ♠♦s
♣tr str ♥②ss ♦ ♠① t sts stt ♦ t rt
♦ r ♦ts s ♣tr ♣r♦♣♦ss ♥ ♦r ♦ t str♥ ♣♣r♦s ♦r ♠① t t ♣ts t t ♦♥ t ♠♣♦rt♥ t♦ ♣r♦ ♠trt♠①tr ♠♦s rs♣t ♦t ♦♦♥ ♦ts
♦ ♣r♦ ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦r ♦♠♣♦♥♥t
♦ ♠♦③ t ♥trss ♣♥♥s
s t♦ ♦ts ♠ t♦ s♠♣② t ♠♦ ♥tr♣rtt♦♥ ♥ t ♣rtt♦♥r s ♥ s r♠♦r ♥ t ♦♥♠♥s♦♥ ♠r♥s ♦ t ♦♠♣♦♥♥tsr ss ♦r♦r ts ss ♥tr♣rtt♦♥ s ♠♦r ♣rs ♥ t ♥trss♣♥♥s r ♠♦③ s t ♠①tr ♠♦s ♣rs♥t ♥ t ♦♦♥♣trs ♠ t rs♣t♥ ♦t ♦ts
t r♦♦t ts ♣rt ♦♥sr t ert t♦r ♦ ♠① rs ♥♦t ② xi = (x1i , . . . , x
ci ,x
c+1i , . . . ,xei ) ❲ ♥♦t ② x
i = (x1i , . . . , xci)
ts sst ♦ t c ♦♥t♥♦s rs ♥ t s♠ ② ♥♦t ② x
i =(xc+1
i , . . . ,xei ) ts sst ♦ t d srt rs r c + d = e ♦t ttt tr♠ srt s♣ ♦r ♣rs♥t ♠♦ ❲ ♥♦t ② m t♥♠r ♦ ♠♦t② r♦ss♥s ♦ t st ♦ t t♦r rs ♦ x
i
trtr ♦ ts ♣tr t♦♥ rst② ♣rs♥ts tr ♥ ♠t♦s r ♥♦t r♥t s♥ t② ♦ ♥♦t ♦♥sr r ♥ ts ♥t s♣ ♦♥②ts st♦♥ ♦r♠ts t♦ ①t♥s♦♥s ♦ ss ♠t♦s t♦ ♣r♦r♠ t str♥②ss ♦ ♠① rs t♦ ♦♦♥ st♦♥s ♣rs♥t t t♦ ♠♦st r♥t♣♣r♦s t♦ str ♠① t t♦♥ ts t ♠①tr ♦ ♦t♦♥ ♠♦s♥ ts ①t♥s♦♥ ♣r ♦ t♦♥ ♥tr♦s t ♥r♥ ss♥ ♠①tr♠♦ ♦♥s♦♥ s ♥ ♥ t♦♥
r ♦ s♠♣ ♠t♦s t♦ str ♠①
t
♠t♦s
♥ ♦rr t♦ ♠♣s③ t ts ♥r♥t t♦ t ♠① t str♥ ♥♠rt tr ♥ t ♥♦t ♥t ♠t♦s ♣r♠t t♦ str s t♦r ♦ ts ♠t♦s ♠♥ r ♥ tr t② ♦ ♥♦trs♣t t ♥ ♦ r tr t② ♦ ♥♦t ♦♥sr r ♥ ts♥t s♣
❲♦ ♦♥t♥♦s ♠t♦ ♥ ♠② t♠♣t t♦ str t srt rss t② r ♦♥t♥♦s s ts ♠t♦ ♦♥ssts ♥ ♦♥rt♥ t t♦r♥ ♦r♥ ttrt s t♦ ♥♠r s ♠t♦ s♣ t♦ t ♦♥t♥♦st s t♥ ♣♣ t♦ ♣r♦r♠ t str ♥②ss s ♣♣r♦ ♠s r②str♦♥ ss♠♣t♦♥ ♦r t ♦r♥ rs ♥ t ss♠s tt tr s ts♠ ♣ t♥ t ♦♣s ♦ sss ♠♦ts ♦r♦r ts ♣♣r♦
r ♦ s♠♣ ♠t♦s t♦ str ♠① t
s t♦ rr ♥ t ♣rs♥ ♦ t♦r rs ♥ t ss♠s ♠♥♥ss ♦rr t♥ t ♠♦ts ♦r ♥st♥ t s ♥♦t ♣♦ss t♦ ♥♠r rs t♦ t♦r s ♦♦r ♦r ♦②
❲♦ srt ♠t♦ s ♠t♦ ♦♥ssts ♥ srt③t♦♥ ♦ t ♦♥t♥♦s rs s ♠t♦ s♣ t♦ t t♦r rs s s t♦ ♣r♦r♠t str ♥②ss ♦s ♦ t ♥♠rs ♦ ♠♦ts ♥ ♦ t ♦♥♦t♦♥s r t ♦r ts ♦s r r s♥ t② ♥♥ trsts ♦ t str ♥②ss ♦r♦r tr r t ♥♠rs ♦ t♦rs tsrt③t♦♥ ♣r♦ss s t♦ ♦ss ♦ ♥♦r♠t♦♥ ♥② t ♥trss ♣♥♥s r t② ♠♦③ s♥ t rs r ♦♥sr s t♦rs Prt
t♦r ♣♣r♦ t♣ t♦r ♥②ss ♠t♦ ♣r♠ts t♦ ♣r♦t♥s sr ② t♦r rs ♥ ♦♥t♥♦s s♣ s ②r♣♥ t srt rs ② tr t♦r ♦♦r♥ts ♥② ♠t♦ s♣ t♦t ♦♥t♥♦s rs ♥ s t♦ str t t st ♦r ♥♦t tt♥ ts s♣ s ♦♥t♥♦s ♥s ♥ t ♦♥② ♥t ♥♠r ♦ ss ♣♥♦♠♥♦♥ s♦ ♥rss t rs ♦ ♥r② ♦r♦r t rsts rss ♠♥♥ s♥ t rs r ♥♦t str ♥ tr ♥t s♣ ♥t ♥tr♣rtt♦♥ ♦ t sss s ♦♥ ② t ♣r♠trs ♦ t t♦r s♣
tr ♠t♦s ♣rs♥t ♦ r ♥♦t ♥t s♥ t② ♦ ♥♦t rs♣tt ♥tr ♦ r s ♥ t ♦♦♥ t st ♠t♦s ♠♦③t strt♦♥ ♦ t rs ♥ tr ♥t s♣
①t♥s♦♥ ♦ ss ♠t♦s ♦r ♠① t
♠♥s ♦rt♠ ♠♥s ♦rt♠ ♦♥② rqrs ♥t♦♥ ♦ st♥ t♥ t ♥s t♦ str ♥② ♥ ♦ t r♥t st♥ ♠srs ♥ st ♦r ♠① t s ♦r ♥st♥ t sst♦♥s ♦ ❩ ♥❬❪ ♥ ♦ ♠ ♥ ② ❬❪ ♦s② ts ♣♣r♦ ♣s trs ♦ t ♦♠tr ♠t♦s sss ♥ t♦♥
♦♥t♦♥ ♥♣♥♥ ♠①tr ♠♦ ♦ rr♥ strt♦♥♦r ♠① t s ♣r♦♠ t♦ ♣r♦r♠ t str ♥②ss t ♠①tr ♠♦ss ♣r♦♠ s s② ♦ ② t ♦♥t♦♥ ♥♣♥♥ ♠♦ s t♦♥ ♥ ♦♠♣♦♥♥t strt♦♥ s ♥ ② t ♣r♦t ♦ ♥rt strt♦♥s s ss strt♦♥s ♥ s s t ♦♥♠♥s♦♥♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥ts s ♣r♦♣♦s ② r ❬❪ ♥ ② ♦st ♥ P♣♦r♦ ❬P❪ ♦♥ssts ♥ stt♥ t ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ② ss strt♦♥s s ♠♦r ♥♦t♦♥ ♦r t ♦♥t♦♥ ♥♣♥♥ ♠♦ s s ♥ ts ss♠♣t♦♥ s ♦t♦ ♣rs♥t t♦ ♠♥ ♣♣r♦s r① t ♦♥t♦♥ ♥♣♥♥ss♠♣t♦♥
♣tr str ♥②ss ♦ ♠① t sts stt ♦ t rt
①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥
♣r ♦s
t ♦t♦♥ ♠①tr ♠♦ ♥tr♦ ② ❲ r③♥♦s ❬r③❪♦s t♦ str t sts t ♦♥t♥♦s ♥ t♦r rs
♥ t ♦♥t♥ts t ♦ t♦r rs ♥t♦ s♥ ♦♥ ♦♦s ♠t♥♦♠ strt♦♥ ♦r♦r t ss♠s tt t ♦♥t♥♦srs ♦♦ ♠trt ss♥ strt♦♥ ♦♥t♦♥② ♦♥ t ss ♥♦♥ ♠♦t② r♦ss♥ ♦r ♣rs② ts ♠♥s ♣♥ ♦♥ ♦t ss ♥t♦r rs s t ♦♥t♦♥ ♣♥♥② t♥ t ♦ rss t♥ ♥t♦ ♦♥t
♦t♦♥ ♠♦
♠ ♥ ♥ t ❬❪ ♥♦t tt t rs♥ r♦♠ ①♣r♠♥tt♦♥s ♥♣s②♦♦② ♦t♥ ♦♥t♥ ♦t srt ♥ ♦♥t♥♦s rs ♦ t② ♥tr♦t ♦t♦♥ ♠♦ t♦ ♠srs ♦ ss♦t♦♥ t♥ t rs ♦ st sts
♥ ♦t♦♥ ♠♦ ♥s t ♠t♥♦♠ strt♦♥ ♦♥ t ♦t♦r rs ♥ ♠trt ss♥ strt♦♥ ♦♥ t ♦♥t♥♦srs ♦♥t♦♥② ♦♥ t t♦r ♦♥s ♦r ♣rs② t st ♦ t t♦r rs x
i s ♦♥sr s ♦♥ t♦r rs ♦♦s r♠t♥♦♠ strt♦♥ Mm(λ1, ..., λm) s λh ♥♦ts t ♣r♦t② tt x
i
ts t ♠♦t② r♦ss♥ h ♦r♦r ♦♥t♦♥② ♦♥ x
i t♥ t ♠♦t②r♦ss♥ h t crt ♦♥t♥♦s r x
i ♦♦s crt ss♥ strt♦♥ Nc(µ
h,Σ) s t t♦r rs ♥♥ t ♠♥ ♦ t ♦♥t♥♦srs t ♥♦t tr s♣rs♦♥
♦tt♦♥s s t st ♦ t t♦r rs x
i s ♦♥sr ② t ♦t♦♥♠♦ s ♦♥ t♦r rs s ♦♠♣t s♥t ♦♥ s sxhi = 1 t ♥ ts t ♠♦t② r♦ss♥ h ♥ xhi = 0 ♦trs
♥t♦♥ ♦t♦♥ ♠♦ t♦r ♦ ♠① rs xi = (x
i ,x
i ) sr♥ ② ♦t♦♥ ♠♦ ts ♣ s rtt♥ s ♦♦s
p(xi;α) =m∏
h=1
(
λhφc(x
i ;µh,Σ)
)xhi ,
r α r♦♣s t s♣rs♦♥ ♠tr① Σ ♥ t t♦r (λh,µh;h = 1, . . . ,m)
①tr ♦ ♦t♦♥ ♠♦s
♦t♦♥ ♠♦ s ①t♥ t♦ t ♠①tr r♠♦r ♥ t ♠①tr♦ ♦t♦♥ ♠♦s s s ♥ sr♠♥♥t ♥②ss ② ❲ r③♥♦s ❬r③❪
①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s
♥ ♥ str ♥②ss ② r♥ ♥ ❲ r③♥♦s ❬❪ ♥ ♦rr t♦t ♥t♦ ♦♥t t ♥trss ♣♥♥s
♥t♦♥ ①tr ♦ ♦t♦♥ ♠♦s t♦r ♦ ♠① t xi =(x
i ,x
i ) s r♥ ② ♠①tr ♦ ♦t♦♥ ♠♦s t ♣ ♦ ts ♦♠♣♦♥♥t ks rtt♥ s ♦♦s ♦r k = 1, . . . , g
p(xi;αk) =m∏
h=1
(
λhkφc(x
i ;µhk,Σ)
)xhi ,
r αk r♦♣s t s♣rs♦♥ ♠tr① Σ ♥ t t♦r (λhk,µhk;h = 1, . . . ,m)
♥trss ♣♥♥s ♠①tr ♦ ♦t♦♥ ♠♦s ♦♥srs t ♥trss ♣♥♥s ♣r ♦♣ ♦ rs s ♦♦s
♦t rs r t♦r tr ♥trss ♣♥♥s r ♠♦③② t ♠t♥♦♠ strt♦♥s
♦t rs r ♦♥t♥♦s tr ♥trss ♣♥♥s r ♠♦③② t rt ss♥ strt♦♥s
♦♥ r s t♦r ♥ ♦♥ r s ♦♥t♥♦s tr ♥trss♣♥♥s r ♠♦③ ② t ♥♥ ♦ t t♦r r ♦♥ t♠♥s ♦ t ss♥ strt♦♥s ♦ t ♦♥t♥♦s r
♥tt② s ♣♦♥t♦t ② ❲s ♥ ♦ ❬❲❪ t ♠①tr ♦♦t♦♥ ♠♦s s ♥♦t ♥t s ♦ t ♥tr♠♥② ♦ ss ♠♠rs♣st ♦t♦♥ ♥ ♦rr t♦ ♦r♦♠ ts ♦ ♥tt② ts t♦rs s♦♠ ♦♥str♥ts ♦♥ t ♠♥ ♣r♠trs ♦ t ss♥ strt♦♥s
♥♠♥s♦♥ ♠r♥ strt♦♥s ❲ ♠♣s③ tt t ♦♥t♦♥ ♥♣♥♥ ♠♦ s ♠♥♥ s♥ ts ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ ts ♦♠♣♦♥♥ts r ss ♦r ♥st♥ t② ♦♥sst ♥ ♠t♥♦♠ ♦r ss♥ strt♦♥s ♦r t ♠①tr ♦ ♦t♦♥ ♠♦s t ♦♥♠♥s♦♥♠r♥ strt♦♥s ♦ t t♦r rs ♦r ♦♠♣♦♥♥t r ss s♥t② r ♠t♥♦♠ strt♦♥s ♦r t ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ t ♦♥t♥♦s rs ♦r ♦♠♣♦♥♥t r ♥♦t ss ♥ t②♦♥sst ♥ ♠①tr ♦ ♦♠♦sst ss♥ t m ♦♠♣♦♥♥ts
Pr♠tr st♠t♦♥ ♥r♥ ♥ s② ♣r♦r♠ ♥ ♦t rq♥tst♥ ②s♥ r♠♦rs ♥ t t♦rs ♦♥② ♣rs♥t t ♥ t rq♥tst ♦♥ ♠ ♥ ♦t♥ ② t ♦♦♥ ♠ ♦rt♠
♣tr str ♥②ss ♦ ♠① t sts stt ♦ t rt
trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ t ♦♥t♦♥ ♣r♦ts
tik(θ[r]) =
π[r]k p(xi;α
[r]k )
p(xi;θ[r])
.
st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦
π[r+1]k =
n[r]k
n, λ
h[r+1]k =
1
n[r]k
n∑
i=1
tik(θ[r])xhi ,
µh[r+1]k =
1
n[r]k
n∑
i=1
tik(θ[r])xhi x
i ,
Σ[r+1] =
1
n[r]
g∑
k=1
mj∑
h=1
n∑
i=1
tik(θ[r])xhi (x
i − µh[r+1]k )′(x
i − µh[r+1]k ),
r n[r]
k =∑n
i=1 tik(θ[r])
♦rt♠ ♠ ♦rt♠ ♦r t ♠①tr ♦ ♦t♦♥ ♠♦s
♠♣ ♥ s② ♦t♥ ② st♥ s ♣r♦r s ♣r♦r ss♠s♥♣♥♥ t♥ t ♣r♠trs ♥ sts ♦♥t strt♦♥s ♦r ♣r♠trs t s s♦ s② t♦ s s♠♣r s♥ t ♣r♠trs ①♣t ♣♦str♦r strt♦♥s
①tr ♦ ♦s ♦ ♦t♦♥ ♠♦
♥ ♥♠r ♦ ♣r♠trs rqr ② t ♠①tr ♦ ♦t♦♥ ♠♦s♥rss t t ♥♠r ♦ t♦r rs ♥ t t ♥♠r ♦ tr♠♦ts s ♦r♥s♥ ♥ ♥t ❬ ❪ ♣r♦♣♦s ♥ ①t♥s♦♥ ♦ts ♠♦ ♥ tr ①t♥s♦♥ t rs r s♣t ♥t♦ ♦♥t♦♥② ♥♣♥♥t♦s s tt ♦ s ♦♠♣♦s t t ♠♦st ♦♥ t♦r r♦r♦r ♦ ♦ rs ♦♦s ♦t♦♥ ♠♦
♥t♦♥ ①tr ♦ ♦s ♦ ♦t♦♥ ♠♦ t♦r xi ♦♠♣♦st ♦♥t♥♦s ♥ t♦r rs rss r♦♠ ♠①tr ♦ ♦s ♦ ♦t♦♥♠♦ t ♣ ♦ ts ♦♠♣♦♥♥t k s rtt♥ s ♦♦s ♦r k = 1, . . . , g
p(xi;αk) =∏
b=1
p(xbi ;αkb),
r αk = (αkb; b = 1, . . . ,) ♥ t ♣ ♦ ♦ b ♦r ♦♠♣♦♥♥t k s rtt♥
①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s
s ♦♦s ♦r b = 1, . . . ,
p(xi;αkb) =
φdkb(xbi ,µkb,Σkb) xb
i s ♦♥t♥♦smb∏
h=1
(λkh)xbhi xb
i s t♦r
mb∏
h=1
(
λhkbφdkb−1(xbi ;µh
kb,Σkb))x
bhi
xbi s ♠①
r x
bi ♥ x
bi r rs♣t② t ♦♥t♥♦s ♥ t t♦r rs
♦ ♦ b
t ♠♦s tr r ♦♥② ♦♥t♥♦s rs c = e ♥ d = 0 ♥ t
rs r t ♥t♦ t s♠ ♦ x1i = xi t♥ t ♠♦
s q♥t t♦ t tr♦sst ss♥ ♠①tr ♠♦ tr r ♦♥② ♦♥t♥♦s rs c = e ♥ d = 0 ♥
♦ s ♦♠♣♦s t ♦♥② ♦♥ r xbi = xbi ♦r b = 1, . . . , c
t♥ t ♠♦ s q♥t t♦ t ss♥ ♠①tr ♠♦ t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ Σk s ♦♥
tr r ♦♥② t♦r rs c = 0 ♥ d = e t♥ t ♠♦s q♥t t♦ t t♥t ss ♠♦ s t♦♥
Pr♠tr st♠t♦♥ ♥r♥ s s② ♣r♦r♠ ♥ t ♦♥t♦♥ ♥♣♥♥ t♥ t ♦s ♦s t♦ ♣t t ♠ ♦rt♠ ♣rs♥t♥ ♦rt♠ ♥ ♦rr t♦ ♦t♥ t ♠ ♦ t ♠①tr ♦ ♦s ♦ ♦t♦♥♠♦ ②s♥ ♥r♥ ♦ ♣r♦r♠ ② s s♠♣r t ♣r♦rsr ss♠ t♦ ♥♣♥♥t ♥ ♦♦ ♦♥t strt♦♥s
♦ st♠t♦♥ ♦ t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s t♦rs st♠t t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s ② ♥ s♥♥ ♠t♦t ① ♥♠r ♦ sss ♥ ♥ ①st ♣♣r♦ s ♥♦t ♦ st♦♥ ♠ ♦ ts s♥♥ ♠t♦ s t♦ ♦♣t♠③ ♥ ♥♦r♠t♦♥ rtr♦♥ s ♠t♦ s ♥t③ ② t ♦② ♥♣♥♥t ♠♦ ♥ r♥t♠♦s r ♣r♦♣♦s ② s♥ t ♥trss ♣♥♥s ♦♠♣t t t rr♥t ♠♦
Pr♦r♠♥s ♦ t ♠①tr ♦ ♦s ♦ ♦t♦♥ ♠♦ s ♣rs♥t ♥❬❪ t ♠①tr ♦ ♦s ♦ ♦t♦♥ ♠♦ ♥ ♦t♣r♦r♠ t ♦② ♥♣♥♥t ♠♦ ♦r ts ♠♦ s t♦ ♠♥ rs rst ♦♥ s♦t t ss ♥tr♣rtt♦♥ s♥ t ♦♥♠♥s♦♥ ♠r♥ strt♦♥ ♦ ♦♠♣♦♥♥t s ♥♦t ss t r s ♦♥t♥♦s s♦♥ ♦♥ s ♦tt t② t♦ ♣r♦r♠ ♠♦ st♦♥ ♥ t ♣r♦♣♦s ♣♣r♦ ♥ s♦♣t♠ t♦ st t r♣rtt♦♥ ♦ t rs ♥t♦ ♦s rtr♠♦r t♦ ♦ t ♦rrt♦♥ ♦♥t t♥ ♦♥t♥♦s r ♥ t♦r
♣tr str ♥②ss ♦ ♠① t sts stt ♦ t rt
♦♥ s st ♦r ts ♦ s r s♥ t tr♠♥s t ♥tsr♥ t ♠♦ st♠t♦♥
❯♥r♥ ss♥ ♠①tr ♠♦
♥ ♥r♥ ss♥ ♠①tr ♠♦ ♥tr♦ ② rtt❬❪ ♣r♦r♠s t str ♥②ss ♦ t sts t ♦♥t♥♦s ♥ ♦r♥ rs ts ♠♥ ss♠♣t♦♥ s tt t ♦sr ♦r♥ ♥ ♥r② rs r♥rt r♦♠ ♥r②♥ ♥♦sr ♦♥t♥♦s rs ♦r♥ t♦ t s ♦ st ♦ trs♦s
♠r t♦r rs r ♥♦t ♦ t♦r rs①♣t t ♥r② ♦♥s ♥♥♦t ♠♦③ ② t ♥r♥ ss♥ ♠①tr♠♦ ♥ ts ♠♦ ss♠s ♥ ♦rr t♥ t ♠♦ts s ♥♦t♣rs♥t ♦r s rs
sr♣t♦♥ ♦ t ♥r♥ ss♥ ♠①tr ♠♦
ss♥ r ❲ ♦♥sr t t♦r yi = (x
i ,y
i ) r y
i s ♦♥t♥♦st♦r ♦ s③ d t♦r yi s ss♠ t♦ r♥ ② t ♦♠♦sst ss♥♠①tr ♠♦ ♦s t ♣ s
p(yi;θ) =
g∑
k=1
πkφe(yi;µk,Σ).
ss♥ t♥t r ♥ ♣rt t rs yi r ♥♦t ♦sr ♦rt② r rt t♦ t st ♦ t ♦sr srt rs x
i s ♦♦s
∀j = c+ 1, . . . , e, xjhi = 1 bjhk < yji ≤ bjh+1k ,
r bjhk < bjh+1k ♦r j = c + 1, . . . , e ♥ h = 1, . . . ,mj ♥ r bj1k = −∞ ♥
bjmj+1k = ∞ ♦♥s bjhk tr♠♥ t ♦sr srt rs r♦♠ tt♥t ♦♥t♥♦s ♦♥s s ♦t♥ tt t ♦sr rs xi = (x
i ,x
i ) t ♦♦♥ ♣
p(xi;θ) =
g∑
k=1
πk
∫
Sk(x
i )
φe(yi,µk,Σ)dyi ,
r Sk(x
i ) s t ♦♠♥ ♦ t ♥trt♦♥ ♦ t♥t ss♥ rs yi rtt♦ t ♦sr srt rs x
i ♦r ♣rs② Sk(x
i ) = Sc+1k (xc+1
i )× . . . ×Sek(xei ) r t ♥tr Sjk(xji ) s ♥ ♦r j = c + 1, . . . , e s s Sjk(xji ) =
]bjhk , bjh+1k ] xjhi = 1
♦♥s♦♥
tr♥t ♦r♠ ♦ t ♣ ♥ tr♥t ♥ ♠♦r r♥② ♦r♠ ♦ t♣ ♥ ② s ♦t♥ ② ♥♦t♥ tt t ♦♥t♦♥ strt♦♥ ♦ x
i
♥ x
i s ♥ ♥r♥ ss♥ ♦♥ s strt♦♥ s t ♠♥ µ|k ♥
t ♦r♥ ♠tr① Σ| r ♥ ②
µ|k = µ
k +ΣΣ−1(x
i − µ
k) ♥ Σ| = Σ −ΣΣ
−1Σ,
r µ
k ♥ µ
k r rs♣t② t ♠♥s ♦ x
i ♥ ♦ x
i ♥ r t ♦
r♥ ♠tr① Σ =
[
Σ Σ
Σ Σ
]
s ♦♠♣♦s ♥t♦ s♠trs ♦r ♥st♥
Σ s t s♠tr① ♦ Σ ♦♠♣♦s ② t r♦s ♥ t ♦♠♥s rt t♦ t♦sr ♦♥t♥♦s rs s tr♥t ♦r♠ ♦ t ♣ ♦s s t♦ ♥t ♥r♥ ss♥ ♠①tr ♠♦
♥t♦♥ ❯♥r♥ ss♥ ♠①tr ♠♦ t xi t t♦r ♦ e ♠①rs r♥ t ♥r♥ ss♥ ♠①tr ♠♦ ts ♣ s rtt♥ s♦♦s
p(xi;θ) =
g∑
k=1
πkφc(x
i ;µ
k,Σ)
∫
Sk(x
i )
φd(y
i ;µ|k ,Σ|)dyi .
r ♦♥t♦♥ ♦r ♠♦ ♥tt② ♥ t t♥t t♦r yi s ♥♦t♦sr tr s ♥♦ ♥♦r♠t♦♥ ♦♥ ts ♠♥ ♥ r♥ s t ♠♦ ss♠stt t ♠♥ts ♦ µ
k r ♥ ♥ tt t ♦♥ ♠♥ts ♦ Σ r q t♦♦♥
st♠t♦♥ ♦ t ♥r♥ ss♥ ♠①tr ♠♦
♥r♥ ② ♠①♠③t♦♥ ♦ t ♦♦♦ ♥t♦♥ s ♥♦t s② s♦ t ♣rs♥ ♦ d♠♥s♦♥ ♥trs ♥ ♥♦ ①♣t ♦r♠ ♥ Σ s ♥♦t♦♥
rtt ♣r♦♣♦ss t♦ ♣r♦r♠ t ♥r♥ ② s♥ s♠♣① ♠t♦ ♦ttt ts ♣♣r♦ ♠ts t ♦r t ♥♠r ♦ srt rs
♦♥s♦♥
❲ ♣♦♥t ♦t t ♠♣♦rt♥ t♦ ♦♥sr t rs ♥ tr ♥ts♣ ♦r t ♠① t r ♥♦t s② str ② ♠① ♠♦s s♦ t ♦ st♥r ♠trt strt♦♥ ♦r s rs s t ♠s t♦ ♣r♦♣♦s r♥t ♠trt strt♦♥ ♦r ♠① t ❲ ♣t tt ♦♥ t ♠♣♦rt♥ tt t ♦♥♠♥s♦♥ ♠r♥s ♦ ts strt♦♥s rss ♥ tt t ♣♥♥s r ♠♦③
♠♦ ♣rs♥t ♥ ♣tr ♦s t♦ ♣r♦r♠ t str ♥②ss ♦ tsts t ♦♥t♥♦s ♥ t♦r rs ② ♥ ts ♦ts ♦ttt ts stt♦♥ t st t ♦♥t♥♦s ♥ t♦r rs s t ♠♦stst ♦♥
♣tr str ♥②ss ♦ ♠① t sts stt ♦ t rt
♥ t ♦r♣② t t♦rs ♦ ♥♦t st② t s ♦ ♠① rs t♥tr s s t ♠♦ ♣rs♥t ♥ ♣tr s ♦ ♥trst ♥ t sr② ♥r s♥ t ♦s t♦ str t sts t ♥② ♥ ♦ rs ♠tt♥
♣tr
♦s str♥ ♦ ss♥
♥ ♦st strt♦♥s
s ♣tr ♥tr♦s s♣rs ♠①tr ♠♦ ♦r tsts t ♦♥t♥♦s ♥ t♦r rs ♦♠♣♦♥♥t strt♦♥s ♦ t ♦♥t♥♦s rs rss♥ ♦r♦r t t♦r rs r ss♠ t♦ ♥♣♥♥t ♦♥t♦♥② ♦♥ t ss ♥♦♥ t ♦♥t♥♦s rs ♥② ♦♥t♦♥② ♦♥t ♦♥t♥♦s rs t ♦♠♣♦♥♥t strt♦♥s ♦t t♦r rs r ♥r ♦st strt♦♥sr ♣r♠trs r ♥♦t ③r♦ ♥ t rrss♦♥ ♠①♠♠ ♦♦ ♥r♥ ♥ t ♠♦ st♦♥ r s♠t♥♦s② ♣r♦r♠ ② ♠ ♦rt♠♦r ① ♥♠r ♦ sss♠r ①♣r♠♥ts strt t ♠♦ r♥ ♥ str ♥②ss ♥ ♥ s♠s♣rs sst♦♥♥ ♥s r
str♠ t♦ts rà ♠♦t t♠♣s ♥♦ ♣♦t rr
r q t♦♠ t♦♠ t♦♠♥ ♦r sr
t strs ♦rt ♣r q ♦ str♦ ♦rt ♣r
r q t♦♠ t♦♠ t♦♠ ♥s ♣♦r♠ rrs st
♥tr♦t♦♥
❲ ♣rs♥t ♠①tr ♠♦ t♦ str t sts t ♦♥t♥♦s ♥ t♦rrs s ♠♦ s ♦ ♦t t♦ ♣r♦ ss ♦♥♠♥s♦♥♠r♥ strt♦♥s ♦r ♦♠♣♦♥♥t ♥ t♦ ♠♦③ t ♥trss ♣♥♥s
♣tr ♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s
♦r s ♠♦ t ♦♥t♥♦s rs ♦♦ ♠trt ss♥ strt♦♥ ♦r ♦♠♣♦♥♥t ♦♥t♦♥② ♦♥ t ss ♥ ♦♥ t ♦♥t♥♦srs t t♦r rs r ss♠ t♦ ♥♣♥♥t② r♥ ② ♥r♦st strt♦♥s rst♥ ♠♦ s s♦ ♥♠ ♠①tr ♠♦ ♦ ss♥♥ ♦st strt♦♥s
♥r ♦st rrss♦♥s r ss② s ② ♠①tr ♠♦s ♦r t♦r t ❬♦r❪ ♦r♦r t ♠t t♥t ss ♠♦ ❬❱r❪ ss t♥t♦♥t♥♦s rs t♦ ♠♦③ t ♥trss ♣♥♥s t♥ t t♦r rs ♦ t s ♥tr t♦ s ♠①tr ♠♦ ♦ ss♥ ♥ ♦ststrt♦♥s ♥ ♦t ♦ t ♦♥t♥♦s ♥ t♦r ♥s ♦ rs r♦sr
♣rs♠♦♥♦s rs♦♥ ♦ ts ♠♦ s ♥tr♦ ② ♥ s♦♠ ♦♥str♥ts ♦♥t ♦st ♣r♠tr s♣ ♦ t rst♥ ♠♦ s ♠♦r s② ♥tr♣rt ♥♥ ♣r♦r♠ ttr tr ♦ t♥ t s ♥ t r♥ ♦r ① ♥♠r♦ sss t st♦♥ ♦ t ♣rs♠♦♥♦s ♠♦ ♥ t ♣r♠tr st♠t♦♥r s♠t♥♦s② ♣r♦r♠ ② ♠ ♦rt♠ ♦♣t♠③s ♥ ♥♦r♠t♦♥rtr♦♥ r♥ ♦r ♥♠r ①♣r♠♥ts strt t r♥ ♦ t rtr♦♥ ♦♠♣r t♦ t rtr♦♥
♥② ♥ t ♠♦ s ♥tr♦ t♦ ♣r♦r♠ str ♥②ss t ♥ ♣♣ ♦r s♠s♣rs sst♦♥ ❬❩+❪ ♥ t s ♥♦♥ ttt ♥rt ♣♣r♦s ♥ ♦t♣r♦r♠ t ♠t♦s s♣ ♦ t sst♦♥♥ s ♣♥♦♠♥♦♥ s ♣rtr② ♦sr ♥ ♦srt♦♥s r ❬❪ ♥ ts ♠t♦s ①♣♦t t ♥♦r♠t♦♥s ♣rs♥t ♥ ♦t ♥ ♥ t sr♠♥t ♣♣r♦s t ♦♥② ♥t♦ ♦♥tt t ♥♦r♠t♦♥ ♥ t② r♥ sst♦♥ r ♦♥② ♦♥ t t
trtr ♦ ts ♣tr s rt s ♦r♥③ s ♦♦s t♦♥ ♣rs♥ts t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♦r t stst ♦♥t♥♦s ♥ t♦r rs s ♠♦ ♣r♦r♠s t str ♥②ss ② ♣r♦♥ ss ♦♥♠♥s♦♥ strt♦♥s ♦r ♦♠♣♦♥♥t ♥ ②♠♦③♥ t ♥trss ♣♥♥s ♥ str♥ r♠♦r t♦♥ s♦t t♦ t ♠①♠♠ ♦♦ st♠t♦♥ t♦♥ ♣rs♥ts t ♠ ♦rt♠ s♠t♥♦s② ♣r♦r♠s t st♠t♦♥ ♦ ♦t ♠♦ ♥ ♣r♠trs② ♦♣t♠③♥ ♥ ♥♦r♠t♦♥ rtr♦♥ t♦♥ ♣rs♥ts r♥t ♥♠r ①♣r♠♥ts ② strt t r♥ ♦ t rtr♦♥ t♦ ♣r♦r♠ t ♠♦st♦♥ ♦r♦r t② s♦ t ♣r♦r♠♥s ♦ t st♠t♦♥ ♦rt♠ ♥t ♠♦ r♦st♥ss t♦♥ ♣rs♥ts ♦♥ ♣♣t♦♥ ♥ str♥ ♥ ♦♥♣♣t♦♥ ♥ s♠s♣rs sst♦♥ ♦♥ t♦ r t sts ♦♥s♦♥ s♥ ♥ t♦♥
t t xi = (x1i , . . . , xci ,x
c+1i , . . . ,xei ) t ert t♦r ♦ ♠①
rs rst c rs r ♦♥t♥♦s ♥ ts sst ♦ rs s ♥♦t② x
i st d rs r t♦r rs s♥ s♥t ♦♥ ♥ts sst ♦ rs s ♥♦t ② x
i ♦t tt c+ d = e
①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s
①tr ♠♦ ♦ ss♥ ♥ ♦st str
t♦♥s
♠ ♠♦ ♣r♦r♠s t str ♥②ss ♦ ♦♥t♥♦s ♥ t♦r tst t ♦ ♦t t♦ ♣r♦ ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦r ♦♠♣♦♥♥t ♥ t♦ ♠♦③ t ♥trss ♣♥♥s
♥ ♣ ♦ ♦♠♣♦♥♥t s ♥ ② t ♣r♦t t♥ t♣ ♦ t ♦ ♦♥t♥♦s rs ♥ t ♣ ♦ t ♦ t♦r rs♦♥t♦♥② ♦♥ t ♦ ♦♥t♥♦s rs ♦r ♣rs② ♦♥r♥♥ t♦♥t♥♦s rs t② ♦♦ ♠trt ss♥ strt♦♥ ♦r ♦♠♣♦♥♥t ♦♥r♥♥ t t♦r rs t ♠♦ ss♠s tr ♥♣♥♥♦♥t♦♥② ♦♥ ♦t t ss ♠♠rs♣ ♥ t ♦♥t♥♦s rs
♥t♦♥ ①tr ♠♦ rt t♦ ♥ t♦r xi s r♥ ② ♠①tr ♠♦ rs♣t♥ ♥ t ♣ ♦ ts ♦♠♣♦♥♥t k s rtt♥ s♦♦s ♦r k = 1, . . . , g
p(xi;αk) = p(x
i ;αk)p(x
i |x
i ;αk)
= φc(x
i ;µk,Σk)e∏
j=c+1
p(xji |x
i ;βkj),
r αk = (µk,Σk,βk) ♥♦ts t ♦ ♦♠♣♦♥♥t ♣r♠trs r t t♦rµk ∈ R
d ♥♦ts t ♠♥ ♦ t ♦♥t♥♦s rs ♦r ♦♠♣♦♥♥t k ♥ r t♠tr① Σk ♥♦ts tr ♦r♥ ♠tr① t♦r βk = (βkj; j = c+ 1, . . . , e)♥♦ts t ♦ ♣r♠trs ♦ ♦♠♣♦♥♥t k r rt t♦ t t♦rrs ♥ t t♦r βkj r♦♣s t ♣r♠trs rt t♦ t strt♦♥♦ t t♦r r x
ji ♦r ♦♠♣♦♥♥t k
♥ ♠♦ ss♠s tt ♦r ♦♠♣♦♥♥t t strt♦♥ ♦ t♦r r s ♥r ♦st rrss♦♥ ♦s t ①♣♥t♦r② rs r t ♦♥t♥♦s ♦♥s
♥t♦♥ ①tr ♠♦ ♦ ♦st rrss♦♥s ❲t t ♥♦tt♦♥ x0 = 1t ♦♠♣♦♥♥t strt♦♥s ♦ xji ♦r j = c+1, . . . , e r ♥ ② t ♦♦♥♣ ♦r k = 1, . . . , g
p(xji |x
i ;βkj) =
mj∏
h=1
exp(
∑cj′=0 β
j′hkj x
j′
i
)
∑mj
h′=1 exp(
∑cj′=0 β
j′h′
kj xj′
i
)
xjhi
,
r t ♣r♠trs βkj = (βj′hkj ; j
′ = 0, . . . , c;h = 1, . . . ,mj) ∈ Rc+1 ♥♦ts t
♦st ♣r♠trs ♦ t t♦r r xji ♦r ss k ♥ ♦rr t♦ ♥sr t
♠♦ ♥tt② ♣t ∀(k, j, j′), βj′1kj = 0 ♥② ♥♦t tt t ♣r♠tr
β0hkj s t ♥tr♣t ♦ t ♦st rrss♦♥ t ♦tr ♣r♠trs βj
′hkj ♦r
j′ = 1, . . . , c r t s♦♣ ♣r♠trs
♣tr ♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s
P♦t♥t r ♥♠r ♦ ♣r♠trs ♠♦ ♥ rqr r ♥♠r♦ ♣r♠trs ♥ t ♥♠r ♦ ♣r♠trs ♥♦ ② t ♠♦ s q t♦
(g − 1) + g
(
c(c+ 3)
2
)
+ ge∑
j=c+1
(mj − 1)(c+ 1).
♥ ♦rr t♦ ♦t♥ ttr sr♥ tr♦ ♥tr♦ ♣rs♠♦♥♦srs♦♥ ♦ t ♠♦ ② r♥ t s♣ ♦ t ♦st ♦♥ts
♣rs ♦st ♥t♦♥s ♦r t ♠① ♦♥t♦♥ ♣♥♥s s♣rst② ♦ t ♠♦ s ♥ ② t srt ♣r♠trs δkj = (δj
′
kj; j′ = 0, . . . , c)
♥ δj′
kj = 1 t♦r r j s ♦♥t♦♥② ♣♥♥t ♦♥ ♦♥t♥♦s
r j′ ♦r ♦♠♣♦♥♥t k ♥ δj′
kj = 0 ♦trs ♦t tt δ0kj = 0 ♥♦s ♥♥tr♣t ♥ t ♦st rrss♦♥ ♦ t♦r r j ♦r ♦♠♣♦♥♥t k sδkj ①s s♦♠ ♦st ♣r♠trs t♦ ③r♦ s♥ βkj ∈ S(δkj) t
S(δkj) =
βkj : ∀(k, j, j′) s s δj′kj = 1, t♥ ∀h βj′hkj = 0
.
♦♥tr♦ ♥♠r ♦ ♣r♠trs ♥♠r ♦ ♣r♠trs rqr ② t♥r ♠♦ ♥ ② (g, δ) s q t♦
(g − 1) + g
(
c(c+ 3)
2
)
+
g∑
k=1
e∑
j=c+1
c∑
j′=0
δj′
kj(mj − 1).
♥♥ ♠♦ ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♣r♦s ♠♥♥ sss ♥ ss ♥ s♠♠r③ ② ♣r♠trs♠♥ ♥ r♥ ♦r t ♦♥t♥♦s rs ♥ ♣r♦t② ♦ ♠♦t② ♦r t♦r r q t♦
p(xjh = 1|zik = 1) =
∫
Rc
φc(x
i ;µk,Σk)exp
(
∑cj′=0 β
j′hkj x
j′
i
)
∑mj
h′=1 exp(
∑cj′=0 β
j′h′
kj xj′
i
)dx
i .
t♦ ts ♥tr s ♥♦t ①♣t t s s② ♣♣r♦①♠t ② ♠♠ ♠t♦rtr♠♦r ♦r ss t ♣♥♥s t♥ t ♦♥t♥♦s rs r♠♦③ ② t ♦rrt♦♥ ♠tr① t t♦r r xji s ♦♥t♦♥②♣♥♥t t t ♦♥t♥♦s ♦♥ xj
′
i δj′
kj = 1
♣♥♥s ♥t♦r r s ♥ ①♠♣ ♦ t ♣♥♥s t♥ rs t♥ ♥t♦ ♦♥t ② t ♠♦ ♥ t♥ rs ♥♦ts ♣♥♥② t♥ rs ♥ ♥ s♥ ♦ ♥ ♥♦ts ♦♥t♦♥ ♥♣♥♥ ♦t tt t ♦sr rs r ♥ t zi tr s qt♥ t ♦♥t♥♦s rs t t♦r rs r ♥♦t ♥ t♦tr♥ t ♥trss ♣♥♥② t♥ t ♦♥t♥♦s ♥ t t♦r rsr ♥ ② t srt ♣r♠trs δ
①♠♠ ♦♦ st♠t♦♥ ♥ ♦rt♠
zi
x1i
x2i
x3i
x4i
r ①♠♣ ♦ t ♣♥♥s t♥ ♥t♦ ♦♥t ② t ♠♦ rx
i = (x1i , x2i ) ♥ x
i = (x3i ,x
4i ) t δ
1k3 = δ2k3 = δ1k4 = 1 ♥ δ2k4 = 0
♥r ♥tt② ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥ss ♥r② ♥t ts ♦ t ♣r♦♦ r ♥ ♥ ♣♣♥① ♠♦♥strt♦♥ s s♣t ♥ t♦ ♣rts rst② s♠ ♦r t ♣♦ss s ♦x
i t♦ ♦t♥ ♠①tr ♦ ss♥ strt♦♥s ♥ t♦ s ts ♥tt② rsts❬ ❨❪ ♦♥② s♦ t ♥tt② ♦ t ♣r♠trs ♦ ♦st♥t♦♥
①♠♠ ♦♦ st♠t♦♥ ♥
♦rt♠
♥ ❲ ♦♥sr t s♠♣ ① = (x1, . . . ,xn) ♦♥ssts ♦ n ♥s ss♠ ♥♣♥♥t② r♥ ② ♠①tr ♠♦ ♦ ss♥ ♥ ♦ststrt♦♥s ♠ s s② ♦t♥ ② t ♦♦♥ ♠ ♦rt♠ s ♦rt♠ tt t ♦ s ♣r♦r♠ ♦r ① ♠♦ ♥ ② t ♦♣(g, δ)
♥r♥ ♦ t ♦st ♣r♠trs t t ♠ st♣ t ♠①♠③t♦♥s ♦♥ t♣r♦♣♦rt♦♥s ♥ ♦♥ t ss♥ ♣r♠trs r s② ♣r♦r♠ ♦r tst♠t♦♥ ♦ t ♣r♠trs rt t♦ t ♦st ♥t♦♥s ♥♦ t♦ s♦ ♥♦♥①♣t qt♦♥s ♦ t② r ss② ♦t♥ ② t♦♥♣s♦♥ ♠t♦♥ t ♠ s st t♦ st♠t t ♦st rrss♦♥ ♣r♠trs r ♥s r♥t ts
♣tr ♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s
trt♥ r♦♠ ♥ ♥t θ[0] trt♦♥ [r] s rtt♥ s st♣ t ♦♥t♦♥ ♣r♦ts
tik(θ[r]) =
π[r]k p(xi;α
[r]k )
p(xi;θ[r])
.
st♣ ♠①♠③t♦♥ ♦ t ①♣tt♦♥ ♦ t ♦♠♣tt ♦♦♦
π[r+1]k =
n[r]k
n, µ
[r+1]k =
1
n[r]k
n∑
i=1
tik(θ[r])xi,
Σ[r+1]k =
1
n[r]k
n∑
i=1
tik(θ[r])(xi − µ
[r+1]k )′(xi − µ
[r+1]k ),
β[r+1]kj = argmax
βkj∈S(δkj)
n∑
i=1
tik(θ[r]) ln p(xji |x
i ;βkj),
r n[r]k =
∑ni=1 tik(θ
[r])
♦rt♠ ♠ ♦rt♠
♦ st♦♥ ♦rt♠
♠ ♠♦ st♦♥ s ♦♠♥t♦r ♣r♦♠ s ♦ t st♠t♦♥ ♦t srt ♣r♠tr δ s ♥t t♦ ♦t♥ t ♣r♠tr δ ♠①♠③st ♥♦r♠t♦♥ rtr♦♥ ♦r ① ♥♠r ♦ sss
♥ t s ♥r② ♠♣♦ss t♦ ♥ t ♠ ♦r δ s♥ t ♥♠r ♦ ♦♠♣t♥ ♠♦s s 2g(c+1)d s t rtr♦♥ s ♣♥③t♦♥ ♦ t♦srt ♦♦♦ ♥ s ♥ ♠ ♦rt♠ ♠①♠③♥ t ♣♥③♦srt ♦♦♦ ❬r❪ s t♦♥ s t ♠ st♣ ♦♥ssts♥ ♠①♠③♥ t ①♣tt♦♥ ♦ t ♣♥③ ♦♠♣tt ♦♦
♦t♦♥ ♦ t ♠ st♣ t trt♦♥ [r] t ♠ st♣ ♦ t ♠ ♦rt♠♠s t tr♠♥♥ (δ
[r+1]kj ,β
[r+1]kj ) s s
(δ[r+1]kj ,β
[r+1]kj ) = argmax
δkj∈0,1c+1
argmaxβkj∈S(δkj)
n∑
i=1
tik(θ[r]) ln p(xji |x
i ;βkj)−νkj2
lnn,
r νkj =∑c
j′=0 δj′
kj ♥ts t ♥♠r ♦ ♣r♠trs rqr ② t ♦strrss♦♥ rt t♦ t♦r r j ♦r ♦♠♣♦♥♥t k
♠r ①♣r♠♥ts ♦♥ s♠t t sts
♠ ♦rt♠ t♦ ♦ t ♦♠♥t♦r ♣r♦♠s s♣ ♦ δs r s♦ ♥ ①st ♣♣r♦ t♦ tr♠♥ δ
[r+1]kj ♦r♥ t♦ s ♥♦t
♦ s ♣rr t♦ s ♠ rs♦♥ ♦ ts ♦rt♠ ♥ ts ♦rt♠t ①♣tt♦♥ ♦ t ♣♥③ ♦♠♣tt ♦♦ s st ♥rs t t ♠st♣ ♦ t trt♦♥ [r] ♦ t ♠ st♣ s s♥♥t ♥ s♥♥t ♠t♦s♥t③ ② (δ
[r]kj ,β
[r]kj)
♥ t ♠♣♦rt♥ ♦ sr ♥t③t♦♥s ♦t tt ts ♣♣r♦ ♣st ss ♣r♦♣rts ♦ t ♠ ♦rt♠ ♦ ts tr♠♥st ♦rt♠ ♦♥rs t♦ ♦ ♦♣t♠♠ ♦ t ♣♥③ ♦srt ♦♦ ♣♥s♦ t ♥t ♦ (δ[0],θ[0]) s t s ♠♥t♦r② t♦ ♣r♦r♠ ts ♦rt♠t r♥t ♥t③t♦♥s t♦ ssr t ♦♥r♥ t♦ ♦ ♠①♠♠ ♦ t♣♥③ ♦srt ♦♦
♠r ①♣r♠♥ts ♦♥ s♠t t sts
♠ ❲ ①♣r♠♥t② st② t ♦r ♦ t ♠ ♦rt♠ ♥ t ♣r♦r♠s t ♠♦ st♦♥ s♥ ♦t ss ♥♦r♠t♦♥ rtr ♥ rsts ttst t♦ t ♦♦ ♦r ♦ t ♠ ♦rt♠ ♦r t s♠t♥♦sst♠t♦♥ ♦ t ♠♦ ♥ t ♣r♠trs ♦r♦r t② s♦ tt t rtr♦♥ ♦t♣r♦r♠s t rtr♦♥ ♥ ts ttr ♦rst♠ts t ♥♠r♦ ♦♠♣♦♥♥ts ♥ t ♦♥t♦♥ ♣♥♥s t♥ ♠① t
trtr ♦ ts st♦♥ rst② t r s♠t ♦r♥ t♦ t ♠①tr♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♦♥② t r s♠t ♦r♥t♦ ♦tr ♠♦s
♠t♦♥s ② t s♣ ♠♦
♥♦♥ ♥♠r ♦ sss
t ♥rt♦♥ t r s♠♣ ♦r♥ t♦ t ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s t t♦ ♦♠♣♦♥♥ts ♥ q ♣r♦♣♦rt♦♥s ♥s rsr t t♦ ♦♥t♥♦s rs ♥ t♦ ♥r② ♦♥s ♠♦ ♣r♠trs r t ♦♦♥
µk = (ε(k − 2), ε(k − 1)), Σk =
[
1 k − 1.5k − 1.5 1
]
, δk1 = (1, 1, 0),
δk2 = (1, 0, 1), βj′2kj = ε,
r t ♣r♠tr ε ♦s t♦ ① t sss ♦r♣s rr s ε t ttrs♣rt r t sss ♦r t♦ sss ♦r♣s ♥ ♦r r♥t s③s ♦ s♠♣s(n = 50, 100, 200, 400) t sts r ♥rt
♣tr ♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s
st♠t♦♥ ♦♥t♦♥s ♦r t st tr ♠♦s r ♥ ♦♠♣tt♦♥ ttr ♠♦ t ♠♦ ♠①♠③♥ t rtr♦♥ ♥ t ♠♦ ♠①♠③♥t rtr♦♥ st♠t♦♥ ♦ t ♣r♠trs rt t♦ t tr ♠♦ s♣r♦r♠ ② t ♠ ♦rt♠ ♠①♠③♥ t ♦♦ ♦r ① ♠♦ ♦rt♠ t♦ ♦tr ♣r♠tr st♠t♦♥s r ♣r♦r♠ ② t ♠ ♦rt♠ ♠①♠③♥ ♥ ♥♦r♠t♦♥ rtr♦♥ ♥ s r♥♦♠ ♥t③t♦♥s♦ t st♠t♦♥ ♦rt♠ r ♦♥ ♦t tt ♥ ♥ ts s♠♣ s t s ♥♦t♦ t♦ ♣r♦r♠ ♥ ♠ ♦rt♠ ♦r ♠♦ s♥ tr r 212 ♠♦s ♥♦♠♣tt♦♥
sts ♥ ♣rs♥t t r r♥ ♦ t tr ♠♦m ♥ ♦ t st ♠♦ ♦r♥ t rs♣t② rtr♦♥ ♥♦t ②m rs♣t② m ❲ ♦♥ t♦ t ♦r ♦ t ♦rt♠ s♦r t ♣r♠tr st♠t♦♥ ♥ t r r♥ t♥s t♦ ③r♦♥ n r♦s ♦r t tr ♣♣r♦s rtr♠♦r ♦r ♥t s♠♣ s③ t♥♦r♠t♦♥ rtr ♦ t♦ r t r r♥ ② st♥ ss♦♠♣① ♠♦s
r♣ 10% 20%n m
m m
♥s ♦ t r r♥s ♥ s♣ ♠♦stt♦♥ r t ♥♠r ♦ sss s ♥♦♥
♠r ♦ sss ♥♥♦♥
t ♥rt♦♥ t r s♠♣ ♦r♥ t♦ ♠♦ sr ♥ t ♣r♦s s♠t♦♥ ♦r tr ♦r♣♣ sss ♥ ♦r r♥t s♠♣ s③s (n =50, 100, 200, 400) t sts r ♥rt
st♠t♦♥ ♦♥t♦♥s ♦r t st t st ♠♦s ♦r♥ t♦ t ♥ t rtr r st♠t ♦r g = 1, . . . , 4 ♥ s r♥♦♠ ♥t③t♦♥s ♦ t ♦rt♠ r ♦♥
sts s♣②s t ♠♥ ♦ t st ♥♠r ♦ sss ♥ t str♥ ♥① ❬❪ ♦♠♣t t t st♠t ♣rtt♦♥ ♦ t st ♥♠r ♦sss ♦r ♦t ♥♦r♠t♦♥ rtr ♦r ♦ t rtr♦♥ s ttrs♥ t ♥rst♠ts t ♥♠r ♦ sss ♥ t s♠♣ s③ s s♠ ♥♥ sss ♦r♣ rtr♠♦r ts ♦♥r♥ t♦ t tr ♥♠r ♦ ssss str t♥ ♦r t rtr♦♥ ♥ ts ttr ♦rst♠ts t ♥♠r ♦sss ♥ t t st s r st ♥ ♥① rt t♦ t ♠♦ ♦♣t♠③ t rtr♦♥ s s♠ s♥ ts ♥① s q t♦ ③r♦ ♥
♠r ①♣r♠♥ts ♦♥ s♠t t sts
g = 1 ♦t tt ts ♥① ts s ♦r t rtr♦♥ ♥ ts ttr♦rst♠ts t ♥♠r ♦ sss s ♥ ♠ tt ♥ t rtr♦♥♦rst♠ts t ♥♠r ♦ sss t s♣ts tr ss ♥t♦ t♦ sss
r♣ 05% 10% 20%n
♥s ♦ t st ♥♠r ♦ sss ♥ ♣♥ ♥ st ♥♥s ♥ ♣r♥tss ♦♠♣t ♦r ♦t ♥♦r♠t♦♥ rtr
♦♥t♦♥ ♥♣♥♥ stt♦♥
t ♥rt♦♥ t r s♠♣ ♦r♥ t♦ t ♦♠♣♦♥♥ts ♠♦ rt t♦r rs r ♦♥t♦♥② ♥♣♥♥t t♦ t ♦♥t♥♦s ♦♥s ts♣r♠trs r
µk = (k − 2, k − 1), Σk =
[
1 k − 1.5k − 1.5 1
]
, δk1 = (1, 0, 0),
δk2 = (1, 0, 0), β02kj = (−1)k/2.
st♠t♦♥ ♦♥t♦♥s ♦r t st t st ♠♦s ♦r♥ t♦ t ♥ t rtr r st♠t ② ♠ ♦rt♠ r♥♦♠② ♥t③ t♠st g = 2
sts s♣②s t ♦♥t r t ♦st ♥tr♣ts r ♥♦t♥ ♥ t ♦♥t r t ♦♥t♦♥ ♣♥♥② rt♦♥s♣ t♥ t♦r r ♥ ♦♥t♥♦s ♦♥ s ③r♦
n
δ0kj = 1 δ(1,2)kj = 1 δ0kj = 1 δ
(1,2)kj = 1
♦♥t r t ♦st ♥tr♣ts r ♥♦t ♥ (δ0kj = 1) ♥ ♦♥t r t ♦♥t♦♥ ♣♥♥② rt♦♥s♣ t♥ t♦r r♥ ♦♥t♥♦s ♦♥ s st♠t (δ
(1,2)kj = 1) ♦r ♦t ♥♦r♠t♦♥ rtr
❲ ♥♦t tt t rtr♦♥ s ttr ♦r s♥ t rtr♦♥♦rst♠ts s♦♠ rt♦♥s♣s t♥ rs ♥ t ♦♥t♦♥ ss♠♣t♦♥s
♣tr ♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s
♠t♦♥s ② ♠ss♣ ♠♦s
t ♥rt♦♥ t sts ♦ s③ r s♠♣ ♦r♥ t♦ t ♦♦♥♦♠♣♦♥♥ts ♦rrt ss♥ ♠①tr ♠♦
πk = 0.5, µ1j = 0.5, µ2j = −µ1j, Σ1 = Σ2 =
1 ε 0.5 εε 1 ε 0.50.5 ε 1 εε 0.5 ε 1
,
r ε s ♥ st♠♥t ♣r♠tr ε = 0 ♠♥② rs r ♦♥t♦♥②♥♣♥♥t ♥ ε s t rs r ♦♥t♦♥② ♣♥♥t t♦ st rs r srt③ t♦ ♦t♥ t♦r t t ♦r s]−∞,−1] ]− 1, 0] ]0, 1] ♥ ]1,∞[
st♠t♦♥ ♦♥t♦♥s st ♦♠♣♦♥♥t ♠♦ ♦r♥ t♦ ♥♦r♠t♦♥ rtr♦♥ s st♠t ② t ♠ ♦rt♠ ♥t③ t♠s
sts s♣②s t ♦♥t ♦ t ♥♦♥ ♦♥ts ♥ t ♦strrss♦♥ ♦r t ♦rrt rs ♥ t ε♦rrt rs ♦r♥t♦ ♦t rtr ♦r r♥t s ♦ ε ❲♥ ε = 0 t ♦♥t ♦ t ♦st rrss♦♥ t♥ t ε♦rrt rs t♦ s t
rtr♦♥ ♦rst♠ts t ♦♥t♦♥ ♣♥♥s ♦r t tts ttr t♣♥♥s t♥ t ♦tr rs
d
♦rrt ε♦rrt ♦rrt ε♦rrt
♦♥t r t ♦♥t♦♥ ♣♥♥s r ♠♦ ② ♦t♥♦r♠t♦♥ rtr
s ♦ t s ♦ t rtr♦♥ s♦♥ r♥ t ♥♠r ①♣r♠♥ts ♦r s t♦ s t rtr♦♥ t♦ ♣r♦r♠ t ♠♦ st♦♥ ♥ ts ttr ♥ ♥t s♦♠ ♦♥t♦♥ ♣♥♥s t♥ ♠① rs
♥②ss ♦ t♦ r t sts
ss t str♥
t st sr♣t♦♥ ♥ rt ss t ❬t❪ sr ♣t♥ts ♣r rs ♦♥t♥♦s t t♦r ♥ ♦♥ ♣rt ttrt ♥ s ♥ t ❯ ♠♥ r♥♥ r♣♦st♦r② ♣rtttrt s ♥r② r ♥t♥ t ♣rs♥ ♦ rt ss ❲ ♥ ts
♥②ss ♦ t♦ r t sts
♥♦r♠t♦♥ r♥ ♦r str♥ rtr♠♦r t s① ♥s ♥ ♠ss♥s r ♦♠tt
♦♠♦♥♦s ♠♦s str♥ ♦t tt ♦t ♥s ♦ rs r♠♣♦rt♥t ♦r t str ♥②ss ♥ t ♥②ss ♣r♦r♠ ♦♥ t ♦♥t♥♦srs ② ♠①tr ♦ ss♥ strt♦♥s sts ♦r sss t ♥②ss♣r♦r♠ ♦♥ t t♦r ♦♥s ② t t♥t ss ♠♦ sts tr sssr s♣②s t st♠t ♣rtt♦♥ ② t ss♥ ♠①tr ♠♦ ♥ trst ♦♠♣♦♥♥t ♠♣ r ♥ ② t ♠t♥♦♠ ♠①tr ♠♦ ♥ trst ♦rrs♣♦♥♥ ♠♣ r s s tt sss ♦r♣ ♥ trst t♦r ♠♣s ♥ ♦ ♥♦t ♥ t♦r ♠♣ r t st♠t sssr s♣rt
−3 −2 −1 0 1 2 3
−3
−2
−1
01
23
4
First principal component analysis
Sec
ond
prin
cipa
l com
pone
nt a
naly
sis
Class 1 Class 2Class 3Class 4
−3 −2 −1 0 1 2 3
−3
−2
−1
01
23
4
First principal correspondence analysis
Sec
ond
prin
cipa
l cor
resp
onde
nce
anal
ysis
Class 1 Class 2Class 3
r Prtt♦♥s st♠t ② t ♦♠♦♥♦s ♠♦ ♣rtt♦♥ ♦ tss♥ ♠①tr ♠♦ r♥ ♥ t rst ♦♠♣♦♥♥t ♠♣ ♣rtt♦♥ ♦ tt♥t ss ♠♦ r♥ ♥ t rst ♦rrs♣♦♥♥ ♠♣
s s♦♥ ② t ♦♥s♦♥ ♠tr① ♣rs♥t ♥ t ♣rtt♦♥s ♦t♥ ② t ♠♦ ♦r ♦♠♦♥♦s rs r r② r♥t r♦♠ t ♣rtttrt ♥ t ♦ t st r♥ ♥① ♦♠♣t t♥ t ♣rtt♦♥ ♦ t ss♥ ♠①tr ♠♦ ♥ t ♣rt ttrt s q t♦ t s q t♦ ♥ t s ♦♠♣t t♥ t ♠t♥♦♠ ♠①tr ♠♦ ♥t ♣rt ttrt st g = 2 t rr♦r rts r ♦r t ♦♥t♥♦ss ♥ ♦r t t♦r s ♥② ♦t ♣rtt♦♥s ♦ t ♦♠♦♥♦s♠♦s r r♥t s♥ tr st r♥ ♥① s r q t♦
tr♦♥♦s ♠♦s str♥ ❲ ♣r♦r♠ t str ♥②ss ♦♥t ♦ t st ② s♥ t♦ ♠♦s t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♥t ♠①tr ♦ ss♥ ♥ ♦st strt♦♥s rsts s♣② ♥ ♠ tt t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ttr
♣tr ♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s
rt ♦♥t♥♦s rs t♦r rsss ss ss ss ss ss ss ss s♥ ♣rs♥
♦♥s♦♥ ts t♥ t ♣rt ttrt ♥ t t♦ ♣rtt♦♥sst♠t ② t ♦♠♦♥♦s ♠①tr ♠♦s
♣♣r♦s t t strt♦♥ ♥ ts ♠♦ sts t♦ sss s♥ ts rtr♦♥ s r t ♦♥ ♦♠♣♦♥♥t ♥ t tr ♦♠♣♦♥♥ts ♦t tt t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♦rst♠ts t ♥♠r ♦sss s♥ t rtr♦♥ sts tr sss t ♦ ② ♦♥sr♥ t ♦ t st t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s♦t♥s ♠♦r ♠♥♥ ♠♦ s♥ t s ss sss ♥ ss ♣r♠trs
♦♥ ♥♣ ♣r♦♣♦s ♠♦ rtr♦♥
♦♦♦ Pr♠trs
❱s ♦ t rtr♦♥ ♥ ♦ t ♦♦♦ ♥t♦♥ ♥ ♥♠r♦ ♣r♠trs ♦r ♦t ♦♠♣♦♥♥t ♠♦s ♥ ♦♠♣tt♦♥
st ♠♦ ♥tr♣rtt♦♥ ♠♦rt② ss 70% r♦♣s ♥s t♥t s♠st s ♦r t ♦♥t♥♦s rs ①♣t ♦r t r t ♦r♦r tr r♥s r s♠r t♥ t♠ ♦ t ♠♥♦rt② ss 30% t♦rrt♦♥ t♥ t ♦♥t♥♦s rs r str♦♥r s s♣② ♥ r ss ♦♥t♥♦s rs ♥♥ t t♦r ♦♥s ♥ ss r t♥♥ ss r r t♥ ♠♣ts t ♠♦st t♦r rs ♥ ss t s t r ♦♣s ♠♣t t ♠♦st t♦rr ♥ ss
r s♣②s t ♣rtt♦♥ ♦t♥ ② t ♦♠♣♦♥♥t ♠①tr ♠♦♦ ss♥ ♥ ♦st strt♦♥s ♥ t rst ♣♥ ♦ t P♠①t ❬❪❲ ♥ s tt t s♦♥ ①s s sr♠♥t t♥ ♦t st♠t sss
♦♠♣rs♦♥ t ♦tr ♣♣r♦s rr♦r rt ♦t♥ ② t ♠①tr♠♦ ♦ ss♥ ♥ ♦st strt♦♥s s 38% ♦r♥ t♦ ❬❪ t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♥ t♠① ♦t♥ ♥ rr♦r rt ♦ 23% ♥②t trt♦♥ rr str♥ ♠t♦s ♠ssst♦♥ rt r♥♥t♥ 22% ♥ 46% ♦t tt t ♠♥s ♣♣r♦ ❬❪ ♦t♥s ♥ rr♦rrt ♦ 15% t t st♦♥ ♦ t ♥♠r ♦ sss s ♠♦r t
♥②ss ♦ t♦ r t sts
age
trestbps
chol
thalach
oldpeach
sex
cp
fbs
restcy
exang
slope
ca
tha
ss
age
trestbps
chol
thalach
oldpeach
sex
cp
fbs
restcy
exang
slope
ca
tha
ss
r ♣♥♥s t♥ t ♦♥t♥♦s rs tr♥s ♦♥ t t♥ t t♦r rs r ♦♥ t rt ♣r ss ♥ ♥♦s ♣♥♥② s♦ ♥♦ ♥ ♦♥t ♥ t ♦st rrss♦♥ ♥ t ss
−2 −1 0 1 2
−1
01
2
First principal component analysis mixte
Sec
ond
prin
cipa
l com
pone
nt a
naly
sis
mix
te
Class 1 Class 2
r Prtt♦♥ r♥ ♥ t rst ♦♠♣♦♥♥t ♠♣ ♦ t P♠①t
♥♦♠ s♠s♣rs sst♦♥
t st sr♣t♦♥ ♥♦♠ s ♥r ♦ s♥ t st srs ♣t♥ts ❬❪ ② ♦r ♦♥t♥♦s rs ♦♠♣♥ t♠ ♥ ②s ♥②rs ②r ♦ ♦♣rt♦♥ ♥ t t♠♦r t♥ss ② t♦ ♥r② rs s① ♥♣rs♥s♥ ♦ r ♥ ② ♦♥ stts r tt ♦t♦♠③ r♦♠ ♠♥♦♠ ♦r ♥♦t
sr♠♥t ♥ ♥rt ♣♣r♦s ♦♠♣rs♦♥ ❲ strt ttt ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♥ ♦t♣r♦r♠ ss
♣tr ♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s
♠t♦s ♦ s♠s♣rs sst♦♥ s♣② ♥ ♦srt♦♥s r ♦ r♥♦♠② ♥ ♣rt ♦ t s ❲ ♦♠♣r t rr♦r rt ♦ t♥ ♣rtt♦♥ ♦t♥ ② ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥st♦ t rr♦r rt ♦t♥ t ♦st rrss♦♥ ♣rs♥ts t ♠♥♦ t ♠sss②♥ rt ♦♠♣t ♦♥ r♥♦♠② ♥ ♣rtt♦♥ ♦r r♥t♣r♥ts ♦ ♠ss♥ s
% ♠ss♥ s
♣r♦♣♦s ♠♦ ♦st
♥ ♦ t ♠sss②♥ rt ♦♠♣t ♦♥ t ♥s ♥ ♠ss♥ ♠♠rs♣
♦ r♥t ♣♣r♦s ♦r t♦ r♥t ♦ts ❲ r♠♥ tt t♦st rrss♦♥ ♠ s t♦ rt② ♠♦③ t ♦rr t♥ t sss ♥ts ♠t♦s s ♦♣ s♣② ♦r t s♠s♣rs sst♦♥ ♠ ♦ t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♠♦s s ♠♦r♠t♦s ♥ t ♠♦③s t ♦ strt♦♥ ♦ t t
♦♠♠♥ts ♣rs♥t rsts r s ①♣t ❲♥ ♠♦rt② ♦ t ♥s s t ♦st rrss♦♥ ♦t♥s ♦r ♠sss②♥ rt ♦r ♥ ♠♦rt② ♦ t ♥s s ♥ t ♠♥ ♥♦r♠t♦♥ s ♦♥t♥ ② ts ♥s s r♥ ts ①♣r♠♥t ♦sr tt t♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♦t♣r♦r♠s t ♦st ♠♦♦r ♥ rt ♦ ♠ss♥ s rtr♠♦r ts rt r♠t② r♦s ♦rt ♦st rrss♦♥ ♥ r② ♥s r ts st②s st♦r t ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♥ s 27.8% ♦t ♥s s r♦♠ ♠♥♦♠ t ♦st rrss♦♥ s ♦s t♦ t ♦rsrr♦r rt ♥ 95% ♦ t s r ♠ss♥
♥♥ sss ♥ str ♥②ss ❲ ♥♦ ♥tr♣rt ♦t sss ♦t♥♥ t ♥s r ♥ ♠♦rt② ss 60% r♦♣s ♥s♥ ♦♥ ♦♠♣♥② t♠ ♥ r♥t② trt ss s ♠♥② ♦♠♣♦s② ②♦♥ ♦♠♥ ♥ s♠ t♠♦r ♥ ts ss t r s t t♠♦r trr s t r rs ♥ t ♠♥♦rt② ss 40% ♥ t ♣t♥ts rt ♦♠♣♥② t♠ s s♦rtr ♦r ♦♠♣♥♠♥t st♦♣♣ ♥ r ttrt♠♥t s ♦ s ♣t♥ts r ♦r ♥r② t t♠♦r ♠♥② ♣rs♥t♦r t ♠♥ ♥ ♥rs♥ t r rs ss ♥tr♣rtt♦♥ s s ♦♥ t♠r♥ ♣r♠trs ♣rs♥t ♥ r
♦♥s♦♥ ♠tr① s♣② ♥ s♦s tt t st♠t ♣rtt♦♥s ♦s t♦ t sr stts ♠♦rt② ss ♥♦s s♠ rs ♦ t r♦♠♥♦♠ ts rs s r ♥ t ♠♥♦rt② ss
♦♥s♦♥
−2000 0 2000 4000 60000.00
000
0.00
020
density of time
C 1 C 2
0 50 100
0.00
00.
010
0.02
0
density of age
C 1 C 2
1960 1965 1970 1975 1980
0.00
0.05
0.10
0.15
density of year
C 1 C 2
−10 −5 0 5 10 15 20
0.00
0.10
0.20
density of thickness
C 1 C 2
female male
0.0
0.4
0.8
barplot of sex
C 1 C 2 C 1 C 2absence presence
0.0
0.4
0.8
barplot of ulcer
C 1 C 2 C 1 C 2
r P♦tt♥ ♦ t ♠r♥ ♣r♠trs ♦ t ♦♠♣♦♥♥t ♠①tr ♠♦♦ ss♥ ♥ ♦st strt♦♥s st♠t ♦♥ t ♥♦♠ t sts
♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s♠♦rt② ss ♠♥♦rt② ss
♥♦t ♦r♠ ♠♥♦♠ ♦r♠ ♠♥♦♠
♦♥s♦♥ ♠tr① t♥ t ♣rtt♦♥ st♠t ② t ♠①tr ♠♦♦ ss♥ ♥ ♦st strt♦♥s ♥ t sr stts
♦♥s♦♥
♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s s ♥ ♥t ♣♣r♦t♦ str t sts t ♦♥t♥♦s ♥ t♦r rs ♦ t s ♦♦ ♥r t♦ t ♥♦♥ ♠♦s ♥ ②s ♥ ♠①tr ♦ ♦t♦♥ ♠♦s ♥ts rs ts rst ♥t s t♦ t ♥t♦ ♦♥t t ♥trss ♣♥♥st♥ t rs s t ♣r♦♣♦s ♠♦ ♦s t ss ♥♦ ②t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥ ts s♦♥ ♥t s t♦ ♣ ssstrt♦♥s ♦r t ♦♥♠♥s♦♥ ♠r♥s ♦ ♦♠♣♦♥♥t ♥ t ♣rtt♦♥r s② s♠♠r③s ss ② t ♣r♠trs ♦ t ss strt♦♥s♥ ② t ♦st ♥t♦♥s
♣rs♠♦♥♦s rs♦♥s ♦ ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♦ t♦ ♠♦ t ♠♥ ♦♥t♦♥ ♣♥♥s t♥ ♠① rss t ss ♥tr♣rtt♦♥ s sr ♠♦ st♦♥ ♥ t ♣r♠tr
♣tr ♦s str♥ ♦ ss♥ ♥ ♦st strt♦♥s
st♠t♦♥ r s♠t♥♦s② ♣r♦r♠ ♠ ♦rt♠ ♠①♠③♥ ♥ ♥♦r♠t♦♥ rtr♦♥ ♦r♥ t♦ ♦r ①♣r♠♥ts ♦r s t♦ s t
rtr♦♥ s t ♥♦r♠t♦♥ rtr♦♥ ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s ♥ s ♥ s♠
s♣rs sst♦♥ ♦r♥ t♦ ♦r ♣♣t♦♥ t ♥ ♦♦ ♥r t♦t ss sr♠♥t♦♥ ♣♣r♦s s♣② ♥ ♥s r
♣tr
♦s str♥ ♦ ss♥
♦♣s ♦r ♠① t
♠①tr ♠♦ ♦ ss♥ ♦♣s s ♣rs♥t t♦str ♠① t r ♥② ♥s ♦ rs r♦ t② ♠t ♠t strt♦♥ ♥t♦♥ s ♣♣r♦ ♦s t♦ strt♦rr② ♥s♠♣ ♠trt ♥trss ♣♥♥② ♠♦s ♣rsr♥ ♥② ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t ♦ ♥trst ♦r t sttst♥ ②♣②♥ ts ♦r t ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t r ss ♣r♠tr ♦♥s ♥ ♦rr t♦ ttt ♠♦ ♥tr♣rtt♦♥ ♥ t♦♥ t ♥trss ♣♥♥s r t♥ ♥t♦ ♦♥t ② t ss♥ ♦♣s ♣r♦ ♦♥ ♦rrt♦♥ ♦♥t ♥ ♦♦♣r♦♣rts ♣r ♦♣ ♦ rs ♥ ♣r sss ♠♦ ♥r③s r♥t ①st♥ ♠♦s ♥♦r ♦♠♦♥♦s ♥ ♠① rs ♥r♥ s♣r♦r♠ tr♦♣♦st♥s s♠♣r ♥ ②s♥ r♠♦r ♠r ①♣r♠♥ts strtt ♠♦ ①t② ♥ ts r♥
s ❩♦rq♥ t ♠ts ♥ ♦♣ s♦
t q t rss♠st♦s s r②♦♥s sr ♥ s ♣♦♥t
♣♦♥tà ♣r♥ ♥tôt P♦rq♦
Pr q ♦r s♦♥ sst ♣s é♣r♣é
sst rss♠ésr ♥ s ♣♦♥t
♠ê♠ s♣rt ♦♠♠♥ t s ♠rs
♥ ♦♥♥tr♥t s♦♥ s♣rtsr ♥ s t ♠ê♠ ♦s
í♦s ③♥t③ás①s ❩♦r
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
♥tr♦t♦♥
♠ ♦ ts ♣tr s t♦ ♣rs♥t ♠♦s str♥ ♦r ♠① t♦ ♥② ♥s ♦ rs ♠tt♥ ♠t strt♦♥ ♥t♦♥ s ♠♦s ♦ ♦t t♦ ♣rsr ss strt♦♥s ♦r ts ♦♥♠♥s♦♥♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t ♥ t♦ ♣rs♠♦♥♦s② ♥ ♠♥♥②♠♦③ t ♥trss ♣♥♥s
s ♦t ♥ ♥tr② ② t s ♦ ♦♣s ❬♦ ❪ ♥ ♦♣s ♠trt ♠♦ ② stt♥ ♦♥ t ♦♥ ♥ t♦♥♠♥s♦♥ ♠r♥s ♥ ♦♥ t ♦tr ♥ t ♣♥♥② ♠♦ t♥rs ♦r ♣rs② t t strt♦♥ s ♣♣r♦ ② ♣r♠tr♠①tr ♠♦ ♦ ss♥ ♦♣s ♦s t ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t r ss ♥ ♦s t ss♥ ♦♣s ❬♦ ❲❪ ♠♦③ t♥trss ♣♥♥s ♦t tt ❬ ❪ r② s ♦♥ ss♥♦♣ t♦ ♥ strt♦♥ ♦ ♠① rs ♣r♦♣♦s ♠♦ s s♦ ♥r③t♦♥ ♦ ts ♣♣r♦ t♦ t ♥t ♠①tr ♠♦ r♠♦r
♥ ♠①tr ♠♦ s ♠♥♥ s♥ t ♣r♠ts tr s♠ ♦s r♥② ♥tr♣rtt♦♥ t ♣r♦♣♦rt♦♥s ♥t t ss ts t ♦♥♠♥s♦♥ ♠r♥ ♣r♠trs ♦ ♦♠♣♦♥♥ts r♦② sr t sss t ♦rrt♦♥ ♠trs r♥ ts sr♣t♦♥ ♥② ② s♥ t ♦♥t♥♦s t♥t strtr ♦ t ss♥ ♦♣s Pt②♣ s③t♦♥ ♣r ss♦s t♦ s♠♠r③ t ♠♥ ♥trss ♣♥♥s ♥ ♣r♦s sttr♣♦t ♦t ♥s ♦r♥ t♦ t ss ♣r♠trs
♦t tt ♦s♠s ♥ rs ❬❪ r♥t② s♠tt ♥ rt ♣r♦♣♦ss t♦ s ♠①tr ♦ ♦♣s t♦ ♣r♦r♠ str ♥②ss t♦rsst② r♥t ♦♣s ♠♦♥ t♠ t ss♥ ♦♣s r ♦♥sr r♠♦ s ♦s t♦ t ♣♣r♦ ♦♣ ♥ ts ♣tr ♦r t♦ ♠♣♦rt♥tr♥s t♦ ♠♥t♦♥ rst② ♣r♦♣♦s ②s♥ ♥r♥ tt♦rs ♣r♦♣♦s ♥ ♣♣r♦ ② ♠①♠♠ ♦♦ ♥r ♦♥str♥ts ♦♥②s♦♠ s③t♦♥s t♦♦s r ♣rs♥t r
trtr ♦ ts ♣tr s ♣♣r s ♦r♥③ s ♦♦s t♦♥ ♣rs♥ts t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♥tr♦ t♦ str ts ♥s tt ①st♥ ♠♦s ♥ ts ♦♥trt♦♥ t♦ t s③t♦♥ ♦ ♠① rs t♦♥ s ♦t t♦ t ♣r♠tr st♠t♦♥ ♥ ②s♥ r♠♦r s♥ t♠①♠♠ ♦♦ st♠t s ♥tt♥ ❬P❪ t♦♥ strts t♦r ♦ t ♦rt♠ ♣r♦r♠♥ t ♥r♥ ♥ s♦ t ♠♦ r♦st♥ss ♦♥♥♠r ①♣r♠♥ts t♦♥ ♣rs♥ts tr ♣♣t♦♥s ♦ t ♥ ♠①tr♠♦ ② str♥ tr r t sts t♦♥ ♦♥s ts ♦r tsrsts r ♣rt ♦ t rt ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠①t ❬❱❪
①tr ♠♦ ♦ ss♥ ♦♣s
①tr ♠♦ ♦ ss♥ ♦♣s
♥t ♠①tr ♠♦
t
t♦r ♦ e ♠① rs s ♥♦t ② xi = (x1i , . . . , xei ) ∈ R
c × X te = c+ d ts rst c ♠♥ts r t st ♦ t ♦♥t♥♦s rs ♥ ♦♥ ts♣ R
c ♥ rtr ♥♦t ② x
i ts st d ♠♥ts r t st ♦ t srtrs ♥tr ♦r♥ ♦r ♥r② ♥ ♦♥ t s♣ X ♥ rtr ♥♦t ②x
i ♦t tt xji s ♥ ♦r♥ r t mj ♠♦ts t♥ t ss ♥♠r
♦♥ 1, . . . ,mj
♦tt♦♥ ❲ r♠♥ tt s t ♥r ♥♦tt♦♥ P (.; .) ♦r t ♠tstrt♦♥ ♥t♦♥s ♥ p(.; .) ♦r t ♣r♦t② strt♦♥ ♥t♦♥ ♣
Pr♦t② strt♦♥ ♥t♦♥
♥t♦♥ ♥t ♠①tr ♠♦ ♦ ♣r♠tr strt♦♥s t xi rs♣♣♦s t♦ r♥ ② t ♠①tr ♠♦ ♦ g ♣r♠tr strt♦♥s ♦s t♣ s rtt♥ s ♦♦s
p(xi;θ) =
g∑
k=1
πkp(xi;αk),
r θ = (π,α) ♥♦ts t ♦ ♣r♠trs t♦r π = (π1, . . . , πg)r♦♣s t ♣r♦♣♦rt♦♥s ♦ ss k ♥♦t ② πk ♥ rs♣t♥ t ♦♦♥♦♥str♥ts 0 < πk ≤ 1 ♥
∑gk=1 πk = 1 t t♦r α = (α1, . . . ,αg) r♦♣s
t ♣r♠trs ♦ ss k ♥♦t ② αk
Pr♦♣rt② t♥t r ♥t ♠①tr ♠♦ ♥ ①♣rss ② s♥t t♥t r zi s t♦r r ♥ts t ss ♠♠rs♣② s♥ ♦♠♣t s♥t ♦♥ ♥ ♦♦s t ♠t♥♦♠ strt♦♥Mg(π1, . . . , πg) s ♥ ♥tr♣rt s t ♠r♥ strt♦♥ ♦ xis ♦♥ t strt♦♥ ♦ t ♦♣ (xi, zi)
ss♥ ♦♣ ♦r ♠① t
♦♠♣♦♥♥t strt♦♥s ♦♦♥ ss♥ ♦♣s
♦♣s ♦ t♦ ♠trt ♠♦ ② stt♥ ♦♥ t ♦♥ ♥ t♦♥♠♥s♦♥ ♠r♥s ♥ ♦♥ t ♦tr ♥ t ♣♥♥② ♠♦ t♥rs ❲ ♥♦ ♣rs♥t t ♠r♥ strt♦♥ ♦ t ♦♠♣♦♥♥ts t♥ ♦s♦♥ t ss♥ ♦♣ s ♦ ♥trst ♦r s s♥ t ♣r♦s ♦♥ ♦rrt♦♥♦♥t ♣r ♦♣ ♦ rs ♥ s♥ t ♦s ♥ s② ♣r♠tr st♠t♦♥
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
♥♠♥s♦♥ ♠r♥s ♦ t ♦♠♣♦♥♥ts
♦r ♦♠♣♦♥♥t ss♠ tt t ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t♦♥s t♦ t ①♣♦♥♥t ♠② ♥ ♦rr t♦ ♣r♦ ♠♥♥ sss
♥t♦♥ ♥♠♥s♦♥ ♠r♥s ♦ t ♦♠♣♦♥♥ts ♠r♥ strt♦♥ ♦ t r xji ♦r ♦♠♣♦♥♥t k ♦♥s t♦ t ①♣♦♥♥t ♠② ♥s p(xji ;βkj) ♦r ♣ ♥ P (x
ji ;βkj) s ♦r ♣rs②
xji s ♦♥t♥♦s ts ♠r♥ ♦ ♦♠♣♦♥♥t k ♦♦s ss♥ strt♦♥t ♠♥ µkj ♥ r♥ σ2
kj xji |zik = 1 ∼ N1(µkj, σ2kj) ♥ βkj =
(µkj, σ2kj) ∈ R× R
+∗ xji s ♥tr ts ♠r♥ ♦ ♦♠♣♦♥♥t k ♦♦s P♦ss♦♥ strt♦♥
xji |zik = 1 ∼ P(βkj) ♥ βkj ∈ R+∗
xji s ♦r♥ ts ♠r♥ ♦ ♦♠♣♦♥♥t k ♦♦s ♠t♥♦♠ strt♦♥ xji |zik = 1 ∼ Mmj
(βkj) βkj ♥ ♥ ♦♥ t s♠♣① ♦ s③ mj
♣♥♥② ♠♦ ♦ t ♦♠♣♦♥♥ts
♠①tr ♠♦ ♦ ss♥ ♦♣s ss♠s tt ♦♠♣♦♥♥t k ♦♦s ss♥ ♦♣ ♦s t ♦rrt♦♥ ♠tr① ♦ s③ e× e s ♥♦t ② Γk ❲♥♦t Φe(.;Γk) t ♦ t ert ♥tr ss♥ strt♦♥ t ♦rrt♦♥♠tr① Γk ♥ Φ−1
1 (.) t ♥rs ♠t strt♦♥ ♥t♦♥ ♦N1(0, 1) s ♦t♥ t ♦♦♥ ♥t♦♥ ♦ t ♦♠♣♦♥♥t
♥t♦♥ ♠t strt♦♥ ♥t♦♥ ♦ t ♦♠♣♦♥♥ts ♦r t ♠①tr ♠♦ ♦ ss♥ ♦♣s t ♦ ♦♠♣♦♥♥t k s rtt♥ s
P (xi;αk) = Φe(Φ−11 (u1k), . . . ,Φ
−11 (uek);0,Γk),
r ujk = P (xji ;βkj) ♥ r αk = (βk,Γk) ♥♦ts t ♦ ♣r♠trs ♦♦♠♣♦♥♥t k t βk = (βk1, . . . ,βke)
Pr♦♣rt② t♥r③ ♦♥t ♦ ♦rrt♦♥ ♣r ss ss♥♦♣ ♣r♦s ♦♥t ♦ ♦rrt♦♥ ♣r ♦♣ ♦ rs s ♦♦♣r♦♣rts ♥ ♥ ♦t rs r ♦♥t♥♦s t s q t♦ t ♣♣r♦♥ ♦ t ♦♥ts ♦ ♦rrt♦♥ ♦t♥ ② t ♠♦♥♦t♦♥ tr♥s♦r♠t♦♥s♦ t rs ❬❲❪ rtr♠♦r ♥ ♦t rs r srt t s qt♦ t ♣♦②♦r ♦♥t ♦ ♦rrt♦♥ ❬s❪
Pr♦♣rt② ♦♥ t♥t r ♠①tr ♠♦ ♦ ss♥ ♦♣s♥♦s s♦♥ t♥t r t♦ t ss ♠♠rs♣ ♦♥ssts♥ ♥ ert ♦♥t♥♦s r ♥♦t ② yi = (y1i , . . . , y
ei ) ∈ R
e ♦♥t♦♥② ♦♥ t ss ♠♠rs♣ ts r ♦♦s ♥ ert ♥tr ss♥strt♦♥ ♥ yi|zik = 1 ∼ Ne(0,Γk) ♥
xji = P−1(Φ1(yj);βkj), ∀j = 1, . . . , e,
t♥ ♦♠♣♦♥♥t k s ss♥ ♦♣ ♦s t s P (xi;αk)
①tr ♠♦ ♦ ss♥ ♦♣s
①tr ♠♦ ♦ ss♥ ♦♣s ♦r ♠① t
❲ ♥tr♦ t ♥t♦♥ Ψ(x
i ;αk) =(xji−µkj
σkj; j = 1, . . . , c
)
♥ t s♣ ♦
t ♥t♥ts ♦ x
i ♦r ss k s ♥♦t Sk(x
i ) = Sc+1k (xc+1
i )× . . .× Sek(xei ) ♥tr Sjk(xji ) =]b⊖k (x
ji ), b
⊕k (x
ji )] s ♥ ♦r j = c + 1, . . . , e ♥ ts ♦♥s r
b⊖k (xji ) = Φ−1
1 (P (xji − 1;βkj)) ♥ b⊕k (x
ji ) = Φ−1
1 (P (xji ;βkj)) ❲ ♥♦ ♥ t ♣♦ t ♦♠♣♦♥♥ts ♦r♥ t♦ s ♣r♦♣♦s ♥ ❬❪
♥t♦♥ ①tr ♠♦ ♦ ss♥ ♦♣s t xi ♦♦s ♠①tr♠♦ ♦ ss♥ ♦♣s ts ♣ s t ♥t ♠①tr ♠♦ ♥ ♥ ♦s t ♣ ♦ ♦♠♣♦♥♥t k s rtt♥ s
p(xi;αk) = p(x
i ;αk)p(x
i |x
i ;αk)
=φc(Ψ(x
i ;αk);0,Γk)∏c
j=1 σkj
∫
Sk(x
i )
φd(u;µ
k ,Σ
k)du,
r Γk =
[
Γk Γk
Γk Γk
]
s ♦♠♣♦s ♥t♦ s♠trs ♦r ♥st♥ Γk s t
s♠tr① ♦ Γk ♦♠♣♦s ② t r♦s ♥ t ♦♠♥s rt t♦ t ♦sr♦♥t♥♦s rs ♦r♦r µ
k s t ♦♥t♦♥ ♠♥ ♦ yi ♥ ② µ
k =ΓkΓ
−1kΨ(x
i ;αk) ♥ Σ
k s ts ♦♥t♦♥ ♦r♥ ♠tr① ♥ ② Σ
k =Γk − ΓkΓ
−1kΓk
Pr♦♣rt② ♥rt ♠♦ ♠①tr ♠♦ ♦ ss♥ ♦♣s ♥♦st ♥rt ♠♦ s♣t ♥t♦ t ♦♦♥ tr st♣s
ss ♠♠rs♣ s♠♣♥ zi ∼ Mg(π1, . . . , πg) ss♥ ♦♣ s♠♣♥ yi|zik = 1 ∼ Ne(0,Γk) sr t tr♠♥st ♦♠♣tt♦♥ xi s ♦t♥ r♦♠
♠rs
♦♠♦sst ♠♦s ❲♥ t s♠♣ s③ s s♠ t tr ♦ t♥t s ♥ t r♥ ♦ t st♠t ♠② ttr s♦♠ ♦♥str♥ts♦♥ t ♣r♠tr s♣ r s ♣r♦♣♦s ♣rs♠♦♥♦s rs♦♥♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s ② ss♠♥ t qt② t♥t ♦rrt♦♥ ♠trs s♦
Γ1 = . . . = Γg.
♦t tt ts ♠♦ s ♥♠ ♦♠♦sst s♥ t ♦r♥ ♠trs♦ t t♥t ss♥ rs r q t♥ sss
♠r ♦ ♣r♠trs tr♦sst rs♣t② ♦♠♦sst♠①tr ♠♦ ♦ ss♥ ♦♣s ♥s ν rs♣t② ν♦ ♣r♠trsr
ν = (g−1)+g
(
e(e− 1)
2+
d∑
j=1
νj
)
♥ ν♦ = (g−1)+e(e− 1)
2+g
d∑
j=1
νj,
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
r νj ♥♦ts t ♥♠r ♦ ♣r♠trs ♦ t ♠r♥ strt♦♥ ♦r j ♦r ♦♥ ♦♠♣♦♥♥t ♦r ♣rs② t t s♣ ♠r♥ strt♦♥ ♦ t ♦♠♣♦♥♥ts νj s q t♦
νj =
2 xj s ♥♠r1 xj s srt
mj − 1 xj s ♦r♥
♦ ♥tt② ♠①tr ♠♦ ♦ ss♥ ♦♣s s ♥t♥ t s♥s ♥ ♥ ❬ ❨❪ t st ♦♥ r s ♦♥t♥♦s♦r ♥tr ♣r♦♦ s ♥ ♥ ♣♣♥①
tr♥ts ♦ t ♠①tr ♠♦
t ♠♦s
♠①tr ♠♦ ♦ ss♥ ♦♣s ♦s t♦ ♥r③ ♠♥② ss♠♦s str♥s ♠♦♥ t♠ ♦♥ ♥ t t ♦♦♥ ♦r
♦s② t ♦rrt♦♥ ♠trs r ♦♥ Γk = I ∀k =1, . . . , g t♥ t ♠①tr ♠♦ ♦ ss♥ ♦♣s s q♥t t♦ t♦♥t♦♥ ♥♣♥♥ ♠①tr ♠♦
t rs r ♦♥t♥♦s c = e ♥ d = 0 t♥ t ♠①tr♠♦ ♦ ss♥ ♦♣s ♦♠s ♠trt ss♥ ♠①tr ♠♦t♦t ♦♥str♥t t♥ t ♣r♠trs ❬❪
♠①tr ♠♦ ♦ ss♥ ♦♣s s ♥ t♦ t ♥♥ ss♥♠①tr ♠♦ ♦r ♥st♥ t s q♥t ♥ t r ♦r♥ t♦ t♠①tr ♠♦ ♦ ❬♦❪ ♥ s s ts ♠♦ s st ② s♦♥ ♦♠♦ts
❲♥ t rs r ♦t ♦♥t♥♦s ♥ ♦r♥ t ♠①tr ♠♦ ♦ss♥ ♦♣s s ♥ ♣r♠tr③t♦♥ ♦ t ♠①tr ♠♦ ♣r♦♣♦s② rtt ❬❪ s t♦♥ ♦r rtt st♠ts rt② ts♣ Sk(x
i ) ♦♥t♥♥ t ♥t♥ts ♦ x
i ♥ ♥♦t t ♠r♥ ♣r♠trss t ♠①♠♠ ♦♦ ♥r♥ s s♦ ♣r♦r♠ s♠♣① ♦rt♠ r♠t② ♠t♥ t ♥♠r ♦ ♦r♥ rs ♦t tt ♦r♣♣r♦ ♦r t ♥r♥ ♦s ts r s ts ♥ t♦♥
t s③t♦♥ ♣r ss ②♣r♦t ♦ ss♥ ♦♣s
❲ ♥ s t ♠♦ ♣r♠trs t♦ ♦t♥ s③t♦♥ ♦ t ♥s ♣rss ♥ t♦ r♥ ♦t t ♠♥ ♥trss ♣♥♥s s ♦r ss k rst②♦♠♣t t ♦♦r♥ts q t♦ E[yi|xi, zik = 1;αk] ♥ s♦♥② ♣r♦t t♠♦♥ t ♣r♥♣ ♦♠♣♦♥♥t ♥②ss s♣ ♦ t ss♥ ♦♣ ♦ ♦♠♣♦♥♥t k♦t♥ ② t s♣tr ♦♠♣♦st♦♥ ♦ Γk
♥s r♥ ② t ♦♠♣♦♥♥t k ♦♦ ♥tr ss♥ strt♦♥♥ t t♦r ♠♣ s♦ t② r ♦s t♦ t ♦r♥ ♦s r♥ ② ♥♦tr♦♠♣♦♥♥t ♥ ①♣tt♦♥ r♥t r♦♠ ③r♦ s♦ t② r rtr t♦ t ♦r♥♥② t ♦rrt♦♥ r s♠♠r③s t ♥trss ♦rrt♦♥s ♦♦♥①♠♣ strts ts ♣♥♦♠♥♦♥
①tr ♠♦ ♦ ss♥ ♦♣s
①♠♣ ①tr ♠♦ ♦ ss♥ ♦♣s ♥ s③t♦♥ ♣r ss tt ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦♠♣♦s t tr rs♦♥ ♦♥t♥♦s ♦♥ ♥tr ♥ ♦♥ ♥r② ♥ ts ♦rr t
π = (0.5, 0.5), β11 = (−2, 1), β12 = 5, β13 = (0.5, 0.5),β21 = (2, 1), β22 = 15,
β23 = (0.5, 0.5), Γ1 =
1 −0.4 0.4−0.4 1 0.40.4 0.4 1
♥ Γ2 =
1 0.8 0.10.8 1 0.10.1 0.1 1
.
−4 −2 0 2 4
510
1520
25
x1
x2
−6 −4 −2 0 2
−2
−1
01
23
first principal component axis
seco
nd p
rinci
pal c
ompo
nent
axi
s
−1.0 −0.5 0.0 0.5 1.0
−1.
0−
0.5
0.0
0.5
1.0
inertia: 60.8 %
iner
tia: 3
2.5
%
continuous
integer
binary
r ①♠♣ ♦ s③t♦♥ sttr♣♦t ♦ t ♥s sr② tr rs ♦♥ ♦♥t♥♦s sss ♦♥ ♥tr ♦r♥t ♥ ♦♥ ♥r②s②♠♦ ♥s sttr♣♦t ♥ t rst ♦♠♣♦♥♥t ♠♣ ♦ ss rs r♣rs♥tt♦♥ ♥ t rst ♦♠♣♦♥♥t ♠♣ ♦ ss ♦♦r ♥tst ss ♠♠rs♣s
s③t♦♥ ♦ ss s ♣rs♥t ♥ r ♦♥r♥♥ t ♥st sttr♣♦t s♦s ♥tr ss t r ♦♥ ♥ s♦♥ ss t ♦♥ ♦t ♦♥ t t s ♦♥r♥♥ t rs t r♣rs♥tt♦♥ ♣♦♥ts ♦t② str♦♥ ♥trss ♦rrt♦♥ t♥ t ♦♥t♥♦s ♥ t ♥tr rs
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
②s♥ ♥r♥ tr♦♣♦st♥s
s♠♣r
♠ ❲ ♦sr t s♠♣ ① = (x1, . . . ,xn) ♦♠♣♦s t n ♥♣♥♥t ♥s xi ∈ R
c×X ss♠ t♦ r♥ ② ♠①tr ♠♦ ♦ ss♥ ♦♣s ♠ s t♦ ♥r t ♣r♠trs ♦r♥ t♦ t t
rq♥tst ♦♥t①t ♥r♥ ② ♠①♠♠ ♦♦ s t ♣r♦♠♦r t ♣r♠tr ♦♣s ♥ t ♠r♥ ♣r♠trs r ♥♥♦♥ ♦ ts ♦t♥ r♣ ② t ♥r♥ ♥t♦♥ ♦r r♥s ♠t♦ ♣r♦r♠♥ t♥r♥ ♥ t♦ st♣s s ♣tr ♦ ❬♦❪ rst st♣ st♠ts t♠r♥ ♣r♠trs ② ♠①♠③♥ ♥rt ♦♦ t s♦♥ st♣st♠ts t ♦rrt♦♥ ♣r♠trs ② ♠①♠③♥ t ♦♦ ♦♥t♦♥② ♦♥t ♠r♥ ♣r♠trs s ♣♣r♦ s s ♥ ❬❪ ♦r t ♠①♠♠♦♦ st♠t ♥ ss♥t② ♦t♥ ♥ t rs r ♦♥t♥♦s② s♥ t ①♣♦♥t ♦rt♠ ♣r♦♣♦s ② ❬❪ ♥ ts ♣♣r♦ ♥♥♦t ①t♥ t♦ t ♠① t stt♥ s ♥ ♠ ♦rt♠ ♥ ♥♦t ♠♣♠♥t t♦ ♦t♥ t ♠①♠♠ ♦♦ st♠ts ♦ ♠①tr ♠♦ ♦ss♥ ♦♣s ♥ t ♠① t s rtr♠♦r ♥ t ♠ st♣ ♦ ①♣t t st♣ ♦ t♦♦ ♠ t♠ ♦♥s♠♥ t srt rsr ♥♠r♦s s ♦ t ♦♠♣tt♦♥ ♦ t ♥tr ♦ ♠♥s♦♥ d ♥ ♥
②s♥ ♦♥t①t ♥ ♦rr t♦ ♦ ♦t ♣r♦s ♣r♦♠s ♣rr t♦ ♦r♥ ②s♥ r♠♦r ❲ rst② ♥ t ♣r♦r strt♦♥s ♥ s♦♥②♣rs♥t t s s♠♣r ♣r♦r♠♥ t ♥r♥
①♠♠ ♣♦str♦r st♠t
Pr♦r strt♦♥s
♥♣♥♥ ss♠♣t♦♥ ss ss♠♣t♦♥ s t♦ s♣♣♦s t ♥♣♥♥ t♥ t ♣r♦r strt♦♥s ts
p(θ) = p(π)
g∏
k=1
(
p(Γk)d∏
j=1
p(βkj)
)
.
Pr♦♣♦rt♦♥s ss ♦♥t ♣r♦r strt♦♥ ♦ t ♣r♦♣♦rt♦♥ t♦rs t r②s ♥♦♥ ♥♦r♠t ♦♥ s rt strt♦♥ ♦s t ♣r♠trs r q t♦
π ∼ Dg
(
1
2, . . . ,
1
2
)
.
r♥ ♣r♠trs ♣r♦r strt♦♥ ♦ t ♠r♥ ♣r♠trs r tss ♦♥t ♦♥s ♦r ♣rs②
②s♥ ♥r♥ tr♦♣♦st♥s s♠♣r
xj s ♦♥t♥♦s t♥ βkj ♥♦ts t ♣r♠trs ♦ ♥rt ss♥strt♦♥ s♦ p(βkj) = p(µkj|σ2
kj)p(σ2kj) t
σ2kj ∼ G−1(c0, C0) ♥ µkj|σ2
kj ∼ N1(b0, σ2kj/N0),
r G−1(., .) ♥♦ts t ♥rs ♠♠ strt♦♥ ❲t ♥ ♠♣r②s♥ ♣♣r♦ t ②♣r♣r♠trs (c0, C0, b0, N0) r ① s ♣r♦♣♦s ② ❬❪ s♦ c0 = 1.28 C0 = 0.36❱r(①j) b0 = 1
n
∑ni=1 x
ji ♥
N0 =2.6
r♠① ①j−r♠♥ ①j xj s ♥tr βkj ♥♦ts t ♣r♠tr ♦ P♦ss♦♥ strt♦♥ ♥
βkj ∼ G(a0, A0).
♦r♥ t♦ ❬❪ t s ♦ ②♣r♣r♠trs a0 ♥ A0 r ♠♣r②① t♦ a0 = 1 ♥ A0 = a0n/
∑ni=1 x
ji
xj s ♦r♥ βkj ♥♦ts t ♣r♠tr ♦ ♠t♥♦♠ strt♦♥ ♥ts r②s ♥♦♥ ♥♦r♠t ♦♥t ♣r♦r ♥♦s tt
βkj ∼ Dmj
(
1
2, . . . ,
1
2
)
.
♦rrt♦♥ ♠trs ♦♥t ♣r♦r ♦ ♦r♥ ♠tr① s t ♥rs❲srt strt♦♥ ♥♦t ② W−1(., .) ♦ t s ♥tr t♦ ♥ t ♣r♦r ♦t ♦rrt♦♥ ♠tr① Γk r♦♠ t ♣r♦r ♦ t ♦rrt♦♥ ♠tr① Λk s♥ Γk|Λk str♠♥st ❬♦❪ ♦
Λk ∼ W−1(s0, S0) ♥ ∀1 ≤ h, ℓ ≤ e, Γk[h, ℓ] =Λk[h, ℓ]
√
Λk[h, h]Λk[ℓ, ℓ],
r (s0, S0) r t♦ ②♣r♣r♠trs ♦r t ss ♣♣r♦ ♦♥sst♥♥ tt♥ t ②♣r♣r♠trs tr♦ ♥ ♠♣r ②s♥ ♣♣r♦ s ♥♦t ♣♦ss s♥ yi s ♥♦t ♦sr ❲ ts ♣t s0 = e + 1 ♥ S0 q t♦ t ♥tt②♠tr① s♥ ♥ ts s t ♠r♥ strt♦♥ ♦ ♦rrt♦♥ ♦♥t s♥♦r♠ ♦♥ ]− 1, 1[ ❬❪
P♦str♦r strt♦♥
②s♥ ♥r♥ s ♣r♦r♠ ② s♠♣♥ sq♥ ♦ ♣r♠trs r♦♠tr ♣♦str♦r strt♦♥ ♥ ♣rt s s s♠♣r s t ♠♦st♣♦♣r ♣♣r♦ t♦ ♣r♦r♠ ②s♥ ♥r♥ ♦ ♠①tr ♠♦ s♥ t ss tt♥t strtr ♦ t t ♥ t tr♥t② s♠♣s t ss ♠♠rs♣s♦♥t♦♥② ♦♥ t ♣r♠trs ♥ ♦♥ t t ♥ t ♣r♠trs ♦♥t♦♥② ♦♥ t ss ♠♠rs♣s ♥ ♦♥ t t ♥ ts stt♦♥r② strt♦♥s p(θ, ③|①) t sq♥ ♦ t ♥rt ♣r♠trs s r♥ ② t ♠r♥♣♦str♦r strt♦♥ p(θ|①) s ♦rt♠ rs ♦♥ t♦ ♥str♠♥t rst ss ♠♠rs♣ ♦ t ♥s ♦ ① ♥♦t ② ③ = (z1, . . . , zn) ♥ tss♥ t♦r ♦ t ♥s ♥♦t ② ② = (y1, . . . ,yn)
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
s s♠♣r
trt♥ r♦♠ ♥ ♥t θ[0] ts trt♦♥ [r] s rtt♥ s
③[r],②[r−1/2] ∼ ③,②|①,θ[r−1]
β[r]kj ,y
j[r][rk] ∼ βkj,y
j[rk]|①,y
[r][rk], ③
[r],β[r]k ,Γ
[r−1]k
π[r] ∼ π|③[r]
Γ[r]k ∼ Γk|②[r], ③[r],
♦rt♠ s s♠♣r
r y[rk] = yi:z
[r]i =k
y[r]i = (y1[r]i , . . . , y
j−1[r]i , y
j+1[r−1/2]i , . . . , y
e[r−1/2]i ) ♥ β
[r]k =
(β[r]k1, . . . ,β
[r]kj−1,β
[r−1]kj+1, . . . ,β
[r−1]ke )
♠r s♠♣♥ ♦ t ss♥ r ss♥ r② s t ♥rt r♥ ♦♥ trt♦♥ ♦ t s s♠♣r t ♦♦s② tsstt♦♥r② strt♦♥ st②s ♥♥ s t s♠♣♥ s ♠♥t♦r② s♦ t str♦♥ ♣♥♥② t♥ ② ♥ ③ ♥ t♥ y
j[rk] ♥ βkj
♠r ♥ t tr♦♣♦st♥s s♠♣r t s♠♣♥s r♦♠ ♥ r ss t t♦ ♦tr ♦♥s r ♠♦r ♦♠♣① ♥ ts♠♣♥ r♦♠ ♥♦s t♦ ♦♠♣t t ♦♥t♦♥ ♣r♦ts ♦ t ss♠♠rs♣s s♦ t♦ ♦♠♣t t ♥tr ♥ ♥ t ♥♠r ♦ srt rs s r ts ♦♠♣tt♦♥ s t♠ ♦♥s♠♥ ♦r t s♠♣♥ r♦♠ ♥ ♥t② ♣r♦r♠ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ ♥ p(zi,yi|xi, t(r−1)) s stt♦♥r② strt♦♥ ♦♥r♥♥t s♠♣♥ ♦r♥ t♦ t s ♣r♦r♠ ♥ t♦ st♣s rst② t ♠r♥♣r♠tr s s♠♣ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ ♥p(βkj|①,y↑j(r)
[rk] , ③(r),β
↑j(r)k ,Γk) s stt♦♥r② strt♦♥ ♦♥② t t♥t s
s♥ t♦r s s♠♣ r♦♠ ts ♦♥t♦♥ strt♦♥
♠r s♠♣♥ ♦ t ss♥ r ss♥ r② s t ♥rt r♥ ♦♥ trt♦♥ ♦ t s s♠♣r t ♦♦s② tsstt♦♥r② strt♦♥ st②s ♥♥ s t s♠♣♥ s ♠♥t♦r② s♦ t str♦♥ ♣♥♥② t♥ ② ♥ ③ ♥ t♥ y
j[rk] ♥ βkj
❲ ♥♦ t t ♦r st♣s ♦ t s s♠♣r ♥ ♣♦♥t ♦t t tst♦ s♠♣ r♦♠ ♥ s ♦t st♣s r ♠♦ t♦ ♦t♥ ttr♦♣♦st♥s s♠♣r t ♥ t ♥①t st♦♥
ss ♠♠rs♣ ♥ ss♥ t♦r s♠♣♥
♠ s t♦ s♠♣ r♦♠ ② s♥ t ♥♣♥♥ t♥ t ♥s t t♦rs (③,②) r s② s♠♣ ♦♥t♦♥② ♦♥ (①,θ[r−1]) ♦r♥t♦
p(③,②|①,θ[r−1]) =n∏
i=1
p(zi|xi,θ[r−1])p(yi|xi, zi,θ[r−1]).
②s♥ ♥r♥ tr♦♣♦st♥s s♠♣r
❲ ♥♦ t ♦t strt♦♥s ♦ t rt s ♦ t ♦ qt♦♥ z
[r]i s ♥♣♥♥t② s♠♣ r♦♠ t ♦♦♥ ♠t♥♦♠ str
t♦♥zi|xi,θ[r−1] ∼ Mg(ti1(θ
[r−1]), . . . , tig(θ[r−1])),
r tik(θ[r−1]) =
π[r−1]k p(xi;α
[r−1]k )
p(xi;θ[r−1])
s t ♣♦str♦r ♣r♦t② tt xi s ♥
r♥ ② ♦♠♣♦♥♥t k t t ♣r♠trs θ[r−1] y
[r−1/2]i s ♥♣♥♥t② s♠♣ ② r♠r♥ tt t rst c ♠♥ts
♦ yi ♥♦t ② yi r tr♠♥st ♦r ① tr♣t (xi, zi,θ[r−1]) t
zik = 1 s s yi = Ψ(x
i ;α[r−1]k ) ts st d ♠♥ts ♥♦t ②
yi r s♠♣ ♦r♥ t♦ drt ss♥ strt♦♥ Nd(0,Γ[r−1]k )
tr♥t ♦♥ t s♣ Sk(x
i )
p(yi |xi, zi,θ[r−1]) ∝g∏
k=1
(
φd(y
i ;µ[r−1]k ,Σ
[r−1]k )1yi ∈Sk(x
i )
)zik,
r µ[r−1]k = Γ
[r−1]k Γ
−1[r−1]k Ψ(x
i ;α[r−1]k )
♠r ts t♦ ♦♠♣t tik(θ[r−1]) ♦t tt t ♦♠♣tt♦♥ ♦
tik(θ[r−1]) ♥♦s t♦ ♦♠♣t t ♥tr ♥ ♥ ♥ t♦♦ ♠
t♠ ♦♥s♠♥ d s r d > 6 s t s♠♣♥ ♦r♥ t♦ ss♦ ♣r♦r♠ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ ♦♥ tst② ♥ t ♥ t ♥①t st♦♥
r♥ ♣r♠tr ♥ ss♥ t♦r s♠♣♥
♠ s t s♠♣♥ r♦♠ ♥ ♦♠♣♦s s ♦♦s
p(βkj,yj[rk]|①,y
[r][rk], ③
[r],β[r]k ,Γ
[r−1]k ) = p(βkj|①,y[r][rk], ③
[r],β[r]k ,Γ
[r−1]k )
× p(yj[rk]|①,y[r][rk], ③
[r],β[r]k ,βkj,Γ
[r−1]k ).
❲ ♥♦ t ♦t strt♦♥s ♦ t rt s ♦ t ♦ qt♦♥ ♦♥t♦♥ strt♦♥ ♦ βkj s ♥ t ♥ ♥♥♦♥ ♥tr♣t
s s
p(βkj |①,y[r][rk], ③[r],β
[r]k ,Γ
[r−1]k ) ∝ p(βkj)
n∏
i=1
(
p(xji |y↑j[r]i , z
[r]i ,Γ
[r−1]k ,βkj)
)z[r]ik
.
♦♥t♦♥ strt♦♥ ♦ xji |y↑j[r]i , z
[r]i ,Γ
[r−1]k t z[r]ik = 1 s ♦♥ t
rt s ♦ t ♦ qt♦♥ s ♥ ②
p(xji |y↑j[r]i , z
[r]i ,Γ
[r−1]k ,βkj) =
φ1(xji−µkjσkj
; µi, σ2i )/σkj 1 ≤ j ≤ c
Φ1(b⊕(xji )−µi
σi)− Φ1(
b⊖(xji )−µiσi
) ♦trs,
r t r µi = Γ[r−1]k [j, ]Γ
[r−1]k [, ]−1y
↑j[r]i s t ♦♥t♦♥ ♠♥ ♦
yji Γk[j, ] ♥ t r♦ j ♦ Γk ♣r ♦ t ♠♥t j ♥ Γk[, ] ♥
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
t ♠tr① Γk ♣r ♦ t r♦ ♥ t ♦♠♥ j ♥ r σ2i s t
♦♥t♦♥ r♥ ♦ yji ♥ ② σ2i = 1−Γ
[r−1]k [j, ]Γ
[r−1]k [, ]−1
Γ[r−1]k [, j]
② t ♥♣♥♥ t♥ t ♥s t ♦♥t♦♥ strt♦♥♦ yj[rk] s ①♣t② ♥ s
p(yj[rk]|①,y[r][rk], ③
[r],β[r]k ,βkj,Γ
[r−1]k ) =n∏
i=1
(
p(yji |xji ,y↑j[r]i , z
[r]i ,βkj,Γ
[r−1]k )
)z[r]ik
.
xj s ♦♥t♥♦s r 1 ≤ j ≤ c ♥ z[r]i = k t
♦♥t♦♥ strt♦♥ ♦ yji s tr♠♥st s s
yj[r]i =
xji − µ[r]kj
σ[r]kj
.
xj s srt r c + 1 ≤ j ≤ e ♥ z[r]ik = 1 t
♦♥t♦♥ strt♦♥ ♦ yji s tr♥t ss♥ strt♦♥ s s
p(yji |xji ,y↑j[r]i , z
[r]i ,β
[r]kj ,Γ
[r−1]k ) =
φ1(yji ; µi, σ
2i )
p(xji ;β[r]kj)
1yji∈[b
⊖[r]k (xji ),b
⊕[r]k (xji )]
,
r b⊖[r]k (xji ) = P (xji − 1;β
[r]kj) ♥ b
⊕[r]k (xji ) = P (xji ;β
[r]kj)
♠r ts t♦ s♠♣ t ♠r♥ ♣r♠trs s♠♣♥ ♦ βkjs ♥♦t s② ♣r♦r♠ s♥ t ♥♦r♠③♥ ♦♥st♥t ♥ ♥ s ♥♥♦♥s st♣ s t♥ r♣ ② ♦♥ trt♦♥ ♦ tr♦♣♦sst♥s ♦rt♠ st ♥ t ♥①t st♦♥ ♦r ♥♦t tt t s♠♣♥ ♦ yj[rk] r♦♠ ss② ♣r♦r♠
❱t♦r ♦ ♣r♦♣♦rt♦♥s s♠♣♥
♠ s t s♠♣♥ r♦♠ s ss ♦r t ♠①tr ♠♦ ♦♥t r②s ♥♦♥ ♥♦r♠t ♣r♦r ♥♦s tt
π|③[r] ∼ Dg
(
♥[r]1 +1
2, . . . , ♥[r]g +
1
2
)
,
r ♥[r]k =∑n
i=1 z[r]ik
♦rrt♦♥ ♠tr① s♠♣♥
♠ s t s♠♣♥ r♦♠ ❲ s t ♣♣r♦ ♣r♦♣♦s ② ❬♦❪♥ t s ♦ s♠♣r♠tr ss♥ ♦♣ s ♥t♦ t♦ st♣srst② ♦r♥ ♠tr① s ♥rt ② ts ①♣t ♣♦str♦r strt♦♥ ♥s♦♥② t ♦rrt♦♥ ♠tr① s ② ♥♦r♠③♥ t ♦r♥ ♠tr①❲♥ (②, ③) r ♥♦♥ r ♥ t ♥♦♥ s ♦ ♠trt ss♥
②s♥ ♥r♥ tr♦♣♦st♥s s♠♣r
♠①tr ♠♦ t ♥♦♥ ♠♥s s t s♠♣♥ ♦r♥ t♦ Γk|②[r], ③[r] s♣r♦r♠ ② t t♦ ♦♦♥ st♣s
Λk|②[r], ③[r] ∼ W−1
s0 + ♥[r−1]
k , S0 +∑
i:z[r]i =k
y[r]Ti y
[r]i
∀1 ≤ h, ℓ ≤ e, Γk[h, ℓ] =Λk[h, ℓ]
√
Λk[h, h]Λk[ℓ, ℓ].
♠r ♠♣♥ ♦ t ♦rrt♦♥ ♠trs ♦r t ♦♠♦sst ♠♦s t ♦♠♦sst ♠♦ ss♠s t qt② t♥ t ♦rrt♦♥ ♠trs♥ s s ♦♥② s♠♣ ♦♥ Λ s♦ s r♣ ②
Λ|②[r], ③[r] ∼ W−1
(
s0 + n, S0 +n∑
i=1
y[r]Ti y
[r]i
)
,
♥ ♣t Λk = Λ ♦r k = 1, . . . , g
♦r♥ t♦ ♦t ♠rs ♥ t rst t♦ st♣s ♦ t s s♠♣r♥♦ ts ♦ ② t ♦♦♥ ②r ♠♠ ♦rt♠
tr♦♣♦st♥s s♠♣r
❲♥ s♦♠ st♣s ♦ s s♠♣r ♥♥♦t s② s♠t t ♠② st♦ ♣r♦r♠ t ♥r♥ ②r ♠♠ ♦rt♠ ❬❪ s s ttr♦♣♦st♥s s♠♣r r♣s ♦t s♠♣♥ r♦♠ ③,②|①,θ[r−1]
♥ βkj|①,y[r][rk], ③[r],β
[r]k ,Γ
[r−1]k ♥ ② ♥ ② ♦♥ trt♦♥ ♦ t♦
tr♦♣♦sst♥s st♣s tt ♥♦ t
ss ♠♠rs♣ ♥ ss♥ t♦r s♠♣♥
st♣ s ♣r♦r♠ ♦♥ trt♦♥ ♦ t tr♦♣♦sst♥s ♦rt♠ s ♦rt♠ s ♥♣♥♥t② ♣r♦r♠ t♦ s♠♣ ♦♣ (zi,yi)s♥ t ♥s r ♥♣♥♥t ts stt♦♥r② strt♦♥ s
p(zi,yi|xi,θ[r−1]) ∝g∏
k=1
(
π[r−1]k φe(yi;0,Γ
[r−1]k )1
yi=Ψ(xi ;α[r−1]k )
1yi ∈Sk(x
i )
)zik.
tr♦♣♦sst♥s ♦rt♠ s♠♣s ♥t (z⋆i ,y
⋆i ) ② t ♥str
♠♥t strt♦♥ q1(.|xi,θ[r−1]) ♥♦r♠② s♠♣s z⋆i t♥ s♠♣sy⋆i |z⋆i s ♦♦s ♦♥t♦♥② ♦♥ z⋆ik⋆ = 1 ts ♥str♠♥t strt♦♥ s tr♠♥st ♦r t rst c ♠♥ts ♦ y⋆i ♥♦t ② y⋆i s s y⋆i = Ψ(x
i ;α[r−1]k⋆ )
t s♠♣s t st d ♠♥ts ♦ y⋆i ♥♦t ② y⋆i ♦r♥ t♦ ♠trt♥♣♥♥t ss♥ strt♦♥ tr♥t ♦♥ Sk⋆(x
i ) s
q1(zi,yi|xi,θ[r−1]) =
g∏
k=1
(
1
g
φd(y
i ;0, I)∏e
j=c+1 p(xji ;β
[r−1]kj )
1yi=Ψ(xi ;α
[r−1]k )
1yi ∈Sk(x
i )
)z⋆ik
.
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
♥t s ♣t t t ♣r♦t②
ρ[r]1i = min
∏gk=1
(
πkφe(y⋆i ;0,Γ
[r−1]k )
)z⋆ik
∏gk=1
(
πkφe(y[r−1]i ;0,Γ
[r−1]k )
)z[r−1]ik
q1(z[r−1]i ,y
[r−1]i |xi)
q1(z⋆i ,y⋆i |xi)
; 1
.
s t trt♦♥ [r] ♦ t ♦rt♠ t s♠♣♥ ♦r♥ t♦ s♣r♦r♠ ♦♥ trt♦♥ ♦ t ♦♦♥ tr♦♣♦sst♥s ♦rt♠
s ♦rt♠ s p(zi,yi|xi,θ[r−1]) s stt♦♥r② strt♦♥ ts s rtt♥ s ♦♦s
(z⋆i ,y⋆i ) ∼ q1(zi,yi|xi)
(z[r]i ,y
[r−1/2]i ) =
(z⋆i ,y⋆i ) t ♣r♦t② ρ[r]1i
(z[r−1]i ,y
[r−1]i ) t ♣r♦t② 1− ρ
[r]1i .
♦rt♠ tr♦♣♦sst♥s
r♥ ♣r♠tr s♠♣♥
st♣ s ♣r♦r♠ ♥ t♦ st♣s rst② t s♠♣♥ ♦ β[r]kj ♦r♥
t♦ s ♣r♦r♠ ♦♥ trt♦♥ ♦ t tr♦♣♦sst♥s ♦rt♠ ♦st stt♦♥r② strt♦♥ s p(βkj|①,y[r][rk], ③
[r],β[r]k ,Γk) ♦♥② t s♠♣♥ ♦
yj[r][rk] s ♣r♦r♠ ♦r♥ t♦ ts ♦♥t♦♥ strt♦♥ ♥ ②
♥str♠♥t strt♦♥ ♦ t tr♦♣♦sst♥s ♦rt♠ q2(.|①, ③) s♠♣s ♥t β⋆kj ♦r♥ t♦ t ♣♦str♦r strt♦♥ ♦ βkj ♥r t ♦♥t♦♥♥♣♥♥ ss♠♣t♦♥ ts strt♦♥ s ①♣t s♥ t ♦♥t ♣r♦r strt♦♥s r s ♦
q2(.|①, ③) = p(βkj|①, ③,Γk = I).
s ♦r♥ t♦ t ♥t β⋆kj s ♣t t t ♣r♦t②
ρ[r]2 = min
p(β⋆kj)q2(β[r−1]kj |①, ③)
p(β[r−1]kj )q2(β
⋆kj |①, ③)
∏
i:z[r]i =k
p(yji |xji ,y
↑j[r]i , zi,β
⋆kj ,Γ
[r−1]k )
p(yji |xji ,y
↑j[r]i , zi,β
[r−1]kj ,Γ
[r−1]k )
; 1
.
s t trt♦♥ [r] ♦ t ♦rt♠ t s♠♣♥ r♦♠ s ♣r♦r♠ ♦♥ trt♦♥ ♦ t ♦♦♥ tr♦♣♦sst♥s ♦rt♠
♠r ①♣r♠♥ts ♦♥ s♠t t sts
s ♦rt♠ s p(βkj|x[rk],y[r][rk], ③,β
[r]k ,Γk) s stt♦♥r② strt♦♥ t
s rtt♥ s ♦♦s
β⋆kj ∼ q2(βkj|①, ③)
β[r]kj =
β⋆kj t ♣r♦t② ρ[r]2
β[r−1]kj t ♣r♦t② 1− ρ
[r]2 .
♦rt♠ tr♦♣♦sst♥s
♠r ♥str♠♥t strt♦♥s ♦t tt t s♠r r t ♥trss ♣♥♥s ♦ t r xi t ♦sr ♦ t stt♦♥r② strt♦♥s rt ♥str♠♥t strt♦♥s ♦ ♦t tr♦♣♦sst♥s ♦rt♠s
st♥ ♣r♦♠
st♥ ♣r♦♠ s ♥r② s♦ ② s♣ ♣r♦rs ❬t❪♦r s ♦♥ t r♠♥t ♦♣ ♥ ❬P❪ ts t♥qs r ♣r♥♣② ♠♣t♥ ♥ g s ♥♦♥
❲♥ t ♠♦ s s t♦ str t ♥♠r ♦ sss s ♥♥♦♥ ♥ t♠♦ st♦♥ s ♣r♦r♠ ② t rtr♦♥ s♠t♥♦s② ♦s t st♥ ♣♥♦♠♥♦♥ ♥ ♦♥ t ♦♥ ♥ ts rtr♦♥ sts qts♣rt sss ♥ t s♠♣ s③ s s♠ s♦ t st♥ s ♥♦t ♣rs♥t♥ ♣rt s ♦ t ss s♣rt② ♥ t ♦tr ♥ ♥ t ♥ st♠♦r sss ♥ t s♠♣ s③ ♥rss t st♥ ♣r♦♠ s stts♥ ts ♣♥♦♠♥♦♥ ♥ss s②♠♣t♦t②
♦s② ♥ t ♥♠r ♦ sss s ① ♥ t s③ ♦ s♠♣ s s♠t st♥ ♣r♦♠ ♥ ♦r ♥ s s ♦r s ♥tr② t♦ st ♣r♦rs t ♥ ❬t❪
♠r ①♣r♠♥ts ♦♥ s♠t t sts
♥ ♦rr t♦ strt t ♣r♦♣rts ♦ t ♠♦ t♦ ♥♠r ①♣r♠♥tsr ♣r♦r♠ rst ♦♥ ♦♥ssts ♥ s♠t♥ t ♦r♥ t♦ t ♣r♦♣♦s♠♦ ♥ t♦ st② t ♦♥r♥ ♦ t st♠ts s♦♥ ♦♥ ♦♥ssts ♥s♠t♥ t ♦r♥ t♦ ♠①tr ♦ P♦ss♦♥ strt♦♥s ❬❪ ♥ ♦rr t♦s♦ t r♦st♥ss ♦ t ♣r♦♣♦s ♠♦ st♠t s ♦♠♣t ② r♥t ♣r♠trs s♠♣ ② t s ♦rt♠
①♣r♠♥t ♦♥t♦♥s
♦r stt♦♥ s♠♣s r ♥rt ♥ t ♦rt♠ s ♥t③t t ♠①♠♠ ♦♦ st♠t ♦ t ♦♥t♦♥ ♥♣♥♥ ♠♦ r♥♥ s ♣r♦r♠ r♥ trt♦♥s ♥ t ♣r♠tr ♥t③t♦♥ sr♥t ♥ t ♥trss ♣♥♥s r s♠ ♦rt♠ s st♦♣♣ tr
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
trt♦♥s ♠①♠♠ ♣♦str♦r st♠t s ♣♣r♦①♠t ② t ♠♥♦ t s♠♣ ♣r♠trs r r♥ s ♣♣r♦①♠t trt♦♥s ♦ ♦♥tr♦ ♠t♦
♠t♦♥ ① rs ♦♥ ♦♥t♥♦s ♦♥ ♥tr ♥ ♦♥ ♥r②❲ ♦♥sr t ♠①tr ♠♦ ♦ ss♥ ♦♣s t ♥ ①♠♣ ♥♦♠♣♦s t ♦♥ ♦♥t♥♦s r ♦♥ ♥tr r ♥ ♦♥ ♥r② rr strts t rs♥ ♦r ♦ t r r♥ ♦t ♠♦ t t ♠①♠♠ ♣♦str♦r st♠t r♦♠ t ♠♦ t t tr ♣r♠trs ♦r♥ t♦ t s♠♣ s③ ♥ t ♠① s s s♠t♦♥ strtst ♦♦ ♦r ♦ t tr♦♣♦st♥s ♦rt♠ rtr♠♦r t ♣♣r♦①♠t♦♥ ♦ t ♠①♠♠ ♣♦str♦r st♠t ② t ♠♥ ♦ t ♣r♠trss♠♣ ② ts ♦rt♠ s ♥t
100 200 400 800 1600
0.00
0.05
0.10
0.15
0.20
size of sample
Kul
lbac
k−Le
ible
r di
verg
ence
r rs ♦ t r r♥ ♦ t ♠♦ t t♠①♠♠ ♣♦str♦r st♠t r♦♠ t ♠♦ t t tr ♣r♠tr
♠t♦♥ ♦st♥ss ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s r♥ts ①♣r♠♥ts t r s♠♣ ♦r♥ t♦ rt P♦ss♦♥ ♠①tr ♠♦❬❪ ♦s t ♠r♥ ♣r♠trs r ♥♦t ② αk = (λk1, λk2, λk3) s♠t♦♥ s ♣r♦r♠ t t ♦♦♥ s ♦ t ♣r♠trs
π = (1/3, 2/3), λ1h = h ♥ λ2h = 3 + h, ♦r h = 1, 2, 3.
rr♦r rt ♦ ts ♠♦ ♦♠♣t t t ②s r s q t♦ 9.5% stss♦ tt t ①t② ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦s t♦ ♥t②t ts s♠t t ♥ t r r♥ ♦♠s r② s♠♥ t s③ ♦ t s♠♣ ♥rss rtr♠♦r t rr♦r rt ♦ t ♠♦ s♠st♦ ♦♥r t♦ st tt t rr t♥ t t♦rt ♦♥ 9.5% ❲ s♦♥♦t tt t ♠r♥ ♣r♠trs ♦ ♦t ♦♠♣♦♥♥ts ♥ t ♦rrt♦♥ ♦♥tss♠ t♦ ♦♥r t♦ tr tr s
♥②ss ♦ tr r t sts
❲ ♥♦ str tr r t sts ② s♥ t ♠①tr ♠♦ ♦ ss♥♦♣s ♣r♠trs r st♠t t tr♦♣♦sts ♦rt♠♥t③ ♦♥ t ♠①♠♠ ♦♦ st♠t ♦ t ♦♥t♦♥ ♥♣♥♥
♥②ss ♦ tr r t sts
50 100 200 400 800
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
size of sample
Kul
lbac
k−Le
ible
r di
verg
ence
50 100 200 400 800
0.10
0.15
0.20
0.25
0.30
size of sample
Bay
es' e
rror
50 100 200 400 800
24
68
10
size of sample
β 11
50 100 200 400 800
0.0
0.2
0.4
0.6
0.8
size of sample
Γ 1[1
,2]
r sts ♦ ♠t♦♥ r r♥ ♦ t st♠t ♠♦ r♦♠ t tr ♦♥ rr♦r rt ♦ t st♠t ♠♦ ❱♦ t rst ♠r♥ ♣r♠tr ♦r t ss ❱ ♦ t ♦rrt♦♥ ♦♥tt♥ ♦t rs ♦r ss
♠♦ r♥♥ s ♣r♦r♠ r♥ trt♦♥s ♥ t ♣r♠tr ♥t③t♦♥ s r♥t ♥ t ♥trss ♣♥♥s r s♠ ♦rt♠ sst♦♣♣ tr trt♦♥s ♥ t st♠t s ♦t♥ ② t♥ t ♠♥ ♦ ts♠♣ ♣r♠trs ♠♦ st♦♥ s ♣r♦r♠ ② s♥ t♦ ♥♦r♠t♦♥rtr rtr♦♥ ❬❪ rtr♦♥ ❬❪ ♦♠♣t ♦♥ t ♠①♠♠ ♣♦str♦r st♠t
r s♦rr t st
t
s t st ❬♦r❪ srs ♥s ② ♦♦ tsts rt♦t t♦ s♥st t♦ r s♦rrs tt ♠t rs r♦♠ ①ss ♦♦♦♥s♠♣t♦♥ ♦♥t♥♦s rs ♥ ② t ♥♠r ♦ qrt♣♥t q♥ts♦ ♦♦ rs r♥ ♣r ② ♦♥ ♥tr r
♦ st♦♥
❲ st♠t t tr ♠①tr ♠♦s ♦♥t♦♥ ♥♣♥♥ ♦♥ tr♦sst ss♥ ♦♣ ♠①tr ♥ ♦♠♦sst ss♥ ♦♣ ♠①tr ♦r r♥t ♥♠rs ♦ sss ♣rs♥ts t s ♦ ♦t s ♥♦r♠t♦♥
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
rtr s ♦ ♦t rtr ♦t♥ t t ♦♠♣♦♥♥t ♦♠♦sst♠①tr ♠♦ ♦ ss♥ ♦♣s r t st ♦♥s ♦r ♥♦t tt t tr♠♦s st t♦ ♦♠♣♦♥♥ts
♦♥ ♥♣t
tr♦ ♦♠♦
♦♥ ♥♣t tr♦ ♦♠♦
❱s ♦ t ♥ rtr ♦r t tr ♠①tr ♠♦s st♠t♦♥ t r s♦rr t st
♥tr♣rtt♦♥ ♦ t st ♠♦
❲ ♥♦ sr t st ♠♦ ♦r♥ t♦ ♦t rtr t ♦♠♦sst♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ② s♥ t ♠r♥ ♣r♠trs♥ t ♥trss ♣♥♥s s♠♠r③ ② r ♠♦ ♦♥srst♦ sss ♦s t ♠♦rt② ♦♥ π1 = 0.60 r♦♣s t ♥s ♥ str♦♥ ♦♦ ♦♥s♠♣t♦♥ β1r♥s = 10.6 ♥ r s ♦ t ♦♦tsts s♣② ♦r t tsts ♦t ♥ ♠♠t ♠♥♦rt② ss r♦♣s t♥s ♥ s♠ ♦♦ ♦♥s♠♣t♦♥ β2r♥s = 1.36 ♥ s♠r s♦ t ♦♦ tsts ♦r ♦t sss t tr ♦♦♥ ♦♦ tsts r ♣♦st②♦rrt t ♣t ♦♣t ♥ ♠♠t t tst s ♣♦st② ♦rrtt t ♥♠r ♦ ♦♦ r♥s
Gammagt
0 50 100 150 200 250 3000.00
00.
005
0.01
00.
015
0 2 4 6 8 11 14 17 20 23 26 29 32 35 38
Drinks
0.00
0.04
0.08
0.12
πkp(xj |βkj , z = k)
−1.0 −0.5 0.0 0.5 1.0
−1.
0−
0.5
0.0
0.5
1.0
inertia: 36 %
iner
tia: 2
1 %
Mcv
Alkphos
Sgpt
SgotGammagt
Drinks
❱rs ♥ t rst ♦rrt♦♥ r ♥ ② Γk
r ♠♠r② ♦ t ♦♠♦sst ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r t r s♦rr t st ss s s♣② ♥ ♥ ss ♥ r
♥②ss ♦ tr r t sts
Prtt♦♥ st②
s t rs r ♥♠r r ♥ s♣② t ♥s ♥tr ss ♠♠rs♣s ♥ t rst ss ♣ ♠♣ ♦r s sss r ♥♦t s♣rt ♥ ts ♠♣ t strtr ♦ t t s ♥♦t r♦t ♦t sr s♣②s t ♥s ♥ t rst ♣ ♠♣ ♦ ss ♥ ts ♠♣sss r ttr s♣rt s♥ t rst ss rs s ♥tr ts♦♥ ss r tr♥s s ♦♥ t t♦♣ ♣rt ♦ t r♣ ♦ t s♦♥ ①ss sr♠♥♥t s s♠♠r② s ♥ r♠♥t t t ss ♥tr♣rtt♦♥ s♥ts ①s s t ② t rs ♥ r♥s r t♠ss sr♠♥♥t♦r♥ t♦ tr ♠r♥ ♣r♠trs
−2 0 2 4 6
−4
−3
−2
−1
01
23
First component (inertia 42%)
Sec
ond
com
pone
nt (
iner
tia 1
8%)
rst ♠♣ ♦ P
−15 −10 −5 0
−6
−4
−2
02
46
inertia 36 %
iner
tia 2
1 %
rst ♠♣ ♦ t P ♦ t ss
r ❱s③t♦♥ ♦ t ♣rtt♦♥ ② t ♦♠♦sst ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r t r s♦rr t st ss s r♥ ② rs ♥ ss ② r tr♥s
♦t tt t ♣rtt♦♥s ♦t♥ ② t tr ♦♠♣♦♥♥t ♠♦s r s♠rt ♥♦t ♥t s s♦♥ ②
tr♦
♦♠♦ ♦♠♦
♦♥ ♥♣t
♦♠♦ ♦♠♦
♦♥s♦♥ ♠trs t♥ t ♣rtt♦♥ ♦t♥ ② t ♦♠♦sst ♦♠♣♦♥♥t ♠♦ ♥ t ♣rtt♦♥ ♦t♥ ② t tr♦sst ♦♠♣♦♥♥t ♠♦ t ♦♥t♦♥ ♥♣♥♥ ♠♦
♦♥s♦♥
♥ ts t st t ♠①tr ♠♦ ♦ ss♥ ♦♣s ttr ts t t♦r♥ t♦ t ♥♦r♠t♦♥ rtr t♥ t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♥
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
♦t ♠♦s st t s♠ ♥♠r ♦ sss ♣ ♣r ss ♦s t♦s♠♠r③ t ♥trss ♣♥♥s ♥ t♦ r♥ ♦t t s♣rt♦♥ ♦ ♦tsss ♥ ② ss ♣
❲♥ t st
t
t st ❬+❪ ♦♥t♥s r♥ts ♦ t P♦rts ❱♥♦ ❱r♥ r ♥s ♥ t ♥s sr ② ♥ ♣②s♦♠♦♥t♥♦s rs ① t② ♦t t② tr t② rs sr♦rs r sr ♦① t♦t ♥st② ♦① ♥st② ♣ s♣ts ♦♦♥ ♦♥ ♥tr r qt② ♦ t ♥ t ② ①♣rts ♥s ♦t ♥s r ♦r t r ♥ ♥ str t t st t tr r♥t♠①tr ♠♦s ♦t tt ♦♥ t ♥ ♥♠r s ① ♦ t st②s♥ t s ♥ ♦tr
♦ st♦♥
❲ st♠t t tr ♠①tr ♠♦s ♦♥t♦♥ ♥♣♥♥ ♦♥ tr♦sst ss♥ ♦♣ ♠①tr ♥ ♦♠♦sst ss♥ ♦♣ ♠①tr ♦r r♥t ♥♠rs ♦ sss ♥ ♣rs♥t t s ♦ ♦t s ♥♦r♠t♦♥ rtr ♥ ♦t rtr st♥t② st t ♦♠♣♦♥♥t tr♦sst ♠①tr♠♦ ♦ ss♥ ♦♣s ❲ ♥♦ s♦ tt ts ♠♦ ♦s t♦ s♣rtt t ♥s r♦♠ t r ♦♥s t♥ t ♠♦ ♥tr♣rtt♦♥
♦♥ ♥♣t
tr♦ ♦♠♦
♦♥ ♥♣t tr♦ ♦♠♦
❱s ♦ t ♥ rtr ♦r t tr ♠①tr ♠♦s st♠t♦♥ t ♥ t st
Prtt♦♥ st②
♣rs♥ts t ♦♥s♦♥ ♠trs ♥ ♦rr t♦ ♦♠♣r t r♥ ♦ tst♠t ♣rtt♦♥s ♦r♥ t♦ t tr ♦♥ ♥ ♦♦r s rsts str♥t♥t tt t ♠♦ st tt♥ t t s t ♦♠♣♦♥♥t tr♦sstss♥ ♦♣ ♠①tr ♠♦s ♥ ts ♣rtt♦♥ s t ♦sst t♦ t tr ♦♥
r s♣②s t ♥s ♥ ♣ ♠♣ ♦ ♦t sss st♠t ②t ♦♠♣♦♥♥t tr♦sst ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r♥ t♦ts sttr♣♦ts sss r s♣rt ❲ ♥♦ t ts ♣r♠trs
♥②ss ♦ tr r t sts
t r ♥
t r ♥
t r ♥
❱s ♦ t st ♥ ♥① ♥ ♦♥s♦♥ ♠trs t♥ ttr ♣rtt♦♥ ♥ t st♠t ♣rtt♦♥ ② t ♦♠♣♦♥♥t tr♦sstss♥ ♦♣ ♠①tr t tr♦♠♣♦♥♥t ♦♠♦sst ss♥ ♦♣♠①tr t ♦r♦♠♣♦♥♥t ♦♥t♦♥ ♥♣♥♥ ♠①tr
−5 0 5
−15
−10
−5
05
inertia 9.7 %
iner
tia 7
.8 %
P ♦ t ss ♠♣
−5 0 5 10
−10
−5
05
inertia 22.5 %
iner
tia 1
8.8
%
P ♦ t ss ♠♣
r ❱s③t♦♥ ♦ t ♣rtt♦♥ ② t tr♦sst ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r t ♥ t st ss s r♥ ② rs ♥ ss ② r tr♥s
♥tr♣rtt♦♥ ♦ t st ♠♦
♦♦♥ ♥tr♣rtt♦♥ s s ♦♥ t ♠r♥ ♣r♠trs ♦ t ♦♠♣♦♥♥ts ♥ ♦♥ t ♥trss ♦rrt♦♥ ♠trs s♠♠r③ ② r ♠♦rt② ss π1 = 0.59 s ♣r♥♣② ♦♠♣♦s t t ♥s s ss srtr③ ② ♦r rts ♦ t② ♣ ♦rs ♥ s♣ts t♥ t♠ ♦ t♠♥♦rt② ss π2 = 0.41 s ♣r♥♣② ♦♠♣♦s ② r ♥s ♠♦rt②ss s rr s ♦r ♦t sr ♦① ♠srs ♥ t ♦♦ rt ♦ttt t ♥ qt② ♦ ♦t sss s s♠r β1qt② = 5.96 ♥ β2qt② = 5.58 ♠♦rt② ss s rtr③ ② str♦♥ ♦rrt♦♥ t♥ ♦t sr ♠srs ♦♣♣♦st t♦ str♦♥ ♦rrt♦♥ t♥ t ♥st② ♥ t② ♠srs ♠♥♦rt② ss ♥r♥s tt t ♥ qt② s ♣♥♥t t rr ♦♦rt ♥ s♠ s ♦r t ♦rs ♥ t② ♠srs
♦♥s♦♥
♥ ts t st t ss♥ ♦♣ ♠①tr ♠♦s ♦s t♦ r t ♥♠r ♦ sss ♥ t♦ ttr t t t rtr♠♦r ts ♠♣t ♦♥ t st♠t
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
volatile.acidity
0.5 1.0 1.5
01
23
residual.sugar
0 10 20 30 40 50 60
0.00
0.04
0.08
0.12
πkp(xj |βkj , z = k)
−1.0 −0.5 0.0 0.5 1.0
−1.
0−
0.5
0.0
0.5
1.0
inertia: 31.9 %
iner
tia: 1
3 %
fxd.
vlt.
ctr.
rsd.
chlrfr..tt..
dnst
pH
slph
alchqlty
❱rs ♥ t rst ♦rrt♦♥r ♥ ② Γ2
r ♠♠r② ♦ t tr♦sst ♦♠♣♦♥♥t ss♥ ♦♣ ♠①tr♠♦ ♦r t ♥ t st ss s r♥ ♥ ♥ ss ♥ r
♣rtt♦♥ s s♥♥t s ♦♥ t ♥ sttr♣♦ts ♥ t ♠♦ ♣ tst♠t sss r r♥t s♥ t② r s♣rt ♥② t st♠t♦♥♦ t ♥trss ♣♥♥s ♣s t ♥tr♣rtt♦♥ s♥ ts ♥r♥s t ♥t♥ t ♥ qt② ♦ t ♠♥♦rt② ss ♥ ts ♣②s♦♠ ♣r♦♣rts
♦rst r t st
t
s t st srs ♦rst rs ❬❪ ♥ t ♥♦rtst r♦♥ ♦ P♦rt② s♥ ♠t♦r♦♦ rs s♥ ♦♥t♥♦s rs ♦r ♦t t ❲s②st♠ ♥ t♦ ♦t t ♠t♦r♦♦② t♠♣rtr ♥rt ♠t② t♦ ♥tr rs rt t♦ t s♣t ♦♦r♥ts ♥ tr♥r② ♦♥s ♥t♥ t ♣rs♥ ♦ r♥ t ss♦♥ s♠♠r ♦r ♥♦t s♠♠r ♥t ② ♥ ♦r ♥♦t ♥
♦ st♦♥
♣rs♥ts t s ♦ ♦t s ♥♦r♠t♦♥ rtr ♦r t tr♠①tr ♠♦s ♦r♥ t♦ ♦t rtr t ♠♦ ttr tt♥ t t s t♦♠♦sst ♠①tr ♠♦ ♦ ss♥ ♦♣s t tr ♦♠♣♦♥♥ts
♥tr♣rtt♦♥ ♦ t st ♠♦
♦♦♥ ♥tr♣rtt♦♥ s s ♦♥ t ♠r♥ ♣r♠trs ♦♥ t ♥trss ♦rrt♦♥ ♠trs s♠♠r③ ♥ r ♠♦rt② ss π1 = 0.57r♦♣s t rs ♦♣ t t♠♣rtr ♥ s♠ rt ♠t② ♠srs ♦ ♥ r s♦♥ ss π2 = 0.26 r♦♣st ♥tr rs s rs r ♦♣ t str♦♥ ♥ ♥ ♥♦ r♥
♥②ss ♦ tr r t sts
♦♥ ♥♣t
tr♦ ♦♠♦
♦♥ ♥♣t tr♦ ♦♠♦
❱s ♦ t ♥ rtr ♦r t tr ♠①tr ♠♦s st♠t♦♥ t ♦rst r t st
t ❲ ♠srs t s♠ s ♠♥♦rt② ss π3 = 0.17) r♦♣s ts♠♠r rs ♦♣ t s ♦ ❲ ♠srs ①♣t t ♦♥ t♠♣rtr s ♠♥ t t rt ♠t② s ♥trss ♦rrt♦♥♠tr① ♥r♥s t ♣♥♥s t♥ t s♠♠r ♥ t♠♣rtr ♥s ♦ ♥ ♥② ♥♦t tt t s♣ ♦♦r♥ts r♦② ♦♦t s♠ strt♦♥ ♥ t tr sss
temp
5 10 15 20 25 30
0.00
0.02
0.04
0.06
no yes0.0
0.2
0.4
summer
C 1 C 2 C 3 C 1 C 2 C 3
πkp(xj |βkj , z = k)
−1.0 −0.5 0.0 0.5 1.0
−1.
0−
0.5
0.0
0.5
1.0
inertia: 23 %
iner
tia: 1
8.4
%
Xaxs
Yaxs
smmr
wknd
FFMCDMCDC
ISI
temp
RH
wind
rain
❱rs ♥ t rst ♦rrt♦♥r ♥ ② Γk
r ♠♠r② ♦ t ♦♠♦sst ♦♠♣♦♥♥t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦r t ♦rst r t st ss s s♣② ♥ r♥ ss ♥ r♥ ss ♥
Prtt♦♥ st②
♦t tt t ♣rtt♦♥s ♦t♥ ② t tr ♠♦s r s♠r t ♥♦t ♥ts s♦♥ ②
♣tr ♦s str♥ ♦ ss♥ ♦♣s ♦r ♠① t
tr♦
♦♠♦ ♦♠♦ ♦♠♦
♦♥ ♥♣t
♦♠♦ ♦♠♦ ♦♠♦
♦♥s♦♥ ♠trs t♥ t ♣rtt♦♥ ♦t♥ ② t ♦♠♦ssttr♦♠♣♦♥♥t ♠♦ ♥ t ♣rtt♦♥ ♦t♥ ② t tr♦sst ♦♠♣♦♥♥t ♠♦ t ♦♥t♦♥ ♥♣♥♥ ♠♦
♦♥s♦♥
♠♦ ♣♦♥ts ♦t tr sss ♦ ♦rst rs t s ♠♦r ♣rs t♥ t♦♥t♦♥ ♥♣♥♥ ♠♦ r♦② s♣rts t s♠♠r rs r♦♠ t♦tr ♦♥s ♥ t ♦♠♦sst ♠①tr ♠♦ ♦ ss♥ ♦♣s ♦♥srst♦ ♥s ♦ s♠♠r rs rstrt♦♥s ♦♥ ♦♥ t ♣r♠trs s♣s ♦ t♦ttr t t t t♥ t tr♦sst ss♥ ♦♣ ♠①tr ♠♦ ♦r♥ t♦ ♦t rtr ts ♠♣t s s♥♥t s♥ t ♥♠rs ♦ sss st② ♦t ♠♦s r r♥t
♦♥s♦♥
♠①tr ♠♦ ♦ ss♥ ♦♣s ss t ♣r♦♣rts ♦ ♦♣s ♥♣♥♥t ♦ ♦ t ♠r♥ strt♦♥s ♥ ♦ t ♣♥♥② rt♦♥s s ts♠①tr ♦s t♦ ① ss strt♦♥s ♦♥♥ t♦ t ①♣♦♥♥t ♠② ♦rt ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ ♦♠♣♦♥♥t ♦r♦r t ts ♥t♦♦♥t t ♥trss ♣♥♥s ♥ ♣♣r♦ s ♦♥ ♣ ♣r ss ♦ tss♥ t♥t r ♦s s♦ t♦ s♠♠r③ t ♠♥ ♥trss ♣♥♥s♥ t♦ s③ t t ② s♥ t ♠♦ ♣r♠trs
r♥ ♦t ♥♠r ①♣r♠♥ts ♥ ♣♣t♦♥s ♣♦♥t ♦t tt ts♠♦ s s♥t② ① t♦ t t r♥ ② ♥ ♦tr ♦♥ rtr♠♦r t ♥r t ss ♦ t ♦♥t♦♥ ♥♣♥♥ ♠♦ ♦r ♥st♥ t rt♦♥♦ t ♥♠r ♦ sss
♥♠r ♦ ♣r♠trs ♥rss t t ♥♠rs ♦ sss ♥ rss♣② s ♦ t ♦rrt♦♥ ♠trs ♦ t ss♥ ♦♣s ♦ ♦ tsr ♣r♦♣♦s ♦♠♦sst rs♦♥ ♦ t ♠♦ ss♠♥ t qt②t♥ t ♦rrt♦♥ ♠trs s ♠♦ ♠② ttr t t t t♥ ttr♦sst ss♥ ♦♣ ♠①tr ♠♦ ♦r t ♥ r ♥ t♥♠r ♦ rs ♥rss ♦ ♠♦r ♣rs♠♦♥♦s ♦rrt♦♥ ♠trs ♦ ♣r♦♣♦s t♦ ♦ ts r ♥ tr ♦rs
♥② t ♠♦ ♥ ♥♦t str ♥♦♥♦r♥ t♦r rs ♥ ♠♦rt♥ t♦ ♠♦ts ♥ ♥ s s t ♠t strt♦♥ ♥t♦♥ s♥♦t ♥ ♥ rt ♦rr t♥ t ♠♦ts ♦ t♦ ♥ ♠t strt♦♥ ♥t♦♥ t ts ♠t♦ s tr ♣♦t♥t ts♦r tt♥t♦♥ s t♦ ♣ t ss♠s rr ♣♥♥s t♥ t
♦♥s♦♥
♠♦ts ♦ t♦ rs ts st♠t♦♥ ♦ s♦ ♦♥ t st♠t♦♥ ♦rt♠♥ ts stt② ♦ t♦ st
♦♥s♦♥ ♦ Prt
❲ s♥ tt t s ♠♣♦rt♥t t♦ ♣r♦r♠ t str ♥②ss ♥ t ♥t s♣ ♦ t rs ♥ ♦rr t♦ ♣r♦ ♠♥♥ rsts ♥ t♠t♦s ♦♥ ♠①tr ♠♦s s r♥t t srs r♦♠ ♦ ♠trtstrt♦♥s ♦r ♠① t
ss♠♣t♦♥ ♦ t ♦♥t♦♥ ♥♣♥♥ t♥ t rs s ♠♥♥ ♠♦ s♥ t ♣r♦s ss strt♦♥s ♦r t ♦♥♠♥s♦♥♠r♥s ♦ t ♦♠♣♦♥♥ts s ♠♦ s r♥t s♣② ♥ t s♠♣ s③ ss♠ ♦r♥ t♦ t ♥♠r ♦ rs ♥ ♥ s s t ♥♦r♠t♦♥♦♥ t ♥trss ♣♥♥② s ♥♦t ♣rs♥t ♥ t t st ♦r t ♥ ♥ssr② t♦ r① t ♦♥t♦♥ ♥♣♥♥ ss♠♣t♦♥
♠①tr ♦ ♦t♦♥ ♠♦s ♥ ts ①t♥s♦♥ ♣r ♦s s s♦ ♥ tr♥t t♦ t ♦♥t♦♥ ♥♣♥♥t ♠♦ ♦t tt ts ①t♥s♦♥ ♣r ♦s♣♣rs t♦ ♠♦r ♥t s♥ t ♥♠r ♦ ♣r♠trs st②s ♠t ♦rt ♠♦ ♥tr♣rtt♦♥ ♥ t t♦ ♣r♦r♠ ② t ♣rtt♦♥r s♥ t♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥ts r ♥♦t ss ♦r t♦♥t♥♦s rs
s♦♥ tr♥t ♦♥ssts ♥ t ♥r♥ ss♥ ♠①tr ♠♦ ♣♣rs s ♠♦r ♠♥♥ ♦r ts ♠t♦ s ♦r t ♣r♠tr st♠t♦♥r♠t② ♠ts t ♥♠r ♦ srt rs
♥ ts ♦♥t①t t♦ ♠♥ ♦ts ♣♣r t♦ s s r t ♠♦ ♠stt♦ ♣r♦ ss ♦♥♠♥s♦♥ ♠r♥ strt♦♥s ♦r ts ♦♠♣♦♥♥ts ♥ t♠st ♣r♦ ♠♥♥ ♦♥ts rt♥ t ♥trss ♣♥♥s s♦♥ ♦t ♦ts ♣r♦♣♦s t♦ ♠①tr ♠♦s
rst ♠♦ ♦s t♦ ♣r♦r♠ t str ♥②ss ♦ t sts t ♦♥t♥♦s♥ t♦r rs t rs r♦♠ t ♠t t♥t ss ♠♦ ♦♣ ♦r ♥trss ♣♥♥t t♦r rs ♥ t ♦♠♣♦♥♥ts ♦ ts♠♦ r ♦♠♣♦s t ss♥ ♥ ② ♦st strt♦♥s s♣t②♦ ♦r ♣♣r♦ s t♦ s♠t♥♦s② ♣r♦r♠ t ♠♦ st♦♥ ♥ t ♣r♠trst♠t♦♥ ♥ ♠ ♦rt♠
s♦♥ ♠♦ s ♠①tr ♦ ss♥ ♦♣s s ♠♦ s r② ♥rs♥ t ♣r♦r♠s t str ♥②ss ♦ ♠① t sts t rs ♠tt♥ ♠t strt♦♥ ♥t♦♥s ♦r♦r t ♣r♦s s♦♠ s③t♦♥ t♦♦s t♦s♠♠r③ t ♥trss ♣♥♥s ♥ t♦ s♣② t ♥s ♦r t♠♦ ♦♠♣①t② ♥rss t t ♥♠r ♦ rs ♥ ♦r t ♦♠♦sstrs♦♥ s ts ♠♦ ♣♣rs s ♥♣♣r♦♣rt ♦r t sts t r ♥♠r♦ rs ♦ ♠♦r ♣rs♠♦♥♦s rs♦♥s ♦ ♦♥sr t♦ str s tsts
♥r ♦♥s♦♥ ♥ ♣rs♣ts
♦♥s♦♥
♥ ts tss ♥ ♥trst ♥ t str ♥②ss ♦ ♦♠♣① t♦r ♣rs② ♦s ♦♥ t t♦r ♥ ♠① t sts ♦t s t♦ ♥tr♦ ♠♦s ♣♣r♦s ♥ ♦rr t♦ str s t ②♠♦③♥ t ♥trss ♣♥♥s ♦r♦r ts ♠♦s t♦ s♠♠r③t t strt♦♥ ② ♣r♠trs t♦ tt t ♥tr♣rtt♦♥
♦ ♠♦s ♥ ♣rs♥t t♦ ♣r♦r♠ t str ♥②ss ♦ t♦rt sts ♠♥ s t♦ r♦♣ t rs ♥t♦ ♦♥t♦♥② ♥♣♥♥t♦s ♥ t♦ ♣t ♣rs♠♦♥♦s strt♦♥ ♦r ♦ ♦♠♥t♦r♣r♦♠s r qt♦s ♥ t t♦r t sts t ♥trss ♣♥♥sr ♥②③ ♦ t ♣rs♥t ♠♦s sr r♦♠ ts ♣r♦♠ r♥ t ♠♦st♦♥ st♣ ♥ t♦ ②s♥ ♣♣r♦s r ts r t s ♥♦trst t♦ ♣r♦r♠ t ts ♠♦s t str ♥②ss ♦ t st t ♦t♦ rs ♦r ♥ t rs r ♦r♥ ♦r ♥r② ♣♦ss ♥srt♦ ts ♣r♦♠ ♥ ♥ ② t ♠♦s ♦♣s
♠①tr ♠♦ ♦ ss♥ ♦♣s s ♥ ♥tr♦ t♦ str ♠①t sts s ♠♦ ♣r♠ts t♦ ♦t♥ ss ♦♥♠♥s♦♥ ♠r♥s ♦r ♦♠♣♦♥♥ts ♥ t♦ ♠♦③ t ♥trss ♣♥♥s ♥r ♠①tr♠♦ ♦ ss♥ ♦♣s ♦s ♥♦t sr r♦♠ ♦♠♥t♦r ♣r♦♠s t♦ ♣r♦r♠t ♠♦ st♦♥ s ts ♠♦ ♥ ♥ ♥t tr♥t t♦ t ♠♦s♣ t♦ t t♦r t ♥ t rs r ♥r② ♦r ♦r♥ ♥ t♦s t ♦♠♥t♦r ♣r♦♠s ♦ t ♠♦ st♦♥
Prs♣ts
r♦♦t ts tss ♥ ss ② t st ♦ t ♥sr♥ ② t s♠ strt♦♥ ♦r tr♥t ♥t♦♥s ♦ ss ♦ s ❬+ ♥❪
♠♦s ♥ ♥tr♦ ♥ str♥ r♠♦r ♦s② t②♥ s ♥ s♠s♣rs ♦r ♥ sst♦♥ ♦♥t①t ♦r ♦♥ ♥①♣t tt ts ♠♦s ♦t♣r♦r♠ t sr♠♥♥t ♣♣r♦s t ♦strrss♦♥ ♦♥② ♥ ♥s r ♥ tr ♦t s ♠♦r
♣tr
♠t♦s t♥ t sr♠♥♥t ♣♣r♦s s♥ t② ♠♦③ t t strt♦♥ t sr♠♥♥t ♣♣r♦s ♦s ♦♥ t ♦♥rs t♥ sss
♠♦s ♥tr♦ ♥ ts tss ♦ ♠♥ t sts t ♠ss♥ s♥ tr st♠t♦♥ ♥ ♣r♦r♠ ② ♥ ♦rt♠ ♦r ② s s♠♣r r ♥♦♥ t♦ ♠♥ s t
♠♦s st♠t ♦r♥ ♠tr① ♥ rqr r ♥♠r ♦♣r♠trs ♦ t s ♠♣♦rt♥t t♦ ♥tr♦ s♦♠ ♣rs♠♦♥♦s rs♦♥s ♦ t♠①tr ♠♦ ♦ ss♥ ♦♣s ♥ ♦rr t♦ ♠♥ t sts t r ♥♠r♦ rs s ♦♥ t ♦♠tr ♣♣r♦s s ♦r t ss♥ ♠①tr♠♦s ❬❪ s♦♠ ♦♥str♥s ♦ ♦♥ t ♦rrt♦♥ ♠tr① ♦ tss♥ ♦♣s ♦r ts ♣♣r♦ ♦ ♠ t st♠t♦♥ rr tr♣rs♠♦♥♦s ♣♣r♦s ♥rt r♦♠ t ss♥ r♠♦r ♦ s♦ s♦r ♥st♥ t ♠♦s ♦r r t sts ❬❪ ♥② ♥♦tr rsr ①s♦ ♦♥sst ♥ ♥r③t♦♥ ♦ t ♠①tr ♠♦ ♦ ♣♥♥② trs ♥t ♦♣s ♥ t rt strt♦♥ ♦r ♥② ♦♣s ♦ ♠① rs t② ♠t ♦r♦r t ♠♦ st♦♥ st♣ ♦ ts ♠t♦ s ss♥② ts ♣♣r♦ ♦ ♦ ♥ ♥r♥ ② ♠①♠③t♦♥ ♦ t ♦♦② s♥ ♠t♦s ♥rt r♦♠ ❬❪ s ♥♦ ♣r♦r ♥♦r♠t♦♥ ♦ t ts ♠t♦ ♦ ♥r♥
♦rrt♦♥ ♦♥t ♦ t ss♥ ♦♣ s ♦♦ ♣r♦♣rts ♥ t♠r♥ strt♦♥s r st♠t ♦r st t ♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥ts ♦r t ♠①tr ♠♦ ♦ ss♥ ♦♣s s s♠♣r♠tr ♣♣r♦ ♦r ♥st♥ s ♦♥ t ♦rs ♦ ❬♦ ❲❪s♥ t ♣r♦♣rts ♦ t ♦rrt♦♥ ♦♥t ♦ s②♠♣t♦t② r♥t
♥② t ♠①tr ♦ t t♦ ①tr♠ ♣♥♥② strt♦♥s ♦ ♥tr♥t t♦ t ss♥ ♦♣s s ♠♦ ♥ ♥ ② s♥ ♦♣s♥ t ♠①♠♠ ♣♥♥② strt♦♥ ♦ ♥ s t strt♦♥ tt♥s t rét♦♥ ♣♣r ♦♥ ② ♥ s♦♠ ♦♥str♥ts t strtr ♥ tr t ♠♦ st♦♥ ♦ s② ♣r♦r♠
t rts t♦ st ②♦ rt ②♦ ♥r ♦♦ ♠
♥ ♦ tr ♥ s♦t s ♥ ♦ ♥ts tr t♦
s s t ♥ ♦♦rs ♥
♣♣♥①
♣♣♥① ♦ Prt
♥r ♥tt② ♦ t ♠①tr ♦ t t♦
①tr♠ ♣♥♥② strt♦♥s
♦ strt♦♥ s ♥r② ♥t ♥ t ♦ ♦♥t♥s t sttr rs ♦r ♥ t ♦ ♦♥t♥s t st t♦ rs ♥ t st tr♠♦ts ♦ ♣r♦ ts ♣r♦♣♦♣rt② rst② s♦ t ♥r ♥tt② ♦t ♠♦ ♥ ♦t ♦ t ♦♦♥ s♠♣ ss t♦ rs t tr ♠♦ts♥ tr ♥r② rs ♥ ♦♥ t♦ t ♥r ♥tt② ♦ t♠♦
Pr♦♣♦st♦♥ ♦ rs t tr ♠♦ts ♠①tr ♠♦ ♦t t♦ ①tr♠ ♣♥♥② strt♦♥s s ♥r② ♥t ♥ dkb = 2m
kb1 = m
kb2 = 3
Pr♦♦ ♣♣♦s tt tr ①stsαkj = (ρkj, ξkb, τ kb, δkb) ♥ αkj = (ρkj, ξkb, τ kb, δkb)s s
∀xkbi p(x
kbi ;αkb) = p(x
kbi ; αkb).
❲ ♠♦♥strt tt ts qt② ♥♦s tt αkj = αkj ♠♦♥strt♦♥s s♣t ♥ tr ♣rts r tr♠♥ ② t tr ♣♦ssts ♦ (δkb, δkb)qt② ♦♥ rt♦♥ q ♦r ♦t ♣r♠trs ♥♦ rt♦♥ q ♦r ♦t ♣r♠trs ❲ s♦ tt ♥♦s t qt② t♥ t ♣♥♥② rt♦♥s δkb = δkb ♥ t♥ t ♦♥t♥♦s ♣r♠trs s ♥♦sαkb = αkb
• qt② ♦ t ♣♥♥② rt♦♥s δkb = δkb❲t♦t ♦ss ♦ ♥rt② ss♠ tt
∀h, h′ ∈ 1, . . . , 3, h 6= h′ : δh2hkb = 1 ♥ δh2h′
kb = 0.
♥ t rt♦♥ ♥ ② s t♦ t ♦♦♥ s②st♠ ♦ ♥♥ qt♦♥s♦r h ∈ 1, . . . , 3 ♥ h′ ∈ 1, . . . , 3 \ h
(1− ρkb)ξ1hkb ξ
2hkb + ρkbτ
hkb = (1− ρkb)ξ
1hkb ξ
2hkb + ρτhkb
(1− ρkb)ξ1hkb ξ
2h′
kb = (1− ρkb)ξ1hkb ξ
2h′
kb .
♣♣♥① ♣♣♥① ♦ Prt
❲ s t s♦♥ ♥ ♦ t ♣r♦s s②st♠ t t ♦♦♥ s ♦ t ♦♣(h, h′) ♥ s ♦t♥ tt
ξ11kbξ12kb
=ξ11kbξ12kb
♥ξ11kbξ13kb
=ξ11kbξ13kb
.
♦ ξ11kb = ξ11kbξ12kbξ12kb
= ξ11kbξ13kbξ13kb
r s ♥tr♣t ε ∈ R+ s tt ε = ξ12kb
ξ12kb=
ξ13kbξ13kb
❲
r♠♥ tt∑3
h=1 ξ1hkb =
∑3h=1 ξ
1hkb = 1 ♦r♦r
3∑
h=1
ξ1hkb = ξ11kbε+ ξ12kbε+ ξ13kbε = ε.
♦ ε = 1 ❲ ♦♥ tt ξ1hkb = ξ1hkb s♠ rs♦♥♥ s s t♦ ♦t♥tt ξ2hkb = ξ2hkb r♦♠ ts ♦t♥ t qt② t♥ ρkb = ρkb ♥ τhkb = τhkb♥② ♦t♥ tt αkb = αkb
• ♥② ♦♥ rt♦♥ s q t♥ ♦t ♣r♠tr③t♦♥s❲t♦t ♦ss ♦ ♥rt② ss♠ tt δ121kb = δ222kb = δ323kb = 1 ♥ δh2h
′
kb = 0♦trs δ122kb = δ221kb = δ323kb = 1 ♥ δh2h
′
kb = 0r♦♠ t s②st♠ ♦ ♥♥ qt♦♥s ♥ ② ①trt t ♦♦♥ s②st♠
(1− ρkb)ξ13kbξ
21kb = (1− ρkb)ξ
13kb ξ
21kb
(1− ρkb)ξ13kbξ
22kb = (1− ρkb)ξ
13kb ξ
22kb
(1− ρkb)ξ11kbξ
21kb + ρkbτ
1kb = (1− ρkb)ξ
11kb ξ
21kb
(1− ρkb)ξ11kbξ
22kb = (1− ρkb)ξ
11kb ξ
22kb + ρkbτ
1kb.
r♦♠ t rst t♦ ♥s ♦ t ♣r♦s qt♦♥ tt ξ22kb = ξ21kbξ22kbξ21kb
❲
♦♥sr t st t♦ ♥s r ξ22kb s r♣ ② ξ21kbξ22kbξ21kb
♥ r t st ♥ s
♠t♣ ② ξ21kbξ22kb
s
(1− ρkb)ξ11kbξ
21kb + ρkbτ
1kb = (1− ρkb)ξ
11kb ξ
21kb
(1− ρkb)ξ11kbξ
21kb = (1− ρkb)ξ
11kb ξ
21kb + ρkbτ
1kbξ21kbξ22kb.
s ρkbτ 1kb+ ρkbτ1kbξ21kbξ22kb
= 0 s rst s ♥ ♦♥trt♦♥ t t strt ♣♦stt②
♦ t tr♠s ♦ t s ♥♦t ♣♦ss t♦ rs♣t ♥ ♦♥② ♦♥ rt♦♥ sq t♥ ♦t ♣r♠tr③t♦♥s
• ♦ rt♦♥ q t♥ ♦t ♣r♠tr③t♦♥s
♥r ♥tt② ♦ t ♠①tr ♠♦ ♦ ♠t♥♦♠ strt♦♥s ♣r
♠♦s
❲t♦t ♦ss ♦ ♥rt② ♦♥sr t ♦♦♥ s②st♠
(1− ρkb)ξ11kbξ
21kb + ρkbτ
1kb = (1− ρkb)ξ
11kb ξ
21kb
(1− ρkb)ξ12kbξ
22kb + ρkbτ
2kb = (1− ρkb)ξ
12kb ξ
22kb
(1− ρkb)ξ13kbξ
23kb + ρkbτ
3kb = (1− ρkb)ξ
13kb ξ
23kb
(1− ρkb)ξ12kbξ
21kb = (1− ρkb)ξ
12kb ξ
21kb + ρkbτ
2kb
(1− ρkb)ξ13kbξ
22kb = (1− ρkb)ξ
13kb ξ
22kb + ρkbτ
3kb
(1− ρkb)ξ11kbξ
23kb = (1− ρkb)ξ
11kb ξ
23kb + ρkbτ
1kb
(1− ρkb)ξ11kbξ
22kb = (1− ρkb)ξ
11kb ξ
22kb
(1− ρkb)ξ12kbξ
23kb = (1− ρkb)ξ
12kb ξ
23kb
(1− ρkb)ξ13kbξ
21kb = (1− ρkb)ξ
13kb ξ
21kb
r♦♠ t ♥s ♥ ♦t♥ tt ξ11kbξ12kb
<ξ11kbξ12kb
r♦♠ t ♥s ♥ ♦t♥
tt ξ11kbξ12kb
>ξ11kbξ12kb
♦ t s ♥♦t ♣♦ss t♦ rs♣t ♥ ♥♦ rt♦♥ s q
t♥ ♦t ♣r♠tr③t♦♥s
Pr♦♣♦st♦♥ r ♥r② rs ♠①tr ♠♦ ♦ t t♦ ①tr♠
♣♥♥② strt♦♥s s ♥r② ♥t ♥ dkb = 3 mkb1 = m
kb2 =
mkb3 = 2
Pr♦♦ ♣♣♦s tt tr ①stαkj = (ρkj, ξkb, τ kb, δkb) ♥ αkj = (ρkj, ξkb, τ kb, δkb)s s
∀xkbi p(x
kbi ;αkb) = p(x
kbi ; αkb).
② rt♥ t s②st♠ t qt♦♥s rt t♦ ♦t♥ tt ∀j =1, . . . , 3 : ξj1kb(1 − ξj1kb) = (1 − ξj1kb)ξ
j1kb s ∀j = 1, . . . , 3 : ξj1kb = ξj1kb ❲
strt♦rr② ♦t♥ t qt② t♥ t ♦trs ♣r♠trs s♦ αkb = αkb
♦♥s♦♥ ♠①tr ♠♦ s st ② s♦♥ ♦ ♠♦ts ♥♦r rs♦ ♦t♥ t ♥r ♥tt② ♦ t ♠♦s ♥ rtt♥ ②s♦♥ ♦ ♠♦ts ♥♦r rs s ♦♥ ♦ t ♦♦♥ ♠♦s t tr ♥r②♦♥ ♥ t t♦ tr♠♦ts ♦♥
♥r ♥tt② ♦ t ♠①tr ♠♦ ♦
♠t♥♦♠ strt♦♥s ♣r ♠♦s
♥r ♥tt② ♦ t ♠♠ ♠♦ t tr ♦s t k0 =r♠♥
kℓkb ♥ t ♠tr① Mb r
Mb(k, h) = ατk0b(h)
kb .
② ♥♦t♥ ② ξb = mink
ℓkb + 1 ♥r②
r♥K Mb = min(g, ξb).
♣♣♥① ♣♣♥① ♦ Prt
♦r♦r② ♣r♠trs ♦ t ♠♠ ♠♦ t tr ♦s r ♥r②♥t ♣ t♦ s♣♣♥ ♣r♦
min(g, ξ1) + min(g, ξ2) + min(g, ξ3) ≥ 2g + 2.
♥r ♥tt② ♦ t ♠♠ ♠♦ t ♠♦r t♥ tr ♦s ♥t s♠ ② tt ❬❪ ♥r③ t rst t ♦s ② ♦sr♥ tt ♦s ♦ t♦r rs ♥ ♦♠♥ ♥t♦ tr t♦r rss ♥ ♣♣② t rs t♦r♠♦r♦r② ❲ ♦♥sr ♠♠ ♠♦ t ♦s r ≥ 3 tr ①sts tr♣rtt♦♥ ♦ t st 1, . . . , ♥t♦ tr s♦♥t ♥♦♥ ♠♣t② ssts S1 S2 ♥S3 s tt γi =
∏
j∈Siξj t
min(g, γ1) + min(g, γ2) + min(g, γ3) ≥ 2g + 2,
t♥ t ♠♦ ♣r♠trs r ♥r② ♥t ♣ t♦ s♣♣♥
♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt
♦♦ ♦ t ♠①tr ♠♦ ♦ ♠t♥♦♠
strt♦♥s ♣r ♠♦s
♥ ts t♦♥ ♣r♦♦ ♦ Pr♦♣♦st♦♥ s ♥ ❲ rst② ♥ ♥♣r♠tr③t♦♥ ♦ t ♦ strt♦♥ tt♥ t ♥trt ♦♠♣tt♦♦ ♦♠♣tt♦♥ ❲ s♦♥② ♥ t ♣r♦r strt♦♥ ♦ t ♥ ♦♣r♠tr③t♦♥ ♦r♥ t♦ t ♦tr ♣r♠tr③t♦♥ r② ♥r♥ trt♦♥ t♥ t ♠ ♠♦s ❲ ♦♥ ② t ♥trt ♦♠♣tt♦♦ ♦♠♣tt♦♥ s t trt rst
♣r♠tr③t♦♥ ♦ t ♦ strt♦♥
❲t♦t ♦ss ♦ ♥rt② ss♠ tt t ♠♥ts ♦ δkb r ♦rr ② rs♥ s ♦ t ♣r♦t② ♠ss ss♦t t♦ t♠ ♥ ♥tr♦ t ♥
♣r♠tr③t♦♥ ♦ akb ♥♦t εkb r εkb ∈ Ekb =[
1mb ; 1
]
×, . . . ,×[
1♠b−ℓkb
; 1]
♥ r εkbh s ♥ ②
εhkb =
aδkbhkb h = 1aδkbhkb∏h−1
h′=1(1−εh
′kb)
♦trs.
♠♠ ♦♥t♦♥ ♣r♦t② ♦ ①b s
p(xb|z, ℓkb, δkb, εkb) =ℓkb∏
h=1
(εhkb)n(h)kb (1− εhkb)
nhkb ,
♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt ♦♦ ♦ t ♠①tr ♠♦
♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s
Pr♦♦
p(xb|z, ℓkb, δkb, εkb) = p(xb|z, ℓkb,αkb)
=♠b∏
h=1
(αhkb)♥hkb
=
ℓkj∏
h=1
(α(h)kb )
♥(h)kb
(
α(ℓkb+1)kb
)♥ℓkbkb
= (ε1kb)n(1)kb
ℓkb∏
h=2
[
(εhkb)♥(h)kb
(
h−1∏
h′=1
(1− εhkb)n(h)kb
)]
ℓkb∏
h=1
(1− εhkb)nℓkbkb
=
ℓkb∏
h=1
(εhkb)n(h)kb (1− εhkb)
nhkb .
Pr♦r strt♦♥
♠♠ ♣r♦r strt♦♥ ♦ εkb s
p(εkb|ω, δkb) =♠b
♠b − ℓkb.
Pr♦♦ ❲ r♠♥ tt akb|ω ∼ Dtℓkb+1
(
1, . . . , 1;♠b)
♥ tt
p(akb, δkb|ω) = p(α|ω) = p(εkb, δkb|ω).
♦ t ♣ ♦ t ♣r♦r strt♦♥ ♦ εkb
p(εkb|δkb,ω) =
∏ℓkbh=1(ε
hkb)
γhkb−1(1− εhkb)∑ℓkb+1
h′=h+1(γh
′
kb−1)
∫
εkb∈Ekb
∏ℓkbh=1(ε
hkb)
γhkb−1(1− εhkb)∑ℓkb+1
h′=h+1(γh
′kb−1)dεkb
.
s εhkb ♦♦s tr♥t t strt♦♥ ♦♥ t ♣r♠trs s♣[
1♠b−h+1
, 1]
♥♦t ② Be(γhkb,∑ℓkb+1
h′=h+1(γh′
kb−1)+1) ♦ ssr t ♣♦stt② ♦ t ♣r♠trs♦ t tr♥t t strt♦♥s ♣t γhkb = 1 s♦
p(εkb|δkb,ω) =♠b
♠b − ℓkb.
t♦♥ t♥ ♠ ♠♦s
♠♠ t t ♠♦ t ℓ⊖kb ♠♦s ♥ t ♣r♠trs (δ⊖
kb, ε⊖kb) ♥ t t
♠♦ t ℓkb ♠♦s ♥ t ♣r♠trs (δkb, εkb) ♦t ♠♦s r ♥ s s
♣♣♥① ♣♣♥① ♦ Prt
tt ℓ⊖kb = ℓkb − 1 tt t ℓ⊖kb ♠♦s ♥ t rst ♣r♦ts t s♠♦t♦♥s ∀h ∈ δ⊖
kb, h ∈ δkb ♥ t s♠ ♣r♦t② ♠sss (ε⊖hkb = εhkb, h < ℓkb)s ♠ ♠♦s ♦♦ ts rt♦♥
p(xb|z, ℓkb, δkb, εkb)p(xb|z, ℓ⊖kb, δ
⊖
kb, ε⊖kb)
=(♠b − ℓkb + 1)♥
ℓkb−1
kb −1
(♠b − ℓkb)♥ℓkbkb
(εℓkb)♥(ℓkb)
kb (1− εℓkb)♥ℓkbkb .
Pr♦♦ ❲ strt ② t ♦♦♥ rt♦♥
p(xb|z, ℓkb,αkb)
p(xb|z, ℓ⊖kb,α⊖kb)
=(αℓkbkb )
♥(ℓkb)
kb (αℓkb+1kb )♥
ℓkbkb
(α⊖ℓkbkb )♥
ℓkb−1
kb
.
♦t tt εhkb = ε⊖hkb ♥ h = 1, . . . , ℓkb − 1 s♥ α(h)kb = α
⊖(h)kb ♥ τℓkb(h) =
τℓkb−1(h) ♥ h = 1, . . . , ℓkb − 1 ♥ ② s♥ t r♣r♠tr③t♦♥ ♥ εkbt ♣r♦♦ s ♦♠♣t
♥trt ♦♠♣tt ♦♦
♥trt ♦♠♣tt ♦♦ s ♥② ♣♣r♦①♠t ② ♥t♥t s♠ ♦r t srt ♣r♠trs ♦ t ♠♦s ♦t♦♥s ♥ ② ♣r♦r♠♥ t①t ♦♠♣tt♦♥ ♦♥ t ♦♥t♥♦s ♣r♠trs ②
p(xb|z, ℓkb) ≈(
1
mb − ℓkb
)♥ℓkbkb
ℓkb∏
h=1
Bi(
1♠b−h+1
; ♥(h)kb + 1; ♥hkb + 1
)
♠b − h,
r Bi(x; a, b) = B(1; a, b)−B(x; a, b) B(x; a, b) ♥ t ♥♦♠♣t t ♥t♦♥ ♥ ② B(x; a, b) =
∫ x
0wa(1 − w)bdw r♦♠ t ♣r♦s ①♣rss♦♥ ts s
strt♦rr t♦ ♦t♥ p(xb, z|ω)
Pr♦♦ ♦ Pr♦♣♦st♦♥ ♦r t ♠♦ t ℓkb − 1 ♠♦s t st ♠♦s♦t♦♥s r ♥♦♥ ♥ ♥ ② δ
⊖
kb t♥ t ♦♥t♦♥ ♣r♦t② ♦ xb ♦r ♠♦ t ℓkb ♠♦s s
p(xb|z, ℓkb, δ⊖
kb, εkb) =1
♠b − ℓkb + 1
∑
τ∈1,...,♠b\δ⊖kb
p(xb|z, ℓkb, δ⊖
kb, τ,α⊖kb, εkb),
s ② ♣♣r♦①♠t♥ ts s♠ ② ts ♠①♠♠ ♠♥t ♦t♥ tt
p(xb|z, ℓkb, δ⊖
kb, εkb) ≈1
♠b − ℓkb + 1p(xb|z, ℓkb, δkb,α⊖
kb, εkb).
② s♥ ♠♠ ♦t♥ tt
p(xb|z, ℓkb, δ⊖
kb, εkb)
p(xb|z, ℓ⊖kb, δ⊖
kb, ε⊖kb)
≈ (♠b − ℓkb + 1)♥ℓkb−1
kb −1
(♠b − ℓkb)♥ℓkbkb
(εℓkbkb )♥(ℓkb)
kb (1− εℓkbkb )♥ℓkbkb .
♦♠♣tt♦♥ ♦ t ♥trt ♦♠♣tt ♦♦ ♦ t ♠①tr ♠♦
♦ ♠t♥♦♠ strt♦♥s ♣r ♠♦s
s p(xb|z, ℓkb = 0) = (mb)−nk ② ♣♣②♥ rrs② t ♣r♦s ①♣rss♦♥ ♦t♥ tt
p(xb|z, ℓkb, εkb) ≈(
1
♠b − ℓkj
)♥ℓkbkb
ℓkb∏
h=1
(εhkb)♥(h)kb (1− εhkb)
♥hkb
♠b − h+ 1.
♣♣♥①
♣♣♥① ♦ Prt
♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥
♥ ♦st strt♦♥s
Pr♦♣♦st♦♥ ♠①tr ♠♦ ♦ ss♥ ♥ ♦st strt♦♥s s♥r② ♥t
Pr♦♦ ♣♣♦s tr r t♦ ♠①tr ♠♦s ♦ ss♥ ♥ ♦st strt♦♥s♥♦t ② p(xi;θ) ♥ p(xi; θ) s tt
∀xi,g∑
k=1
πkp(xi;αk) =
g∑
k=1
πkp(xi; αk), 0 < πk, πk ≤ 1,
g∑
k=1
πk =
g∑
k=1
πk = 1.
♠ s t♦ ♣r♦ tt θ = θ ♠♦♥strt♦♥ s s♣t ♥ t♦ ♣rts ♥ trst ♦♥ s♦ t qt② ♦ t ss♥ strt♦♥s ♣r♠trs ♥ ♦ t♣r♦♣♦rt♦♥s ♥ t s♦♥ ♦♥ s♦ t qt② ♦ t ♣r♠trs ♦ t ♦strrss♦♥s
♦♥t♥♦s ♣r♠trs ♥ ♣r♦♣♦rt♦♥s❲ s♠ qt♦♥ ♦r t ♣♦ss s ♦ x
i s♦ ♦t♥ tt
∀x
i ,
g∑
k=1
πkφ(x
i ;µk,Σk) =
g∑
k=1
πkφ(x
i ; µk, Σk), 0 < πk, πk ≤ 1,
g∑
k=1
πk =
g∑
k=1
πk = 1.
♥tt② ♦ t ♥t ss♥ ♠①trs ♠♦s s ❬❪ ♦r t♥rt s ♥ ❬❨❪ ♦r t ♠trt s ♥♦s tt g = gπk = πk µk = µk ♥ Σk = Σk
Pr♠trs ♦ t ♦st rrss♦♥st s r ❬❪ tt ∀j = 1 + c, . . . , e ∀(x
i ,xji )
g∑
k=1
fk(x
i )p(xji |x
i ;βkj) =
g∑
k=1
fk(x
i )p(xji |x
i ; βkj)
♣♣♥① ♣♣♥① ♦ Prt
♥♦s tt βkj = βkj r fk(x
i ) = πkφ(x
i ;µk,Σk) t♥ ♠①tr ♠♦♦ ss♥ ♥ ♦st strt♦♥s s ♥tt t t♦r ♦ s③ c ♥♦t ② yi = (y1, . . . , yc) r t ♠♥tsr ③r♦ ①♣t t ♠♥t j′ s q t♦ a ❲t♦t ♦ss ♦ ♥rt② ♦♥sr tt t fk(yi ) r ♦rr s tt Σ−1
k (j′, j′) < Σ−1k+1(j
′, j′)r♦♠ qt♦♥ tt
g∑
k=1
fk(y
i )α1(βkj|yi ) =g∑
k=1
fk(y
i )α1(βkj|yi ),
t(
α1(βkj|yi ))−1
= 1 +∑mj
h=2 exp(β0hkj + βj
′hkj a) ❲ t ♦
qt♦♥ ② f1(yi )α1(β1j|yi ) ts
1 +
g∑
k=2
fk(y
i )α1(βkj|yi )f1(yi )α1(β1j|yi )
=α1(β1j|yi )α1(β1j|yi )
+
g∑
k=2
fk(y
i )α1(βkj|yi )f1(yi )α1(β1j|yi )
.
tt♥ a → ∞∑g
k=2
fk(y
i )α1(βkj |y
i )
f1(yi )α1(β1j |y
i )= 0 ♥
∑gk=2
fk(y
i )α1(βkj |y
i )
f1(yi )α1(β1j |y
i )= 0 s♥
t fk(yi ) r ♦rr ❲t♦t ♦ss ♦ ♥rt② mj > 2 ss♠ ttβj
′h1j > βj
′h+11j 1 < h < mj
lima→∞
α1(β1j|yi )α1(β1j|yi )
= lima→∞
exp(
(βj′2kj − βj
′2kj )a+ (β02
kj − β02kj ))
= 1.
♦ qt♦♥ ♥♦s tt βj′2kj = βj
′2kj ♥ β02
kj = β02kj ② r♣t♥
ts r♠♥t ♦r h = 3, . . . ,mj t♥ ♦r j = 1, . . . , ♦♥ ttβ1j = β1j ② r♣t♥ ts r♠♥t ♦r j = 1 + , . . . , + t♥ ♦rk = 2, . . . , g ♦♥ tt qt♦♥ s tr t♥ θ = θ
♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥
♦♣s
♠♦ ♥tt② s ♣r♦ ② t♦ ♣r♦♣♦st♦♥s rst ♣r♦♣♦st♦♥♣r♦s t ♠♦ ♥tt② ♥ t rs r ♦♥t♥♦s ♥♦r ♥trs ♣r♦♣♦st♦♥ ♣rs♥ts t rs♦♥♥ ♥ s♠♣ s s♥ t ♦s ♥♦t ♦♥srt ♦r♥ rs s♦♥ ♣r♦♣♦st♦♥ ♣r♦s tt t ♠♦ rqrs t st♦♥ ♦♥t♥♦s ♦r ♥tr r t♦ ♥t
Pr♦♣♦st♦♥ ♥tt② t ♦♥t♥♦s ♥ ♥tr rs ♠①tr ♠♦ ♦ ss♥ ♦♣s s ② ♥t ❬❪ t rs r♦♥t♥♦s ♥ ♥tr ♦♥s t ♠r♥ strt♦♥s ♦ t ♦♠♣♦♥♥ts rss♥ ♦r P♦ss♦♥ strt♦♥s s
∀x ∈ Rc × N
d,
g∑
k=1
πkp(x;αk) =
g′∑
k=1
π′kp(x;α
′k)
⇒ g = g′, π = π′, α = α′.
♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s
Pr♦♦ ♥tt② ♦ t ♠trt ss♥ ♠①tr ♠♦s ♥ ♦ t♥rt P♦ss♦♥ ♠①tr ♠♦ ❬ ❨❪ ♥♦s tt ♠♣s
g = g′, π = π′, βkj = β′kj ♥ Γk = Γ
′k.
❲ ♥♦ s♦ tt Γk = Γ′k ♥ Γk = Γ
′k
t j ∈ 1, . . . , c ♥ h ∈ c + 1, . . . , e ❲ ♥♦t ② ρk = Γk(j, h)
ρ′k = Γ′k(j, h) vk = Φ−1
1 (P (xj;βkj)) εk(xj) = πk
φ1(vk)σkj
ak =b⊕k (xj)−ρkvk√
1−ρ2k♥
a′k =b⊕k (xj)−ρ′kvk√
1−ρ′2k ❲t♦t ♦ss ♦ ♥rt② ♦rr t ♦♠♣♦♥♥ts s s
σkj > σk+1j ♥ σkj = σk+1j t♥ µkj > µk+1j t♥ ♠♣s tt
1 +
g∑
k=2
(εk(xj)Φ(ak))/(ε1(x
j)Φ(a1)) =
g∑
k=1
εk(xj)Φ(a′k)/(ε1(x
j)Φ(a1)).
t γt = (xj, xh) ∈ R× N : a1 = t ♥ tt♥ xh → ∞ s s (xj, xh) ∈ γt
∀t,∫ a′1tφ(u)du
Φ(t)= 0.
s a′1 = a1 s♦ ρ′1 = ρ1 ♣t♥ ts r♠♥t ♦r k = 2, . . . , g ♥ ♦r t♦♣s (j, h) ♦♥ tt Γk = Γ
′k
❲♥ ♦t rs r ♥tr s t s♠ r♠♥t t γ(t,ξ) = (xj, xh) ∈N × N : a1 ∈ B(t, ξ) ♦t tt ρ1 6= ρ′1 t♥ ∃n0 s s ∀xj > n0 a
′1 > t + ξ
tt♥ xh → ∞ s s (xj, xh) ∈ γ(t,ξ) ♦t♥ t ♦♦♥ ♦♥trt♦♥
∫ a′1t+ξ
φ(u)du
Φ(t− ξ)= 0 ♥
∫ a′1t+ξ
φ(u)du
Φ(t− ξ)> 0.
♦ a′1 = a1 t♥ ρ1 = ρ′1 ♣t♥ ts r♠♥t ♦r k = 2, . . . , g ♥ ♦r t♦♣s (j, h) ♦♥ tt Γk = Γ
′k
Pr♦♣♦st♦♥ ♥tt② ♦ t ♠①tr ♠♦ ♦ ss♥ ♦♣s ♠①tr ♠♦ ♦ ss♥ ♦♣s s ② ♥t ❬❪ t st ♦♥ r s ♦♥t♥♦s ♦r ♥tr
Pr♦♦ ♥ ts ♣r♦♦ ♦♥sr ♦♥② ♦♥ ♦♥t♥♦s r ♥ t♦ ♥r② rs ♦s② t s♠ rs♦♥♥ ♥ ①t♥ t♦ t ♦tr ss ❲ ♥♦s♦ tt Γk = Γ
′k ♥ Γk = Γ
′k
t j = 1 ♥ t h ∈ 2, 3 ❲ ♥♦t ρk = Γk(j, h) ρ′k = Γ′k(j, h) vk =
Φ−11 (P (xj;βkj)) εk(x
j) = πkφ(vk;0,1)σkj
ak =b⊕k (xj)−ρkvk√
1−ρ2k♥ a′k =
b′⊕k (xj)−ρ′kvk√1−ρ′2k
❲t♦t
♦ss ♦ ♥rt② ♦rr t ♦♠♣♦♥♥ts s s σkj > σ[k+1]j ♥ σkj = σ[k+1]j
t♥ µkj > µ[k+1]j ♦t tt ♠♣s tt
1 +
g∑
k=2
(εk(xj)Φ(ak))/(ε1(x
j)Φ(a1)) =
g∑
k=1
εk(xj)Φ(a′k)/(ε1(x
j)Φ(a1)).
♣♣♥① ♣♣♥① ♦ Prt
tt♥ x1 → ∞ ♥ ss♠♥ tt ρk > 0 t♥ Φ(a′k)
Φ(ak)= 1 ♦ s♥(ρk) = s♥(ρ′k).
② ♥♦t♥ κ = lima→∞
φ(a)Φ(a)
♥ tt♥ x1 → ∞ κ 1κ
φ(a′k)
φ(ak)= 1. s a′1 = a1 s♦ ρ′1 = ρ1
♥ b⊕k (xj) = b′⊕k (xj) s♦ βkh = β′
kh♦t tt t s♠ rst ♥ ♦t♥ ② t♥♥ x1 t♦ −∞ s ρk < 0
♣t♥ ts r♠♥t ♦r k = 2, . . . , g ♥ ♦r t ♦♣s (j, h) ♦♥tt Γk = Γ
′k t♥ Γk = Γ
′k
♦r♣②
❬❪ ③③♥ ♥ ❲ ♦♠♥ ♦♦ t s♦♠ t ♦♥ t t②sr ♣♣ ttsts
❬❪ P ♥rs♥ ♦r♥ ♥ ♥ ttst ♠♦s s ♦♥ ♦♥t♥ ♣r♦sss ♣r♥r rs ♥ ttsts ♣r♥r❱r ❨♦r
❬❪ ♠ ♥ ② ♠♥ str♥ ♦rt♠ ♦r ♠① ♥♠r♥ t♦r t t ♥♦ ♥♥r♥
❬r❪ rst t♦r t ♥②ss ♦♠ ♦♥ ❲② ♦♥s
❬❪ ♥♦r♠t♦♥ t♦r② ♥ ♥ ①t♥s♦♥ ♦ t ♠①♠♠♦♦ ♣r♥♣ ♥ ♦♥ ♥tr♥t♦♥ ②♠♣♦s♠ ♦♥ ♥♦r♠t♦♥ ♦r② ss♦r ♣s é♠ ó ♣st
❬❪ ♠♥ ts ♥ ♦s ♥tt② ♦ ♣r♠trs♥ t♥t strtr ♠♦s t ♠♥② ♦sr rs ♥♥s♦ ttsts
❬❱❪ rtr ♥ ❱ssts ♠♥s ♥ts ♦ rs♥ ♥ Pr♦♥s ♦ t t♥t ♥♥ s②♠♣♦s♠ ♦♥ srt ♦rt♠s ♣s ♦t② ♦r ♥str♥ ♣♣ t♠ts
❬❪ r ♣r♦st str♥ ♠♦ ♦r rs ♦ ♠① t②♣t② ♥ ♥tt②
❬❪ ♦②r♦♥ ♥ r♥t ♦s str♥ ♦ ♠♥s♦♥ t r ♦♠♣tt♦♥ ttsts ♥ t ♥②ss
❬❪ r♥ ① ♥ ♦rt ssss♥ ♠①tr ♠♦ ♦rstr♥ t t ♥trt ♦♠♣t ♦♦ Pttr♥ ♥②ss♥ ♥ ♥t♥ r♥st♦♥s ♦♥
♦r♣②
❬❪ r♥ ① ♥ ♦rt ①t ♥ ♦♥t r♦ t♦♥s ♦ ♥trt ♦♦s ♦r t t♥t ss ♠♦ ♦r♥ ♦ttst P♥♥♥ ♥ ♥r♥
❬❪ ♥ ♥ ♥tr ♥ ♦rt♠ ♦rs♠ ♥ ♥♦♥♣r♠tr st♠t♦♥ ♥ ♠trt ♠①trs ♦r♥♦ ♦♠♣tt♦♥ ♥ r♣ ttsts
❬❨❪ ♥ ♥tr ♥ ❨♦♥ ♠①t♦♦s ♥ ♣ ♦r ♥②③♥ ♥t ♠①tr ♠♦s ♦r♥ ♦ ttst♦tr
❬r❪ P r♥ sr② ♦ str♥ t ♠♥♥ t♥qs ♥ r♦♣♥♠t♠♥s♦♥ t ♣s ♣r♥r
❬❪ ♦♥♥♥s rt ♠r ♥ st③ ♠r ♣t♠③t♦♥ t♦rt ♥ ♣rt s♣ts
❬❪ r♥ ♥r② ♥ t ♠①♠♠ ♦♦ st♠t♦♥ ♦ ♥rt ss♥ ♠①trs ♦r r♦♣ t ♥ ♦r ♦ t ♦rt♠ ♥♥♥ ♦r♥ ♦ ttsts
❬❪ r♥ ♥ ♦r♠ ss♥ Prs♠♦♥♦s str♥ ♦s ♥r♥t ♥ t ② Pr♦t♦♥ ttsts ♥ ♦♠♣t♥♣ ♥ ♣rss
❬❪ r♥r ♦ ♥ ❳ ♥ ♦♥ ♦r♥ ♠trs ♥ tr♠s ♦ st♥r t♦♥s ♥ ♦rrt♦♥s t ♣♣t♦♥t♦ sr♥ ttst ♥
❬♦③❪ ♦③♦♥ ♦ st♦♥ ♥ s ♥♦r♠t♦♥ rtr♦♥ t ♥r t♦r② ♥ ts ♥②t ①t♥s♦♥s Ps②♦♠tr
❬❪ ♥ ♥ tr② ♦s ss♥ ♥ ♥♦♥ss♥ str♥ ♦♠trs ♣s
❬❪ r♥t ♥ ♦♥ rt♦♥ ♦ t ♦rt♠ Prss ♣s♦♥ ♦rt♠ ♦♠♣tt♦♥ ttsts ♥ t ♥②ss
❬+❪ P r② tr② ① ♦ ♥ ♦ttr♦ ♦♠♥♥ ♠①tr ♦♠♣♦♥♥ts ♦r str♥ ♦r♥ ♦ ♦♠♣tt♦♥♥ r♣ ttsts
❬r❪ ❱ rt♥♦ Prs♦♥ ♦♠♠♥t♦♥ s♦r s♠
❬+❪ P ♦rt③ rr ♠ t♦s ♥ s ♦♥ ♥ ♣rr♥s ② t ♠♥♥ r♦♠ ♣②s♦♠ ♣r♦♣rtss♦♥ ♣♣♦rt ②st♠s
♦r♣②
❬+❪ ① ♦t t ♥ ♦rt♠s ♦r ♠①trsttst ♥ ♥♠r s♣ts ♣♣♦rt r ♥r
❬❪ ① ♥ ♦t st♦st ♣♣r♦①♠t♦♥ t②♣ ♦rt♠ ♦r t ♠①tr ♣r♦♠ t♦sts ♥ ♥tr♥t♦♥ ♦r♥♦ Pr♦t② ♥ t♦st Pr♦sss
❬❪ ① ♥ ♦rt str♥ rtr ♦r srt t ♥t♥t ss ♠♦s ♦r♥ ♦ sst♦♥
❬❪ ① ♥ ♦rt sst♦♥ ♦rt♠ ♦r str♥♥ t♦ st♦st rs♦♥s ♦♠♣tt♦♥ ttsts t ♥②ss
❬❪ ① ♥ ♦rt ss♥ ♣rs♠♦♥♦s str♥ ♠♦sPttr♥ ♦♥t♦♥
❬❪ ♥tr ♥ ♥ st♠t♦♥ ♦r ♦♥t♦♥♥♣♥♥ ♠trt ♥t ♠①tr ♠♦s
❬❪ ♥t ❱ ♥t③♠♦♥t ♥ r♦ rt♦♦♥ r♦tt♦♥ ♥P❳ ♥s ♥ t ♥②ss ♥ sst♦♥
❬❪ ♦ ♥ ♣♣r♦①♠t♥ srt ♣r♦t② strt♦♥st ♣♥♥ trs ♥♦r♠t♦♥ ♦r② r♥st♦♥s ♦♥
❬❪ P ♦rt③ ♥ ♦rs t ♠♥♥ ♣♣r♦ t♦ ♣rt ♦rst rss♥ ♠t♦r♦♦ t
❬❪ ♦rt ♥ r st♠t♦♥ ♥ srt ♣r♠tr ♠♦s ttst ♥
❬❩+❪ ♣ ö♦♣ ❩♥ t ♠s♣rs r♥♥♦♠ ♣rss ♠r
❬❩❪ ③r♥ ♥ ❩r③② ♣♣t♦♥ ♦ r♦ sts ♥ t ♣rs♠♣t ♥♦ss ♦ r♥r② s②st♠ sss rt ♥t♥ ♥ rt② ♥ ♦♠♣t♥ ②st♠s t ♥tr♥t♦♥ ♦♥r♥Pr♦♥s ♣s
❬t❪ tr♥♦ ♥ rt ss t ♦♥ ♥ ♥♥ ♦♥t♦♥
❬❪ P ♠♣str r ♥ ♥ ①♠♠ ♦♦ r♦♠♥♦♠♣t t t ♦rt♠ ♦r♥ ♦ t ♦② ttst♦t② rs t♦♦♦ ♣s
♦r♣②
❬❪ ♥ r♣② ♥ ♦♥② ❯s♥ ♥ t t♦ ♣t sst♦♥ rs t ♣♣t♦♥s ♥ ♦♦ t♥tt② sts♦r♥ ♦ t ♦② ttst ♦t② rs ♣♣ ttsts
❬❪ s♣♥ ♥ ♥♠♥ ❯s♥ t♥t ss ♦s t♦rtr③ ♥ ssss t rr♦r ♥ srt sr♠♥ts ♦♠trs ♣♣
❬❪ rtt ♥ s ♥ t str ♥②ss♦♥♦♥ ② t♦♥
❬❪ rtt ♥t ♠①tr ♠♦ ♦r t str♥ ♦ ♠①♠♦t ttsts Pr♦t② ttrs
❬♦r❪ ♦rs②t P srs tt♣rs♠❯P sr t
❬♦r❪ ♦r♠♥♥ ♥r ♦st t♥t ss ♥②ss ♦r ♣♦②t♦♠♦s t♦r♥ ♦ t ♠r♥ ttst ss♦t♦♥
❬❪ r② ♥ tr② ❯ rs♦♥ ♥ ♣ ♦r♥♦r♠ ♠①tr ♠♦♥ ♥ ♠♦s str♥ ♥ r♣♦rt ♦♠♥t
❬❲❪ r ②♥ ♥ ❲②s ②s♥ ♠♦ st♦♥ ♦r tt♥t ♣♦st♦♥ str ♠♦ ♦r s♦ ♥t♦rs r❳ ♣r♣r♥tr❳
❬❪ rürt♥ttr ♥t ♠①tr ♥ r♦ st♥ ♠♦s♣r♥r
❬❪ rürt♥ttr ♥t ①tr ♥ r♦ t♥ ♦s
❬❲❪ r ♥ ❲②s st♠t♥ t ♥ r ttstr♥
❬❪ ♥st ♥ r r②t♥ ②♦ ②s ♥t t♦ ♥♦♦t ♦♣ ♠♦♥ t r r t♦ s ♦r♥ ♦ ②r♦♦♥♥r♥
❬❪ ♦♥ ♥ r♣② ①tr ♦ t♥t trt ♥②③rs ♦r ♠♦s str♥ ♦ t♦r t ttsts ♥ ♦♠♣t♥ ♣s
❬❪ ♦rt ♥ ♦ str♥ t r♥♦ ♠①tr ♠♦s ♦♠♣rs♦♥ ♦ r♥t ♣♣r♦s ♦♠♣tt♦♥ ttsts t ♥②ss
♦r♣②
❬❪ ♦rt ♥ t♥t ♦ ♠♦ ♦r ♦♥t♥♥② t ♦♠♠♥t♦♥s ♥ ttsts♦r② ♥ t♦s
❬♦♦❪ ♦♦♠♥ ①♣♦rt♦r② t♥t strtr ♥②ss s♥ ♦t ♥t ♥ ♥♥t ♠♦s ♦♠tr
❬♦❪ ♦t ❯tst♦♥ s ♠♦ès ♠é♥ ♣♦r sst♦♥ t♦♠tq ♦♥♥és ♦r♥s P tss ❯♥rsté ♥♦♦ ♦♠♣è♥
❬♦❪ ♦rt t ♥②ss ♦♠ ❲②
❬r❪ P r♥ ♥ s ♦ t ♦r ♣♥③ ♦♦ st♠t♦♥ ♦r♥ ♦ t ♦② ttst ♦t② rs t♦♦♦ ♣s
❬r❪ P r♥ rs ♠♣ r♦ ♥ ♦♥t r♦ ♦♠♣tt♦♥♥ ②s♥ ♠♦ tr♠♥t♦♥ ♦♠tr
❬❪ rt ♥ P r ♦♠♣r♥ ♣rtt♦♥s ♦r♥ ♦ sst♦♥
❬❪ ♥rs t♥t strtr ♠♦s t rt ts t♥♥t♦rs ♦ ♣♥♥ ♠♦s ♦♦♦ t♦s sr
❬r❪ r♣r ♦ ♣♥♥ t♥t strtr ♠♦s Ps②♦♠tr
❬❪ ♥ ❲ ♦♥ ♥ ❨ ♥ ♥ ①t♥s♦♥ ♦ ♠t♣ ♦rrs♣♦♥♥ ♥②ss ♦r ♥t②♥ tr♦♥♦s sr♦♣s ♦ rs♣♦♥♥ts Ps②♦♠tr
❬♥❪ ♥♥ t♦s ♦r ♠r♥ ss♥ ♠①tr ♦♠♣♦♥♥ts ♥s ♥ t ♥②ss ♥ sst♦♥
❬❪ ♥t ♥ ♦r♥s♥ ♦r② t♦s ①tr ♠♦ str♥ s♥ t ❯❳ ♣r♦r♠ str♥ ❩♥ ♦r♥♦ ttsts
❬❪ ♥t ♥ ♦r♥s♥ str♥ ♠① t ❲② ♥trs♣♥r② s t ♥♥ ♥ ♥♦ s♦r②
❬❲❪ P ♦ ❳ ♥ ❲♥r ♥♦r♠t♦♥ ♦♥s ♦r ss♥♦♣s r❳ ♣r♣r♥t r❳
❬♦❪ P ♦ ①t♥♥ t r♥ ♦♦ ♦r s♠♣r♠tr ♦♣st♠t♦♥ ♥♥s ♦ ♣♣ ttsts ♣s
♦r♣②
❬❪ ❩ ♥ ①t♥s♦♥s t♦ t ♠♥s ♦rt♠ ♦r str♥ rt ts t t♦r ❱s t ♥♥ ♥ ♥♦ s♦r②
❬❲❪ ♥tr ❲♥ ♥ P tt♠♥s♣rr ♥r♥ ♦r ♠①trs♦ s②♠♠tr strt♦♥s ♥♥s ♦ ttsts ♣s
❬❨❪ ♥ ♥ ❨ ♦ts ②s ♦t ♦ t♣ tr ♥tr♥t♦♥ ttst
❬❪ qs ♥ r♥ ♦s str♥ ♦r ♠trt♣rt r♥♥ t ♣♣♦rt r ♥r
❬❪ ♦r♥s♥ ♥ ♥t ①tr ♠♦ str♥ ♦ t sts tt♦r ♥ ♦♥t♥♦s rs ♥ Pr♦♥s ♦ t ♦♥r♥ ♦♠ ♣s
❬♦❪ ♦ trt ♠♦s ♥ ♠trt ♣♥♥ ♦♥♣ts ♦♠ Prss
❬P❪ qs ♥ Pr ♦s str♥ ♦ ♠trt ♥t♦♥ t ♦♠♣tt♦♥ ttsts ♥ t ♥②ss
❬r❪ r♥ ♦♥sst♥t st♠t♦♥ ♦ t ♦rr ♦ ♠①tr ♠♦s♥② r
❬❪ ♦s♠s ♥ rs ♦s str♥ s♥ ♦♣s t♣♣t♦♥s r❳ ♣r♥ts
❬❪ ♥ r ♥ ♥♦r♠t♦♥ ♥ s♥② ♥♥s ♦ t♠t ttsts
❬❪ rs ♥ ♦ts♦ ♥t ♠①trs ♦ ♠trt P♦ss♦♥ strt♦♥s t ♣♣t♦♥ ♦r♥ ♦ sttst P♥♥♥ ♥♥r♥
❬r❪ ♦s♣ rs ♥ t s♦rtst s♣♥♥♥ str ♦ r♣ ♥ ttr♥ ss♠♥ ♣r♦♠ Pr♦♥s ♦ t ♠r♥ t♠ts♦t②
❬r③❪ ❲ r③♥♦s ♦t♦♥ ♠♦ ♦r ♠①trs ♦ t♦r ♥♦♥t♥♦s rs ♦r♥ ♦ sst♦♥
❬❪ rs ♥ P s♠②rt③s ①t ②s♥ ♠♦♥ ♦r rtP♦ss♦♥ t ♥ ①t♥s♦♥s ttsts ♥ ♦♠♣t♥
❬❲❪ ss♥ ♥ ❲♥r ♥t st♠t♦♥ ♥ t rt♥♦r♠ ♦♣ ♠♦ ♥♦r♠ ♠r♥s r st ♦r r♥♦
♦r♣②
❬❪ P t♦ r♠é ♥ ♠r♦s ②s♥ ♠t♦s ♦r r♣str♥ ♥ ♥s ♥ t ♥②ss t ♥♥ ♥ s♥ss♥t♥ ♣s ♣r♥r
❬❪ ♠♥♥ ♥ s ♦r② ♦ ♣♦♥t st♠t♦♥ ♦♠ ♣r♥r
❬+❪ rt ♦ ♥r♦♥t r♥ ① ♥ ♦rt ♠①♠♦ P ♦ t ♦s ❯♥s♣rs ♣rs ♥ ♠♣rs sst♦♥ ①♠♦ rr② Pr♣r♥t
❬❪ r♥ ♥ ❲ r③♥♦s ①tr s♣rt♦♥ ♦r ♠①♠♦ t ttsts ♥ ♦♠♣t♥
❬♦❪ ♦② st sqrs q♥t③t♦♥ ♥ P ♥♦r♠t♦♥ ♦r② r♥st♦♥s ♦♥
❬❪ ♠ ❲❨ ♦ ♥ ❨ ♦♠♣rs♦♥ ♦ ♣rt♦♥ r② ♦♠♣①t② ♥ tr♥♥ t♠ ♦ trt②tr ♦ ♥ ♥ sst♦♥ ♦rt♠s ♥ r♥♥
❬❱❪ r r♥ ♥ ❱ ❱♥ ♦s str♥♦r ♦♥t♦♥② ♦rrt t♦r t ♦r♥ ♦ sst♦♥
❬❱❪ r r♥ ♥ ❱ ❱♥ ♦s str♥♦r ♦♥t♦♥② ♦rrt t♦r t ♣♣♦rt rr
❬❱❪ r r♥ ♥ ❱ ❱♥ ♥t ♠①tr ♠♦ ♦♦♥t♦♥ ♣♥♥s ♠♦s t♦ str t♦r t ♠tt
❬❱❪ r r♥ ♥ ❱ ❱♥ ♦s str♥♦ ss♥ ♦♣s ♦r ♠① t ♠tt
❬❪ rr② ♥s♦♥ r♥ ♥ s ②s♥ ss♥♦♣ t♦r ♠♦s ♦r ♠① t ♦r♥ ♦ t ♠r♥ ttstss♦t♦♥
❬❪ ♥ ♦r♥ r♥♥ t ♠①trs ♦ trs ♦r♥♦ ♥ r♥♥ sr
❬❪ ♥ ♥ rs♥♥ ♦rt♠ ❲② rs ♥Pr♦t② ♥ ttsts ♣♣ Pr♦t② ♥ ttsts ❲②♥trs♥ ❨♦r
❬❪ r♥ ♥rs♥ ♥ P ♦rt ②s♥ ♠♦♥ ♥♥r♥ ♦♥ ♠①trs ♦ strt♦♥s ♥♦♦ ♦ sttsts
♦r♣②
❬P❪ ♥ ♥ P ♥t ♠①tr ♠♦s ❲② rs ♥Pr♦t② ♥ ttsts ♣♣ Pr♦t② ♥ ttsts ❲②♥trs♥ ❨♦r
❬P❪ ♦st ♥ P♣♦r♦ t♥t ss ♠♦s ♦r ♠① rst ♣♣t♦♥s ♥ r♦♠tr② ♦♠♣tt♦♥ sttsts t♥②ss
❬t❪ té♥ t♥t r ②rs r ♦ ♦ ♥ ♥ ♠♦s♥s ♥ t♥t r ♠①tr ♠♦s
❬❪ s♥ ♥ ♥tr♦t♦♥ t♦ ♦♣s ♣r♥r
❬s❪ ❯ ss♦♥ ①♠♠ ♦♦ st♠t♦♥ ♦ t ♣♦②♦r ♦rrt♦♥♦♥t Ps②♦♠tr
❬❪ ♥ ♥ t trt ♦rrt♦♥ ♠♦s t ♠① srt ♥ ♦♥t♥♦s rs ♥♥s ♦ t♠t ttsts♣s
❬Pr❪ Pr③♥ ♥ st♠t♦♥ ♦ ♣r♦t② ♥st② ♥t♦♥ ♥ ♠♦♥♥s ♦ ♠t♠t sttsts
❬P❪ Ptt ♥ ♥ ♦♥ ♥t ②s♥ ♥r♥ ♦r ss♥ ♦♣ rrss♦♥ ♠♦s ♦♠tr
❬P❪ Prs♦♥ ♦♥trt♦♥s t♦ t ♠t♠t t♦r② ♦ ♦t♦♥P♦s♦♣ r♥st♦♥s ♦ t ♦② ♦t② ♦ ♦♥♦♥
❬P❪ P ♥ ♥ ♦st ♠①tr ♠♦♥ s♥ t tstrt♦♥ ttsts ♥ ♦♠♣t♥
❬❪ ❨ ♥ ♥ t♥r ♥♦♠ ts ♦s ♥ t♥tss ♥②ss ♦r t♥ r② ♦ ♥♦st sts ♦♠trs♣♣
❬❪ tr② ②♣♦tss tst♥ ♥ ♠♦ st♦♥ ♥ r♦ ♥♦♥t r♦ ♥ ♣rt ♣s ♣r♥r
❬❪ ♠♦♥r♥② ♦♥♣t rs♦♥ ♦ t ♠♥s ♦rt♠Pttr♥ ♦♥t♦♥ ttrs
❬❪ P ♦rt ♥ s ♦♥t r♦ sttst ♠t♦s ♦♠ tsr
❬❪ rs♦♥ ♥ P r♥ ♥ ②s♥ ♥②ss ♦ ♠①trs t♥ ♥♥♦♥ ♥♠r ♦ ♦♠♣♦♥♥ts t sss♦♥ ♦r♥ ♦ t♦② ttst ♦t② rs ttst t♦♦♦②
♦r♣②
❬❲❪ ♦ss♥ ♣ ♥ ❲♦s♦♥ ♦② ♣♥♥t t♥tss ♠♦s t ♦rts ♥ ♣♣t♦♥ t♦ ♥r r♥♥ ♥t ❯ ♦r♥ ♦ t ♦② ttst ♦t② rs ttsts♥ ♦t②
❬♦❪ P ♦rt ②s♥ ♦ r♦♠ s♦♥t♦rt ♦♥t♦♥st♦ ♦♠♣tt♦♥ ♠♣♠♥tt♦♥ ♣r♥r
❬+❪ ♦ss♥ ❨ ♦♥ rst ♦♠♥ ♥ ❲♦s♦♥ t♥t ss ♥②ss ♦ ♥r ♣r♦♠ r♥♥ ♥r♦♠ ♦♠♠♥t② s♠♣ ♦ ②r ♦s r ♥ ♦♦ ♣♥♥
❬❪ r③ st♠t♥ t ♠♥s♦♥ ♦ ♠♦ ♥♥s ♦ ttsts
❬❪ P ❳ ♦♥ ❨ ♥ ♥ s ①♠③t♦♥ ② ♣rts♥ ♦♦ ♥r♥ ♦r♥ ♦ t ♠r♥ ttst ss♦t♦♥
❬❪ ♠t ♥ st♠t♦♥ ♦ ♦♣ ♠♦s t srt ♠r♥s ②s♥ t ♠♥tt♦♥ ♦r♥ ♦ t ♠r♥ttst ss♦t♦♥
❬+❪ trss ♥s♦♣ st♦♥r s rs♥ ♥ ❯s♥ t♥t ss ♥②ss t♦ ♥t② ♣ttr♥s ♦ ♣tts sr ♣r♦s♦♥ ♥ rr trt♠♥t ♣r♦r♠s ♥ t s r♥ ♦♦ ♣♥♥
❬t❪ t♣♥s ♥ t st♥ ♥ ♠①tr ♠♦s ♦r♥ ♦ t ♦② ttst ♦t② rs ttst t♦♦♦②
❬t❪ t♣♥s ♥ t st♥ ♥ ♠①tr ♠♦s ♦r♥ ♦ t ♦② ttst ♦t② rs ttst t♦♦♦②
❬❪ r ♥tt② ♦ ♥t ♠①trs ♥♥s ♦ t♠tttsts
❬❪ r ♥tt② ♦ ♠①trs ♦ ♣r♦t ♠srs ♥♥s ♦t♠t ttsts
❬❱r❪ ❱r♠♥t t t♥t ss ♠♦s ♦♦♦ ♠t♦♦♦②
❬❱r❪ ❱r♠♥t t ♠①tr t♠ rs♣♦♥s t♦r② ♠♦s ♥♣♣t♦♥ ♥ t♦♥ tst♥ Pr♦♥s ♦ t t sss♦♥ ♦ t♥tr♥t♦♥ ttst ♥sttt s♦♥ P♦rt ♣s
♦r♣②
❬❱❪ P ❱♥ tt♠ ♥ ♦t♥ rt ♠♥tt♦♥ ❯s♥ r♥trt② sr ②s♥ ♥r♥ t s♣t t♦ ①trs ♦ ♦♥r ♦s ♦r♥ ♦ sst♦♥
❬❱❪ ❱r♥ ♥ ♦♥ ♠♣ ♥ ♦② ♦♥r♥t ♠t♦s♦r rt♥ t ♦♥r♥ ♦ ♥② ♦rt♠ ♥♥♥♦r♥ ♦ ttsts
❬❲❪ ❲s ♥ ♦ ♥t ♥t ♠①trs ♦ ♦t♦♥ ♠♦s♦r str♥ ♠①♠♦ t ttsts ♥ ♦♠♣t♥
❬❲❪ ❲ ♥ t ♦♥r♥ ♣r♦♣rts ♦ t ♦rt♠ ♥♥s ♦ ttsts
❬❨❪ ❨♦t③ ♥ ♣r♥s ♥ t ♥tt② ♦ ♥t ♠①trs♥♥s ♦ t♠t ttsts
❬❩❪ ❩ ❩♠♦♠ ♥ s ♦ r♥ ♥st② st♠t♦♥ t♣♣t♦♥s t♦ ♦♥♦♠trs r❳ ♣r♣r♥t r❳