View
235
Download
0
Category
Preview:
Citation preview
1
Page 1
Modeling Modeling StatistikStatistik untukuntukComputer VisionComputer Vision
sumbersumber: :
- Forsyth+Ponce Chap. 7.
- Standford Vision & Modeling
Agenda
•Statistical Models (baca Forsyth+Ponce Chap. 7.)
- Bayesian Decision Theory
- Density Estimation
• PCA (Principal Component Analysis
• EM (Expectation Maximazation)
Modeling Modeling StatistikStatistik untukuntuk Computer Computer VisionVision
2
Page 2
ContohContoh aplikasiaplikasi model model statistikstatistik: : segmentasisegmentasi dengandengan EMEM
Segmentasi Warna
3
Page 3
ContohContoh recognition recognition dengandengan PCA:PCA:
Face Recognition dengan PCA (Turk+Pentland, ):
ContohContoh contour tracking contour tracking dengandengantheorematheorema BayesBayes
Snake TrackingSnake Tracking
E + βΩ ln p(x|c) + ln p(c)
4
Page 4
• Model Statistical : model yg merepresentasikan
Uncertainty and Variability
• Probability Theory: menjelaskan tentang mekanisme dari
Uncertainty
• Lihat contoh2 pada file pdf buku elektronik, pada CD.
(Forsyth+Ponce Chap 6)
Statistical Models / Probability TheoryStatistical Models / Probability Theory
• Fakta mengakan bahwa Segala sesuatu adalah merupakan
Variabel Random
Statistical Models / Probability TheoryStatistical Models / Probability Theory
5
Page 5
PengantarPengantar DesisiDesisi Optimal Optimal BayesBayes
DenganDengan berbagaiberbagai aplikasiaplikasi untukuntuk prosesproses klasifikasiklasifikasi
TeoriTeori DesisiDesisi BayesBayes ((BayesBayes Decision Decision Theory)Theory)
ContohContoh: Character Recognition:: Character Recognition:
TujuanTujuan: : MengklasifikasikanMengklasifikasikan karakterkarakter sedemikiansedemikian ruparupasehinggasehingga dapatdapat meminimalisasimeminimalisasi probabilitiprobabiliti kesalahankesalahanklasifikasiklasifikasi (minimize probability of misclassification)(minimize probability of misclassification)
6
Page 6
TeoriTeori DesisiDesisi BayesBayes
)( kCP• konsep #1: Priors (prob. anggapan awal)
a a b a b a a b ab a a a a b a a b aa b a a a a b b a b a b a a b a a
P(a)=0.75P(b)=0.25
?
TeoriTeori DesisiDesisi BayesBayes
• Konsep #2: Conditional Probability / Likelihood
)|( kCXP)|( aXP
)|( bXP# black pixel
# black pixel
7
Page 7
TeoriTeori DesisiDesisi BayesBayes
Contoh:
)|( aXP )|( bXP
X=7
?=kC
TeoriTeori DesisiDesisi BayesBayes
Contoh:
)|( aXP )|( bXP
X=8
?=kC
8
Page 8
TeoriTeori DesisiDesisi BayesBayes
• Contoh:
)|( aXP )|( bXP
X=8
Karena…P(a)=0.75P(b)=0.25
aCk =
TeoriTeori DesisiDesisi BayesBayes
• Contoh:
)|( aXP )|( bXP
X=9P(a)=0.75P(b)=0.25
?=kC
9
Page 9
TeoriTeori DesisiDesisi BayesBayes
• Teorema Bayes :
)(
)()|()|(
XP
CPCXPXCP kk
k =
TeoriTeori DesisiDesisi BayesBayes
• Teorema Bayes :
)(
)()|()|(
XP
CPCXPXCP kk
k =
∑=
jjj
kk
CPCXP
CPCXP
)()|(
)()|(
10
Page 10
TeoriTeori DesisiDesisi BayesBayes
• Teorema Bayes :
Posterior = Likelihood x prior
Normalization factor
TeoriTeori DesisiDesisi BayesBayes
• Contoh:
)|( aXP )|( bXP
11
Page 11
TeoriTeori DesisiDesisi BayesBayes
• Contoh:
)()|( aPaXP)()|( bPbXP
TeoriTeori DesisiDesisi BayesBayes
• Contoh:
)|( XbP)|( XaP
X>8 sehingga termasuk kelas b
12
Page 12
TeoriTeori DesisiDesisi BayesBayes
TujuanTujuan: : MengklasifikasikanMengklasifikasikan karakterkarakter sedemikiansedemikianruparupa sehinggasehingga dapatdapat meminimalisasimeminimalisasi probabilitiprobabilitikesalahankesalahan klasifikasiklasifikasi (minimize probability of (minimize probability of misclassification)misclassification)
Batas2 Batas2 desisidesisi (Decision boundaries):(Decision boundaries):
kjxCPxCP jk ≠> allfor )|()|(
TeoriTeori DesisiDesisi BayesBayes
BatasBatas--batasbatas desisidesisi::
kjxCPxCP jk ≠> allfor )|()|(
kjCPCxPCPCxP jjkk ≠> allfor )()|()()|(
13
Page 13
TeoriTeori DesisiDesisi BayesBayes
DaerahDaerah desisidesisi : : cRR ,...,1
R1 R2 R3
TeoriTeori DesisiDesisi BayesBayes
TujuanTujuan:: minimize probability of misclassificationminimize probability of misclassification
),(),()error( 2112 CRxPCRxPP ∈+∈=
14
Page 14
TeoriTeori DesisiDesisi BayesBayes
TujuanTujuan:: minimize probability of misclassificationminimize probability of misclassification
),(),()error( 2112 CRxPCRxPP ∈+∈=
)()|()()|( 221112 CPCRxPCPCRxP ∈+∈=
TeoriTeori DesisiDesisi BayesBayes
TujuanTujuan:: minimize probability of misclassificationminimize probability of misclassification
),(),()error( 2112 CRxPCRxPP ∈+∈=
)()|()()|( 221112 CPCRxPCPCRxP ∈+∈=
∫ ∫+=2 1
)()|()()|( 2211
R R
dxCPCxpdxCPCxp
15
Page 15
TeoriTeori DesisiDesisi BayesBayes
TujuanTujuan:: minimize probability of misclassificationminimize probability of misclassification
∫ ∫+=2 1
)()|()()|( 2211
R R
dxCPCxpdxCPCxp
TeoriTeori DesisiDesisi BayesBayes
MengapaMengapa (Posteriori Probability) (Posteriori Probability)
menjadimenjadi sangatsangat--sangatsangat pentingpenting ??
)()|( kk CPCxp
16
Page 16
TeoriTeori DesisiDesisi BayesBayes
MengapaMengapa jadijadi pentingpenting sekalisekali ??
ContohContoh #1: Speech Recognition#1: Speech Recognition
)()|( kk CPCxp
7189
= x y ε [/ah/, /eh/, .. /uh/]FFTmelscalebank
apple, ...,zebra
TeoriTeori DesisiDesisi BayesBayes
ContohContoh #1: Speech Recognition#1: Speech Recognition
FFTmelscalebank
/t/ /t/ /t/ /t/
/aal/ /aol/ /owl/
17
Page 17
TeoriTeori DesisiDesisi BayesBayes
ContohContoh #1: Speech Recognition#1: Speech Recognition
Bagaimana manusia dapatmengenali dengan mudah?
Apakah mesin bisa ???
TeoriTeori DesisiDesisi BayesBayes
ContohContoh #1: Speech Recognition#1: Speech Recognition
7189
= x yFFTmelscalebank
)|( kCxp
18
Page 18
TeoriTeori DesisiDesisi BayesBayes
ContohContoh #1: Speech Recognition#1: Speech Recognition
7189
= x yFFTmelscalebank
)|( kCxp
P(“wreck a nice beach”) = 0.001P(“recognize speech”) = 0.02
Language Model
)( kCP
TeoriTeori DesisiDesisi BayesBayes
MengapaMengapa pentingpenting ??
ContohContoh #2: Computer Vision#2: Computer Vision
)()|( kk CPCxp
Low-LevelImageMeasurements
High-LevelModelKnowledge
)|( kCxp )( kCP
19
Page 19
BayesBayes
MengapaMengapa pentingpenting??
ContohContoh #3: Curve Fitting#3: Curve Fitting
)()|( kk CPCxp
E + βΩ ln p(x|c) + ln p(c)
BayesBayes
MengapaMengapa pentingpenting??
ContohContoh #4: Snake Tracking#4: Snake Tracking
)()|( kk CPCxp
E + βΩ ln p(x|c) + ln p(c)
20
Page 20
•Statistical Models (Forsyth+Ponce Chap. 6)
- Bayesian Decision Theory
- Density Estimation
EstimasiEstimasi DensitasDensitas (Density Estimation)(Density Estimation)
Probability Density EstimationProbability Density Estimation
)|( Cxp
Data koleksi: x1,x2,x3,x4,x5,...
x
x
?
Estimasi:
21
Page 21
Probability Density EstimationProbability Density Estimation
Beberapa metode estimasi dengan:
•Parametric Representations• Non-Parametric Representations• Mixture Models
Probability Density EstimationProbability Density Estimation
• Parametric Representations- Normal Distribution (Gaussian)- Maximum Likelihood- Bayesian Learning
22
Page 22
Normal DistributionNormal Distribution
mean=µσ
variance=σ
Multivariate Normal DistributionMultivariate Normal Distribution
23
Page 23
Multivariate Normal DistributionMultivariate Normal Distribution
Mengapa Gaussian, apa istimewanya ?
• Punya properti sederhana:- linear transformasi Gaussians adalah Gaussian juga- marginal dan conditional densities dari Gaussians adalah Gaussian- Moment dari densitas Gaussian secara explisit merupakan fungsi dari µ
• “Good” Model of Nature:- Central Limit Theorem: Mean of M random variables is distributed
normally in the limit.
Multivariate Normal DistributionMultivariate Normal Distribution
Discriminant functions:
)(ln )|(ln )( kkk CPCxpxy +=
24
Page 24
Multivariate Normal DistributionMultivariate Normal Distribution
Discriminant functions:
)(ln )|(ln )( kkk CPCxpxy +=
equal priors + cov:Jarak Mahalanobis
Multivariate Normal DistributionMultivariate Normal Distribution
Bagaimana "Belajar" dari contoh ?Bisa dilakukan dengan :
• Maximum Likelihood
• Bayesian Learning
25
Page 25
Maximum LikelihoodMaximum Likelihood
Bagaimana "Belajar" dari contoh ?:
x
x
?
?
Maximum LikelihoodMaximum Likelihood
Likelihood dari model densitas θ untuk menghasilkan data X:
)|()|()( Likelihood1
θθθ n
N
nxpXpL
=∏=≡
26
Page 26
Maximum LikelihoodMaximum Likelihood
)|()|()( Likelihood1
θθθ n
N
nxpXpL
=∏=≡
∑=
−=−=N
nnxpLE
1
)|(ln )(ln :convenient more θθ
Likelihood dari model densitas θ untuk menghasilkan data X:
Maximum LikelihoodMaximum Likelihood
“Belajar” = Proses optimasi (maximizing likelihood / minimizing E):
∑=
−=−=N
nnxpLE
1
)|(ln )(ln :convenient more θθ
27
Page 27
Maximum LikelihoodMaximum Likelihood
Maximum Likelihood untuk Gaussian density:
∑=
−=−=N
nnxpLE
1
)|(ln )(ln :convenient more θθ
∑=
=N
nnx
N 1
1µSolusi singkatnya:
∑=
−−=∑N
n
Tnn xx
N 1
)ˆ)(ˆ(1ˆ µµ
Probability Density EstimationProbability Density Estimation
• Parametric Representations• Non-Parametric Representations• Mixture Models
Recommended