Upload
apollo
View
38
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Inference. Probabilistic Graphical Models. Variable Elimination. Complexity Analysis. Eliminating Z. Reminder: Factor Product. N k =|Val( X k )|. Cost: (m k -1) N k multiplications. Reminder: Factor Marginalization. N k =|Val( X k )|. Cost: ~ N k additions. - PowerPoint PPT Presentation
Citation preview
Daphne Koller
Variable Elimination
Complexity Analysis
ProbabilisticGraphicalModels
Inference
Daphne Koller
Eliminating Z
Daphne Koller
a1 b1 0.5
a1 b2 0.8
a2 b1 0.1
a2 b2 0
a3 b1 0.3
a3 b2 0.9
b1 c1 0.5
b1 c2 0.7
b2 c1 0.1
b2 c2 0.2
a1 b1 c1 0.5·0.5 = 0.25
a1 b1 c2 0.5·0.7 = 0.35
a1 b2 c1 0.8·0.1 = 0.08
a1 b2 c2 0.8·0.2 = 0.16
a2 b1 c1 0.1·0.5 = 0.05
a2 b1 c2 0.1·0.7 = 0.07
a2 b2 c1 0·0.1 = 0
a2 b2 c2 0·0.2 = 0
a3 b1 c1 0.3·0.5 = 0.15
a3 b1 c2 0.3·0.7 = 0.21
a3 b2 c1 0.9·0.1 = 0.09
a3 b2 c2 0.9·0.2 = 0.18
Reminder: Factor Product
Cost: (mk-1)Nk multiplications
Nk =|Val(Xk)|
Daphne Koller
a1 b1 c1 0.25
a1 b1 c2 0.35
a1 b2 c1 0.08
a1 b2 c2 0.16
a2 b1 c1 0.05
a2 b1 c2 0.07
a2 b2 c1 0
a2 b2 c2 0
a3 b1 c1 0.15
a3 b1 c2 0.21
a3 b2 c1 0.09
a3 b2 c2 0.18
a1 c1 0.33
a1 c2 0.51
a2 c1 0.05
a2 c2 0.07
a3 c1 0.24
a3 c2 0.39
Reminder: Factor Marginalization
Cost: ~Nk additions
Nk =|Val(Xk)|
Daphne Koller
Complexity of Variable Elimination• Start with m factors–m n for Bayesian networks– can be larger for Markov networks
• At each elimination step generate • At most elimination steps• Total number of factors: m*
Daphne Koller
Complexity of Variable Elimination• N = max(Nk) = size of the largest factor• Product operations: k (mk-1)Nk
• Sum operations: k Nk
• Total work is linear in N and m*
Daphne Koller
Complexity of Variable Elimination• Total work is linear in N and m• Nk =|Val(Xk)|=O(drk) where– d = max(|Val(Xi)|)– rk = |Xk| = cardinality of the scope of the
kth factor
Daphne Koller
Complexity ExampleC
DC DCCD ),()()(1
D
G DDIGIG )(),,(),( 12
I
IS IGIISGS ),()(),(),( 23
H
H JGHJG ),,(),(4
),(),(),(),,( 435 JGGSGLSLJG
L
SL
J SLJSLJJ,
56 ),,(),,()(
C
D I
SG
L
JH
D
Daphne Koller
Complexity and Elimination Order
),,(),,(),( JGHDIGGL HGG
L C
D I
SG
L
JH
D
• Eliminate: G
)(),()(),,(),,(),(),(),,(,,,,,,
CDCIJGHDIGISGLSLJ CDCDIHGSL
IHGSLJ
Daphne Koller
Complexity and Elimination OrderA
C
B1
B2 B3 Bk…Eliminate A first:
Eliminate Bi‘s first:
A
C
B1
B2 B3 Bk…
Daphne Koller
Summary• Complexity of variable elimination linear
in– size of the model (# factors, # variables)– size of the largest factor generated
• Size of factor is exponential in its scope• Complexity of algorithm depends heavily
on elimination ordering
Daphne Koller
END END END