Upload
camilla-briggs
View
219
Download
0
Embed Size (px)
Citation preview
Discrete OptimizationLecture 3 – Part 1
M. Pawan Kumar
Slides available online http://cvn.ecp.fr/personnel/pawan/
Energy Minimization
Va Vb Vc Vd
2
5
4
2
6
3
3
7
0
1 1
0
0
2
1
1
4 1
0
3
Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)
Label l0
Label l1
Energy Minimization
Va Vb Vc Vd
2
5
4
2
6
3
3
7
0
1 1
0
0
2
1
1
4 1
0
3
Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)
2 + 1 + 2 + 1 + 3 + 1 + 3 = 13
Label l0
Label l1
Energy Minimization
Va Vb Vc Vd
2
5
4
2
6
3
3
7
0
1 1
0
0
2
1
1
4 1
0
3
Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)
Label l0
Label l1
Energy Minimization
Va Vb Vc Vd
2
5
4
2
6
3
3
7
0
1 1
0
0
2
1
1
4 1
0
3
Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)
5 + 1 + 4 + 0 + 6 + 4 + 7 = 27
Label l0
Label l1
Energy Minimization
Va Vb Vc Vd
2
5
4
2
6
3
3
7
0
1 1
0
0
2
1
1
4 1
0
3
Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b)
f* = arg min Q(f; )
q* = min Q(f; ) = Q(f*; )
Label l0
Label l1
Min-Marginals
Va Vb Vc Vd
2
5
4
2
6
3
3
7
0
1 1
0
0
2
1
1
4 1
0
3
f* = arg min Q(f; ) such that f(a) = i
Min-marginal qa;i
Label l0
Label l1
Min-Marginals and MAP• Minimum min-marginal of any variable = energy of MAP labelling
minf Q(f; ) such that f(a) = i
qa;i mini
mini ( )
Va has to take one label
minf Q(f; )
RecapWe only need to know two sets of equations
General form of Reparameterization
’a;i = a;i
’ab;ik = ab;ik
+ Mab;k
- Mab;k
+ Mba;i
- Mba;i
’b;k = b;k
Reparameterization of (a,b) in Belief Propagation
Mab;k = mini { a;i + ab;ik }
Mba;i = 0
Dynamic Programming
3 variables 2 variables + book-keeping
n variables (n-1) variables + book-keeping
Start from left, go to right
Reparameterize current edge (a,b)
Mab;k = mini { a;i + ab;ik }
’ab;ik = ab;ik+ Mab;k - Mab;k’b;k = b;k
Repeat
• LP Relaxation and its Dual
• TRW Message Passing
Outline
Integer Programming Formulation
min ∑a ∑i a;i ya;i + ∑(a,b) ∑ik ab;ik yab;ik
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
Integer Programming Formulation
min Ty
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
= [ … a;i …. ; … ab;ik ….]
y = [ … ya;i …. ; … yab;ik ….]
Linear Programming Relaxation
min Ty
ya;i {0,1}
∑i ya;i = 1
yab;ik = ya;i yb;k
Two reasons why we can’t solve this
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
yab;ik = ya;i yb;k
One reason why we can’t solve this
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
∑k yab;ik = ∑kya;i yb;k
One reason why we can’t solve this
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
One reason why we can’t solve this
= 1∑k yab;ik = ya;i∑k yb;k
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
∑k yab;ik = ya;i
One reason why we can’t solve this
Linear Programming Relaxation
min Ty
ya;i [0,1]
∑i ya;i = 1
∑k yab;ik = ya;i
No reason why we can’t solve this *
*memory requirements, time complexity
Dual
Let’s try to understand it intuitively
Dual of the LP RelaxationWainwright et al., 2001
Va Vb Vc
Vd Ve Vf
Vg Vh Vi
Va Vb Vc
Vd Ve Vf
Vg Vh Vi
Va Vb Vc
Vd Ve Vf
Vg Vh Vi
1
2
3
4 5 6
i
Dual of the LP RelaxationWainwright et al., 2001
q*(1)
i
q*(2)
q*(3)
q*(4) q*(5) q*(6)
q*(i)
Dual of LP
Va Vb Vc
Vd Ve Vf
Vg Vh Vi
Va Vb Vc
Vd Ve Vf
Vg Vh Vi
Va Vb Vc
Vd Ve Vf
Vg Vh Vi
max
Dual of the LP RelaxationWainwright et al., 2001
i
max q*(i)
I can easily compute q*(i)
I can easily maintain reparam constraint
So can I easily solve the dual?
• LP Relaxation and its Dual
• TRW Message Passing
Outline
Things to Remember
• Forward-pass computes min-marginals of root
• BP is exact for trees
• Every iteration provides a reparameterization
TRW Message PassingKolmogorov, 2006
Va Vb Vc
Vd Ve Vf
Vg Vh Vi
VaVb Vc
VdVe Vf
VgVh Vi
1
2
3
4 5 6
i
q*(i)
Pick a variable Va
TRW Message PassingKolmogorov, 2006
i
q*(i)
Vc Vb Va
1c;0
1c;1
1b;0
1b;1
1a;0
1a;1
Va Vd Vg
4a;0
4a;1
4d;0
4d;1
4g;0
4g;1
TRW Message PassingKolmogorov, 2006
1 + 4 + rest
q*(1) + q*(4) + K
Vc Vb Va Va Vd Vg
Reparameterize to obtain min-marginals of Va
1c;0
1c;1
1b;0
1b;1
1a;0
1a;1
4a;0
4a;1
4d;0
4d;1
4g;0
4g;1
TRW Message PassingKolmogorov, 2006
’1 + ’4 + rest
Vc Vb Va
’1c;0
’1c;1
’1b;0
’1b;1
’1a;0
’1a;1
Va Vd Vg
’4a;0
’4a;1
’4d;0
’4d;1
’4g;0
’4g;1
One pass of Belief Propagation
q*(’1) + q*(’4) + K
TRW Message PassingKolmogorov, 2006
’1 + ’4 + rest
Vc Vb Va Va Vd Vg
Remain the same
q*(’1) + q*(’4) + K
’1c;0
’1c;1
’1b;0
’1b;1
’1a;0
’1a;1
’4a;0
’4a;1
’4d;0
’4d;1
’4g;0
’4g;1
TRW Message PassingKolmogorov, 2006
’1 + ’4 + rest
min{’1a;0,’1a;1} + min{’4a;0,’4a;1} + K
Vc Vb Va Va Vd Vg
’1c;0
’1c;1
’1b;0
’1b;1
’1a;0
’1a;1
’4a;0
’4a;1
’4d;0
’4d;1
’4g;0
’4g;1
TRW Message PassingKolmogorov, 2006
’1 + ’4 + rest
Vc Vb Va Va Vd Vg
Compute average of min-marginals of Va
’1c;0
’1c;1
’1b;0
’1b;1
’1a;0
’1a;1
’4a;0
’4a;1
’4d;0
’4d;1
’4g;0
’4g;1
min{’1a;0,’1a;1} + min{’4a;0,’4a;1} + K
TRW Message PassingKolmogorov, 2006
’1 + ’4 + rest
Vc Vb Va Va Vd Vg
’’a;0 = ’1a;0+ ’4a;0
2
’’a;1 = ’1a;1+ ’4a;1
2
’1c;0
’1c;1
’1b;0
’1b;1
’1a;0
’1a;1
’4a;0
’4a;1
’4d;0
’4d;1
’4g;0
’4g;1
min{’1a;0,’1a;1} + min{’4a;0,’4a;1} + K
TRW Message PassingKolmogorov, 2006
’’1 + ’’4 + rest
Vc Vb Va Va Vd Vg
’1c;0
’1c;1
’1b;0
’1b;1
’’a;0
’’a;1
’’a;0
’’a;1
’4d;0
’4d;1
’4g;0
’4g;1
’’a;0 = ’1a;0+ ’4a;0
2
’’a;1 = ’1a;1+ ’4a;1
2
min{’1a;0,’1a;1} + min{’4a;0,’4a;1} + K
TRW Message PassingKolmogorov, 2006
’’1 + ’’4 + rest
Vc Vb Va Va Vd Vg
’1c;0
’1c;1
’1b;0
’1b;1
’’a;0
’’a;1
’’a;0
’’a;1
’4d;0
’4d;1
’4g;0
’4g;1
’’a;0 = ’1a;0+ ’4a;0
2
’’a;1 = ’1a;1+ ’4a;1
2
min{’1a;0,’1a;1} + min{’4a;0,’4a;1} + K
TRW Message PassingKolmogorov, 2006
Vc Vb Va Va Vd Vg
2 min{’’a;0, ’’a;1} + K
’1c;0
’1c;1
’1b;0
’1b;1
’’a;0
’’a;1
’’a;0
’’a;1
’4d;0
’4d;1
’4g;0
’4g;1
’’1 + ’’4 + rest
’’a;0 = ’1a;0+ ’4a;0
2
’’a;1 = ’1a;1+ ’4a;1
2
TRW Message PassingKolmogorov, 2006
Vc Vb Va Va Vd Vg
’1c;0
’1c;1
’1b;0
’1b;1
’’a;0
’’a;1
’’a;0
’’a;1
’4d;0
’4d;1
’4g;0
’4g;1
min {p1+p2, q1+q2} min {p1, q1} + min {p2, q2}≥
2 min{’’a;0, ’’a;1} + K
’’1 + ’’4 + rest
TRW Message PassingKolmogorov, 2006
Vc Vb Va Va Vd Vg
Objective function increases or remains constant
’1c;0
’1c;1
’1b;0
’1b;1
’’a;0
’’a;1
’’a;0
’’a;1
’4d;0
’4d;1
’4g;0
’4g;1
2 min{’’a;0, ’’a;1} + K
’’1 + ’’4 + rest
TRW Message Passing
Initialize i. Take care of reparam constraint
Choose random variable Va
Compute min-marginals of Va for all trees
Node-average the min-marginals
REPEAT
Kolmogorov, 2006
Can also do edge-averaging
• Preliminaries
• LP Relaxation and its Dual
• TRW Message Passing– Examples– Primal Solution– Results
Outline
Example 1
Va Vb
0
1 1
0
2
5
4
2l0
l1
Vb Vc
0
2 3
1
4
2
6
3
Vc Va
1
4 1
0
6
3
6
4
5
6
7
Pick variable Va. Reparameterize.
Example 1
Va Vb
-3
-2 -1
-2
5
7
4
2
Vb Vc
0
2 3
1
4
2
6
3
Vc Va
-3
1 -3
-3
6
3
10
7
5
6
7
Average the min-marginals of Va
l0
l1
Example 1
Va Vb
-3
-2 -1
-2
7.5
7
4
2
Vb Vc
0
2 3
1
4
2
6
3
Vc Va
-3
1 -3
-3
6
3
7.5
7
7
6
7
Pick variable Vb. Reparameterize.
l0
l1
Example 1
Va Vb
-7.5
-7 -5.5
-7
7.5
7
8.5
7
Vb Vc
-5
-3 -1
-3
9
6
6
3
Vc Va
-3
1 -3
-3
6
3
7.5
7
7
6
7
Average the min-marginals of Vb
l0
l1
Example 1
Va Vb
-7.5
-7 -5.5
-7
7.5
7
8.75
6.5
Vb Vc
-5
-3 -1
-3
8.75
6.5
6
3
Vc Va
-3
1 -3
-3
6
3
7.5
7
6.5
6.5
7 Value of dual does not increase
l0
l1
Example 1
Va Vb
-7.5
-7 -5.5
-7
7.5
7
8.75
6.5
Vb Vc
-5
-3 -1
-3
8.75
6.5
6
3
Vc Va
-3
1 -3
-3
6
3
7.5
7
6.5
6.5
7 Maybe it will increase for Vc
NO
l0
l1
Example 1
Va Vb
-7.5
-7 -5.5
-7
7.5
7
8.75
6.5
Vb Vc
-5
-3 -1
-3
8.75
6.5
6
3
Vc Va
-3
1 -3
-3
6
3
7.5
7
Strong Tree Agreement
Exact MAP Estimate
f1(a) = 0 f1(b) = 0 f2(b) = 0 f2(c) = 0 f3(c) = 0 f3(a) = 0
l0
l1
Example 2
Va Vb
0
1 1
0
2
5
2
2
Vb Vc
1
0 0
1
0
0
0
0
Vc Va
0
1 1
0
0
3
4
8
4
0
4
Pick variable Va. Reparameterize.
l0
l1
Example 2
Va Vb
-2
-1 -1
-2
4
7
2
2
Vb Vc
1
0 0
1
0
0
0
0
Vc Va
0
0 1
-1
0
3
4
9
4
0
4
Average the min-marginals of Va
l0
l1
Example 2
Va Vb
-2
-1 -1
-2
4
8
2
2
Vb Vc
1
0 0
1
0
0
0
0
Vc Va
0
0 1
-1
0
3
4
8
4
0
4 Value of dual does not increase
l0
l1
Example 2
Va Vb
-2
-1 -1
-2
4
8
2
2
Vb Vc
1
0 0
1
0
0
0
0
Vc Va
0
0 1
-1
0
3
4
8
4
0
4 Maybe it will decrease for Vb or Vc
NO
l0
l1
Example 2
Va Vb
-2
-1 -1
-2
4
8
2
2
Vb Vc
1
0 0
1
0
0
0
0
Vc Va
0
0 1
-1
0
3
4
8
f1(a) = 1 f1(b) = 1 f2(b) = 1 f2(c) = 0 f3(c) = 1 f3(a) = 1
f2(b) = 0 f2(c) = 1
Weak Tree Agreement
Not Exact MAP Estimate
l0
l1
Example 2
Va Vb
-2
-1 -1
-2
4
8
2
2
Vb Vc
1
0 0
1
0
0
0
0
Vc Va
0
0 1
-1
0
3
4
8
Weak Tree Agreement
Convergence point of TRW
l0
l1
f1(a) = 1 f1(b) = 1 f2(b) = 1 f2(c) = 0 f3(c) = 1 f3(a) = 1
f2(b) = 0 f2(c) = 1
• Preliminaries
• LP Relaxation and its Dual
• TRW Message Passing– Examples– Primal Solution– Results
Outline
Obtaining the Labelling
Only solves the dual. Primal solutions?
Va Vb Vc
Vd Ve Vf
Vg Vh Vi
’ = i
Fix the labelOf Va
Obtaining the Labelling
Only solves the dual. Primal solutions?
Va Vb Vc
Vd Ve Vf
Vg Vh Vi
’ = i
Fix the labelOf Vb
Continue in some fixed order
Meltzer et al., 2006
Computational Issues of TRW
• Speed-ups for some pairwise potentials
Basic Component is Belief Propagation
Felzenszwalb & Huttenlocher, 2004
• Memory requirements cut down by half
Kolmogorov, 2006
• Further speed-ups using monotonic chains
Kolmogorov, 2006
Theoretical Properties of TRW
• Always converges, unlike BP
Kolmogorov, 2006
• Strong tree agreement implies exact MAP
Wainwright et al., 2001
• Optimal MAP for two-label submodular problems
Kolmogorov and Wainwright, 2005
ab;00 + ab;11 ≤ ab;01 + ab;10
• Preliminaries
• LP Relaxation and its Dual
• TRW Message Passing– Examples– Primal Solution– Results
Outline
ResultsBinary Segmentation Szeliski et al. , 2008
Labels - {foreground, background}
Unary Potentials: -log(likelihood) using learnt fg/bg models
Pairwise Potentials: 0, if same labels
1 - exp(|da - db|), if different labels
ResultsBinary Segmentation
Labels - {foreground, background}
Unary Potentials: -log(likelihood) using learnt fg/bg models
Szeliski et al. , 2008
Pairwise Potentials: 0, if same labels
1 - exp(|da - db|), if different labels
TRW
ResultsBinary Segmentation
Labels - {foreground, background}
Unary Potentials: -log(likelihood) using learnt fg/bg models
Szeliski et al. , 2008
Belief Propagation
Pairwise Potentials: 0, if same labels
1 - exp(|da - db|), if different labels
ResultsStereo Correspondence Szeliski et al. , 2008
Labels - {disparities}
Unary Potentials: Similarity of pixel colours
Pairwise Potentials: 0, if same labels
1 - exp(|da - db|), if different labels
ResultsSzeliski et al. , 2008
Labels - {disparities}
Unary Potentials: Similarity of pixel colours
Pairwise Potentials: 0, if same labels
1 - exp(|da - db|), if different labels
TRW
Stereo Correspondence
ResultsSzeliski et al. , 2008
Labels - {disparities}
Unary Potentials: Similarity of pixel colours
Belief Propagation
Pairwise Potentials: 0, if same labels
1 - exp(|da - db|), if different labels
Stereo Correspondence
ResultsNon-submodular problems Kolmogorov, 2006
BP TRW-S
30x30 grid K50
BP TRW-S
BP outperforms TRW-S
Code + Standard Data
http://vision.middlebury.edu/MRF