Embed Size (px)
Citation preview
Submodular Optimization and
Approximation Algorithm
Satoru Iwata
岩田 覚
RIMS, Kyoto University
京都大学数理解析研究所
Outline
• Submodular Functions
Examples
Discrete Convexity
• Submodular Function Minimization
• Approximation Algorithms
• Submodular Function Maximization
• Approximating Submodular Functions
Submodular Functions
• Cut Capacity Functions
• Matroid Rank Functions
• Entropy Functions
)()()()( YXfYXfYfXf
VYX ,
VX Y
R2: Vf
:V Finite Set
Cut Capacity Function
Cut Capacity
} leaving :|)({)( XaacX
X
s t
Max Flow Value＝Min Cut Capacity
0)( ac
Matroid Rank Functions
:
)()(
||)(,
YXYX
XXVX
],[rank)( XUAX
Matrix Rank Function
Submodular
A
X
V
U
Whitney (1935)
Entropy Functions
:)(Xh
0)( h
X
Information Sources
0
)()()()( YXhYXhYhXh
Entropy of the Joint Distribution
Conditional Mutual Information
Positive Definite Symmetric Matrices
0)( f
A ][XA][detlog)( XAXf
X
)()()()( YXfYXfYfXf
Ky Fan’s Inequality
Extension of the Hadamard Inequality
Vi
iiAAdet
Discrete Concavity
)(}){()(}){( TfuTfSfuSf
TS
V
T
Diminishing Returns
S u
Discrete Convexity
)()()()( YXfYXfYfXf
V
X Y
Convex Function
x
y
Discrete Convexity
},{ ba
},,{ cbaV
}{b
}{a
}{c
},{ cb
Lovász (1983)
f̂
f
f̂
： Linear Interpolation
： Convex
： Submodular
:]1,0[ Chosen Uniformly at Random
},)(|{: vxvX )](E[)(ˆ Xfxf
Submodular Function Minimization
X
)(Xf
?}|)(min{ VYYf
Minimization Algorithm
Evaluation Oracle
Minimizer
0)( fAssumption:
Submodular Function Minimization
)( 87 nnO )log( 5 MnO
)log( 7 nnO
Grötschel, Lovász, Schrijver (1981, 1988)
Iwata, Fleischer, Fujishige (2000) Schrijver (2000)
Ellipsoid Method
Cunningham (1985)
|)(|max XfMVX
: Time for Function Evaluation
Base Polyhedra
Submodular Polyhedron
)}()(,,|{)( YfYxVYxxfP V R
Yv
vxYx )()(
}|{ RR VxV
)}()(),(|{)( VfVxfPxxfB
Base Polyhedron
Greedy Algorithm
v
))((
))((
))((
)(
)(
)(
111
0
11
001
2
1
2
1
nn vLf
vLf
vLf
vy
vy
vy
}){)(())(()( vvLfvLfvy
Edmonds (1970) Shapley (1971)
Extreme Base
)(vL
)( Vv
:y
2v1v nv
Theorem
Min-Max Theorem
)}(|)(max{ )(min fBxVxYfVY
)}(,0min{:)( vxvx
)()()( YfYxVx
Edmonds (1970)
Combinatorial Approach
Extreme Base
Convex Combination
Cunningham (1985)
L
L
L yx
)( fByL
x
|)(|max XfMVX
)log( 6 nMMnO
)()()( TfTxVx
Combinatorial Approach
Tvvx
Tvvx
,0)(
,0)(
:Ly
L
. ),()( LTfTyL
L
L
L yx
).()( TfTx
T
Extreme Base
:T Minimizer
Submodular Function Minimization
)( 87 nnO )log( 5 MnO
)log( 7 nnO
Grötschel, Lovász, Schrijver (1981, 1988)
Iwata, Fleischer, Fujishige (2000) Schrijver (2000)
Iwata (2003)
Fleischer, Iwata (2000)
)log)(( 54 MnnO
Orlin (2007)
)( 65 nnO
Iwata (2002)
Fully Combinatorial
Ellipsoid Method
Cunningham (1985)
Iwata, Orlin (2009)
)( subject to
Minimize2
fBx
x
The Fujishige-Wolfe Algorithm Fujishige (1984)
sol. opt. :x
}0)(|{: vxvS
)(min)()( XfTfSfVX
}0)(|{: vxvT
Minimal Minimizer
Maximal Minimizer
x
Evacuation Problem （Dynamic Flow）
:)(Xo
:)(a
Hoppe, Tardos (2000)
:)(vb
:)(ac
SX XT \
TSXXoXb ),()(
TS
Capacity
Transit Time
Supply/Demand
Maximum Amount of Flow from to .
Feasible
Multiterminal Source Coding
X XR
Y
Encoder
Encoder YRDecoder
)(YH
)(XH
)|( XYH
)|( YXH
),( YXH
),( YXHXR
YR
Slepian, Wolf (1973)
Multiclass Queueing Systems
ii
Server
Control Policy
Arrival Rate Service Rate
Multiclass M/M/1
Preemptive
Arrival Interval Service Time
Performance Region
VXS
Xi
i
Xi
ii
Xi
ii
,1
:s
,: iii
YRjs
is
:js Expected Staying Time of a Job in j
Achievable
Coffman, Mitrani (1980)
:s1
Vi
i
A Class of Submodular Functions Vzyx R,,
))(()()()( XxhXyXzXf
:h
VXS
Xi
i
Xi
ii
Xi
ii
,1
Nonnegative, Nondecreasing, Convex
)( VX
i
iiy
:
xxh
1
1:)(
iii Sz :
iix :
Submodular
Itoko & Iwata (2007)
Zonotope in 3D
))(),(),(()( XzXyXxXw
}|)({conv VXXwZ
)(),,(~
xyhzzyxf
Zonotope
} ofPoint ExtremeLower :),,(|),,(~
min{
}|)(min{
Zzyxzyxf
VXXf
Remark: ),,(~
zyxf is NOT concave!
z
Line Arrangement
iii zyx
Topological Sweeping Method
Edelsbrunner, Guibas (1989) )( 2nO
Enumerating All the Cells
Symmetric Submodular Functions
. ),\()( VXXVfXf
)( 3nO
RVf 2:
Symmetric
Symmetric Submodular Function Minimization
Minimum Cut Algorithm by MA-ordering
Nagamochi & Ibaraki (1992)
Minimum Degree Ordering Nagamochi (2007)
?}|)(min{ VXXf
Queyranne (1998)
Submodular Partition
)( jiVV ji
k
i
iVf1
)(Minimize
subject to kVVV 1
Greedy Split
)11(2 k -Approximation
:f Monotone or Symmetric
Zhao, Nagamochi, Ibaraki (2005)
Questions
What Kind of Approximation Algorithms
Can Be Extended to Optimization Problems
with Submodular Cost or Constraints ?
Cf. Submodular Flow (Edmonds & Giles, 1977)
How Can We Exploit Discrete Convexity
in Design of Approximation Algorithms?
Cf. Ellipsoid Method
(Grötschel, Lovász & Schrijver, 1981)
Submodular Vertex Cover
Graph ),( EVG
Submodular Function
RVf 2:
)(SfFind a Vertex Cover VS Minimizing
2-Approximation Algorithm
Goel, Karande, Tripathi, Wang (FOCS 2009)
Iwata & Nagano (FOCS 2009)
Relaxation Problem
)( 0)( Vvvx
Convex Programming Relaxation (CPR)
)),(( 1)()( Evuevxux
)(ˆ xfMinimize
subject to
CPR has a half-integral optimal solution.
2-Approximation Algorithm
Submodular Cost Set Cover
U).(Xf
NX UN
Find Covering
with Minimum
||max: uUu
N
Rounding Algorithm
-Approximation
Primal-Dual Algorithm
uuN
Submodular Multiway Partition
)( jiVV ji
k
i
iVf1
)(Minimize
subject to kVVV 1
),...,1( kiVs ii
Chekuri & Ene (FOCS 2011)
:f Nonnegative Submodular
Relaxation Problem
),,...,1( 0)( Vvkivxi
Convex Programming Relaxation
k
i
i Vvvx1
)( 1)(
k
i
ixf1
)(ˆMinimize
subject to
Ellipsoid Method
),...,1( 1)( kisx ii
Optimal Solution :
ix
Rounding Scheme
),...,( 1 kAA
3/2-Approximate Solution
:]1,0[
),,...,1( })(|{: kivxvA ii
Chosen Uniformly at Random
Uncross
Return
i
k
iAVU
1:
Symmetric Submodular :f
),,...,( 11 UAAA kk
Rounding Scheme
2-Approximate Solution
:]1,(21
),,...,1( })(|{: kivxvA ii
Chosen Uniformly at Random
Return
i
k
iAVU
1:
Nonnegative Submodular :f
),,...,( 11 UAAA kk
Improvement over the -Approximation
by Zhao, Nagamochi, & Ibaraki (2005)
)1( k
Submodular Function Maximization
Monotone Submodular Functions
Nemhauser, Wolsey, Fisher (1978)
Cardinality Constraint
(1-1/e)-Approximation
Calinescu, Chekuri, Pál, Vondrák (IPCO 2007)
Vondrák (STOC 2008)
Filmes and Ward (FOCS 2012)
Matroid Constraint
(1-1/e)-Approximation
Greedy Approximation Algorithm
}{:
}){(maxarg:
:
1
1
0
jjj
jj
vTT
vTfv
T
1jv1vGreedy Algorithm
)(SfMaximize subject to kS ||
kj ,...,1
1jT
Nemhauser, Wolsey, Fisher (1978)
:f Monotone Submodular Function
Greedy Approximation Algorithm
),...,1( kj
)(: *Sf:*S
1
1
j
i
ijk
jj
TSu
jjj
j
kTf
TfuTfTf
TSfSf
j
)(
)(}){()(
)()( )
1
\
111
1
**
1*
Optimal Solution
1
1
j
i
i
k
i
ikTf1
)(
)()(: 1 jjj TfTf
Greedy Approximation Algorithm
),...,1( 0 kjj
j
jkk
11:ˆ
),...,1( kj
kk
k
k
2
1
11
0
1
00
Minimize
k
i
i
1
subject to
e
11
111ˆ
1
kk
i
ik
)(: *Sf
k
i
ki Tf1
)(
Submodular Welfare Problem
(Monotone, Submodular) kff ,...,1Utility Functions
)( jiVV ji
k
i
ii Vf1
)(Maximize
subject to kVVV 1
e)11( -Approximation Vondrák (2008)
Submodular Function Maximization
Nonnegative Submodular Functions
Feige, Mirrokni, Vondrák (FOCS 2007)
2/5-Approximation
Lee, Mirrokni, Nagarajan, Sviridenko (STOC 2009)
1/4-Approximation (Matroid Constraint)
1/5-Approximation (Knapsack Constraints)
Buchbinder, Feldman, Naor, Schwartz (FOCS 2012)
1/2-Approximation
Double-Greedy Algorithm Buchbinder, Feldman, Naor, Schwartz (FOCS 2012)
Initialize: .: ,: ,: DVBA
Invariants: .\ , ABDBA
Iteration: grow A or shrink B.
Output: either A or B that has the greater value.
AB
V
Double-Greedy Algorithm
BDA While
Select ).(\ ADBu
}.0),(}){(max{: AfuAf
}.0),(}){\(max{: BfuBf
with probability ,
}{: uAA
with probability .
}{\: uBB
If ,0 then
}.{: uDD Else
Double-Greedy Algorithm
The expectation )](2)()(E[ ZfBfAf
never decreases in the process.
) Expected changes of
.0)(2 2
at each iteration is at least
)(2)()( ZfBfAf
Approximating Submodular Functions
X
)(Xf
Algorithm Evaluation
Oracle
Function f̂Y
)(ˆ Yf
Algorithm with
for monotone submodular functions
Construct a set function such that
For what function is this possible?
Approximating Submodular Functions
. ),(ˆ)()()(ˆ VXXfnXfXf
,0)( f . 0,)( VXXf
f̂
)log()( nnOn
Goemans, Harvey, Iwata & Mirrokni (SODA 2009)
Ellipsoidal Approximation
:K
)( KxKx
Centrally Symmetric Convex Body
Maximum Volume Inscribed Ellipsoid
(The John Ellipsoid)
)()( AEnKAE
:)(AE
Submodular Load Balancing
?)(maxmin},...,{ 1
jjjVV
Vfm
Xv
jj vpXf )(:)(
:,...,1 mff
Svitkina & Fleischer (FOCS 2008)
Monotone Submodular Functions
Scheduling
2-Approximation Algorithm
Lenstra, Shmoys, Tardos (1990)
)log( nnO -Approximation Algorithm
Learning Submodular Functions
Balcan & Harvey (STOC 2010)
Construct a set function such that
holds for most ( fraction) of
For what function is this possible with
probability in time?
)(ˆ)()()(ˆ XfnXfXf
1
f̂
1 )/1,/1,poly( n
.X
PMAC-learning
Summary
• Submodular Functions Arise Everywhere.
• Discrete Analogue of Convexity.
• Combinatorial Algorithms for Minimization.
• Exploit Special Structures of Problems.
• Approx. Algorithms by Discrete Convexity
• Approx. Algorithms for Maximization
• Learning Submodular Functions
