Exact and Partial Energy Minimization in Computer Visioncmp.felk.cvut.cz/~shekhovt/publications/shekhovtsov-thesis-slides.pdf · 4/21 MINCUT Introduction CPU Mem CPU Mem ... Desmet

Supervisor: Václav HlaváčSupervisor-specialist: Tomáš Werner

Czech Technical University in PragueFaculty of Electrical Engineering, Department of Cybernetics

Ph.D. programme: Electrical Engineering and Information TechnologyBranch of study: Mathematical Engineering, 3901V021

Exact and Partial Energy Minimization inComputer Vision

Alexander Shekhovtsov

Ph.D. Thesis

2/21Overview

Discrete Optimization in Computer Vision (Energy Minimization)

Contribution: Distributed mincut/maxflow algorithm

Cases Reducible to Minimum Cut

General NP-hard case

Contribution: Methods to find a Part of Optimal Solution

3/21

MINCUT

Minimum Cut Problem

4/21

MINCUT

Introduction

CPU

Mem

CPU

Mem

Quick

Slow

Distributed Sequential

CPU

Mem

Quick

Slow

Disk

Distributed Parallel

Distributed Model – Divide Computation AND Memory

Solve large problem on a single computer

on more computers

Split data in parts:

5/21

MINCUT

Introduction

Sequential Algorithms

Parallel Algorithms

Our goal is to improve on this

6/21

MINCUT

Main Result

Main Result:

t

s

7/21

MINCUT

Main Idea

New distance function

length of the path = number of boundary edges

distance = length of a shortest path to the sink

d∗B(u) = 2

d∗B(v) = 0

t

corresponds to costly operations

Algorithm: push-relabel between regions, augmenting path insideregions

8/21

MINCUT

Experimental Confirmation

push-relabel (with heuristics)

proposed method

Synthetic instances: Grid graph with random capacities,partitioned into 4 regions

sweep = synchronously send messages on all boundary arcs

9/21

MINCUTInstances in Computer Vision

Dataset Published by Vision Group at University of Western Ontario

10/21

MINCUT

speedup: Ours/BK

Sequential Variant for Limited Memory Model

sometimes faster than BK

(CPU time excluding I/O)

robust over partition size

uses BK (Boykov and Kolmogorov) inside regions

11/21

MINCUT

Messages (sweeps) speedup over push-relabel (distributed version of Delong and Boykov 2008)

Sequential Variant for Limited Memory Model

12/21

MINCUT

Parallel Variant

Competitive with shared memory model methods

Speedup bounded by memory bandwidth

13/21

MINCUT

Conclusion

New distributed algorithm

Terminates in at most B2+1 sweeps (few in practice)

Sequential Algorithm

1) competitive with sequential solvers

2) uses few sweeps (= loads/unloads of regions)

3) suitable to run in the limited memory model

Parallel Algorithm

1) competitive with shared memory algorithms

2) uses few sweeps (= rounds of message exchange)

3) suitable for execution on a computer cluster

Implementation can be specialized for regular grids (less memory/faster)

(?) no good worst case complexity bound in terms of elementary operations

14/35

Part. OptimalityMINCUT

Discrete Energy Minimization Problem

s t

fst(1, 1) ft(1)

fs(0)fst(0, 0)

fst(1, 0)

fst(0, 1)

x

t0s t

xs xt

15/35


Partial Optimality

Energy model for stereo,minimization NP-hard

Find optimal solution in some pixels

solution unknown

solution globally optimal

16/35


s tt0

y

Partial Optimality

17/35


Partial Optimality (Multilabel)

s tt0

18/35


Overview

19/35


Dead End Elimination

Desmet et al. (1992), Goldstein (1994), Lasters et al. (1995),Pierce et al. (2000), Georgiev et al. (2006)

s

t

t0

y

t00

α

β

20/35


Improving Mapping

s tt00

1

2

3

21/35


Linear Embedding

s t

x

t0

22/35


Linear Embedding

s t

x

t0

23/35


Linear Embedding of Maps

24/35



25/35



26/35



27/35


More General Projections/Maps

s

t

α

3

67

6

7

3

0.5

0.5

Example of fractional map

28/35


Λ-Improving Characterization

29/35


Special Cases

DEE conditions by Desmet (1992) and Goldstein (1994)

(Weak/strong) Persistency in Quadratic Pseudo-Boolean Optimization (QPBO) by Nemhauser & Trotter (1975), Hammerr et al. (1984), Boros et al. (2002)

Multilabel QPBO Kohli et al. (2008), Shekhovtsov et al. (2008)

Submodular Auxiliary problems by Kovtun (2003, 2010)

Iterative Pruninig by Swoboda et al. (2013)

Methods that can be explained by the proposed condition:

Common properties, Only (M)QPBO was previously related to LP relaxation

30/35


Maximum Λ-Improving Projections

Problem: Find the mapping that maximizes domain reduction

31/35


[1] Nemhauser & Trotter (1975), Hammerr et al. (1984), Boros et al. (2002)

[2] Picard & Queyranne (1977) (Vertex Packing)

Maximum Λ-Improving Projections

Follow-up work, submitted to CVPR

Thesis

32/35


Conclusions

33/35


Higher Order

-Adams, W. P., Lassiter, J. B., and Sherali, H. D. (1998). Persistency in 0-1 polynomial programming.-Kolmogorov, V. (2012). Generalized roof duality and bisubmodular functions.-Kahl, F. and Strandmark, P. (2012). Generalized roof duality.-Lu, S. H. and Williams, A. C. (1987). Roof duality for polynomial 0-1 optimization.-Ishikawa, H. (2011). Transformation of general binary MRF minimization to the first-order case.-Fix, A. et al. (2011). A graph cut algorithm for higher-order Markov random fields.

34/35


Generalized Potts (5 labels) Fully Random (4 labels)

Experiments: solution completeness on random problems

Algorithm proposed:

New

Follow-up Work

35/35


Follow-up Work

Experiments: solving large scale problems by parts

Restrict the method to a local window

Find globally optimal reduction

partial labeling # remaining labels

New

Documents

Exact and Partial Energy Minimization in Computer Visioncmp.felk.cvut.cz/~shekhovt/publications/shekhovtsov-thesis-slides.pdf · 4/21 MINCUT Introduction CPU Mem CPU Mem ... Desmet