24 CV MeanShift-GCuts 0605users.ics.forth.gr/~argyros/cs472_spring20/24_CV... · 2020. 5. 6. · σ=.1 σ=.2 σ=1 σ=.2 Data points Affinity matrices Slide credit: Kristen Grauman

24 – Mean shift and Graph Cuts

CS472 – Computer Vision, spring 20201

Mean Shift

Wednesday, 06/05/2020

Antonis Argyrose-mail: [email protected]

Mean shift clustering and segmentation

• An advanced and versatile technique for

clustering-based segmentation

D. Comaniciu and P. Meer, Mean Shift: A Robust Approach toward Feature

Space Analysis, PAMI 2002.



What is Mean Shift ?

PDF in feature space

• Color space

• …

• Actually any feature space you can conceive

A tool for:

Finding modes in a set of data samples, manifesting an

underlying probability density function (PDF) in RN

Intuitive Description

Distribution of identical billiard balls

Region of

interest

Center of

mass

Mean Shift

vector

Objective : Find the densest region





Region of

interest

Center of

mass

Mean Shift

vector




Region of

interest

Center of

mass

Mean Shift

vector






Region of

interest

Center of

mass

Mean Shift

vector




Region of

interest

Center of

mass

Mean Shift

vector






Region of

interest

Center of

mass

Mean Shift

vector




Region of

interest

Center of

mass




What is Mean Shift ?

Non-parametric

Density Estimation

Non-parametric

Density GRADIENT Estimation

(Mean Shift)

Data

Discrete PDF Representation

PDF Analysis

A tool for:

Finding modes in a set of data samples, manifesting an

underlying probability density function (PDF) in RN

Non-Parametric Density Estimation

Assumption : The data points are sampled from an underlying PDF

Assumed Underlying PDF Real Data Samples

Data point density

implies PDF value !









Parametric Density Estimation

Assumption : The data points are sampled from an underlying PDF

Assumed Underlying PDF

2

2

( )

2

i

PDF( ) =

i

i

ic e

x-μ

x

Estimate

Real Data Samples

Real Modality Analysis

Tessellate the space

with windowsRun the procedure in parallel



Real Modality Analysis

The blue data points were traversed by the windows towards the mode

Clustering

Attraction basin : the region for which all trajectories lead to the same mode

Cluster : All data points in the attraction basin of a mode

Mean Shift : A robust Approach Toward Feature Space Analysis, by Comaniciu, Meer



Mean Shift Mode Detection

Updated Mean Shift Procedure:

• Find all modes using the Simple Mean Shift Procedure

• Prune modes by perturbing them (find saddle points and plateaus)

• Prune nearby – take highest mode in the window

What happens if we

reach a saddle point

?

Perturb the mode position

and check if we return back

ClusteringSynthetic Examples

Simple Modal Structures

Complex Modal Structures



ClusteringReal Example

L*u*v space representation

ClusteringReal Example

Initial window

centers

Modes found Modes after

pruning

Final clusters

Feature space:

L*u*v representation



SegmentationExample

…when feature space is only

gray levels…

SegmentationExample



SegmentationExample

SegmentationExample



SegmentationExample

SegmentationExample



SegmentationExample

Mean-Shift Object TrackingTarget Representation

Choose a

reference

target model

Quantized

Color Space

Choose a

feature space

Represent the

model by its

PDF in the

feature space

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

1 2 3 . . . m

color

Pro

bab

ilit

y

Kernel Based Object Tracking, by Comaniniu, Ramesh, Meer



,f p y q � �

Mean-Shift Object TrackingTarget Localization Algorithm

Start from the

position of the

model in the

current frame

q�

Search in the

model’s

neighborhood

in next frame

p y�

Find best

candidate by

maximizing a

similarity func.

Mean-Shift Object TrackingResults



Tracking Through Scale SpaceResults

Fixed-scale

Tracking through scale space

Mean shift

• Pros:– Does not assume shape on clusters

– One parameter choice (window size, aka “bandwidth”)

– Generic technique

– Find multiple modes

• Cons:– Selection of window size

– Does not scale well with dimension of feature space



Image Segmentation with Graph Cuts

q

Images as graphs

• Fully-connected graph

– node (vertex) for every pixel

– link between every pair of pixels, p,q

– affinity weight wpq for each link (edge)

• wpq measures similarity

• similarity is inversely proportional to difference (in color and

position…)

p

wpq

w

Source: Steve Seitz



Segmentation by Graph Cuts

• Break Graph into Segments

– Want to delete links that cross between segments

– Easiest to break links that have low similarity (low weight)

• similar pixels should be in the same segments

• dissimilar pixels should be in different segments

w

A B C

Source: Steve Seitz

q

p

wpq

Measuring affinity

• One possibility:

Small sigma:

group only

nearby points

Large sigma:

group distant

points



Measuring affinity

σ=.1 σ=.2 σ=1

σ=.2

Data points

Affinity

matrices

Slide credit: Kristen Grauman

Cuts in a graph: Min cut

• Link Cut

– set of links whose removal makes a graph

disconnected

Cost of a cut:

AB

Find minimum cut• gives you a segmentation

• fast algorithms exist for doing this

Source: Steve Seitz

BqAp

qpwBAcut,

,),(



Minimum cut

• Problem with minimum cut:

Weight of cut proportional to number of edges in the cut;

tends to produce small, isolated components.

J. Shi and J. Malik, Normalized Cuts and Image Segmentation, CVPR, 1997

Documents

24 CV MeanShift-GCuts 0605users.ics.forth.gr/~argyros/cs472_spring20/24_CV... · 2020. 5. 6. · σ=.1 σ=.2 σ=1 σ=.2 Data points Affinity matrices Slide credit: Kristen Grauman