Upload
namnl2706
View
222
Download
0
Embed Size (px)
Citation preview
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
1/67
Lecture 6 -Fei-Fei Li
Lecture6:Clusteringand
Segmentation Part2
ProfessorFeiFei Li
StanfordVisionLab
11Jan111
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
2/67
Lecture 6 -Fei-Fei Li
Recap:GestaltTheory
Gestalt:wholeorgroup
Wholeisgreaterthansumofitsparts
Relationshipsamongpartscanyieldnewproperties/features
Psychologistsidentifiedseriesoffactorsthatpredisposesetof
elementstobegrouped(byhumanvisualsystem)
Untersuchungen zur Lehre von der Gestalt,
Psychologische Forschung, Vol. 4, pp. 301-350, 1923http://psy.ed.asu.edu/~classics/Wertheimer/Forms/forms.htm
I st and at t he window and see a house, t rees, sky.Theoret ical ly I might say t here were 327 br ightnessesand nuances of colour. Do I have "327"? No. I have sky, house,and t rees.
Max Wertheimer(1880-1943)
11Jan112
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
3/67
Lecture 6 -Fei-Fei Li
Recap:GestaltFactors
Thesefactorsmakeintuitivesense,butareverydifficulttotranslateintoalgorithms.
11Jan113
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
4/67
Lecture 6 -Fei-Fei Li
Recap:ImageSegmentation
Goal:identifygroupsofpixelsthatgotogether
11Jan114
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
5/67
Lecture 6 -Fei-Fei Li
Recap:KMeansClustering
Basicidea:randomlyinitializethekclustercenters,and
iteratebetweenthetwostepswejustsaw.
1. Randomlyinitializetheclustercenters,c1,...,cK2. Givenclustercenters,determinepointsineachcluster
Foreachpointp,findtheclosestci. Putpintoclusteri
3. Givenpointsineachcluster,solveforci
Setci tobethemeanofpointsinclusteri4. Ifci havechanged,repeatStep2
Properties
Will
always
converge
to
some solution Canbealocalminimum
Doesnotalwaysfindtheglobalminimumofobjectivefunction:
11Jan115
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
6/67
Lecture 6 -Fei-Fei Li
Recap:ExpectationMaximization(EM)
Goal
Findblobparametersthatmaximizethelikelihoodfunction:
Approach:
1. Estep: givencurrentguessofblobs,computeownershipofeachpoint
2. Mstep: givenownershipprobabilities,updateblobstomaximizelikelihood
function
3. Repeatuntilconvergence
11Jan116
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
7/67
Lecture 6 -Fei-Fei Li
Recap:MeanShiftAlgorithm
IterativeModeSearch1. Initializerandomseed,andwindowW
2. Calculatecenterofgravity(themean)ofW:
3. Shiftthesearchwindowtothemean
4. RepeatStep2untilconvergence
11Jan117
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
8/67
Lecture 6 -Fei-Fei Li
Recap:MeanShiftClustering
Cluster:alldatapointsintheattractionbasinofamode
Attractionbasin:theregionforwhichalltrajectoriesleadto
the
same
mode
11Jan118
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
9/67
Lecture 6 -Fei-Fei Li
Recap:MeanShiftSegmentation
Findfeatures(color,gradients,texture,etc)
Initializewindowsatindividualpixellocations
Performmeanshiftforeachwindowuntilconvergence
Mergewindowsthatendupnearthesamepeakormode
11Jan119
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
10/67
Lecture 6 -Fei-Fei Li
BacktotheImageSegmentationProblem
Goal:identifygroupsofpixelsthatgotogether
Uptonow,wehavefocusedonwaystogrouppixelsintoimagesegmentsbasedontheirappearance
Segmentationasclustering. Wealsowanttoenforceregionconstraints.
Spatialconsistency
Smoothborders
11Jan1110
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
11/67
Lecture 6 -Fei-Fei Li
Whatwewilllearntoday? Graphtheoreticsegmentation
NormalizedCuts
Usingtexturefeatures
SegmentationasEnergyMinimization
MarkovRandomFieldsGraphcutsforimagesegmentation
stmincut algorithm
ExtensiontononbinarycaseApplications
(Midtermmaterials)
11Jan1111
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
12/67
Lecture 6 -Fei-Fei Li
Whatwewilllearntoday?
Graphtheoreticsegmentation
NormalizedCutsUsingtexturefeatures
SegmentationasEnergyMinimization
MarkovRandomFields
Graphcutsforimagesegmentation
stmincut algorithm
Extensiontononbinarycase
Applications
11Jan1112
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
13/67
Lecture 6 -Fei-Fei Li
ImagesasGraphs
Fullyconnectedgraph Node(vertex)foreverypixel
Linkbetweeneverypairofpixels,(p,q)
Affinityweightwpq foreachlink(edge) wpq measuressimilarity Similarityisinverselyproportionaltodifference
(incolorandposition)
q
p
wpq
w
Slide credit: Steve Seitz
11Jan1113
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
14/67
Lecture 6 -Fei-Fei Li
SegmentationbyGraphCuts
BreakGraphintoSegments Deletelinksthatcrossbetweensegments
Easiesttobreaklinksthathavelowsimilarity(lowweight) Similarpixelsshouldbeinthesamesegments
Dissimilarpixelsshouldbeindifferentsegments
w
A B C
Slide credit: Steve Seitz
11Jan1114
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
15/67
Lecture 6 -Fei-Fei Li
MeasuringAffinity
Distance
Intensity
Color
Texture
2 212( , ) exp daff x y x y
2 212( , ) exp ( ) ( )daff x y I x I y
(some suitable color space distance)
221
2( , ) exp ( ), ( )
d
aff x y dist c x c y
Source:Forsyth&
Ponce
221
2( , ) exp ( ) ( )
d
aff x y f x f y
(vectors of filter outputs)
11Jan1115
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
16/67
Lecture 6 -Fei-Fei Li
ScaleAffectsAffinity
Small :grouponlynearbypoints
Large :groupfarawaypoints
Slide credit: Svetlana Lazebnik Small Medium Large 11Jan1116
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
17/67
Lecture 6 -Fei-Fei Li
GraphCut:usingEigenvalues
Extractasinglegoodcluster
Whereelementshavehighaffinityvalueswitheachother
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
18/67
Lecture 6 -Fei-Fei Li
points
matrix
eigenvector
GraphCut:usingEigenvalues
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
19/67
Lecture 6 -Fei-Fei Li
Extractasinglegoodcluster
Extractweightsforasetofclusters
GraphCut:usingEigenvalues
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
20/67
Lecture 6 -Fei-Fei Li
GraphCut:usingEigenvalues
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
21/67
Lecture 6 -Fei-Fei Li
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
22/67
Lecture 6 -Fei-Fei Li
GraphCut
Setofedgeswhoseremovalmakesagraphdisconnected
CostofacutSumofweightsofcutedges:
AgraphcutgivesusasegmentationWhatisagoodgraphcutandhowdowefindone?
Slide credit: Steve Seitz
A B
BqAp
qpwBAcut,
,),(
11Jan1122
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
23/67
Lecture 6 -Fei-Fei Li
GraphCut
Image Source: Forsyth & Ponce
Here, the cut is nicely
defined by the block-diagonal
structure of the affinity matrix.
How can t his be general ized?
11Jan1123
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
24/67
Lecture 6 -Fei-Fei Li
MinimumCut
Wecandosegmentationbyfindingtheminimumcutinagraph
aminimumcut ofagraphisacutwhosecutset hasthesmallestnumber
ofelements(unweighted case)orsmallestsumofweightspossible.
Efficientalgorithmsexistfordoingthis
Drawback:
Weightofcutproportionaltonumberofedgesinthecut
Minimumcuttendstocutoffverysmall,isolatedcomponents
IdealCut
Cutswith
lesser
weightthanthe
idealcut
Slidecredit:Khurram
Hassan-Sha
fique
11Jan1124
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
25/67
Lecture 6 -Fei-Fei Li
NormalizedCut(NCut)
Aminimumcutpenalizeslargesegments
Thiscanbefixedbynormalizingforsizeofsegments
Thenormalizedcutcostis:
TheexactsolutionisNPhardbutanapproximationcanbe
computedbysolvingageneralizedeigenvalueproblem.
assoc(A,V)=sumofweightsofalledgesinVthattouchA
),(
),(
),(
),(
VBassoc
BAcut
VAassoc
BAcut
J.ShiandJ.Malik.Normalizedcutsandimagesegmentation. PAMI2000
11Jan1125
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
26/67
Lecture 6 -Fei-Fei Li
InterpretationasaDynamicalSystem
Treat
the
links
as
springs
and
shake
the
system Elasticityproportionaltocost Vibrationmodescorrespondtosegments
Cancomputethesebysolvingageneralizedeigenvectorproblem
Slidecredit:Ste
veSeitz
11Jan1126
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
27/67
Lecture 6 -Fei-Fei Li
NCuts asaGeneralizedEigenvectorProblem
Definitions
RewritingNormalizedCutinmatrixform:
,: ( , ) ;
: ( , ) ( , );
: {1, 1} , ( ) 1 .
the affinity matrix,
the diag. matrix,
a vector in
i j
j
N
W W i j w
D D i i W i j
x x i i A
Slide credit: Jitendra Malik
0
(A,B) (A,B)(A,B)
(A,V) (B,V)
( , )(1 ) ( )(1 ) (1 ) ( )(1 );
1 1 (1 )1 1 ( , )
...
i
T Tx
T T
i
cut cut NCut
assoc assoc
D i ix D W x x D W xk
k D k D D i i
11Jan1127
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
28/67
Lecture 6 -Fei-Fei Li
SomeMoreMath
Slidecredit:JitendraMalik
11Jan1128
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
29/67
Lecture 6 -Fei-Fei Li
NCuts asaGeneralizedEigenvalueProblem
Aftersimplification,weget
ThisisaRayleighQuotient Solutiongivenbythegeneralized eigenvalueproblem
Solvedbyconvertingtostandardeigenvalueproblem
Subtleties
Optimalsolutionissecondsmallesteigenvector Givescontinuousresultmustconvertintodiscretevaluesofy
( )( , ) , with {1, }, 1 0.
T
T
iT
y D W y NCut A B y b y D
y Dy
Slide credit: Alyosha Efros
This is hard,
as y is discrete!
Relaxation:
continuous y.
,1 1 1
2 2 2 D (D W)D z z where z D y
11Jan1129
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
30/67
Lecture 6 -Fei-Fei Li
NCuts Example
Smallest eigenvectors
Image source: Shi & MalikNCuts segments
11Jan1130
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
31/67
Lecture 6 -Fei-Fei Li
Discretization
Problem:eigenvectorstakeoncontinuousvalues Howtochoosethesplittingpointtobinarize theimage?
Possibleproceduresa) Pickaconstantvalue(0,or0.5).
b) Pickthemedianvalueassplittingpoint.
c) LookforthesplittingpointthathastheminimumNCutvalue:1. Choosen possiblesplittingpoints.
2. ComputeNCutvalue.
3. Pickminimum.
Image Eigenvector NCut scores
11Jan1131
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
32/67
Lecture 6 -Fei-Fei Li
NCuts:OverallProcedure
1. ConstructaweightedgraphG=(V,E)fromanimage.
2. Connecteachpairofpixels,andassigngraphedgeweights,
3. Solve forthesmallestfeweigenvectors.This
yields
a
continuous
solution.4. Thresholdeigenvectorstogetadiscretecut
Thisiswheretheapproximationismade(werenotsolvingNP).
5. Recursively
subdivide
if
NCut value
is
below
a
pre
specified
value.
( )W y Dy
( , ) Prob. that and belong to the same region.W i j i j
Slidecredit:JitendraMalik
NCuts Matlab codeavailableat
http://www.cis.upenn.edu/~jshi/software/
11Jan1132
C l I S i i h NC
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
33/67
Lecture 6 -Fei-Fei Li
ColorImageSegmentationwithNCuts
ImageSource:Shi&Malik
11Jan1133
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
34/67
Lecture 6 -Fei-Fei Li
UsingTextureFeaturesforSegmentation
Texturedescriptorisvectoroffilterbankoutputs
J.Malik,S.Belongie,T.LeungandJ.Shi."ContourandTextureAnalysisforImageSegmentation".
IJCV43(1),727,2001.Slidecredit:Sv
etlanaLazebnik
11Jan1134
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
35/67
Lecture 6 -Fei-Fei Li
UsingTextureFeaturesforSegmentation
Texturedescriptoris
vectoroffilterbankoutputs.
Textons arefoundby
clustering.
Slide credit: Svetlana Lazebnik
11Jan1135
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
36/67
Lecture 6 -Fei-Fei Li
UsingTextureFeaturesforSegmentation
Texturedescriptorisvectoroffilterbankoutputs.
Textons arefoundbyclustering.
Affinitiesaregivenbysimilaritiesoftextonhistogramsoverwindowsgivenbythelocalscaleofthetexture.
Slide credit: Svetlana Lazebnik
11Jan1136
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
37/67
Lecture 6 -Fei-Fei Li
ResultswithColor&Texture
11Jan1137
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
38/67
Lecture 6 -Fei-Fei Li
Summary:NormalizedCuts
Pros:Genericframework,flexibletochoiceoffunctionthat
computesweights(affinities)betweennodes
Doesnotrequireanymodelofthedatadistribution
Cons:
Time
and
memory
complexity
can
be
high Dense,highlyconnectedgraphs manyaffinitycomputations Solvingeigenvalueproblemforeachcut
Preferenceforbalancedpartitions Ifaregionisuniform,NCutswillfindthe
modesofvibrationoftheimagedimensions
Slide credit: Kristen Grauman
11Jan1138
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
39/67
Lecture 6 -Fei-Fei Li
Whatwewilllearntoday? Graphtheoreticsegmentation
NormalizedCutsUsingtexturefeatures
SegmentationasEnergyMinimization
MarkovRandomFieldsGraphcutsforimagesegmentation
stmincut algorithm
Extensiontononbinarycase
Applications
11Jan1139
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
40/67
Lecture 6 -Fei-Fei Li
MarkovRandomFields
Allowrichprobabilisticmodelsforimages
Butbuiltinalocal,modularway
Learnlocaleffects,getglobaleffectsout
Slidecredit:William
Freeman
Observed evidence
Hidden true states
Neighborhood relations
11Jan1140
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
41/67
Lecture 6 -Fei-Fei Li
MRFNodesasPixels
Reconstruction
from MRF modeling
pixel neighborhood
statistics
Degraded imageOriginal image
Slidecredit:BastianLeibe
11Jan1141
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
42/67
Lecture 6 -Fei-Fei Li
MRFNodesasPatches
Image
Scene
Image patches
Scene patches
( , )i ix y
( , )i jx x
Slidecredit:William
Freeman
11Jan1142
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
43/67
Lecture 6 -Fei-Fei Li
Network
Joint
Probability
,( , ) ( , ) ( , )i i i j
i i j x y x y x x Scene
Image
Image-scene
compatibilityfunction
Scene-scene
compatibilityfunction
Neighboring
scene nodesLocal
observations
Slidecredit:William
Freeman
11Jan1143
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
44/67
Lecture 6 -Fei-Fei Li
EnergyFormulation Jointprobability
Takingthelogp(.)turnsthisintoanEnergyoptimizationproblem
Thisissimilartofreeenergyproblemsinstatisticalmechanics
(spinglasstheory).WethereforedrawtheanalogyandcallEanenergy function.
andarecalledpotentials.
,
( , ) ( , ) ( , )i i i ji i j
P x y x y x x
,
,
log ( , ) log ( , ) log ( , )
( , ) ( , ) ( , )
i i i j
i i j
i i i j
i i j
x y x y x x
E x y x y x x
Slidecredit:BastianLeibe
11Jan1144
l
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
45/67
Lecture 6 -Fei-Fei Li
EnergyFormulation
Energyfunction
Singlenodepotentials
Encodelocalinformationaboutthegivenpixel/patch Howlikelyisapixel/patchtobelongtoacertainclass
(e.g.foreground/background)?
Pairwisepotentials
Encodeneighborhoodinformation Howdifferentisapixel/patchslabelfromthatofitsneighbor?(e.g.
basedonintensity/color/texturedifference,edges)
Pairwise
potentials
Single-node
potentials
( , )i iy
( , )i jx
,
( , ) ( , ) ( , )i i i j
i i j
E x y x y x x
Slidecredit:BastianLeibe
11Jan1145
E Mi i i i
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
46/67
Lecture 6 -Fei-Fei Li
EnergyMinimization
Goal: InfertheoptimallabelingoftheMRF.
Manyinferencealgorithmsareavailable,e.g. Gibbssampling,simulatedannealing Iteratedconditionalmodes(ICM) Variationalmethods
Beliefpropagation Graphcuts
Recently,GraphCutshavebecomeapopulartool
Onlysuitableforacertainclassofenergyfunctions Butthesolutioncanbeobtainedveryfastfortypicalvisionproblems(~1MPixel/sec).
( , )i iy
( , )i jx
Slidecredit:BastianLeibe
11Jan1146
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
47/67
Lecture 6 -Fei-Fei Li
What
we
will
learn
today? Graphtheoreticsegmentation
NormalizedCuts
Usingtexturefeatures
Extension:Multilevelsegmentation
SegmentationasEnergyMinimizationMarkovRandomFields
Graphcutsforimagesegmentation
stmincut algorithm
ExtensiontononbinarycaseApplications
11Jan1147
Graph Cuts for Optimal Boundary Detection
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
48/67
Lecture 6 -Fei-Fei Li
GraphCutsforOptimalBoundaryDetection
Idea:convertMRFintosourcesinkgraph
n-links
s
t a cuthard
constraint
hard
constraint
Minimumcostcutcanbe
computedinpolynomialtime
(maxflow/mincutalgorithms)
2
2
exp
pq
pq
Iw
pqI
Slide credit: Yuri Boykov
11Jan1148
Si l E l f E
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
49/67
Lecture 6 -Fei-Fei Li
SimpleExampleofEnergy
Npq
qppq
p
pp LLwLDLE )()()(
},{ tsLp
tlinks nlinks
BoundarytermRegionalterm
(binaryobjectsegmentation)
Slide credit: Yuri Boykov
22
exp
pq
pq
Iw
pqI
s
t a cut
)(sDp
)(tDp
11Jan1149
Addi R i l P ti
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
50/67
Lecture 6 -Fei-Fei Li
AddingRegionalProperties
pqw
n-links
s
ta cut)(tDp
)(sDp
NOTE:hardconstrainsarenotrequired,ingeneral.
Regional biasexample
Suppose aregiven
expectedintensities
ofobjectandbackground
ts II and 22 2/||||exp)( spp IIsD
22
2/||||exp)( t
pp IItD
Slide credit: Yuri Boykov
11Jan1150
Adding Regional Properties
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
51/67
Lecture 6 -Fei-Fei Li
AddingRegionalProperties
pqw
n-links
s
ta cut)(tDp
)(sDp
22 2/||||exp)( spp IIsD
22
2/||||exp)( t
pp IItD
EMstyleoptimization
expected intensities of
object and background
can be re-estimated
ts
II and
Slide credit: Yuri Boykov
11Jan1151
Adding Regional Properties
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
52/67
Lecture 6 -Fei-Fei Li
AddingRegionalProperties
Moregenerally,regionalbiascanbebasedonany
intensitymodelsofobjectandbackground
a cut ( ) logPr( | ) p p p p D L I L
givenobjectandbackgroundintensityhistograms
)(sDp
)(tDps
t
)|Pr( sIp
)|Pr( tIp
pI
Slide credit: Yuri Boykov
11Jan1152
How to Set the Potentials? Some Examples
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
53/67
Lecture 6 -Fei-Fei Li
HowtoSetthePotentials?SomeExamples
Colorpotentials e.g.modeledwithaMixtureofGaussians
Edgepotentials e.g.acontrastsensitivePottsmodel
where
Parameters, needtobelearned,too!
[Shotton & Winn, ECCV06]
( , , ( ); ) ( ) ( )T
i j ij ij i jx x g y g y x x
2
2 i javg y y 2
( ) i jy y
ij g y e
( , ; ) log ( , ) ( | ) ( ; , )i i i i i k k k
x y x k P k x N y y
11Jan1153
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
54/67
Lecture 6 -Fei-Fei Li
What
we
will
learn
today? Graphtheoreticsegmentation
NormalizedCuts
Usingtexturefeatures
Extension:Multilevelsegmentation
SegmentationasEnergyMinimizationMarkovRandomFields
Graphcutsforimagesegmentation
stmincut algorithm
ExtensiontononbinarycaseApplications
11Jan1154
G hC t A li ti G bC t
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
55/67
Lecture 6 -Fei-Fei Li
GraphCutApplications:GrabCut
Usersegmentationcues
Additionalsegmentation
cues
InteractiveImageSegmentation[Boykov &Jolly,ICCV01] Roughregioncuessufficient
Segmentationboundarycanbeextractedfromedges
Procedure Usermarksforegroundandbackgroundregionswithabrush.
Thisisusedtocreateaninitialsegmentationwhichcanthenbecorrectedbyadditionalbrushstrokes.
Slide credit: Matthieu Bray
11Jan1155
GrabCut: Data Model
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
56/67
Lecture 6 -Fei-Fei Li
Obtainedfrominteractiveuserinput Usermarksforegroundandbackgroundregionswithabrush
Alternatively,usercanspecifyaboundingbox
GrabCut:DataModel
Globaloptimumofthe
energy
Background
color
Foreground
color
Slide credit: Carsten Rother
11Jan1156
GrabCut: Coherence Model
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
57/67
Lecture 6 -Fei-Fei Li
GrabCut:CoherenceModel
Anobjectisacoherentsetofpixels:
Howtochoose ?
Slide credit: Carsten Rother
Error (%) over training set:
25
2
( , )
( , ) e m ny y
n m
m n C
x y x x
11Jan1157
Iterated Graph Cuts
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
58/67
Lecture 6 -Fei-Fei Li
IteratedGraphCuts
Energyafter
eachiteration
Result
Foreground &
Background
Background G
RForeground
Background G
R
1 2 3 4
Color model
(MixtureofGaussians)
Slide credit: Carsten Rother
11Jan1158
GrabCut: live demo
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
59/67
Lecture 6 -Fei-Fei Li
GrabCut:livedemo
IncludedinMSOffice2010(letstryit)
Reported
results
11Jan1159
GrabCut: live demo
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
60/67
Lecture 6 -Fei-Fei Li
GrabCut:livedemo
IncludedinMSOffice2010(letstryit)
Reportedresults
11Jan1160
GrabCut: live demo
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
61/67
Lecture 6 -Fei-Fei Li
GrabCut:livedemo
IncludedinMSOffice2010(letstryit)
11Jan1161
GraphCut ImageSynthesisResults
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
62/67
Lecture 6 -Fei-Fei Li 62
Source:Vivek
Kwatra
11Jan11
Application:TextureSynthesisintheMedia
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
63/67
Lecture 6 -Fei-Fei Li
Currently,stilldonemanually
Slide credit: Kristen Grauman
11Jan1163
ImprovingEfficiencyofSegmentation
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
64/67
Lecture 6 -Fei-Fei Li
p g y g
Problem:Imagescontainmanypixels Evenwithefficientgraphcuts,anMRF
formulationhastoomanynodesforinteractiveresults.
Efficiencytrick:Superpixels Grouptogethersimilarlooking
pixelsforefficiencyoffurther
processing. Cheap,localoversegmentation
Importanttoensurethatsuperpixelsdonotcrossboundaries
Severaldifferentapproachespossible Superpixelcodeavailablehere http://www.cs.sfu.ca/~mori/research/superpixels/
Image source: Greg Mori
11Jan1164
Superpixels forPreSegmentation
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
65/67
Lecture 6 -Fei-Fei Li
SpeedupGraph structure
11Jan1165
Summary: Graph Cuts Segmentation
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
66/67
Lecture 6 -Fei-Fei Li
Summary:GraphCutsSegmentation
ProsPowerfultechnique,basedonprobabilisticmodel(MRF).
Applicableforawiderangeofproblems.Veryefficientalgorithmsavailableforvisionproblems.
Becomingadefactostandardformanysegmentationtasks.
Cons/IssuesGraphcutscanonlysolvealimitedclassofmodels
Submodularenergyfunctions
CancaptureonlypartoftheexpressivenessofMRFsOnlyapproximatealgorithmsavailableformultilabelcase
6611Jan11
8/6/2019 Lecture6 Clustering and Seg p2 Cs223b
67/67
Lecture 6 -Fei-Fei Li
What
we
have
learned
today?
11Jan1167
Graphtheoreticsegmentation
NormalizedCuts
Usingtexturefeatures
SegmentationasEnergyMinimization
MarkovRandomFields
Graphcutsforimagesegmentation
stmincut algorithm
Extensiontononbinarycase
Applications
(Midtermmaterials)