DiscreteInferenceandLearningLecture4

MVA2020–2021

http://thoth.inrialpes.fr/~alahari/disinflearn

SlidesbasedonmaterialfromNikosKomodakis,M.PawanKumar

Outline

•  Previousclasses– Graphcuts,Beliefpropagationandvariants–  (Inference)

•  Today– Quickrecapofthecourse– Learningparameters

Beforemovingon…

Projectsuggestions(alsosentbyemail)

•  ImplementBPontrees,thengraph,extendtoTRW,compare•  Implementgraphcut+extension(Ishikawa,othermulti-label)or

variationofimplementation+smallapplication•  Complexapplicationofgraphcut,requiringmodelling(e.g.,

sequenceofimages)•  Geometricscenelabellingwithgraphcuts•  Jointmodellingoftwolabellingproblems(e.g.,segmentation+

detection)•  Implementfastprimal-dualalgorithm+evaluate•  Implementdeformablepartsmodelforobjectdetection•  …•  Oryourown(butcheckwithusfirst)•  Selectprojectsbefore25thJanuaryandemailus

(karteek.alahari@inria.fr,guillaume.charpiat@inria.fr)

Projects

•  Chooseprojectsbefore25/1(Monday!)

•  Presentationson31/3–  InEnglishorFrench– 15min,includingquestions

•  Reportdueon30/3

Recap

•  Whatinferencealgorithmwouldyouusefor– agraphwithonlychains•  2-labelproblem?•  Multi-labelproblem?

– Treestructuredgraph•  2labelproblem?•  Multi-labelproblem?

Recap•  Basics:problemformulation– EnergyFunction– MAPEstimation– Computingmin-marginals– Reparameterization

•  Solutions– BeliefPropagationandrelatedmethods[Lecture3]– Graphcuts[Lecture2]

Outline

•  Recapofthecourse

•  Learningparameters

ConditionalRandomFields(CRFs)•  Ubiquitousincomputervision•  segmentation stereomatchingopticalflow imagerestorationimagecompletion objectdetection/localization...

•  andbeyond•  medicalimaging,computergraphics,digitalcommunications,physics…

•  Reallypowerfulformulation

ConditionalRandomFields(CRFs)

•  Extensiveresearchformorethan20years

•  Keytask:inference/optimizationforCRFs/MRFs

•  Lotsofprogress

•  Graph-cutbasedalgorithms•  Message-passingmethods•  LPrelaxations•  DualDecomposition•  ….

•  Manystate-of-the-artmethods:

MAPinferenceforCRFs/MRFs

•  Hypergraph– Nodes– Hyperedges/cliques

•  High-orderMRFenergyminimizationproblem

high-orderpotential(oneperclique)

unarypotential(onepernode)

hyperedges

nodes

CRFtraining•  ButhowdowechoosetheCRFpotentials?

•  Throughtraining•  Parameterizepotentialsbyw•  Usetrainingdatatolearncorrectw

•  Characteristicexampleofstructuredoutputlearning[Taskar],[Tsochantaridis,Joachims]

•  Equally,ifnotmore,importantthanMAPinference•  Betteroptimizecorrectenergy(evenapproximately)•  Thanoptimizewrongenergyexactly

•  SupervisedLearning

•  ProbabilisticMethods

•  Loss-basedMethods

•  Results

Outline

ImageClassification

Isthisanurbanorruralarea?

Input:d Output:x∈{-1,+1}

ImageClassification

Isthisscanhealthyorunhealthy?

Input:d Output:x∈{-1,+1}

ImageClassification

X

d

LabelingX=x LabelsetL={-1,+1}

ImageClassification

Whichcityisthis?

Input:d Output:x∈{1,2,…,h}

ImageClassification

Whattypeoftumordoesthisscancontain?

Input:d Output:x∈{1,2,…,h}

ObjectDetection

Whereistheobjectintheimage?

Input:d Output:x∈{Pixels}

ObjectDetection

Whereistheruptureinthescan?

Input:d Output:x∈{Pixels}

ObjectDetection

X

d

LabelingX=x LabelsetL={1,2,…,h}

Segmentation

Whatisthesemanticclassofeachpixel?

Input:d Output:x∈{1,2,…,h}|Pixels|

car

roadgrass

treesky

sky

Segmentation

Whatisthemusclegroupofeachpixel?

Input:d Output:x∈{1,2,…,h}|Pixels|

Segmentation

X1

d1

X2

d2

X3

d3

X4

d4

X5

d5

X6

d6

X7

d7

X8

d8

X9

d9

LabelingX=x LabelsetL={1,2,…,h}

Segmentation

X1

d1

X2

d2

X3

d3

X4

d4

X5

d5

X6

d6

X7

d7

X8

d8

X9

d9

LabelingX=x LabelsetL={1,2,…,h}

CRFtraining•  Stereomatching:•  Z:left,rightimage•  X:disparitymap

Z X

f :

argf = parameterizedbyw

Goaloftraining:estimateproper

w

CRFtraining•  Denoising:•  Z:noisyinputimage•  X:denoisedoutputimage

Z X

f :

argf = parameterizedbyw

Goaloftraining:estimateproper

w

CRFtraining•  Objectdetection:•  Z:inputimage•  X:positionofobjectparts

Z X

f :

argf = parameterizedbyw

Goaloftraining:estimateproper

w

CRFtraining(somefurthernotation)

vectorvaluedfeaturefunctions

Learningformulations

Riskminimization

Ktrainingsamples

RegularizedRiskminimization

RegularizedRiskminimization

ReplaceΔ(.)witheasiertohandleupperboundLG(e.g.,convexw.r.t.w)

Choice1:Hingeloss

§  UpperboundsΔ(.)

§  Leadstomax-marginlearning

Max-marginlearning

energyofgroundtruth

anyotherenergy

desiredmargin

slack

Max-marginlearning

subjecttotheconstraints:

energyofgroundtruth

anyotherenergy

desiredmargin

slack

Max-marginlearning

subjecttotheconstraints:

energyofgroundtruth

anyotherenergy

desiredmargin

slack

Max-marginlearning

subjecttotheconstraints:

orequivalently

CONSTRAINED

UNCONSTRAINED

Choice2:logisticloss

§  Canbeshowntoleadtomaximumlikelihoodlearning

partitionfunction

Max-marginvsMaximum-likelihoodmax-margin

maximumlikelihood

Max-marginvsMaximum-likelihoodmax-margin

maximumlikelihood

soft-max

Solvingthelearningformulations

Maximum-likelihoodlearning

§  Differentiable&convex

partitionfunction

§  Globaloptimumviagradientdescent,forexample

Maximum-likelihoodlearning

gradient

Recallthat:

Maximum-likelihoodlearning

gradient

§  RequiresMRFprobabilisticinference§  NP-hard(exponentiallymanyx):approximationvialoopy-BP?

Max-marginlearning(UNCONSTRAINED)

§  Convexbutnon-differentiable§  Globaloptimumviasubgradientmethod

Subgradient

x2

subgradientatx1

g(x2)+h2·(x-x2)

subgradientatx2=gradientatx2

Subgradient

Subgradient

x

Subgradient

subgradientofLG =

Max-marginlearning(UNCONSTRAINED)

totalsubgr. =

Repeat1.computeglobalminimizersatcurrentw 2.computetotalsubgradientatcurrentw3.updatew bytakingastepinthenegativetotalsubgradient direction

untilconvergence

Subgradientalgorithm

Max-marginlearning(UNCONSTRAINED)

partialsubgradient=

Repeat1.pickkatrandom2.computeglobalminimizeratcurrentw 3.computepartialsubgradientatcurrentw4.updatew bytakingastepinthenegativepartialsubgradient direction

untilconvergence

Stochasticsubgradientalgorithm

MRF-MAPestimationperiteration(unfortunatelyNP-hard)

Max-marginlearning(CONSTRAINED)

subjecttotheconstraints:

Max-marginlearning(CONSTRAINED)

subjecttotheconstraints:

linearinw

•  Quadraticprogram(great!)•  Butexponentiallymanyconstraints(notsogreat)

•  Whatifweuseonlyasmallnumberofconstraints?

•  ResultingQPcanbesolved•  Butsolutionmaybeinfeasible

Max-marginlearning(CONSTRAINED)

•  onlyfewconstraintsactiveatoptimalsolution!!(variablesmuchfewerthanconstraints)

•  Constraintgenerationtotherescue

•  Giventheactiveconstraints,restcanbeignored•  Thenletustrytofindthem!

1.Startwithsomeconstraints

Constraintgeneration

2.SolveQP

3.Checkifsolutionisfeasiblew.r.t.toallconstraints

4.Ifyes,wearedone!

5.Ifnot,pickaviolatedconstraintandaddittothecurrentsetofconstraints.Repeatfromstep2.(optionally,wecanalsoremoveinactiveconstraints)

•  Keyissue:wemustalwaysbeabletofindaviolatedconstraintifoneexists

Constraintgeneration

•  Recalltheconstraintsformax-marginlearning

•  Tofindviolatedconstraint,wethereforeneedtocompute:

(justlikesubgradientmethod!)

1.InitializesetofconstraintsC toempty

Constraintgeneration

2.SolveQPusingcurrentconstraintsC andobtainnew(w,ξ)

4.Foreachk,ifthefollowingconstraintisviolatedthenaddittosetC:

5.Ifnonewconstraintwasaddedthenterminate.Otherwisegotostep2.

3.Computeglobalminimizersatcurrentw

MRF-MAPestimationpersample(unfortunatelyNP-hard)

Max-marginlearning(CONSTRAINED)

subjecttotheconstraints:

•  Alternatively,wecansolveaboveQPinthedualdomain

•  dualvariables↔primalconstraints•  Toomanyvariables,butmostofthemzeroatoptimalsolution

•  Useaworking-setmethod(essentiallydualtoconstraintgeneration)