Sampling for Part BasedObject Models

Daniel HuttenlocherSeptember, 2006

2

Part Based Object Recognition

Matching constellation models, pictorial structures, etc.– Dominated by energy minimization approaches

• Local or global methods depending on problem definition

• MAP estimation view

Computationally tractable global optimization depends on models that factor– Appearance of parts independent

– Spatial model with low tree width

3

State of the Art?

Model introduced error– Model overly simplistic in order to be tractable

Computationally introduced error– Model “right thing” but don’t know how

computational results related

Often not explicit about these sources of error– Precise description of what want to compute

and what actually computing

4

Sampling

Statistical method for using tractable (factored) models as means of estimating intractable ones

Proposal distribution– Samples from distribution using factored model

evaluated according to more general one

– Want “enough” probability mass distributed around in proposal distribution• “Promiscuous” – likes multiple things

• E.g., smoothing a distribution (temperature)

– Does more than k-best

5

More Concrete

Part based graphical model, M=(V,E)– Parts V=(v1, …, vm)

– Spatial relations (undirected edges) E={eij}

For detection, consider all configurations L

PM(I) ≈ maxL PM(I|L) PM(L)

Efficient when factors

PM(I|L) = viV PM(I|li)

PM(L) = C M(LC)

For small cliques C, e.g, 2-cliques for tree

6

A Model that Doesn’t Factor

Patchwork-of-parts (POP) model proposed by Amit and Trouve– Star model with latent reference part

– Account for part overlap by averaging probabilities for parts covering an image pixel• PM(I|L) no longer factors (sum over parts)

Use likelihood that factors for proposal distribution – overcounting (promiscuous)– Sample from posterior distribution and

compute POP probability for these samples• Efficiently approximating MAP estimate

7

Sampling Example for Tree [FH05]

8

Comparison with Direct Minimization

Using posterior distribution for factored model – efficient to:– Compute marginals (box sum)

– Generate samples• For tree, sample location for root from marginal,

then sample children conditioned on root location

– Evaluate general model on samples

As opposed to trying to optimize general model directly– Using difficult to characterize techniques

9

Simple Experiments

Pictorial structure model using oriented edge part templates

Star topology

Factored appearance model for proposal distribution vs. POP model

Six parts and Caltech-4 data, for comparison with some earlier results using similar models (without POP likelihood)– CFH05, same topology and part models

– FPZ05, same topology

10

Detection Results

Single class detection (equal ROC error)– MAP of factored model vs. sampling from factored model– Significant at 95% confidence level except bikes

Airplanes Cars (rear) Faces Motorbikes

MAP

(hill climb)

94.3% 94.4% 98.0% 98.6%

Sample 94.8% 95.0% 98.4% 98.8%

CFH05FPZ05

93.0%

93.6%

90.3%84.2%

96.4%90.3%

93.3%97.3%

11

Not Limited to Appearance

Sampling is a general technique for approximating intractable distributions– Even easier when using to approximate MAP of

those distributions

Tractable distributions can make explicit aspects of problem structure– Over-counting of scene evidence

– Importance of kinematic tree spatial constraints for humans, vs. limb coordination