Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape

Correspondence

Yusuf Sahillioğlu and Yücel YemezComputer Eng. Dept., Koç University, Istanbul, Turkey

Problem Definition & Apps2 / 24

Shape interpolation

Shape registration

Shape matching

Time-varying recon.

Statistical shape analysis

Goal: Find a mapping between two isometric shapes

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Attribute transfer

Contributions

Avoid embedding

C2F joint sampling of evenly-spaced salient vertices

geodesic curvatureintegral

Euclideanembedding

Non-Euclideanembedding

3 / 24

O(NlogN) time complexity for dense correspondenceYusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Isometry

Our method is purely isometric Intrinsic global property

Similar shapes have similar metric structures

Metric: geodesic distance

4 / 24

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Isometric Distortion

Given , measure its isometric distortion:

5 / 24

in the most general setting.

: normalized geodesic distance b/w two vertices.

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Isometric Distortion6 / 24

g ggggg

g g

average for .

in action:

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Minimizing Isometric Distortion

N = |S| = |T| for perfectly isometric shapes. N! different mappings; intractable.

7 / 24

Solution: Patch-by-patch matching to reduce search space. Optimal mapping maps nearby vertices in source to

nearby vertices in target.

Recursively subdivide matched patches into smaller patches (C2F sampling) to be matched (combinatorial search).

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Coarse-to-Fine Sampling

: set of base vertices sampled from at level .

Sampling radii s.t. for k=0,1,..,K. at level defines patch : all vertices within a

distance from the base .

8 / 24

greens inherited from level k−1blues are all vertices ( )patches being defined ( )

blacks + greens =

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Correspondence Algorithm

Correspondence at level k is obtained in two steps: Match level k bases inside the patch pairs matched at level

k−1. Merge patch-based local correspondences into one global

correspondence over whole surface.

9 / 24

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Patch-based Matching ( )

Ensure base vertices fall into each patch to allow combinatorial matching.

Patch radius to select for such an :

10 / 24

, area of the largest patch at level k−1.

M=5 samples with circular

patches to cover blue area

(enlarge a bit to cover whites)

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

5M

M

M

Patch-based Matching ( )

11 / 24

Combinatorial matching

greens inherited from level k−1

blacks + greens =

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Correspondence Merging ( )

Merge patch-to-patch correspondences into one global correspondence that covers the whole surface.

12 / 24

Multi-graph single graph. Also, diso values made available.1st pass over source samples to keep only one match per sample, the one

with the min diso.

2nd pass over target samples to assign one match per isolated sample, the

one with the min diso.

Trim matches with diso > 2Diso, i.e., outliers.

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Insight to the Algorithm13 / 24

Conditions for the algorithm to work correctly High-resolution sampling on two perfectly isometric

surfaces Evenly-spaced sampling s.t. every vertex is in at least one

patch Distortion is a slowly changing convex function around

optimum One optimal solution (no symmetric flips)

Optimal mapping assigns si to tj which is as nearest to the

ground-truth ti as possible

Inclusion assertion is then expected to apply:

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Inclusion assertion (demonstration)14 / 24

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Computational Complexity15 / 24

Saliency sorting

C2F sampling

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Computational Complexity16 / 24

Patch-based combinatorial matching

Merging

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Computational Complexity17 / 24

Overall

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Experimental Results18 / 24

Details captured, smooth flow Many-to-one

Two meshes at different resolutions

red line: the worst match

w.r.t. isometric distortion

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

6K vs. 16K

Experimental Results19 / 24

red line: the worst match w.r.t. isometric distortion

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Experimental Results20 / 2

for four more pairs:

red line: the worst match w.r.t. isometric distortiongreen line: the worst match w.r.t. ground-truth

distortionYusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Experimental Results21 / 24

Comparisons

GMDS O(N2logN)[Bronstein et al.]

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Spectral O(N2logN)[Jain et al.]

Nonrigid world dataset

Our method O(NlogN)

Our method O(NlogN)

Future Work22 / 24

Symmetric flip issue Purely isometry-based methods naturally fail at symmetric

inputs Not intrinsically symmetric only one optimal solution

Our method may still occasionally fail to find the optimum due to initial coarse sampling

Solution suggested

A solution for symmetric flips due to initial coarse sampling:

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

Conclusion23 / 24

Computationally efficient C2F dense isometric shape correspondence algorithm (O(NlogN)).

Isometric distortion minimized in the original 3D Euclidean space wherein isometry is defined.

Accurate for isometric and nearly isometric pairs. Different levels of detail thanks to the C2F joint

sampling. No restriction on topology. Symmetric flips may occasionally occur due to

initial coarse sampling (but can be healed as proposed).

Yusuf Sahillioğlu & Yücel Yemez, Coarse-to-Fine Combinatorial Matching for Dense Isometric Shape Correspondence, SGP’11.

People

Assoc. Prof. Yücel Yemez, supervisor

Yusuf, PhD student

24 / 24