1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval...

1Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval

Optimal invariant metrics for shape retrieval

Michael Bronstein

Department of Computer ScienceTechnion – Israel Institute of Technology

Tagged shapes

Shapes withoutmetadata

Man, person, humanPersonText search

Content-based search

3D warehouse

Outline

Feature descriptor

Geometric words

Bag of words?

Invariance

Local geodesic distance histogram

Rigid Inelastic Topology

Gaussian curvature

Wang, B 2010

Heat kernel signature (HKS)

Scale-invariant HKS (SI-HKS)

Heat kernels

Heat kernel is a fundamental solution of the heat equation with

point heat source at (heat value at point after time )

Initial conditions: heat distribution at time

Solution : heat distribution at time

Heat equation governs heat propagation

on a surface

Heat kernel signature

Sun, Ovsjanikov & Guibas SGP 2009

Multiscale local shape descriptor

can be interpreted as probability of Brownian motion to return to

the same point after time (represents “stability” of the point)

Time (scale)

Heat kernel signature

Sun, Ovsjanikov, Guibas SGP 2009Ovsjanikov, BB & Guibas NORDIA 2009

Heat kernel signatures represented in RGB space

Scale invariance

B, Kokkinos CVPR 2010

Original shape Scaled by

HKS= HKS=

Not scale invariant!

Scale-invariant heat kernel signature

Log scale-space

Scaling = shift and multiplicative

constant in HKS

log + d/d

Undo scaling

Fourier transformmagnitude

Undo shift

0 100 200 300-15

t0 100 200 300

t0 2 4 6 8 10 12 14 16 18 20

Scale invariance

Heat Kernel Signature Scale-invariantHeat Kernel Signature

Scale invariance

Heat Kernel Signature Scale-invariantHeat Kernel Signature

Modeling vs learning

Wang, B 2010

Learning invariance

Positives P Negatives N

Similarity learning

with high probability

false positive

false negative

positive

negative

Similarity-preserving hashing

-1 -1 +1 -1-1 -1 -1 +1-1 -1 +1 +1

+1 +1 -1 +1+1 +1 +1 -1+1 +1 +1 +1

with low probability

Collision:

= # of distinct bits

Gionis, Indik, Motwani 1999Shakhnarovich 2005

Boosting

Construct 1D embedding

Similarity is approximated by

Downweight pairs with

Upweight pairs with

BBK 2010; BB Ovsjanikov, Guibas 2010Shakhnarovich 2005

Boosting

BBK 2010; BB Ovsjanikov, Guibas 2010Shakhnarovich 2005

Construct 1D embedding

Similarity is approximated by

Upweight pairs with

SHREC 2010 dataset

SHREC 2010 datasetBB et al, 3DOR 2010

Total dataset size: 1K shapes (715 queries)

Positives: 10K

Negatives: 100K

ShapeGoogle with HKS descriptorBB et al, 3DOR 2010

ShapeGoogle with SI-HKS descriptorBB et al, 3DOR 2010

Similarity sensitive hashing (96 bit)BB et al, 3DOR 2010

WaldHash

B2, Ovsjanikov, Guibas 2010

Construct embedding by maximizing

Remove pairs with

Upweight pairs with

Early decision

and sample in new pairs into the training set

positive

negative

B2, Ovsjanikov, Guibas 2010

Incommensurablespaces!

BB, Michel, Paragios CVPR 2010

Cross-modal similarity

Modality 1 Modality 2

How to compare apples to oranges?

Triangular meshes Point clouds

Cross-modality embedding

Cross-modality hashing

-1 -1 +1 -1-1 -1 -1 +1-1 -1 +1 +1

+1 +1 -1 +1+1 +1 +1 -1+1 -1 -1 +1

with low probability

Collision:

8x8 dimensionalbag of expressions

Database

32-dimensionalbag of words

Cross-representation 3D shape retrieval

1052 shapes

Number of bits

Retrieval performance

1 Michael Bronstein Shape Google: geometric words and expressions for invariant shape retrieval...

Documents

3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation

Shape Based Image Retrieval Using Fourier Descriptors

SVBRDF-Invariant Shape and Reflectance Estimation … we derive a spatially-varying (SV)BRDF-invariant theory for recovering 3D shape and reﬂectance from light-ﬁeld cameras. Our

Pose Invariant Deformable Shape Priors Using L1 Higher Order Sparse Graphs · 13 hours ago · Pose Invariant Deformable Shape Priors Using L 1 Higher Order Sparse Graphs Bo Xiang

1 Numerical Geometry of Non-Rigid Shapes Feature-based methods & shape retrieval problems Feature-based methods and shape retrieval problems © Alexander

Content-based Image Retrieval using intuitive Shape Partitioning

Automated crater shape retrieval using weakly-supervised

Texture and Shape for Image Retrieval – Multimedia

Partial 3D shape retrieval by Reeb pattern unfolding

Sketch-Based 3D Shape Retrieval Using Convolutional Neural …users.cecs.anu.edu.au/~fwang/publications/cvpr-2015-sbsr.pdf · 2015-07-28 · Sketch-based 3D Shape Retrieval using

A Pose Invariant Statistical Shape Model for Human Bodies3dbodyscanning.org › cap › papers › 2014 › 14327_26colaianni.pdf · A Pose Invariant Statistical Shape Model for Human

Two-dimensional shape retrieval using the …graphics.cs.ucdavis.edu/~hamann/NiuBremerLindstromHamann...Two-dimensional (2D) shape retrieval is an important problem in computer vision

Digital generation of shape-invariant Bessel-like beams · Digital generation of shape-invariant Bessel-like ... Bowman, N. Muller, ... Introduction Bessel beams (BBs)

Graph Matching for Shape Retrieval

ANSIG An Analytic Signature for Permutation Invariant 2D Shape Representation

Sketch-Based Shape Retrieval

Relevance Feedback for 3D Shape Retrieval - Technionwebee.technion.ac.il/labs/cgm/Computer-Graphics-Multimedia/... · Relevance Feedback for 3D Shape Retrieval George Leifman,

Cue-invariant shape recognition in rats as tested with ...jov.arvojournals.org/pdfaccess.ashx?url=/data/journals/jov/934653/... · Cue-invariant shape recognition in rats as tested

SHape REtrieval Contest 2007: Watertight Models Track

Estimates of shape by eye, or, The little invariant that could