Efficiently searching for similar images ( Kristen Grauman )

Efficiently searching for similar images (Kristen Grauman)

Efficiently searching for similar images (Kristen Grauman)

Universidad Catlica San Pablo

Cristina Patricia Cceres Juregui

[email protected]

1MotivationFast image search is a useful component for a number of vision problems.

Plenty of nuisance parameters (lighting, pose, background clutter, etc.)

2Nuisance parameters

3OutlineScalable image search

Fast correspondence-based search with local features

Fast similarity search for learned metrics

4Local image features

5How to handle sets of features?Want to compare, index, cluster, etc. local representations, but: Each instance is unordered set of vectors Varying number of vectors per instance

6Comparing sets of local featuresPrevious strategies:

Match features individually, vote on small sets to verify Explicit search for one-to-one correspondences

Bag-of-words: Compare frequencies of prototype features

7Pyramid match kernel

optimal partial matching

Optimal match: O(m3)Pyramid match: O(mL)

m = # featuresL = # levels in pyramid

8Pyramid match: main idea

descriptor space

Feature space partitions serve to match the local descriptors within successively wider regions.

9Pyramid match: main idea

Histogram intersection counts number of possible matches at a given partitioning.

10Image search with matching-sensitive hash functions Main idea: Map point sets to a vector space in such a way that a dot product reflects partial match similarity (normalized PMK value). Exploit random hyperplane properties to construct matching-sensitive hash functions. Perform approximate similarity search on hashed examples.11Locality Sensitive Hashing (LSH)Q111101110111110101h r1rkXiNh r1rk