CVPR 2006 New York City
Spatial Random Partition for Common Visual Pattern Discovery
Junsong Yuan and Ying Wu EECS Dept. Northwestern Univ.
{j-yuan,yingwu}@northwestern.edu
ICCV 2007
ICCV 2007 Rio de Janeiro, Brazil 3
Challenges
No prior knowledge of the common patterns– What are they ? appearances– Where are they ? locations– How large are they ? scales– How many of them ? number of
instances
Computationally demanding– Exponentially large solution space– Large image dataset
Robust similarity matching
ICCV 2007 Rio de Janeiro, Brazil 4
Related Work Pattern Discovery by Matching Visual Words
– J. Sivic and A. Zisserman, CVPR04– J. Yuan, Y. Wu and M.Yang, CVPR07– S. Nowozin, K. Tsuda, T. Uno, T. Kudo and G. Bakir,
CVPR07– T. Quack, V. Ferrari, B. Leibe and L. V. Gool, ICCV07– …
Pattern Discovery by Direct Matching – O. Boiman and M. Irani, ICCV05, NIPS06– K. Grauman and T. Darrell, CVPR06, NIPS07– K.-K. Tan and C.-W. Ngo, ICCV05 – N. Ahuja and S. Todorovic, CVPR06, ICCV07– …
ICCV 2007 Rio de Janeiro, Brazil 5
Spatial Random Partition
DI
DR
SR
Votingmaps
CommonPatterns
RandomPartition
Matching
Voting
Localization
ICCV 2007 Rio de Janeiro, Brazil 6
Visual Primitives
Visual Primitives: Scale Invariant Feature Transformation (SIFT, D.Lowe,
IJCV04 )
Locality Sensitive Hashing (LSH) for matching visual primitives– For each visual primitive, search for its
matches from other images, based on Euclidean distance
ICCV 2007 Rio de Janeiro, Brazil 7
Matching Subimages A many-to-many assignment problem.
Fast approximation by set intersection:
where
is the # of visual primitives in the subimage
ICCV 2007 Rio de Janeiro, Brazil 8
Another View: Max Flow
VR MVQ
VR VQ..
..
.
... .. .
..
....
. ... . ..
Visual primitives
subimage
Problem: Matching two sets of m and n points (feature vectors)
Fast Approximate Solution: set intersection (linear complexity)
ICCV 2007 Rio de Janeiro, Brazil 9
An Example Final estimation of similarity score:
= 3
T K
A EB DC F
S
P
P Z
H
Z X
I R
QG
Y
J
O
SX
Z
W
L
M
Y
VR
VQ
MVQ
N
Z
X
Y=
LSH for Fast Query
N
ICCV 2007 Rio de Janeiro, Brazil 11
Asymptotic Property Theorem: Given two pixel i and j, where
i locates in a common pattern while j locates in the background, let and the total votes i and j receives regarding to K random partitions. Both and are discrete random variables and we have
Proof: using the weak law of large numbers, see the Appendix for details
ICCV 2007 Rio de Janeiro, Brazil 15
Image Irregularity Detection
Differences from common pattern discovery– disocver unpopular subimages instead of popular
ones– Adjust voting weight proportional to the subimage
size: the larger the unpopular subimage, the more possible it contains an irregular pattern
ICCV 2007 Rio de Janeiro, Brazil 16
Evaluation Collect 8 image datasets, each contains 4-8 images. An image
dataset contains 1-3 common patterns each has 2-4 instances (* indicates the dataset containing multiple common patterns )
Comparisons of computational complexity, around 12 sec. for 2 images
J. Sivic & A. Zisserman
04O. Boiman & M. Irani
05
ICCV 2007 Rio de Janeiro, Brazil 17
Conclusion A novel spatial random partition
method for common pattern discovery and irregularity detection in images– No construction of visual vocabularies– Trade-off of performance and efficiency
by the total number of random partitions– Efficient by using LSH and approximate
matching between subimages– Theoretically justified by the asymptotic
property of the algorithm