Fast Pattern Matching. Fast Pattern Matching: Presentation Plan Pattern Matching: Definitions Classic Pattern Matching Algorithms Fast pattern matching

Fast Pattern Matching

Fast Pattern Matching:Presentation Plan

• Pattern Matching: Definitions

• Classic Pattern Matching Algorithms

• Fast pattern matching algorithms

Pattern Matching:Definition

• Pattern Matching:– The problem of locating a

specific pattern inside raw data.

• Pattern:– A collection of strings

described in some formal language.

Image Pattern Matching:Definition

• Pattern Matching:– Finding occurrences of a

particular pattern in an image.

• Pattern:– Typically a 2D image

fragment.

– Much smaller than the image.

Pattern Matching: Variations

• Detecting a set of patterns in a single image:– A set of distinct patterns.

– A set of transformations of a single pattern.

• Finding a particular pattern in a set of images:– Example: frames in a video sequence.

Pattern Matching:Main Tasks

• Search in Spatial Domain:– The pattern may appear in any

location in the image.

• Search in Transformation Domain:– The pattern may be subjected

to any transformation (in a transformation group).

Image Similarity Measures

• Image Similarity Measure:– A function that assigns a

nonnegative real value to two given images.

– Small measure high similarity

d( - ) ≥ 0

1 ( , )( , )

0

d P Q thresholdf P Q

otherwise

– Can be combined with thresholding.

Image Similarity Measure: The Euclidean Distance

• Pros:– Computed fast by convolution.

– Allows reducing the representations of linear spaces using Principal Component Analysis (PCA) approaches.

• PCA preserves Euclidean distance.

• Cons:– Not in accord with human perception.

– May change drastically when a small transformation is applied.

2( , )d P Q P Q

Pattern Matching: Applications

• Image Compression

• Video Compression

• Optical Character Recognition (OCR)

• Medical Image Registration

• Fingerprint Identification

• …

Pattern Matching: Real Time Applications

• Robot Vision

• Vehicle tracking

• Surveillance

• Industrial Inspection:– Automatic Part Inspection– Automatic Circuit Inspection


• Pattern Matching: Definition

• Classical Pattern Matching Algorithms

• Fast pattern matching algorithms:– Search in Spatial Domain– Search in Transformation Domain

The “Naïve” Approach

• Scan the entire image, pixel by pixel.• For each pixel, evaluate the similarity between its

local neighborhood and the pattern.

Naïve Approach using Euclidean Distance

• Given:– k×k pattern P(x,y)

– n×n image I(x,y)

• For each pixel (x,y), we compute the distance:

• Complexity: 2 2( )O n k

Improvement: FFT

Fixed 2( * )I P

• Convolution can be applied rapidly using FFT.• Complexity: 2( log )O n n

2 * mask of 1'sI k kOver all pixels:

Naïve and FFT Approaches: Performance

NaïveFFT

Time Complexity

Space

Integer ArithmeticYesNo

Run time for 16×161.33 Sec.3.5 Sec.



2( log )O n n2 2( )O n k2n2n

Performance table for a 1024×1024 image, on a 1.8 GHz PC:

Far too long for real-time applications!

Normalized Grayscale Correlation

• NGC:– A similarity measure, based on a normalized cross-

correlation function.– Maps two given images to [0,1] (absolute value).– Identical images are mapped to 1.

Given two images I and T, NGC is computed as follows:

Pattern Matching using NGC

+ =

• Pattern Matching using NGC:– Create a correlation map:

• Shift the pattern over the image.

• For each position, compute NGC(pattern,window).

– Find peaks in the correlation map.

Pattern Matching using NGC: Evaluation

• Pros:– Very accurate:

• By interpolating the correlation map, a pattern can be located in 1/16 pixel accuracy.

– Invariant to linear changes in brightness.

• Cons:– Computationally expensive:

• Matching k×k pattern with n×n image takes O(n2k2).

– Limited tolerance to variations in scale, orientation and illumination.

Multi Resolution Approach

Pattern Matching using a Gaussian Pyramid

First suggested by: Burt & Adelson (1984)

Multi Resolution Approach: Scale Invariant Search

• The same pattern is searched in each pyramid level.

• problems:– Computationally expensive.– Limited scale resolution.

Adelson, Anderson, Bergen, Burt, Ogden, “Pyramid methods in image processing” (1984)





Gradient Descent Search in an Image Pyramid

A technique for fast search in the spatial domain

Suggested by:

Maclean & Tsotsos (2000)

MacLean, Tsotsos, “Fast pattern recognition using gradient-descent search in an image pyramid” (2000)

Multi Resolution Schemes:Gradient Descent Search

• Create pyramid for pattern and image.

• Compute NGC between top level image and pattern.


– Find candidates using thresholding.

– For each candidate, estimate location in next level.

– Refine location by correlation gradient descent, using estimated location as starting point.

Iterate search untiltop-level

Gradient Descent Search: Evaluation

• Pros:– Full NGC is performed only in top level (small).

– The pattern pyramid is created off-line.

• Cons:– All inherent NGC limitations:

• Only 8-10% invariance to scale and rotation.


Heuristics: Focus of Attention

• Assumptions on the location of the pattern within the image.

• Used for motion estimation in video compression.

Divide a frame into blocks

In the next frame, search each block within its surrounding area

Code only motion vectors for each block

Pattern Matching using Projection Kernels

Y. Hel-Or, H. Hel-Or, "Real Time Pattern Matching Using Projection Kernels“ (2002)

A new technique for fast search in the spatial

domain

Suggested by:

Y. Hel-Or and

H. Hel-Or (2002)

Motivation

• Euclidean distance as a similarity measure.• Fast computation of a lower bound on the distance.• Fast rejection of non-matching windows.

Distance Measure in Sub-space

• Given:– k×k pattern p.

– k×k image window w.

• Both w and p can be represented as vectors in Rkxk:

|| - ||222 ( , )Ed p w p w


Distance Measure in Sub-space (cont.)

• Assume that p and w are not given.• However, we know their projection values onto a unit

vector u.• From the Cauchy-Schwarz Inequality:

2 2( , ) ( , )T TE Ed p w d u p u w

u p

w



• If we project p and w onto two unit vectors u and v, we can tighten the lower bound:

u

p

wv



• By projecting on a set of mutually orthonormal vectors U:

2T Ti i

i

LB u p u w • When U forms a basis, LB = Euclidean distance.

u p

w

v


Choosing Projection Kernels• A good choice of projection kernels U can expedite

the distance calculations.


– Projection on U should be fast to apply.

Natural Images

u1

– U should have high probability of being parallel to the vectors p - w.

• The first few kernels should capture a large proportion of the distance.

u pw

The Walsh-Hadamard Kernels


• The first 28 kernel vectors.

• Displayed in order of increasing spatial frequency.

Projection Kernels: Walsh-Hadamard

• Properties:– Fast Transform calculations:

• The basis vectors contain only 1.

• Transform calculations require only integer additions and subtractions.

– First few vectors capture high proportion of image energy.


Lower bound on distance between pattern and windowvs.

Number of projection kernels used

The Walsh-Hadamard Kernels (cont.)


Walsh-Hadamard Standard basis

Efficient Transformation Computation

• The fast transformation is due to:– Computation of one window is exploited in its

neighbor.– WH kernels have a recursive structure:

• Projection on one kernel is exploited in the next kernel.

– Projection onto the first few WH kernels is usually sufficient.


The Walsh-Hadamard Tree (1D)

+-+ -

- +

+

+ ++ -

+ + + ++ + - -+ - + - + - - ++ - +- + - +-

+ - + - - + - ++ - + - + - + - + + + + + + + ++ + + + - - - -+ + - - + + - - + + - - - - + ++ - - + + - - ++ - - + - + + -


A tree for computing projections onto first 8 WH basis vectors:

The root:Contains the original signal

A node:Contains intermediate computations

The i’th leaf:Contains a vector of projection values of all signal windows onto the i’th WH kernel

Tree height = log8

The Walsh-Hadamard Tree (2D)

+

+ - + +

+ + + +

+ + - -

+ - + -

+ - - +

+ - + +

+ +

+-

+-

+ +


• In 2D:– The projections are performed in a similar manner.

– Height of the tree: log(k) 2log(k).

– Number of leaves: k k2

The Pattern Matching Algorithm

1) Project all image windows on the first WH kernel.– Find a lower bound on the distance between the pattern and each

window.

2) Reject all windows whose lower bound exceeds a given threshold.

3) Project only the remaining windows on the next kernel.– Tighten the lower bounds of the remaining windows.

4) Return to step (2), unless either:– All k kernels have been processed.

– The number of remaining windows reached a predefined value.


Iterate

Performance


Average ops/pixel

SpaceInteger Arithmetic

Run Time

NaiveO(k2)n2Yes4.86 Sec

FFTO(logn)n2No3.5 Sec

Projection Kernels

O(logk)n2 log kYes78 MSec

• Performance table for a 1024×1024 image and a 32×32 pattern.

• Performed on a 1.8 GHz PC.

Example

Image (256×256) Pattern (16×16)

Initially: 65536 candidates


Example (cont.)

Image Pattern

After the 1st projection: 563 candidates


Example (cont.)

Image Pattern

After the 2nd projection: 16 candidates


Example (cont.)

Image Pattern

After the 3rd projection: 1 candidate


Tolerance to Noise

Original Noise Level = 40 Detected Patterns


Results of the algorithm when applied to a noisy image:

• Requires a large rejection threshold: More candidates Longer running time

Illumination Invariance

• The DC of all windows is given by projection on the first WH kernel.


+

+ - + +

+ + - -

+ - + -

+ - - +

+ - + +

+-

+-

+ +

+ + + +

+ +

• We can disregard the DC by skipping the first kernel.• The rest of the process continues as before.

Illumination Invariance (cont.)

Original Image Illumination gradient added

Detected patterns


• Results of the algorithm (without DC):

• Only 5 projections were performed.

P. Matching using Projection Kernels: Summary

• Fast projection computations.

• Fast rejection of non-pattern windows.

• Invariant to:– Noise– Illumination variations






Search in Transformation Domain

• Represent a k×k pattern P as a point in kxk:

• Let T(α)P be a transformation T(α) applied to P:

kxkP =

kxkT(α)P =

• T(α)P for all α forms an orbit in kxk:

T()P

T(0)P

T(2)P

T(1)P

P

Transformation Domain:Example

• A pattern under 2D rotations in Euclidean space.

• Sampled at equal rotation angles.

• Projected on its three most significant components.

Similarly perceived patterns may be distant in Pattern Space!

Search in Transformation Domain: Problem Formulation

• Given:– Distance metric d(Q,P)

– k×k image window W

• We want to find:

(W,P) = min d(W,T()P) W

P

(W,P)

• If (W,P) < threshold, W matches P.

Search in Transformation Domain: Approaches

• Orbit Simplification• Dimensionality Reduction• Fast Search• Orbit Decomposition

Orbit Simplification:Transformation Invariant Functions

• First suggested by Hu (1961).

• Find a function over the pattern space, which is constant for some transformation parameters.

M. Hu, “Pattern recognition by moment invariants” (1961)

Transformation Invariant Functions (cont.)

• Create a blob image (by thresholding).

• A blob enhances low frequency components:– Size

– Orientation

– Low-level shape

M. Hu, “Pattern recognition by moment invariants” (1961)

• Hu derived a set of functions, based on the central blob’s moments.

• Function’s output is independent of:– Translation

– Rotation

– Reflection

Dimensionality Reduction

• Reduce the dimensionality of the pattern space:– Create sparse representation using Wavelets and

DCT transform:• Energy of natural patterns is concentrated in few

coefficients.

– Principal Component Analysis (PCA):• Find a reduced linear basis for the pattern space.• Search the reduced space.

Fast Exhaustive Search in the Transformation Domain

• Apply an exhaustive search in some of the transformation domains.

• Requires a fast search technique.

• Example:– Fast search for different translations and scales

using:• Pyramidal representation.

• Fast implementation of convolution.

Pattern Matching using Orbit Decomposition

A new technique for fast search in a

pattern orbit

Suggested by:

Y. Hel-Or and

H. Hel-Or (2002)

Y. Hel-Or, H. Hel-Or, “Generalized Pattern Matching using Orbit Decomposition”

Orbit Distance

• Recall:– d(Q,P) is a distance metric, where Q,Pn.

– We want to find the orbit distance:(Q,P) = min d(Q,T()P)

Q

P

R


• But:– In the general case, (Q,P) is not a metric.

• It doesn’t satisfy the triangle inequality:

Orbit Distance (cont.)

• Observe:– If the distance d(Q,P) is transformation invariant:

d(Q,P)= d(T()Q, T()P)

Then:

Q

P

S


1 (Q,P) is a metric.2) point-to-orbit distance

is equal toorbit-to-orbit distance.

Orbit Decomposition• In practice T() is sampled into T(i) = T(i), i=1,2,…

QP

(Q,P)

T(i)P

P’

• We can divide T(i)P into two sub-orbits:

T2(i)P and T2(i)P’ where P’= T(1)P

2

T2(i)P


Orbit Decomposition (cont.)

T2(i)P T2(i)P’

PQPQMinPQ ,,,, 22

Q

T(i)P

(Q,P)

P

P’

2(Q,P) 2(Q,P’)

P

P’



T2(i)P T2(i)P’

Q

2(Q,P) 2(Q,P’)

P

P’


2 (P,P’)

2 is a metric

2 (P,P’) can be calculated in advance

We can save calculations using the triangle inequality constraint


The decomposition can be applied recursively:


Orbit Tree


Threshold = 100

Example


Example (cont.)


Example (cont.)


Example (cont.)


Example (cont.)


Example (cont.)


Example (cont.)


Rejection Rate

# Distance Calculations5 10 15 20 25 30

0102030

40506070

8090100%

Im

age

Pix

els

Rem

aini

ng


Average number of distance computations per pixel is 2.868

P. Matching using Orbit Decomposition:Summary

• Orbit distance is a metric– When point distance is transformation invariant.

• Fast search in orbit distance space– Using recursive orbit decomposition.

• Fast rejection of distant patterns.

Documents

Fast Pattern Matching. Fast Pattern Matching: Presentation Plan Pattern Matching: Definitions Classic Pattern Matching Algorithms Fast pattern matching