DESCRIPTORS(DESCRIPTION OF INTEREST REGIONS WITH LOCAL

BINARY PATTERNS)

Yu-Lin Cheng(03/07/2011)

OUTLINE Scale Invariant Feature Transform (SIFT)

Descriptor

Local Binary Pattern (LBP) Descriptor Center-Symmetric LBP (CS-LBP) Descriptor

Histogram of Oriented Gradients (HOG) Descriptor

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

SIFT Algorithm:

descriptor

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Scale-space Extrema Detection: Stable feature points ----- (scale invariant)

Principle: A local maximum over scales by using combination

of normalized derivatives can be treated as a characteristic point of local structure

Use LoG to find maximum

scale

bad

scale

Good !

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Scale-space Extrema Detection: Use DoG instead of LoG ---- (computational efficiency)

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Scale-space Extrema Detection:

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Scale-space Extrema Detection: Local extrema detection:

Compare to 26 neighbors

Keep the same keypoint in all scale

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Scale-space Extrema Detection: Reject points with low contrast

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Accurate keypoints localization:Quadratic function to interpolate the

location of maximum

Eliminate edge response:

r: threshold, H: Hessian matrix

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Orientation Assignment: Assign a consistent orientation to achieve

orientation invariant

Method:

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Orientation Assignment: Calculate gradient magnitude and direction of

neighboring pixels

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Orientation Assignment: Calculate weighted orientation histogram

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Orientation Assignment: Calculate weighted orientation histogram

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Orientation Assignment: Calculate weighted orientation histogram

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Keypoints Descriptor: Empirical result:

Cell size: 44 pixels Block size: 44 cells Dimension: 44 (cells) 8 (bins) = 128

Weighted magnitude

SIFT(SCALE INVARIANT FEATURE TRANSFORM )

Keypoints Descriptor: Avoid all boundary effect

Use trilinear interpolation

Normalization: (illumination invariant) Normalize to unit length Threshlod the maximum value to 0.2

Match the magnitudes for large gradients is no longer important

Renormalize to unit length

LBP(LOCAL BINARY PATTERN)

A powerful mean of texture description

LBP operator: Standard LBP:

Illustration:

LBP(LOCAL BINARY PATTERN)

Example:

Parameters: P : Number of neighboring pixels R : Radius

LTP(LOCAL TRINARY PATTERN)

LTP operator:

t : threshold

Illustration:

CS-LBP(CENTER-SYMMETRIC LOCAL BINARY PATTERN)

CS-LBP operator:

Illustration:

CS-LBP DESCRIPTOR Flow diagram:

CS-LBP DESCRIPTOR Interest Region Detection:

Detectors: 1. Hessian-Affine (blob-like structure) 2. Harris-Affine (corner-like structure) 3. Hessian-Laplace (scale-invariant version) 4. Harris-Laplace (scale-invariant version)

4141

CS-LBP DESCRIPTOR Feature Extraction:

CS-LBP operator: Parameters:

R: radius R = 1, 2

N: number of neighboring pixels N = 6, 8

T: threshold T = 0.2

Descriptor Construction: Location grids

33 cells/44 cells Avoid boundary effects:

Using ‘bilinear interpolation’

4141

CS-LBP DESCRIPTOR Descriptor Normalization: (illumination invariant)

Normalize to unit length Thresholding Renormalize to unit length

24× (4×4 )=256

COMPARISON(SIFT V.S. CS-LBP)

Assumption: Computations cannot be reused from detection

algorithm

Comparison:

Conclusion: Computational efficiency and better performance

than SIFT

HOG(HISTOGRAM OF ORIENTED GRADIENTS)

HOG(HISTOGRAM OF ORIENTED GRADIENTS)

Gradient Computation:

HOG(HISTOGRAM OF ORIENTED GRADIENTS)

Gradient Computation:

HOG(HISTOGRAM OF ORIENTED GRADIENTS)

Spatial/Orientation Binning: Weighted votes

Function of magnitude Avoid aliasing

Interpolation

Parameters: Number of orientation bins Cell size Block size

Cell Block

HOG(HISTOGRAM OF ORIENTED GRADIENTS)

Spatial/Orientation Binning: Parameters:

Number of orientation bins: 9 bins/18bins Cell size: 88 pixels Block size: 22 cells

HOG(HISTOGRAM OF ORIENTED GRADIENTS)

Normalization: Group cells to larger blocks and normalize each

block separately (illumination invariant)

Normalization Schemes:

HOG(HISTOGRAM OF ORIENTED GRADIENTS)

Normalization: Normalization Schemes:

COMPARISON(SIFT V.S. HOG)

Comparison:

HOG VARIATION

‘Object Detection with Discriminatively Trained Part Based Models’

Pixel-Level Feature Maps: Use [-1, 0, 1] to calculate gradient Contrast sensitive(B1), Contrast insensitive(B2)

,(p = 9)

Quantize into orientation bins

r: gradient magnitude

HOG VARIATION

Spatial Aggregation: Rectangular cell: 88 pixels Cell-based feature map:

Reduce the size of feature map Avoid aliasing:

Bilinear interpolation

Normalization:

HOG VARIATION

Truncation:

maximum 0.2 No renormalization

Dimension: 9 bins 4 different normalization = 36 (contrast

insensitive)

HOG VARIATION

PCA analysis:

Top 11 eigenvectors captures most of information of HOG

HOG VARIATION

PCA analysis: Top eigenvectors lie (approximately) in a linear

subspace

13-dimensional features: Project 36-dimensional HOG feature into uk, vk

Projection into uk : sum over 4 normalization over fixed orientation

Projection into vk : sum over 9 orientation over fixed normalization

HOG VARIATION

For Contrast Insensitive(B2): 9 bins 4 different normalization = 36 (contrast

insensitive)

For Contrast Sensitive(B1): 18 bins 4 different normalization = 72 (contrast

insensitive)

Reduce to (18 + 9) + 4 = 31 dimension

REFERENCE “Description of Interest Regions With Local Binary

Patterns”, Pattern Regonization ’09 Marko Heikkilä http://www.tele.ucl.ac.be/~devlees/ref_ELEC2885/projects/Ro

IdescriptionLBP-pr-accepted.pdf

“Effective Pedestrian Detection Using Center-symmetric Local Binary/Trinary Patterns”, Youngbin Zheng

“Scale-space Theory” Tony Lindeberg “Histogram of Oriented Gradients for Human

Detection”, CVPR ‘05 Navneet Dalal “Finding People in Images and Videos”, Navneet Dalal “Feature matching” Yung-Yu Chuang “Scale & Affine Invariant Interest Point Detectors”,

IJCV ’04 Krystian Mikolajczyk

REFERENCE “Object Detection with Discriminatively Trained Part

Based Models” “Distinctive Image Features from Scale-Invariant

Keypoints”, IJCV ’04 David G. Lowe http://

citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.157.3843&rep=rep1&type=pdf