1

Detecting Pedestrians by Learning Shapelet Features

Payam Sabzmeydani and Greg MoriVision and Media Lab

School of Computing Science

Simon Fraser University

2

Problem Given a still image, we want to find and locate

the pedestrians in the image Clothing (color, appearance) Body pose

Applications: Automated surveillance systems Image search and retrieval Robotics Intelligent vehicles

3

Problem

4

Problem Classification-based detection

Classify a window as pedestrian or non-pedestrian Search exhaustively the scale-space image

Different cues Wavelet coefficients (Mohan et al., PAMI 2001) Oriented gradients (Dalal and Triggs, CVPR 2005) SIFT features (Leibe et al., CVPR 2005) Edgelet features (Wu and Nevatia, ICCV 2005) “Shapelet features” (Sabzmeydani and Mori, CVPR 2007)

5

Datasets MIT : Standing pose, simple background, no occlusion

INRIA : Standing pose, complex background, partial occlusions

6

Previous Work Dalal & Triggs (CVPR 2005)

HOG features + SVM

7

Previous Work Wu & Nevatia (ICCV 2005)

Edgelet features: short line and curve segments AdaBoost

8

Our Method

Compute low-level gradient features Oriented filter responses

Learn mid-level features for detecting pedestrians “Shapelet features”

Build final classifier from shapelet features

9

Low-level Features Filter responses

Image gradient in different directions

10

Low-level Features Smoothed gradient responses in different

directions

11

Shapelet Features A weighted set of low-level gradient features

inside a sub-window of the detection window Characteristics

Simple and low-dimensional Learned exclusively for our object classes Highly discriminative Local effective area : useful to model separate parts

instead of the whole body

12

Shapelet Features

13

Learning Shapelet Features Learned using AdaBoost (Viola and Jones,

2001) Extract low-level features in sub-window Select subset of features using AdaBoost

Find those which discriminate between pedestrian and background classes

14

AdaBoost Algorithm

W

w

15

Low-level features as weak classifiers Each low-level feature can provide us many

weak classifiers:

AdaBoost will combine weak classifiers to form a better classifier:

16

Shapelet features Train classifiers in sub-windows Use the output of a classifier as the shapelet

feature response:

17

Shapelet Features

18

Shapelet Features

19

Final Classifier Take all shapelet features

Learned at many sub-windows of detection window

Run AdaBoost again to select weighted subset of shapelet features for final classifier

20

Final Classifier

21

Shapelet Feature Size Small, Medium, and Large features

Capture different scales of information

22

Why normalize? Different lighting, shadows, different contrast, …

How to normalize? Per shapelet feature : L2-norm

Normalization

23

Normalization

24

Results on INRIA Dataset

25

Most non-pedestrian-like pedestrians (false negatives)

Most pedestrian-like non-pedestrians (false positives)

Error examples

26

Future work

Detecting other objects Use image context or segmentation Pyramid of features

27

28

References N. Dalal and B. Triggs. “Histograms of oriented

gradients for human detection”. CVPR 2005. B. Wu and R. Nevatia. “Detection of multiple, partially

occluded humans in a single image by bayesian combination of edgelet part detectors”. ICCV 2005.

P. Viola and M. Jones. “Rapid object detection using a boosted cascade of simple features”. SCTV 2001.

29

Problem

30

Problem

31

Bootstrapping

32

Mid-level Features