Human tracking and counting using the KINECT range sensor based on Adaboost and Kalman Filter

Human tracking and counting using the KINECT range sensor based on Adaboost and Kalman Filter

ISVC 2013

Problem

• Human tracking

Avoid occlusion

Human Detection

• Observations:– There is an empty space in the front and back of

head– The right side of right shoulder and the left side of

left shoulder are also empty– There is a height difference between the head and

the two shoulders

How to describe the spatial information of 3D HASP

• Those criteria can be formulated as the difference of two pixel areas in the depth map – Haar-like feature

• Adaboost is introduced to construct a strong classifiers from those weak criteria

Human Detection

Human Detection by Adaboost

• Framework

Spatial feature

• Processing window– 20 redefined sub-windows

Spatial feature

• Four Haar-like features

Depth integral image

• The sum of rectangle pixel values from the top-left corner to a pixel in depth image– To speed up the computation of Haar-like features

• All pixel intensity values of D:( ) (4) (3) (2) (1)areaValue D dd dd dd dd

Adaboost algorithm

• Construct a strong classifier by a weighted linear combination of weak classifiers

1, * ( ) *

0, * ( ) *

1, * ( )H( , , , )

1,

j j

j j

H jF

H j

wherep h x p

h x potherwise

Our Classifier

• Challenge– Human can stand and face all directions with many

postures

• Solutions– Combine a horizontal strong classifier and a

vertical strong classifier

( ) ( ) | ( )C hor verwin win winF F F

Horizontal Strong Classifier

• Formulation

1, * ( ) *( )

0, * ( ) *j j

horj j

H jwin

H jF

Vertical Strong Classifier

• Formulation

1, * ( ) *( )

0, * ( ) *j j

verj j

H jwin

H jF

Training

• Took many depth maps of each object by rotating a certain degree

• 720 positive images + 288 negative images

Results

• Testing on three datasets:– Dataset 1: only one human object standing in

different directions– Dataset 2: Two human objects– Dataset 3: three or more human objects

Results (Dataset 1)

Results (Dataset 2)

Results (Dataset 3)

Choice of window sizes

Limitation

• Fails if detected humans are standing two very close to each other– Improve tracking accuracy by incorporating

Kalman Filter, since the closing time is short in real tracking application.

Conclusion

• We construct a real-time human detection based the depth image from Kinect sensor

• Head and Shoulder Profile described by some Haar-like features is incorporated into Adaboost algorithm to detect human objects.

• Detection time for each image is about 33 ms.