Dept. of Electrical and Computer Engineering University of Missouri, Columbia, MO Page 1 Silhouette Analysis-Based Gait Recognition for Human Identification

Dept. of Electrical and Computer EngineeringUniversity of Missouri, Columbia, MO

Page 1

Silhouette Analysis-Based Gait Recognitionfor Human Identification

Liang Wang, Tieniu Tan, Huazhong Ning, and Weiming HuDecember 2003

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE

Presenter:Xi Chen


Page 2

Background and Research Problem

• Biometrics is a technology that makes use of the physiological or behavioral characteristics to authenticate the identities of people.

• The interest of vision-based human identification at a distance is driven by the need for automated person identification systems for visual surveillance and monitoring applications in security-sensitive environments such as banks, parking lots, and airports

• So far, gait is probably the only perceivable biometric feature from a great distance. In comparison with other first-generation biometric features such as fingerprint and iris, gait has the advantage of being unobtrusive.

• Gait recognition, also called gait-based human identifications, is a relatively new research direction in biometrics, which aims to discriminate individuals by the way they walk.


Page 3

Related Work

• Current gait recognition approaches may be classified into two main classes, model-based and motion-based methods.

• Model-based approaches aim to model human body or motion and perform model matching in each frame of a walking sequence.

• Motion-based approaches can be divided into two main classes. The first class, state-space method considers gait motion to be composed of a sequence of static body poses. The second class, spatiotemporal methods, characterizes the spatiotemporal distribution generated by gait motion in its continuum.

• A number of approaches have already shown that it is possible to recognize people by gait.


Page 4

Overview of Approach


Page 5

Purpose and Contribution

The proposed method has several desirable properties:

• It is easy to comprehend and implement.

• It is insensitive to the color and texture of cloth as a silhouette-based approach.

• Some additional features related to pace, stride, and build are used to improve recognition accuracy.

• Experimental results demonstrate that it has a relatively low computational cost.


Page 6

Human Detection and Tracking

Background Modeling: • The main assumption made here is that the camera is static, and the only moving object in video sequences is the walker.• Here, the LMedS (Least Median of Squares) method is used to construct the background. Let I represent a sequence including N images. The resulting background can be computed by:

2)(minarg pImedb txyt

pxy

Where p is the background brightness value to be determined for the pixel location (x,y), med represents the median value, and t represents the frame index ranging within 1-N.

xyb


Page 7


Differencing between the Background and the Current Image:

In this paper, the following extraction function is used to indirectly perform differencing:

)256()256(

)256)(256(2

)1()1(

)1)(1(21),(

ba

ba

ba

babaf

Where a(x,y) and b(x,y) are the brightness of current image and the background at the pixel position (x,y).

This function can detect the change sensitivity of the difference value according to the brightness level of each pixel in the background image.


Page 8



Page 9


Postprocessing and Tracking:

• Differencing process is independently performed for each component R, G, and B. For a given pixel, if one of the three components determines it as the changing point, then the pixel will be set to the foreground.

• No change detection algorithm is perfect. So, it is imperative to remove as much noise and distortion as possible from the segmented foreground. Morphological operators such as erosion and dilation are used to further filter spurious pixels, and small holes inside the extracted silhouettes are then filled.


Page 10

Feature Extraction

Silhouette Representation:

• For the sake of computational efficiency, we convert these 2D silhouette changes into an associated 1D signals to approximate temporal pattern gait.

• After the shape centroid is obtained, by choosing it as the reference origin, we can unwrap the outer contour counterclockwise to turn it intoa distance signal.


Page 11

Feature Extraction

Silhouette Representation:


Page 12

Feature Extraction

Principal Component Analysis (PCA) Training:

• The purpose of PCA training is to obtain several principal components to represent the original gait features from a high-dimensional measurement space to a low-dimensional eigenspace.

• Given s classes for training, and each one represents a sequence of distance signals of one subject’s gait.

• Let Dij be the jth distance signal in class i and Ni the number of such distance signals in the ith class. The total number of training samples is

st NNNN ...21


Page 13

Feature Extraction

PCA Training:

• The mean and the global covariance matrix of such a data set can be calculated as:

s

i

N

j

Tdjidji

t

s

i

N

jji

td

i

i

mDmDN

DN

m

1 1,,

1 1,

))((1

1


Page 14

Feature Extraction

PCA Training:

• We can compute the N nonzero eigenvalues and the associated eigenvectors based on SVD (Singular Value Decomposition).

• Considering the memory efficiency in practical applications, only those bigger eigenvalues and corresponding eigenvectors are kept using a threshold Ts=0.95.

N ,...,, 21

Neee ,...,, 21

s

N

ii

k

iik TW

11

Where Wk is the accumulated variance of the first k largest eigenvalues with respect to all eigenvalues.


Page 15

Feature Extraction

Projection to the Eigenspace:

• Taking only k<N largest eigenvalues and their associated eigenvectores, the transform matrix can be constructed to project an original distance signal into a point in the k-dimensional eigenspace.

]...[ 21 keee

jiD , jiP,

jiT

kji DeeeP ,21, ]...[ • k is usually much smaller than then original data dimension N. That is to say, eigenspace analysis can drastically reduce the dimensionality of input samples. For each training sequence, the projection centroid is given by:

iN

jji

ii P

NC

1,

1


Page 16

Recognition

Similarity Measures:

(1) Spatiotemporal Correlation: for two input sequences, they are projected into and in the eigenspace. The similarity between two such vector sequences can be computed by:

)(1 tP )(2 tP

T

tab

batPtPd1

2

212 )(')(min

)('2 batP )(2 tPWhere is a dynamic time warping vector from with respect to time stretching and shifting for an approximation of the temporal alignment between the two sequences. The selection of the parameters a and b depends on the relative stride frequency and phase difference within a gait period respectively.


Page 17

Recognition

Gait Period Analysis: serves to determine the frequency and phase of each observed sequence so as to align sequences before matching.


Page 18

Recognition

)(1 tP


• The selection of the parameters a and b depends on the relative stride frequency and phase difference within a gait period respectively.

• Let and denote the frequencies of the two gait sequences, then we can get .

• By cropping a subsequence of length from the second sequence vector and stretching it with a, we may obtain its correlation with .

• The minimum of all prominent valleys of the correlation results determines their similarity.

1f 2f

12 ffa

2f


Page 19

Recognition


(2) The Normalized Euclidean Distance (NED): the computational cost will increase quickly if the similarity is performed in the spatiotemporal domain, especially when time stretching and shifting is taken into account. The NED is used to measure the similarity of two gait sequences only with the projection centroids.

Each projection centroid implicitly represents a principal structural shape of certain subject in the eigenspace. The normalized Euclidean distance between the two sequential projection centroids can be defined by:

2

2

2

1

12

C

C

C

Cd

iN

jji

ii P

NC

1,

1


Page 20

Recognition

Classifier:

• The classification process is carried out via two simple different classification methods, the nearest neighbor classifier (NN) and the nearest neighbor classifier with respect to class exemplars (ENN).

• represents a test sequence and represents the ith reference sequence. This test sequence is classified into class that can minimize the similarity distance between the test sequence and all reference patterns by:

T iRc

),(minarg iii

RTdc


Page 21

Experiments

Data Acquisition:

• A new gait database, called NLPR database, is established for the experiment. • All subjects walk along a straight-line path in three different views with respect to the image plane, laterally, obliquely, and frontally.• The resulting NLPR database includes 20 subjects and 4 sequences for each viewing angle per subject.


Page 22

Experiments

Preprocessing and Training:• For each image sequence, motion segmentation and tracking are performed to extract the silhouette of the walking subjects.• The extracted silhouettes are converted into an associated sequence of 1D distance signals before training and projection.


Page 23

Experiments

Preprocessing and Training:• The first 15 eigenvalues and their associated eigenvectors are kept to form the eigenspace transformation matrix.• Then each silhouette image can be mapped to one point in a 15-dimensional eigenspace.


Page 24

Results and Analysis

Identification Mode:

• A useful classification performance measure, defined as the cumulative probability that the real class of a test measurement is among its top k matches, is used. To show the performance graphically, the rank k is plotted in the horizontal axis, and the vertical axis is the match score.

• The leave-one-out cross-validation rule is used with the NLPR to estimate the performance of the proposed method.

• After computing the similarity between the test sample and the training data, the NN and ENN are then applied for classification.


Page 25




Page 26



• The identification performance using NED is, in general, better than that using STC. In theory, STC should better capture spatiotemporal characteristics of gait motion than NED. This result probably due to the fact that segmentation errors maybe accumulated into a quick-enlarged match error.

• The NED based on the exemplar projection centroid performs better than NED using only a single projection centroid.

• The recognition performance under frontal walking is the best. This result is probably due to the averaging associated with the silhouette shape analysis because there are less severe variations of silhouette appearances in such gait patterns compared with other views.


Page 27


Verification Mode:• In this paper, the FAR (False Acceptance Rate) and FRR (False Reject Rate) are also estimated via the leave-one-out rule in verification mode.


Page 28


Validation Based on Physical Features:

• In experiments, recognition errors often occurred when the two smallest values of similarity function are very close.

• When the difference between the last two minima is lower than a predefined threshold, some additional features available from the training sequences are introduced to validate the final decision.

• The features can be body height, build, and stride length, which can be obtained in the training process.


Page 29




Page 30



• The experiment results shows that the additional physical features improve the performance of the classifier without validation.


Page 31

Comparisons

• Compared with other algorithms, the algorithm proposed has betterperformance and the advantage of lowest computational cost.


Page 32

Discussion and Future Work

• To provide a general approach to automatic person identification in unconstrained environments, much remains to be done.

• Further evaluation on a much larger and most-varied database is stillneeded. We are planning to set up such a database with more subjects, more sequences and more variations in conditions.

• It is more efficient for recognition to extract dynamic information such as the oscillatory trajectories of joints. Future work may try to combine both static and dynamic features of gait for recognition.

• Also, seeking better similarity measures, designing more sophisticatedclassifiers, gait segmentation, and the evaluation of different scenariosdeserve more attention in future work.

Documents

Dept. of Electrical and Computer Engineering University of Missouri, Columbia, MO Page 1 Silhouette Analysis-Based Gait Recognition for Human Identification