International Conference on Automatic Face and Gesture Recognition, 2006

A Layered Deformable Model for Gait Analysis

Haiping Lu, K.N. Plataniotis and A.N. Venetsanopoulos

The Edward S. Rogers Sr.Department of Electrical and Computer Engineering

University of Toronto

FG2006, Southampton, UK 2Haiping Lu

Outline

Motivation Overview The layered deformable model (LDM) LDM body pose recovery Experimental results Conclusions

FG2006, Southampton, UK 3Haiping Lu

Motivation

Automated Human identification at a distance• Visual surveillance and monitoring applications

• Banks, parking lots, airports, etc.

USF HumanID Gait Challenge problem Articulated human body model for gait

recognition Manually labeled silhouettes

• Layered, deformable

FG2006, Southampton, UK 4Haiping Lu

Overview

Manual labeling

LDM recovery

Automatic extraction

LDM recovery

FG2006, Southampton, UK 5Haiping Lu

The Layered Deformable Model (LDM)

Trade-off:• Complexity Vs. descriptiveness

Match manual labeling: • Close to human’s subjective perception

Assumptions: • Fronto-parallel, from right to left.

FG2006, Southampton, UK 6Haiping Lu

LDM – 22 Parameters

Ten segments

Static:• Lengths (6)

• Widths (3)

Dynamic• Positions (4)

• Angles (9)

FG2006, Southampton, UK 7Haiping Lu

LDM –Layers and deformation

Four

layers

Deformation:

FG2006, Southampton, UK 8Haiping Lu

LDM – Summary

Summary: Realistic with moderate complexity

• Compact: 13 dynamic parameters

• Layered: model self-occlusion

• Deformable: realistic limbs

• Resemblance to manual labeling

FG2006, Southampton, UK 9Haiping Lu

Manual silhouettes pose estimation(ground truth & statistics)

Limb joint angles:• Reliable edge orientation

• Spatial–Orientation mean-shift (mode-seeking): dominant modes limb orientation

Others: • Joint positions, limb widths and lengths

• Simple geometry

• Torso: bounding box:

• Head: “head top” and “front face”

FG2006, Southampton, UK 10Haiping Lu

Post-processing

Human body constraints: • Parameter variation limits

• Limb angles inter-dependency

Temporal smoothing• Moving average filtering

FG2006, Southampton, UK 11Haiping Lu

Automatic pose estimation

Silhouette extraction (ICME06, Lu, et al.) Static parameters

• Coarse estimations: statistics from Gallery set

Silhouette information extraction based on ideal human proportion: • Height, head and waist center, joint spatial-

orientation domain modes of limbs

FG2006, Southampton, UK 12Haiping Lu

Ideal proportion of the human eight-head-high figure in drawing

FG2006, Southampton, UK 13Haiping Lu

Automatic pose estimation

Dynamic parameters: • Geometry on static parameters and silhouette

information, constraints.

Limb switching detection• Thighs & lower legs: variations of angles.

• Arms: opposite of thighs

• Frames between successive switch

Post-processing: smoothing

FG2006, Southampton, UK 14Haiping Lu

Experimental results

285 sequences from five data sets, one gait cycle each sequence.

Imperfection due to silhouette extraction noise and estimation algorithm

Feedback LDM recovery to silhouette extraction process may help.

FG2006, Southampton, UK 15Haiping Lu

LDM recovery results

Raw

LDM

manual

LDM

auto

FG2006, Southampton, UK 16Haiping Lu

LDM recovery example (revisit)

Manual labeling

LDM recovery

Silhouette extraction

LDM recovery

FG2006, Southampton, UK 17Haiping Lu

Angle estimation – left & right thighs

From manual silhouettes From automatically extracted silhouettes

FG2006, Southampton, UK 18Haiping Lu

Error rate (in percentage) for lower limb angles

FG2006, Southampton, UK 19Haiping Lu

Conclusions

A layered deformable model for gait analysis • 13 Dynamic and 9 static parameters

• Body pose recovery from manual (ground truth) and automatically extracted silhouettes.

Average error rate for lower limb angles: 7% Overall: close match to manual labeling,

accurate & efficient model for gait analysis Future work: model-based gait recognition

FG2006, Southampton, UK 20Haiping Lu

Acknowledgement

Thanks Prof. Sarkar from the University of South Florida (USF) for providing the manual silhouettes and Gait Challenge data sets.