Experimental ResultsAbstractFingerspelling is widely used for education and communication among signers. We propose a new static fingerspelling recognition framework based on a left to right Hidden Markov model (HMM):

An overlapping frame-based analysis is used to generate features;

Histogram of Oriented Gradient (HOG) features are extracted.

Sub-gesture models are used that allow for precise modeling of background noise.

Three design issues were investigated:

the number of sub-gestures;

the number of states in the HMMs;

signer-dependent (SD) and signer-independent (SI) training.

Our best singer-independent system achieves an error rate of XX.X% on the <database> which represents the best published results on this data.

Shuang Lu and Dr. Joseph PiconeDepartment of Electrical and Computer Engineering, Temple University

Fingerspelling Recognition Using sub-gesture Hidden Markov Model

College of EngineeringTemple University

Recognition Architecture

Sub-gesture HMMs Error Analysis Conclusions Sub-gesture HMMs performance of

52.4% represents best published results on the <database name>.

Completely automated training based only on the identity of a gesture.

Performance is most sensitive too… and least sensitive too…

The architecture can be extended to continuous sign language recognition.

Future Work State something about the problems

that need to be solved… State something about the problems

that need to be solved…

Applications

www.isip.piconepress.com

Education

Entertainment

Communication

Robotic control

Human computer interaction

1 2 3 4 5 6 7 8 9 10 Avg0.0

3.0

6.0

9.0

12.0

15.0

18.0Subject1Subject2Subject3Subject4Subject5

Signer dependent test

Data:

5 signers

24 fingerspelling gestures

…explain what they are

over 500 tokens per gesture

variations in backgrounds … say something about your

cross-validation approach

Best system configuration:

No. of sub-gestures: 11

HMM order for sub-gestures: 13

Long background order: 11

Short background order: 1

Frame and window size: 5 / 30

No. of HOG feature bins: 9

No. Gaussian Mixtures: 16

Training Mode:

Best SD error rate: 10.9%

Best SI error rate: 52.4%

7/11/11/1 7/13/9/1 7/13/11/1 7/13/11/3 7/15/11/1 9/13/11/1 11/13/11/1

13/13/11/1

13/15/11/1

0.0

20.0

40.0

60.0

80.0Subject 1

Subject 2

Subject 3

Subject 4

Subject 5

Signer independent test

Sub-gesture HMM Topology

Model parameters

Left-Right models

Grammar: LB -> gesture-> LB (LB: long background )

Fig. 1. Histogram of Oriented Gradient (HOG) features are … say something important in three lines…… say something important in three lines…

Benefits: …say something about training

… say something about segmentation

Robust to variations in background

Similarities between gestures:

Variations in signers:

Segmentation: