1

Gesture recognition

Using HMMs and size functions

2

Approach

Combination of HMMs (for dynamics) and size functions (for pose representation)

3

Size functionsTopological representation of contours

4

Measuring functions

Functions on the contour to which the size function is computed

real image

measuring function

family of lines

5

Feature extraction 1

An edge map is extracted from the image

real image

edge map

… and …

6

Feature extraction 2a family of measuring functions is chosen

… the szfc are computed, and their means form the feature vector

7

Hidden Markov modelsFinite-state model of gestures as sequences of a small number of poses

8

Four-state HMM

Gesture dynamics -> transition matrix A

Object poses -> state-output matrix C

9

EM algorithm

feature matrices: collection of feature vectors along time

EM A,C

learning the model’s parameters through EM

two instances of the same gesture

10

EM algorithm -> learning the model’s parameters

Learning algorithm

11

Gesture classification

…

HMM 1

HMM 2

HMM n

the new sequence is fed to the learnt gesture’s models

they produce a likelihoodthe most likely model is chosen (if above a threshold)