By: Jonathan LebronAdvisor: Nick Webb

SENIOR PROJECT – COMPUTER SCIENCE – 2013

RECOGNIZING MILITARY GESTURES

DEVELOPING A GESTURE RECOGNITION INTERFACE

MOTIVATION• Scenarios where speech would not be possible or optimal

solution

• Giving natural gestures to Robots that resemble Human-to-Human interaction

• No need for special training

• Incorporating both speech an gesture would be ideal, but this problem is out of scope

RESEARCH QUESTION• How do you give a robot a sequence of natural gestures to

interpret and act upon?

Xbox Kinect

HOW IT WORKS• Perform Gesture

• Kinect reads skeleton data

• Data gets transformed into angles

• Classifier reads angles and determines gesture using C4.5 decision tree

left_shoulder: 0.35, 0.12, 2.19left_elbow: -0.18, -0.10, 2.02left_hand: -0.13, -0.34, 1.924

CLASSIFICATION

GesturesAbreast Enemy

Freeze Stop

Listen Rifle

Cover

ClassifierAngleshandTheta

handPhi

elbowTheta

elbowPhi

CLASSIFICATION• How do we handle erroneous data?

• When do we know when to execute the sequence?

• What actions can we expect from the robot?

SOFTWARE DESIGN

ADDING GESTURES• Create new gestures

• Replace old gestures

• Combining gestures

CONCLUSION• Future Work

• Wrap-up

• Q&A