1. The Image-Based Data Glove Vitor F. Pamplona Leandro A. F. Fernandes Joo L. Prauchner Luciana P. Nedel [email_address] Manuel M. Oliveira

2. Outline

Data gloves

Our idea

The tracking system

The virtual hand representation and animation

The prototype

Device evaluation

Results and conclusion

3. Outline

Data gloves

Our idea

The tracking system

The virtual hand representation and animation

The prototype

Device evaluation

Results and conclusion

4. Data Gloves P5 Glove $59.00 6 DOF of motion Fingers flexion0.5 resolution (90 )60Hz refresh ratePinch Glove $1,899.00 6 DOF of motion Pinch gestures only DG5-VHand Glove $485.00 Fingers flexion (10-bit per finger) 5. Data Gloves MidiGlove $1,200.00 Fingers flexion (10-bit per finger) 100-200Hz refresh rate X-IST Data Glove$1,200.00-$1,700.00 Pitch and roll motion (+ $125.00) Fingers flexion (10-bit per finger) 100-200Hz refresh rate 5DT Data Glove $895.00-$1495.00 Pitch and roll motion Fingers flexion (8-bit per finger) 75Hz-200Hz refresh rate 6. Data Gloves CyberGlove II 6 DOF of motion 0.5 resolution 90 records per second ShapeHand $7,500.00-$16,200.00 6 DOF of motion Fingers flexion and adduction 7. Outline

Data gloves

Our idea

• The Image-based data glove (IBDG)

The tracking system

The virtual hand representation and animation

The prototype

Device evaluation

Results and conclusion

8. The Idea

Alow cost full-featured device

Markers on each finger tip

Asingle web camfixed on hand

A software to

• track the position of those markers

• estimate the position of the finger joints

• returns thefull status of the user fingers

Webcam Visual Marker 9. Research Challenges

Build animage-baseddata glove

Finger tips trackingwith flexure and adduction

Suitable for existing manipulation techniques

Requirements

• Low Cost

• Comfort

• Accuracy andPrecision

• Real timeperformance

10. Contributions

Single cameraper hand

Continuous trackingof fingers position

Reproduction of fingersflexionandadduction

Low cost, real time andprecise

11. Problems to be solved

Tracking system

• The markers aresmall

• The camera hasblurandfocus lost

• The tracking system needs to bereal time

Virtual hand representationand animation

• The dimensions of the virtual hand must be real

• The inverse kinematics should be real time

Prototype assembling (hardware)

• The camera position and orientation can not change

• The markers need to be planar

• The markers position and orientation can not change

12. Outline

Data gloves

Our idea

The tracking system

The virtual hand representation and animation

The prototype

Device evaluation

Results and conclusion

13. Tested Tracking Systems

ARToolKit

• Misclassification among patterns

• Sensitive to focus changes

• Unstable tracking(jitter)

ARTag

• Needs at least 2 different markers to detects the same object

ARToolKitPlus

• Improved pattern recognition system

• Stable tracking (less jitter)

• Automatic thresholding

• Improved camera calibration model

14. Tested Tracking Systems 10x10 mm, ARToolKitPlus BCHpatterns withthin borders 10x10 mm, ARToolKitPlus simple patterns 7x7 mm, ARToolKit patterns 15. Tested Webcam Genius Trek 310 Genius VideoCam NB Creative WebCam PD1001 Creative WebCam Live! Ultra CIF CMOS PC Camera Logitech QuickCam for Notebooks Pro 16. Tracking System and Calibration M k M c M h 17. Outline

Data gloves

Our idea

The tracking system

The virtual hand representation and animation

The prototype

Device evaluation

Results and conclusion

18. Virtual Hand Representation

Hand model fromV-ART library

Blender Inverse Kinematics Module

19. Outline

Data gloves

Our idea

The tracking system

The virtual hand representation and animation

The prototype

Device evaluation

Results and conclusion

20. The Prototype

Weight 195g

21. The Dices! 22. The FLEA Camera

Point Grey Research

Developers Experience

30 31 mm size

High quality images(1024 768 pixels) at 30 fps

3.5-10.5 mm Computar Varifocal Lenses.

Total weight: 115 g

23. Using The Prototype 24. Summary

Hardware

• Camera to track finger tips

• Paper and printer to make visual markers

• Glove, leather and sticks to build the camera support

• Dices and thimbles

Software

• C++

• ARToolKitPlus library

• V-ART library

• Blender inverse kinematics module

25. Outline

Data gloves

Our idea

The tracking system

The virtual hand representation and animation

The prototype

Device evaluation

Results and conclusion

26. Device Evaluation

Testbed application

15 Subjects

• 2 women and 13 men

• 13 right handed and 2 left handed

• 21 to 38 years old

Goals :

• to verify the quality of the finger tips tracking

• to verify the gesture reproduction

• to collect some impressions of the users about IBDG

27. Testbed Application

Given a hand gesture, the user must imitate it.

28. Methodology

Pre-test questions

Calibration section

Warm up section

• Unlimited time for training

Task description

• Six pre-defined hand poses

• The user imitates them

• The movements are logged

Post-test questions

29. Data Collection and Analysis

Store

• Useridentification

• Taskparameters

• Position and orientation ofeach finger along time

• Elapsed time

Analyze

• Precision

• Performance

• Subjective measures(post-test questions)

30. Outline

Data gloves

Our idea

The tracking system

The virtual hand representation and animation

The prototype

Device evaluation

Results and conclusion

31. Precision Analisis

Mean observed error: 0.59

Confidence interval of 99%;

32. Usability

Latency about237ms(2.4 GHz PC with 2gb of memory)

Velocity about23 fps(640 x 480)

Weight about195g(58% because of the FLEA camera)

Maximum size of hands for using our prototype21.5 cm

User opinions to our prototype

• Comfort: 2.73(5 means comfortable and 1 uncomfortable)

• Precision: 2.87(5 means precise and 1 means imprecise)

80%of the volunteers said they can build their own IBDG

33. Discussion

The tracking system (ARToolkitPlus) worked well with

• Smooth variations inlighting conditions

• Shadowsover the visual patterns

For dark places one can use aninfrared camera

FLEA Camera can be replaced by anewest web cams

• Improving comfortwith a lighter one

• Decreasing the prototype cost with acheaper one

• Wireless or bluetoohenabled camera

Thumb can be tracked using a device with agreater field of view

The prototype needs a second version!

34. Questions?

Luciana Porcher Nedel

[email_address]

The Image-Based Data Glove