Markless inspection of augmented reality objects using fingertip tracking

Hello!I am Tharuka DilaniI am here for my image processing presentation.

AgendaAugmented Reality ?Marker and Markerless based ARIntroduction to the inspectionMethodalogiesApplicationDiscussion

WHAT IS AUGMENTED REALITY (AR) ?

An integration of digital information with live video or the users environment in real time .

Why I used augmented reality?To become a smart human from an ordinary human being.

Marker And Marker Less Based ARMaker-based AR is when the tracked object is black-white square marker.

Mark less augmented reality is when the tracked object can be anything else: picture, human body, head, eyes, hand or fingers etc. and on top of that you add virtual objects.

Maker-based AR

Marker less AR

Markerless Inspection Of Augmented Reality Objects Using Fingertip Tracking

Inspecting a vertual bunny on top of the users hand from different viewing angles

Flowchart

(a) given a captured image

(b) skin color segmentation

(c) distance transform

(d) extract a single connected component of the hand

1. Hand Segmentation

2. Fingertip Detection PART(I)

The points with higher curvature values than a threshold(on any scale level) are selected as candidates points.Computation of candidate points

Directions are indicate by the cross product of two vectors.

(a) Ellipses are fitted to contour based on candidates points

For initial detection hand has to be held fingers up. 10 frames are used in initial fingertip detection

At last fingertips are ordered based on the index of the thumb(b) Fingertips are ordered based on detected thumb2. Fingertip Detection PART(II)

3. Fingertip TrackingMatching the newly detected fingertips to the previously tracked fingertips.Using matching algorithm that minimizes the displacement of pairs of fingertips over two frames.The matching cost is minimized as

4. Pose Estimation from FingertipsRemove the errors in tracking fingertipsBy using Kalman filter the estimate camera pose is smoothed

summary

DiscussionIllumination changes causing different hand skin color and for out door scenes.

Implementation will not successfully track two hands in similar area.When fingertip are not visible, this system determines that it has lost tracking the fingertips.

CreditsSpecial thanks to all the people who made and released these awesome resources for free:

Handy AR: Markerless Inspection of Augmented Reality Objects Using Fingertip TrackingM. Kolsch and M. Turk. Fast 2D hand tracking with flocks of features and multi-cue integration. In Vision for HumanComputer Interaction, page 158, 2004.A. A. Argyros and M. I. A. Lourakis. Vision-based interpretation of hand gestures for remote control of a computer mouse. In Computer Vision in Human-Computer Interaction, pages4051, 2006.Letessier, Julien and Berard, Francois. Visual tracking of bare fingers for interactive surfaces. In Proceedings of the ACM Symposium on User Interface Software and Technology, Interactive surfaces, pages 119122, 2004.Z. Y. Zhang. A flexible new technique for camera calibration. IEEE Trans. Pattern Analysis and Machine Intelligence, 22(11):13301334, Nov. 2000.

Thanks!Any questions?You can find me at tharukasiriwardena27@gmail.com