Slide 1

Wrist Recognition and the Center of the Palm Estimation Based on Depth Camera Zhengwei Yao ; Zhigeng Pan ; Shuchang Xu Virtual Reality and Visualization (ICVRV), 2013 International Conference on 1

Slide 2

Outline Introduction Related work Proposed method Experimental results Conclusion 2

Slide 3

Introduction 3

Slide 4

Problem: Can not separate a hand from a forearm using color and depth information Solution: Find wrist to recognize hand 4

Slide 5

Related Work 5

Slide 6

Hand segmentation and extraction Color [11,12] Depth threshold [13,14] The location of other body parts [15~17] Wrist Wear wristband[14] Palm detection[18] 6

Slide 7

Reference [13] D. Uebersax, J. Gall, and M. Van den Bergh, and L. Van Gool, Realtime sign language letter and word recognition from depth data. International Conference on Computer Vision Workshops (ICCV Workshops), 2011 [14] Z. Ren, J. Yuan, and Z. Zhang, Robust hand gesture recognition based on finger- earth mover's distance with a commodity depth camera. ACM international conference on Multimedia, 2011 [15] T. I. Cerlinca and S. P. Pentiuc, Robust 3D Hand Detection for Gestures Recognition. Proc. the 5th International Symposium on Intelligent Distributed Computing, Delft, 2012 [16] M. Van den Bergh and L. Van Gool, Combining RGB and ToF cameras for real-time 3D hand gesture interaction. Workshop on Applications of Computer Vision (WACV), Kona, 2011 [17] K. Fujimura and L. Xia, Sign recognition using depth image streams. Automatic Face and Gesture Recognition, 2006 [18] U. Lee and J. Tanaka, Hand Controller: Image Manipulation Interface Using Fingertips and Palm Tracking with Kinect Depth Data. Proc. of 10th Asia Pacific Conference on Computer Human Interaction(APCHI), Matsue, 2012 7

Slide 8

Related Work Candescent NUI(Natural User Interface) project Hand and finger tracking Develop by Stefan Stegmueller, Swiss Open source: Use the OpenNI framework with the Kinect sensor http://blog.candescent.ch/ http://candescentnui.codeplex.com/ Finger direction detection Blue : cluster centroid Green : palm center Red : fingertips Yellow : hand contour Long lines : finger directions A Robust Method of Detecting Hand Gestures Using Depth Sensors, Yan Wen; Chuanyan Hu; Guanghui Yu; Changbo Wang, 2012 IEEE International Workshop on HAVE http://www.camdemy.com/media/11513http://www.camdemy.com/media/11513 8

Slide 9

Proposed Method 9

Slide 10

Hand segmentation and palm estimation Wrist recognition The center of the palm estimation 10

Slide 11

Hand Segmentation and Palm Estimation (1/5) a. Cluster the hand data K-means clustering algorithm Specify the depth range: 0.5~0.8m 11

Slide 12

Hand Segmentation and Palm Estimation (2/5) b. Compute the Convex hull of the hands The Graham scan algorithm 12

Slide 13

Hand Segmentation and Palm Estimation (3/5) c. Detect the hand contours Moor-Neighbor tracking algorithm 13

Slide 14

Hand Segmentation and Palm Estimation (4/5) d. Detect the fingertips Find all candidate points that are both on the convex hull and the contour The distance of P 0 and P> threshold 14

Slide 15

Hand Segmentation and Palm Estimation (5/5) e. Estimate the center of the palm The biggest circle inside the hand contour 15

Slide 16

Wrist Recognition Wrist: pit points Find an obvious pit point in the contour of hand Create an appropriate to find another wrist point. inscribed rectangle 16

Slide 17

Wrist Recognition (1/4) Step 1: Find candidate lines of wrist The ends of the candidate line should not be both fingertips. The distance of the candidate line should not be less than a specific value. 17

Slide 18

Wrist Recognition (2/4) Step 2: Find the corresponding candidate contours whose ends are the ends of the candidate lines. 18

Slide 19

Wrist Recognition (3/4) Step 3: Find one of the wrist points Calculate the maximum distance between the candidate line and the corresponding candidate contour. The largest distance from these maximum distances. The point with the largest distance is one of the wrist points 19

Slide 20

Wrist Recognition (4/4) Step 4: Find another wrist point Connect this wrist point to each point in the hand contour, and take these connecting lines as the diagonals of rectangles. If the rectangle is not inside the hand contour, the corresponding point in the contour is not another wrist point. Find out the point with the shortest rectangle diagonal as another wrist point. 20

Slide 21

Wrist Recognition Candidate lines Corresponding contour Find one of the wrist points Find another wrist point 21

Slide 22

Estimating the Center of the Palm (1/4) Step 1: Select three points from the hand contour The three points (P 1, P 2, P 3 ) form an acute triangle. 22

Slide 23

Estimating the Center of the Palm (2/4) Step 2: Find circumcenter O j of the triangle The O j coordinate The radius of circle : 23

Slide 24

Step 3: Determine the center Calculate the distances from each point in the hand contour to the center Formula. Condition A: The number of distance R ji > R j is bigger than the threshold Condition B: R j >minR (minR=> the minimum radius of the palm ) If A or B is not satisfied, Step 4 Estimating the Center of the Palm (3/4) 24

Slide 25

Step 4: Find another appropriate palm center One end-point of these two intersectant chords is replaced by point P min Repeat step 2 to step 4 until the ending condition is true Estimating the Center of the Palm (3/4) 25

Slide 26

Proposed Method Hand segmentation and palm estimation Wrist recognition The center of the palm estimation Fingertips detection 26

Slide 27

Experimental Results 27

Slide 28

Experimental Results Device: AMD Athlon(tm)Formula Dual Core Processor Formula CPU, 4GB RAM, NVIDIA GeForce 9600GT Graphics card and Window7 32bit OS Threshold setting Depth: 0.5-0.8m Minimum distance of line: 50 Minimum radius of the palm: 33 #hand contour inside circle: 25~50 28

Slide 29

Experimental Results Before After 29

Slide 30

Experimental Results Divide into three groups based on the number of the points inside the hand contour. #contour#inside contour a6257085 b4705919 c5057335 d107411205 e135612362 f115513047 g140716716 h147817961 i102418990 30

Slide 31

Experimental Results 31

Slide 32

Experimental Results Improved original algorithm: every 8 th point The new algorithm: proposed method 32

Slide 33

Conclusion 33

Slide 34

Conclusion Propose the wrist recognition algorithm to separate the hand from the forearm, Propose a new algorithm of estimating the center of the palm to reduce the computing time. Without Kinect SDK 34