Comparing Hand-Gesture and Finger-Touch Interfacing toNavigate Bulk Image-Sequence Data

V. Du Preez, E.P.Clarkson, S. Innes, D.Q. Quach and K.A. HawickComputer Science, Massey University, North Shore 102-904, Auckland, New Zealand

email: {dupreezvictor, dara.quach, elliot.clarkson}@gmail.comredmansteve2@hotmail.co.uk, k.a.hawick@massey.ac.nz

Tel: +64 9 414 0800 Fax: +64 9 441 8181

June 2013

ABSTRACT

Modern interface devices such as depth field cameras andtouch sensitive screen offer new scope for navigation largeand complex image data sets. Image sequences frommovies or simulations can be treated as hyper-bricks ofnavigable data. We have implemented an image naviga-tion interface framework that supports a compatible setof both depth-field hand gestures and touch-screen fingermovements. We report on how these two apparently dis-parate interfaces can be combined in a unifying softwarearchitecture and explore the human computer interactionspace of suitable gestural and touch idioms and metaphorssuitable for rapid interactive navigation and rendering of asequence of images from a simulation, from photographicstills, or frames of a movie. We compare the two sorts ofinteraction and discuss a descriptive vocabulary for theseand suggest some directions for development and use inother bulk data navigation interfaces.

KEY WORDSimage data navigation; gestures; touch screen; HCI.

1 IntroductionImprovements in and cheaper costs of image capture de-vices are making the problem of navigating and manipu-lating large sequences of image data more common. Sim-ulations too, often generate large quantities of image data.Interacting with regular hyper-bricks of data in real inter-active time is quite computationally feasible using modernprocessing technologies if one can find the right interac-tion metaphors to allow a user to express appropriate navi-gational commands. In this paper we investigate the inter-action technologies such as depth field cameras and touchscreens so that users can interact with image sequence datausing both gestures and multi-finger touches.Human-Computer Interaction is a relatively mature dis-cipline [1] and many of the key guiding principles have

Figure 1: One of the authors (DQQ) demonstrating theprototype system using gestural interfaces that are de-tected by a Kinect depth-field camera, and used to directthe operations of a simulation model.

been well studied [2–4]. However, the widespread avail-ability of new interface devices is leading to hitherto unex-plored interaction mechanisms. The multi-touch capabil-ity of tablet computers [5] is a particularly interesting areathat is still being explored by new communities of usersfor various disciplines.The Kinect depth-field camera [6] is another commodity-priced device that has attracted a lot of attention as anenabler of innovative HCI modes for gaming applica-tions [7–9], but also for applications including geospa-tial navigation [10], handicap and visual impairment sup-port [11–13], and also interactive learning [14].Widely available devices like the Kinect make possible arange of human interaction possibilities. The Kinect itselfis a well integrated set of sensors [15] including cameras,orientation devices and sound capabilities. These systemsand their software frameworks support detection of spe-cific devices like paddles [16] or wands, but more inter-esting - and indeed natural, is for the user to use ges-

tures [17–21] to interact with a sequence or brick or im-ages or simulation model.Gestural interfaces are not new [22] although with the veryrapid product commoditization of touch sensitive tabletcomputing the research and textbook literature on multi-touch and gestural systems has not yet caught up and thereare surprisingly few accounts of multi-touch applicationsand associated experiences [23].In this present paper we are particularly interested in en-abling different devices to support image navigation, withsets of either r hand gestures or screen touches having anintuitive relationship with one another. Figure 1 shows ...HCI experiments and applications [24] have been widelyreported in the literature for tablet computing [25] on dataentry [26], database interaction [27], interactive training[28, 29] and simulation interaction [30, 31]. Software de-velopment work is also reported on HCI frameworks thatwill further enable these applications [32].Much recently reported research has focused on interact-ing with 3D objects [33, 34]. Our present paper focuseson the human user as a 3D entity [35] that must be recog-nized and the detected [36] human activities [37] used asfeedback into any interesting running simulation. Humandetection [38] requires tracking the whole skeletal body[39, 40] as well as specifics such as head and hands [41].There appears to be a wealth of work to be done in thefield of HCI in appropriately categorising and naming ap-propriate 3D gestures so that suitable simulation drivinglibraries can be formulated. As we discuss in this paper, itis not necessarily feasible or indeed desirable to solely usegestures and a hybrid approach using some mix of gesturaland speech/sounds [42] may be more natural for an imagesequence navigator.Our article is structured as follows: In Section 2 we de-scribe the client-server architecture of our software com-bining interpretation of Kinect gestures or touch screen in-teractions. We present a set of photos explaining the set ofhand gestures and multi finger clicks that e developed toexpress image sequence navigation in Section 3. In Sec-tion 4 we discus s some of the implications for this sort ofdevice agnostic architecture and offer some conclusionsand areas for further investigation in Section 5.

2 MethodRather than modifying existing simulations to supportgesture-based input, a plug-in application was developedwhich would interpret gesture information sent over thenetwork and convert this to simulated key-presses on thehost operating system. Java was chosen as the languagefor the server for platform independent testing. To in-terface with the server, separate software was written for

Figure 2: Architecture

each input device to decode gesture input. This recogni-tion software can be on the input device itself (In the caseof an Android tablet), on the host machine (in the case ofthe Kinect), or anywhere on the reachable network, pro-vided it has fast access to the raw data provided by theinput device. When gestures are recognised, a short stringdescribing the detected gesture is sent over the networkto the server application. These loosely coupled systemscommunicating over a network allows existing simulationsoftware to quickly and easily make use of gesture input,often without any modification to the source code.While this approach is flexible and easy to both implementand use, it is limited in that it can only offer as much con-trol as the target program provides with hot-keys. Thismeans that gestures such as rotation must be done itera-tively; simulations will not be able to respond in real-timeto the rotational degree of the gesture without modifica-tions to the target simulation.

2.1 Input Device - AndroidThe Android API has large support for gestures, it beingthe main way to interact with most Android devices, andwas very easy to implement. Gesture listeners, which caterfor most generic gesture inputs, have been a part of the APIsince its creation. There is still some modification neededin order to recreate more specific gestures, such as flingleft and fling right, but these were easy to build on top ofthe provided interface methods which cover a lot of themore common gestures.

Figure 3: Gesture to Key-press server, ready to turn ges-tures from the network into key-presses on the host ma-chine.Swipe Left Wait for a touch event to be placed and record

the position. If the finger is raised in a different posi-tion on the screen with a lesser x dimension than thedown even was recorded having then a swipe left hasoccurred.

Swipe Right Wait for a touch event to be placed andrecord the position. If the finger is raised in a differ-ent position on the screen with a greater x dimensionthan the down even was recorded having then a swiperight has occurred.

Pinch When 2 fingers are placed on the screen their po-sitions and the distance between them is recorded.When the fingers are lifted off of the screen take thepositions of each and find their distance apart fromeach other. If the second recorded distance is lessthan the first it is a pinch gesture

Spread When 2 fingers are placed on the screen their po-sitions and the distance between them is recorded.When the fingers are lifted off of the screen take thepositions of each and find their distance apart fromeach other. If the second recorded distance is greaterthan the first it is a spread gesture

Touch If a touch down event is recorded followed veryquickly by an off touch event with very similar posi-tions then a touch gesture has occurred.

Long Touch If a touch down event is recorded followedby an off touch event after 2 seconds and these two

events have similar recorded positions then a longtouch gesture has occurred.

2.2 Input Device - KinectThe Kinect is made for tracking general body movementsrather than fine manipulators. This restricts the variety ofgestures the device is capable of accurately reporting. It isimportant to keep this in mind when designing gestures tobe recognised by this device, using broad and deliberatemotions to account for the lack of accuracy. We usedthe skeletal tracking to invoke these gestures using jointsthat represent major points in the skeleton body. Thisallowed us to track regions in the recognition with somepositioning using x and y coordinates.

We achieved a gesture vocabulary that does not detect un-wanted gestures. This makes it difficult to implement ges-tures that are consistent and reliable. We therefore imple-mented a rule-set that subtlety detects the required gesture.Figure 4 show how the implemented zoom-in gesture,similar to Android’s pinch-to-zoom gesture is performedby bringing both hands above the neck and spreading themapart at a greater distance than the users shoulder width.Once the gesture has been performed the software idlesuntil a new gesture is recognised.

3 Experimental Simulation ResultsThe system was tested on three applications, with two sep-arate input devices. The server was run on a Mac hostmachine, and loaded with configurations of hot-keys forVLC media player, Xee image viewer, and Animaux, anagent based simulator running the Ising model. Figure 6displays the gestures used and how they were mapped to

Figure 5: Kinect implemented gesture, showing the swipeof the motion gesture with the left and right hands on avideo or image application.

Figure 4: Performing the ‘zoom in’ gesture with the Kinect alongside its homomorphic equivalent on the Android tablet.

Kinect Gesture Android Gesture VLC Xee Ising ModelSwipe Left Medium Forwards Jump Next Image Increase Critical Coupling

Swipe Right Medium Backwards Jump Previous Image Decrease Critical CouplingPinch - Zoom Out -Spread - Zoom In -

Hand Raise Tap Pause/Play - Next IterationLong Hand Raise Long Tap Stop - Run Simulation

Figure 6: Mapping gestures to actions in programs.

Figure 7: Diagrams illustrating all the gestures imple-mented in to Kinect gesture recognition.

different applications.All combinations of device and application follow a sim-ilar format. A typical example with Xee and the Androidtablet could go as follows: An Android tablet running acustom App for gesture recognition and network commu-nication connects to the server. A ‘fling left’ gesture, whenmade on the Android tablet, would send a string describ-ing the gesture just made to the server. The server, having

previously been configured with Xee’s keyboard shortcutfor ‘next image’, uses Java’s Robot class to send thesekey-presses to the active window. Xee will then change todisplay the next image.

3.1 Simulations SoftwareThe system exhibited positive results when tested with ad-vanced agent-based simulation software running the Isingmodel, making it appropriate for displaying the model toan audience or for allowing observers to interact with themodel in an intuitive way. The existing software interfacehad a number of problems. It was crowded with intim-idating elements, and the key combinations were modelspecific and occasionally esoteric. In comparison, the ges-ture overlay remained simple to operate provided it wasconfigured correctly, completely bypassing the traditionalmethods of control. Configurations specific for particularmodels could be loaded and mapped to familiar gesturesthat reflect the action taken on the model.A users existing knowledge can be transferred from evennon-scientific applications. Knowing that performing a‘hand raise’ gesture on the Kinect triggered a pause/playcommand in a media player can be directly applied to theagent-based simulation model, with the implication thatthe same gesture in the agent-based simulator would resultin some kind of temporal manipulation (a ‘next iteration’command for the Ising model.)It was even possible to define a gesture to switch config-urations on the gesture server, making it easier still to in-teract with complex models. Although introducing modesinto an already complex system would result in a poor user

experience, [43] it makes sense to switch configuration tomatch the model displayed in the simulation software ifit changes. This allows users to perform the same familiarset of gestures and have it map to the control configurationsuited for that model.Hiding the interface and providing a standard set of inter-action gestures means that non-scientists can interact withand manipulate models without having to have a deep un-derstanding of the parameters that modify their behavior.This encourages exploration and investigation into whatwould otherwise be an intimidating concept to someoneunfamiliar with complex systems.

3.2 Input DevicesWhile the two input devices tested are able to share manyconcepts such as the same motion to execute gestures (Fur-ther discussed in section 4.1), they also differ significantlyin other areas. It is quite simple for an Android device todetect the start and end of a gesture, as these events cor-relate with the introduction and removal of fingers fromthe touch-screen. The Kinect is harder to work with inthis regard, as the points of interest for gestures are almostalways visible. The requirement for a voice command toinitiate a gesture was briefly considered, but this was ul-timately rejected in favour of comparing the location ofinterest points in relation to others. For example, a ‘swipeleft’ gesture would only start when the left hand was belowthe waist, and the right hand between the waist and shoul-ders, and further away from the body than the right shoul-der. This means transition from one device to the otheris a more natural process, as there are no additional stepsrequired; If the user is relaxed and acting in a sensiblemanner, a gesture on one device occupies the same log-ical space as on another. Gestures become more device-agnostic and universal, which means less to remember forthe user.An interesting observation is that both devices tested havesome kind of processing capability. The Kinect outputsskeletal data which requires only minimal processing toconvert into a useful gesture, and Android devices arecompletely capable of doing all gesture processing them-selves and simply passing along a message indicating agesture has been detected to the server. This may not al-ways be the case, for example if the host machine is en-gaged in processing gestures from a web-cam. However,when it is the case, this lightens the load on the host ma-chine, meaning its CPU resources are free to run more in-tensive simulation software. This could be important ininterfacing with real-time simulations.

4 DiscussionGestural computer interfacing is still in a relatively earlystage of development and there is great scope for identify-ing and naming standard gestures for more widespread usein applications. Some of the touch and gestural notionslike “pinch to zoom” have now become widely acceptedand understood. Other named intuitive gestures will likelyemerge from ongoing work in this area.

4.1 Homomorphic GesturesSome gestures have a very clear method of executionacross various devices. A ‘pinch’ gesture, for example,can be intuitively executed on the Kinect by bringing bothhands together. The same gesture can be executed on An-droid by pinching the fingers together on the touch surface.These gestures can be considered homomorphic. Homo-morphism is a desirable trait because such gestures facil-itate information transfer - Skills learned on one platformbecome obvious and intuitive on another. The system ar-chitecture supports this paradigm by easily linking thesegestures to the same action on the host machine, easilybringing together the motion with the desired intent.

4.2 Representing Continuous DataSome gestures are well suited for delivering continuousoutput - That is, a continuously changing stream of data,which could be interpreted from the height of a hand, orthe distance between fingers, for example. This is desir-able because the rich data it provides would be suitable formodifying parameters of simulations in real time. How-ever, there are some difficulties in transparently convey-ing the information to a simulation application. The im-plemented hot-key-based approach is only really suitablefor discrete information packets, and a continuous streamwould have to be represented as a plethora of key-presses.Something like a constantly changing floating point valueis difficult to represent at all with this system.In order for a program to receive continuous informationin an input device agnostic manner, the program could lis-ten for information on a stream input such as the stdinfile descriptor, which could be provided either by a serverlistening for streams of gesture data over the network, orbeing piped in from a program which talks directly withan input device, bypassing the network entirely.A potential problem with continuous data gestures is therequirement for some kind of termination of the gesture.For example, an intuitive end of a rotation gesture using atouch device is to simply remove ones hand form the touchsurface. This stops the continuous stream of data, and thefinal value of any variable bound to that data can be setas the last value received. However, for the Kinect, thesituation is slightly less obvious - There is no clear wayto abruptly end a gesture such as hand elevation. Lower-

ing ones hand out of the active gesture area is impossiblewithout unwanted modification of gesture data. To remedythis, a further signal must be given to the device withoutmodification of the interest point the Kinect is monitor-ing. This could be done with a voice command, or themovement of another, non-tracked interest point. How-ever, this introduces the problem that performing this ges-ture is quite different between devices, which could makethe process less intuitive.

5 ConclusionsWe implemented gestures on the Kinect based of con-ventional touch-device gestures, we saw how these ges-tures relate and if there are some commonalities. We haveshown that the Kinect could be used in the same manner asyou would a touch-device, using only hand based gestures.The software developed shows that this could replace con-ventional key-bindings and change the way humans inter-act with computers. Hardware such as the Kinect showsthat this could easily be achieved.This will allow users to interact with computer applica-tions as they would a touch-device. However the technol-ogy has some limitations, such as no finger recognitionwhich restricts the amount of gestures that can be imple-mented. These gestures work very well with all the appli-cations as a navigation tool for streams of data in pictureand video software. The Kinect works well for the imple-mented gestures and shown very accurate and reliable datahowever only a small number of gestures could be imple-mented due to restrictions within the device.Considering there is no set gesture vocabulary describinghow humans should interact with these applications, webelieve the gestures implemented show how ease of use issimilar to accustomed finger gestures. Since these fingergestures such pinch to zoom are conventional for all touchdevices these gestures we implemented should have thesame effect with HCI.We believe these gestures will become more generalisedfor humans to interact with a more similar technology.This will likely be available in standard user applications.We have shown that these uses could support video, imag-ing and simulation applications as a manipulation tool.The implemented gesture methods mechanics that couldbe much deeper refined, allowing scope to investigate ahigher hierarchy of gestures using different devices andmore complex gestures.New technology such as the Leap Motion and Kinect2.0 could allow whole new spectrum of gestures for hu-man computer interaction. Increased precision and perfor-mance in finger recognition should allow these gesturesused in touch devices to be more generalised.

References[1] Dix, A., Finlay, J., Abowd, G., Beale, R.: Human-

Computer Interaction. Prentice Hall (1993)[2] Scogings, C.J.: The Integration of Task and Dialogue Mod-

elling in the Early Stages of User Interface Design. PhDthesis, Massey University (2003)

[3] Diaper, D., Stanton, N., eds.: The Handbook of Task Anal-ysis for Human-Computer Interaction. IEA (2004)

[4] Scogings, C., Philips, C.: Linking Task and Dialogue Mod-eling: Toward an Integrated Software Engineering Method.In: The Handbook of Task Analysis for Human-ComputerInteraction. IEA (2004) 551–568

[5] Coulter, R.: Tablet computing is here to stay, and will forcechanges in laptops and phones. Mansueto Ventures (2011)

[6] McEwan, T.: Being kinected. ITNow 53 (2011) 6–7[7] Nacke, L.E., Kalyn, M., Lough, C., Mandryk, R.L.:

Biofeedback game design: Using direct and indirect phys-iological control to enhance game interaction. In: Proc.Computer Human Interaction (CHI 2011), Vancouver, BC,Canada (2011) 103–112

[8] Staiano, A.E., Calvert, S.L.: The promise of exergames astools to measure physical health. Entertainment Computing2 (2011) 17–21

[9] Turner, J., Browning, D.: Workshop on hci and game inter-faces: A long romance. In: Proc. OZCHI 2010 : Design,Interaction, Participation, Brisbane, Queensland, Australia,Queensland University of Technology (2010)

[10] Boulos, M.N.K., Blanchard, B.J., Walker, C., Montero, J.,Tripathy, A., Gutierrez-Osuna, R.: Web gis in practice x:a microsoft kinect natural user interface for google earthnavigation. Int. J. Health Geograpics 10 (2011) 45

[11] Chang, Y.J., Chen, S.F., Huang, J.D.: A kinect-based sys-tem for physical rehabilitation: A pilot study for youngadults with motor disabilities. Research in Developmen-tal Disabilities 32 (2011) 2566–2570

[12] Cameirao, M.S., Badia, S.B., Oller, E.D., Verschure,P.F.M.J.: Virtual reality based rehabilitation and game tech-nology. J. NeuroEngineering and Rehabilitation 7 (2010)48

[13] Zollner, M., Huber, S., Jetter, H.C., Reiterer, H.: Navi - aproof-of-concept of a mobile navigational aid for visuallyimpaired based on the microsoft kinect. In: Proc. 13th IFIPTC 13 Int. Conf. on Human-Computer Interaction (INTER-ACT’11), Lisbon, Portugal (2011)

[14] DePriest, D., Barilovits, K.: Live: Xbox kinect virtul real-ities to learning games. In: Proc. 16th Annual Technology,Colleges and Community Online Conference (TCC’11).Number ISSN 1937-1659, University of Hawaii (2011) 48–54

[15] Martynov, I., Kamarainen, J.K., Lensu, L.: Projector cal-ibration by inverse camera calibration. In: ScandinavianConf. on Image Analysis (SCIA2011). Number 6688 inLNCS, Ystad Saltsjobad, Sweden, Springer (2011) 536–544

[16] Rambone, F.: Table tennis paddle detection library for mi-crosoft kinect. Master’s thesis, Universita della SvizzeiraItaliana, Faculty of Informatics (2011)

[17] Tang, M.: Recognizing hand gestures with microsoft?skinect. Technical report, Stanford University, Dept. Elec-trical Engineering (2011)

[18] Deshayes, R., Mens, T.: Statechart modelling of inter-active gesture-based applications. In: Proc. First Interna-tional Workshop on Combining Design and Engineering ofInteractive Systems through Models and Tools (ComDeis-Moto),, Lisbon, Portugal (2011) INTERACT 2011, 13thIFIP TC13 Conf. on HCI.

[19] Doliotis, P., Stefan, A., McMurrough, C., Eckhard, D.,Athitsos, V.: Comparing gesture recognition accuracy us-ing color and depth information. In: Proc. Conference onPervasive Technologies Related to Assistive Environments(PETRA), Crete, Greece (2011)

[20] Schwaller, M., Lalanne, D., Khaled, O.A.: Pygmi?creationand evaluation of a portable gestural interface. In: Proc. 6thNordic Conf. on Himan Computer Interaction (NordiCHI2010), Reykjavik, Iceland (2010) 773–776

[21] Ulribe, A., Alves, S., Rosario, J.M., Fiho, H.F., Perez-Gutierrez, B.: Mobile robotic teleoperation using gesture-based human interfaces. In: Robotics Symposium, 2011IEEE IX Latin American and IEEE Colombian Conferenceon Automatic Control and Industry Applications (LARC),Bogota, Volombia (2011) 1–6

[22] Kortum, P.: HCI Beyond the GUI - Design for Hapric,Speech, Olfactory and other Nontraditional Interfaces.Morgan Kaufmann (2008)

[23] Rautaray, S.S., Kumar, A., Agrawal, A.: Human com-puter interaction with hand gestures in virtual environment.In: Proc. First Indo-Japan Conf. on Perception and Ma-chine Intelligence ( PerMin 2012). Number 7143 in LNCS,Kolkata, India, Springer (2012) 106–113

[24] Capra, R., Golovchinsky, G., Kules, B., Russell, D., Smith,C.L., Tunkelang, D., White, R.W.: Hcir 2011: The fifthinternational workshop on human-computer interaction andinformation retrieval. ACM SIGIR Forum 45 (2011) 102–107

[25] Preez, V.D., Pearce, B., Hawick, K.A., McMullen, T.H.:Human-computer interaction on touch screen tablets forhighly interactive computational simulations. In: Proc.International Conference on Human-Computer Interaction,Baltimore, USA., IASTED (2012) 258–265

[26] Castellucci, S.J., MacKenzie, I.S.: Gathering text entrymetrics on android devices. In: Proc. Computer HumanInteractions (CHI2011), Vancouver, BC, Canada (2011)1507–1512

[27] Buchanan, N.: An examination of electronic tablet basedmenus for the restaurant industry. Master’s thesis, Univer-sity of Delaware (2011)

[28] Johnson, M.G.B., Hawick, K.A.: Teaching computa-tional science and simulations using interactive depth-of-field technologies. In: Proc. Int. Conf on Frontiers inEducation: Computer Science and Computer Engineering(FECS’12), Las Vegas, USA, CSREA (2012) 339–345 a.

[29] MacDonald, J.E., Foster, E.M., Divina, J.M., Donnelly,D.W.: Mobile Interactive Training: Tablets, Readers, andPhones? Oh, My! In: Proc. Interservice/Industry Train-ing, Simulation and Education Conference (I/ITSEC 2011).

Number 11038, Orlando, Florida, USA (2011) 1–9[30] Preez, V.D., Pearce, B., Hawick, K.A., McMullen, T.H.:

Software engineering a family of complex systems simula-tion model apps on android tablets. In: Proc. Int. Conf. onSoftware Engineering Research and Practice (SERP’12),Las Vegas, USA, SERP12-authors.pdf, CSREA (2012)215–221

[31] Pearce, B.T., Hawick, K.A.: Interactive simulation and vi-sualisation of falling sand pictures on tablet computers. In:Proc. 10th International Conference on Modeling, Simula-tion and Visualization Methods )MSV’13). Number CSTN-196, Las Vegas, USA, WorldComp (2013) MSV2341

[32] Ali, S.I., Jain, S., Lal, B., Sharma, N.: A framework formodeling and designing of intelligent and adaptive inter-faces for human computer interaction. Int. J. Applied In-formation Systems 1 (2012) 20–25

[33] Krainin, M., Henry, P., Ren, X., Fox, D.: Manipulatorand object tracking for in-hand 3d object modeling. Int. J.Robotics Research Online (2011) 1–17

[34] Steinicke, F., Benko, H., Daiber, F., Keefe, D., de la Riv-iere, J.B., eds.: Proc. Special Interest Group Touching the3rd Dimension (T3D). CHI 2011, Vancouver, BC, Canada,ACM (2011) ISBN 978-1-4503-0268-5/11/05.

[35] Schouten, B.A.M., Tieben, R., Ven, A., Schouten, D.W.:Human Behavior Analysis in Ambient Gaming and PlayfulInteraction. In: Computer Anaylsis of Human Behavior.Springer (2011) 387–403

[36] Luber, M., Spinello, L., Arras, K.O.: Learning to detect andtrack people in rgbd data. In: Proc. Workshop on AdvancedReasoning with Depth Cameras, Robotics Science and Sys-tems (RSS), University of Souther California (2011)

[37] Sung, J., Ponce, C., Selman, B., Saxena, A.: Human activ-ity detection from rgbd images. In: Proc. AAAI Workshopon Plan, Activity and Intent Recognition (PAIR 2011), SanFrancisco, USA (2011) 47–55

[38] Xia, L., Chen, C.C., Aggarwal, J.K.: Human detectionusing depth information by kinect. In: Proc. 2011 IEEEComp. Soc. Conf. on Computer Vision and Pattern Recog-nition Workshops (CVPRW), Colorado Springs, Colorado,USA (2011) 15–22

[39] Suma, E.A., Lange, B., Rizzo, A.S., Krum, D.M., Bola, M.:Faast: The flexible action and articulated skeleton toolkit.In: Proc. IEEE Virtual Reality, Singapore (2011) 247–248ISBN 978-1-4577-0038-5/11.

[40] Kar, A.: Skeletal tracking using microsoft kinect. Method-ology 1 (2010) 1–11

[41] Garstka, J., Peters, G.: View-dependent 3d projection usingdepth-image-based head tracking. In: Proc. 8th IEEE Int.Workshop on Projector-Camera Sytems, Colorado Springs,USA. (2011) 52–58

[42] Molenaar, G.: Sonic gesture. Master’s thesis, University ofAmsterdam (2010)

[43] Andre, A., Degani, A.: Do You Know What Mode You’reIn? An Analysis of Mode Error in Everyday Things. In:2nd Conference on Automation Technology and HumanPerformance. (1996)