Upload
tom-mens
View
191
Download
1
Embed Size (px)
DESCRIPTION
Citation preview
PetriNect : Modeling gestural interactions with executable Petri nets
PetriNect : Modeling gestural interactions withexecutable Petri nets
Romuald Deshayes1 Philippe Palanque2 Tom Mens1
Université de Mons – UMONS, [email protected]
Université de Toulouse, [email protected]
4 september 2012
8 dec 2011
Romuald Deshayes – UMONS 1 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Table of contents
1 IntroductionGoalCase StudyTools and formalisms used
2 Petshop and ICO
3 Modeling of the case study
4 Discussion
5 Future Work and improvements
6 Conclusion
Romuald Deshayes – UMONS 2 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Introduction
Introduction
IntroductionGoalCase StudyTools and formalisms used
Romuald Deshayes – UMONS 3 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Introduction
What is it about ?
Formal specificationtechniques
VS
(Informal) Games
Romuald Deshayes – UMONS 4 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Introduction
Goal
Goal of the talk
Why and how to use formal specification techniques for interactionwith games ?
Why ?Because it is not morecomplicated thancodingAnd it is safer ! Gamesalso have the right tosecurity
How ?By using toolsallowing to executeformal modelsSee what’s next !
Romuald Deshayes – UMONS 5 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Introduction
Case Study
Case study
Pong GameAlmost like the classic gameUses KinectWith free hands only (gestural interaction)⇒ Very simple to play
Move the hand to control the paddleClose it to grab the ball
Interaction modeled with executable Petri-nets
Romuald Deshayes – UMONS 6 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Introduction
Case Study
Gestural interaction
Gestural Interaction ?In a general way : control a computer with the handsIn our case study : move or manipulate virtual objectsdisplayed on the screen with gestures
Possible GesturesFlyOverGrabReleaseDragZoom (in/out)
Romuald Deshayes – UMONS 7 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Introduction
Case Study
Kinect
Kinect : what and why ?3D sensorEquipped with powerful real-time tracking algorithmsDetects hand position in metric space (point in space)Homemade algorithm to detect if hand is open/closed
For our case study, we will use the position and status of the handsand the head
Romuald Deshayes – UMONS 8 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Introduction
Tools and formalisms used
Tools and formalisms used
Work done in IRIT LabPetri-net rulesFormalism called ICO based on Petri-netsTool called Petshop to create ICO’s and execute them
Romuald Deshayes – UMONS 9 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Petshop and ICO
Petshop and ICO
Petshop and ICO
Romuald Deshayes – UMONS 10 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Petshop and ICO
Petri-net
Directed bipartite graphNode correspond to Places and TransitionsDirected arcs connect places and transitionsTokens travel from place to placeState of the Petri net is represented by the position of thetokens in the placesNon-deterministic executionlocality in transitions
Romuald Deshayes – UMONS 11 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Petshop and ICO
ICO
ICO : main key conceptsDefined as a formal description technique designed to thespecification, modeling and implementation of interactivesystemsUses high-level Petri nets (HLPN) to describe interactivesystems’ behavioural aspectsIn HLPN, tokens can carry informationsIn ICO, tokens can store Java objects (e.g. Kinect data)In ICO, transitions can contain Java code (some processing onKinect data)
Romuald Deshayes – UMONS 12 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Petshop and ICO
ICO example
Romuald Deshayes – UMONS 13 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Petshop and ICO
Petshop
PetshopTool suite dedicated to the engineering of interactive systems.Supports ICOAllows the editing, simulation and dynamic execution ofmodels.
Use of petshop to create our models, link them together, executethem, and interact with Java code
Romuald Deshayes – UMONS 14 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Petshop and ICO
Petshop
Romuald Deshayes – UMONS 15 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Modeling of the case study
Modeling of the case study
Modeling of the case study
Romuald Deshayes – UMONS 16 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Modeling of the case study
Modeling of the case study
Models used to process kinect raw dataTransform data into abstract eventsEvents are interpreted by virtual objects models
Romuald Deshayes – UMONS 17 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Modeling of the case study
Modeling of the case study
Use Petshop to create a 3 layered ICO modelEncapsulate the data of hands and head of each player intotokens (position and state)Models are created to be independent of the number of players(ID for each player)Only one instance of each layer whatever the amount of virtualobjects to controlEach virtual object can be modeled using petshop (4th layer)
Romuald Deshayes – UMONS 18 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Modeling of the case study
Raw data to positions
Layer 1 : Raw data to positionsReceives positions of hands and head of each player at regulartime intervals3 tokens are generated at each kinect frame refresh (1 for thehead, 1 for each hand)Informations about head and hand positions are stored in thetokensICO model joins data from head and hand and create a newtoken for each handTokens are sent to the next layer
Romuald Deshayes – UMONS 19 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Modeling of the case study
Raw data to positions
Romuald Deshayes – UMONS 20 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Modeling of the case study
Positions to gestures
Layer 2 : Positions to gestures
Receives absolute positions of hands (1 token per hand perplayer)Uses 2 consecutive tokens to calculate relative movementDetects state changes of the hands (open/closed)Sends open/close/move events to the 3rd layer
Romuald Deshayes – UMONS 21 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Modeling of the case study
Positions to gestures
Romuald Deshayes – UMONS 22 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Modeling of the case study
Gestures to events
Layer 3 : Gestures to events
Receives move/open/close events from layer 2Stores the state of each player in a 4 places sub Petri netJoins information about the player and events from layer 2 togenerate eventsGenerates 5 different events
GrabReleaseFlyoverDragZoom (in/out)
Sends events to modeled objects
Romuald Deshayes – UMONS 23 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Modeling of the case study
Gestures to events
Romuald Deshayes – UMONS 24 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Discussion
Discussion
Discussion
Romuald Deshayes – UMONS 25 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Discussion
Discussion
Modeling has a lot of advantages :Reusable models, easy to maintain or extend (dynamicity)Models hide some technical complexityAmenable to formal analysisCan deal with Multi-userEasy to integrate with many different virtual objectsGoal : toolkit for gestural interaction
Romuald Deshayes – UMONS 26 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Future Work and improvements
Future Work and improvements
Future Work and improvements
Romuald Deshayes – UMONS 27 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Future Work and improvements
Future work and improvements
Future Work and improvements :Specify the game (rules and graphics) with Petri netsPerformance issue
Threads tweaking (constant token flow)Compile for deployment
Extend the models to deal with more gesturesCreate a DSL over ICO to simplify the specification of virtualobjects behaviour
Romuald Deshayes – UMONS 28 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Conclusion
Conclusion
Conclusion
Romuald Deshayes – UMONS 29 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Conclusion
Conclusion
Wrap-upPossible to specify gestural interaction of a game with formalmethodsFormal methods might help for safety issues in gaming tooLot of other advantages to resort to executable modelinginstead of coding
Dynamicity lead to productivity increaseHides some technical detailsEasy to read and extend
Petshop is a cool tool suite, USE IT !
Romuald Deshayes – UMONS 30 / 31
PetriNect : Modeling gestural interactions with executable Petri nets
Conclusion
Thank you
Thank you for your attention !
Questions ?
Romuald Deshayes – UMONS 31 / 31