Upload
kennan
View
54
Download
0
Tags:
Embed Size (px)
DESCRIPTION
TACTUS: A Hardware and Software Framework for Research in Multi-Touch Interaction. Paul Varcholik, Joseph J. LaViola Jr., Denise Nicholson Institute for Simulation and Training University of Central Florida [email protected] , [email protected] , [email protected]. - PowerPoint PPT Presentation
Citation preview
Paul Varcholik, Joseph J. LaViola Jr., Denise NicholsonInstitute for Simulation and TrainingUniversity of Central Florida
[email protected], [email protected], [email protected]
TACTUS: A Hardware and Software Framework for Research in Multi-Touch Interaction
Multi-Touch: Introduction
Framework Motivation Primary multi-touch components
1. Physical interaction surface2. Software system for collecting and
interpreting points of contact
Present a barrier to entry for researchers focused on higher-level interface issues or application development
Multi-Touch Origins (pre-2000) Keyboards Touchscreens Pen-based computing 1984 – Bob Boie (Bell Labs) perhaps the first
multi-touch screen 1985 – Bill Buxton, Multi-Touch Tablet 1990 – Sensor Frame, Paul McAvinney (CMU)
Optical Sensor 3 fingers (some trouble with ambiguous finger positions)
1998 – Fingerworks
Multi-Touch Origins (2000 to present) 2001 – Paul Deitz – Mitsubishi DiamondTouch 2003 – Jazz Mutant 2004 – Andy Wilson – TouchLight Oct. 2005 – Jeff Han – FTIR, UIST Paper Feb. 2006 – Jeff Han – TED Talk 2006 – Jeff Han – Perceptive Pixel (CNN Magic Wall) Spring 2007 – Microsoft Surface Summer 2007 – Apple iPhone 2008 – Open-Source Community 2009 – Windows 7
Framework RequirementsEssential Component DescriptionMulti-Touch Surface Commercial-off-the-shelf hardware, requiring
near zero pressure to detect an interaction point
Software Hit-Testing Ability to determine the presence and location of each point of surface contact; supporting at least four concurrent users
Software Point Tracking Identifying a continuous point of contact and reporting its velocity and duration
Framework RequirementsSecondary Component Description
Application Programming Interface (API)
A software system upon which multiple multi-touch applications can be developed
Multi-Platform Support The ability to access multi-touch interaction data from different computing environments (e.g. languages, OS’s, etc.)
Reconfiguration Modifying the software system without recompilation
Software Service Allowing multiple applications to access multi-touch interaction data simultaneously, including over a computer network
Presentation-Layer Independence
Isolating the multi-touch interaction data from the system to graphically present such data, allowing any GUI to be employed when developing multi-touch applications
Mouse Emulation Support for controlling traditional ‘Window, Icon, Menu, Pointing Device’ interaction through a multi-touch surface
Tangible Interfaces The ability to detect and interact with physical devices placed on or near the multi-touch surface
Customizable Gesture System Support for training arbitrary multi-touch gestures and mapping them to software events
Shape Detection e.g. Finger, blob, hand
Multi-Touch Hardware Frustrated Total Internal Reflection (FTIR)
1. Optical waveguide2. Supporting structure (e.g. cabinet)3. IR sensing camera4. Projector and diffuser5. IR emission source6. Computer
Multi-Touch Efforts: Hardware
Software Framework Image processing Presentation layer Multi-platform communication Pen/writing style interaction (Ink) Gesture recognition
Image Processing Pipeline
Webcam RotateFlip Background Filter
Threshold
Scaling
Blob Detection GrayScale
Grayscale Image 8bpp
Binary Image 1bpp
Calibration Point Tracking
ProcessedFrame
Camera Frame24bpp
Image Processing
Raw Camera Frame
Background Filtered Threshold Processed
Points
Multi-Touch Efforts: Software
Mouse EmulatorFunction Gesture DescriptionLeft Click Quick tap on the surface with one finger.
Left Click (alternate) While holding down a finger, tap another finger to the left side of the first.
Drag Perform a Left Click (alternate) but do not release the left side press. Drag both fingers to the destination and release.
Right Click While holding down a finger, tap another finger to the right side of the first.
Double Click Tap two fingers at the same time.
Mouse Wheel Scroll While holding down a finger, drag another finger vertically and to the right side of the first. Dragging up scrolls the mouse wheel up and vice versa.
Alt-Tab While not a mouse command, the Alt-Tab command is a useful Windows feature that switches between applications. To perform an Alt-Tab, hold down a finger and drag another finger horizontally above the first. Dragging to the left moves backward through the list of active applications and dragging to the right moves forward.
Multi-Touch Efforts: Video Montage
Multi-Touch Efforts: Additional Videos Demo Explorer Improved Calibration Multi-Touch Starcraft Mouse Emulator Ink Demonstration Particle System Demonstration SurfaceSimon SurfaceCommand
Conclusions The Framework:
In development since Fall 2007Open-source availability since Spring 2008Now in its fourth major releaseAnswers most of the specified requirementsSuccessfully employed by UCF and colleagues
from Brown UniversityDownloaded extensively by the open-source
communityProvides a robust, extensible platform for
research in multi-touch interaction
Questions?
Paul Varcholik, Joseph J. LaViola Jr., Denise NicholsonInstitute for Simulation and TrainingUniversity of Central Florida
[email protected], [email protected], [email protected]
TACTUS: A Hardware and Software Framework for Research in Multi-Touch Interaction
Resources: Commercial Platforms Microsoft Surface Mitsubishi DiamondTouch Apple iPhone & iPod Touch Perceptive Pixel N-trig MultiTouch OY (Finland) Jazz Mutant HP TouchSmart Dell Latitude XT2 TacTable
Resources: Open-Source Platforms TouchLib (NUI Group) tBeta (NUI Group) reacTIVision Bespoke Multi-Touch Framework Sparsh UI (Iowa State) BBTouch Touche`
Resources: Online NUI Group TouchKit by NOR_/D Multi-Touch Systems that I Have Known
and Loved – Bill Buxton, Microsoft Research
Jeff Han’s TED Talk Bespoke Software