Upload
intel-software
View
166
Download
2
Tags:
Embed Size (px)
Citation preview
Intel® RealSense™ Technology | Intel® Software
Kevin Arthur, Senior User Experience Researcher
Meghana Rao, Developer Evangelist
Augmented Reality with the Intel® RealSense™ SDK and R200 Camera
User Experience and Development Best Practices
Intel® RealSense™ Technology | Intel® Software 2
Outline
Part 1, Kevin
• Overview of R200 camera and tablet augmented reality use cases
• User experience guidelines highlights
Part 2, Meghana
• SDK overview
• Sample code and demos
Intel® RealSense™ Technology | Intel® Software 3
New R200 Depth Camera For Tablets, Peripheral Dev Kit Available Now
Intel® RealSense™ Technology | Intel® Software 4
R200 View Volume and SDK Features
R200 with Intel® RealSense™ SDK
• Scene Perception Module, enables scene-aware AR
• Camera tracking and localization
• Mesh reconstruction
• Other Modules
• 3D capture
• Depth-enhanced photo and video
• Measurement
• Face detection and tracking
• Speech (Windows SDK only)
Intel® RealSense™ Technology | Intel® Software 5
Use Cases for R200 Mixed and Augmented Reality
Gaming and Play Education and Training Visualization
Intel® RealSense™ Technology | Intel® Software 6
Video
Intel® RealSense™ Technology | Intel® Software 7
Video – ToyZ Game
Shows real-time scene perception for collision and occlusion (no pre-scan)
Try it at the Intel booth
By Shachar Oz, Omek Studio at Intel
Intel® RealSense™ Technology | Intel® Software 8
Video – Procedural Island
Shows scan as part of “capture and play”
Illustrates procedural shaders and set dressing
By Eddy Ortega, Garrett Stevens, Perceptual Computing at Intel
Intel® RealSense™ Technology | Intel® Software
UX Guidelines for R200 Tablet ARDesigning real, usable apps for a mass market
Intel® RealSense™ Technology | Intel® Software 10
Lesson 1: Give People a Reason to Move, or They Won’t
Tablet as window into a virtual space vs. tablet as fixed screen
Address with motion hints
Intel® RealSense™ Technology | Intel® Software 11
Motion Hints – Explicit
Registered with scene
Or registered with window
Intel® RealSense™ Technology | Intel® Software 12
Motion Hints – Implicit, Part of Experience
Lead the user with content
Example: “Windy Day” (Google Spotlight Stories)
Intel® RealSense™ Technology | Intel® Software 13
Lesson 2: But Let People Relax Too
Support bothActive Camera Mode
• Tiring
Inactive Camera Mode
• Less tiring
Intel® RealSense™ Technology | Intel® Software 14
Active-Camera and Inactive-Camera Modes
Active Camera Mode
• Touch interaction less comfortable, less precise
Inactive Camera Mode
• Touch interaction more comfortable, more precise
Make the main controls easy to reach with the thumbs during Active Camera modes .
During Inactive Camera modes, placing controls elsewhere is acceptable.
AVOID OKOKOK
Make the main controls easy to reach with the thumbs during Active Camera modes .
During Inactive Camera modes, placing controls elsewhere is acceptable.
AVOID OKOKOK
Touch Zones Touch Zones
Intel® RealSense™ Technology | Intel® Software 15
Two Styles of Mixed-Reality Games
Augmented Reality Capture and Play
Active Camera
Inactive Camera
Intel® RealSense™ Technology | Intel® Software 16
Lesson 3: Plan for the Scene
• Consider size of play space, and use appropriate voxel resolution
Intel® RealSense™ Technology | Intel® Software 17
Game Design Considerations
• What objects does the user need?
• Level design has more unknowns
• Enhance and transform everyday objects in interesting ways
Intel® RealSense™ Technology | Intel® Software 18
Procedural Set Dressing
Intel® RealSense™ Technology | Intel® Software 19
Plan for the Scene
• Understand the camera limitations. Depth data is less accurate on
• Very bright areas
• Clear glass
• Black surfaces
• Give relevant feedback
• Fail gracefully, don’t prevent play
Intel® RealSense™ Technology | Intel® Software 20
More UX Guidelines
software.intel.com/articles/realsense-ux-design-guidelines
Intel® RealSense™ Technology | Intel® Software
Developing Augmented Reality applications with R200
Intel® RealSense™ Technology | Intel® Software 22
What is Scene Perception?• Provides Augmented Reality experiences through
• Adding virtual content in real time to live camera feed
• Handling occlusions of virtual objects with physical objects
• Providing motion estimation and metric measurement
• Intel® RealSense™ SDK library for Scene Perception
• libPXCScenePerception
Intel® RealSense™ Technology | Intel® Software 23
Scene Perception in Action
Intel® RealSense™ Technology | Intel® Software 24
Augmented Reality in Action
Intel® RealSense™ Technology | Intel® Software 25
Metric Measurements in Action
Intel® RealSense™ Technology | Intel® Software 26
Requirements for Scene Perception• The scene being captured has to be static
• Works within the bounds of a close object / cubicle space/ living room
• You can track camera movement in 3D space along six degrees of freedom
-courtesy: http://en.wikipedia.org/wiki/Six_degrees_of_freedom
Intel® RealSense™ Technology | Intel® Software
Modalities you will learn about ( R200 )• Using Scene Perception for Augmented Reality
• Configuring Scene Perception
• Setting Initial Camera Pose
• Setting Meshing Threshold
• Setting Voxel Resolution
• Starting Scene Quality
• Checking Scene Quality
• Enabling Scene Reconstruction
• Tracking Data
• Camera Pose
• Volume Preview
• Mesh Data
Intel® RealSense™ Technology | Intel® Software 28
Guidelines for using Scene Perception
• Works best in the depth camera working range
• The observed environment is assumed to be containing some visual texture and/or having some 3D structure:
• Tracking may be inaccurate when tracking reflective or monotonous surfaces, flat walls, dark black areas or scene with limited geometric structure
• Using appropriate VoxelResolution::XYZ_RESOLUTION settings:
• VoxelResolution::HIGH_RESOLUTION for object size use cases
• VoxelResolution::LOW_RESOLUTION for room sized environments
Intel® RealSense™ Technology | Intel® Software
Intel® RealSense™ Front-Facing 3D Camera (F200) and Augmented Reality Usages
Intel® RealSense™ Technology | Intel® Software 30
Augmented Reality apps can make use of:
• Contour Mode
• Hand Data
• Joint Data
• Finger Data
• Extremity points
• Hand Openness
• Usages
• Nail Salon
• Virtual Shopping – Jewelry
• Games
AR Usages Using Hand Module
Intel® RealSense™ Technology | Intel® Software 31
Augmented Reality apps can make use of:
• Landmark
• Pose detection
• Eye Movement
• Expression Detection
Usages:
• Face Makeup
• Virtual Shopping for glasses
• Games/Entertainment
• Plastic Surgery
AR Usages using Face Module
Intel® RealSense™ Technology | Intel® Software 32
Resources
software.intel.com/realsense
software.intel.com/articles/realsense-ux-design-guidelines
[email protected], @karthur
[email protected], @RaoIntel
Intel® RealSense™ Technology | Intel® Software