Embed Size (px)
Citation preview
LECTURE 9: AR TECHNOLOGY: DISPLAYS
AND TRACKING
COMP 4010 – Virtual Reality Semester 5 – 2016
Bruce Thomas, Mark Billinghurst University of South Australia
October 11th 2016
Augmented Reality Definition • Defining Characteristics [Azuma 97]
• Combines Real and Virtual Images • Both can be seen at the same time
• Interactive in real-time • The virtual content can be interacted with
• Registered in 3D • Virtual objects appear fixed in space
Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.
Augmented Reality Technology
! Combining Real and Virtual Images • Display technologies
! Interactive in Real-Time • Input and interactive technologies
! Registered in 3D • Viewpoint tracking technologies
Display
Processing
Input Tracking
AR DISPLAYS
Display Technologies ! Types (Bimber/Raskar 2003)
! Head attached • Head mounted display/projector
! Body attached • Handheld display/projector
! Spatial • Spatially aligned projector/monitor
Display Taxonomy
HEAD MOUNTED DISPLAYS
Head Mounted Displays (HMD) • Display and Optics mounted on Head • May or may not fully occlude real world • Provide full-color images • Considerations
• Cumbersome to wear • Brightness • Low power consumption • Resolution limited • Cost is high?
Types of Head Mounted Displays
Occluded See-thru
Multiplexed
Immersive VR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
Non see-thru!
Image source &
optics
virtual object
Virtual World
See-thru AR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
see-thru!combiner
Virtual Image superimposed!over real world object
real world
Image source
Optical see-through head-mounted display Virtual images from monitors
Real World
Optical Combiners
Optical See-Through HMD
Epson Moverio BT-200
▪ Stereo see-through display ($700) ▪ 960 x 540 pixels, 23 degree FOV, 60Hz, 88g ▪ Android Powered, separate controller ▪ VGA camera, GPS, gyro, accelerometer
View Through Optical See-Through HMD
Strengths of optical see-through AR
• Simpler (cheaper) • Direct view of real world
• Full resolution, no time delay (for real world) • Safety • Lower distortion
• No eye displacement • but some video see-through displays avoid this
Video AR Architecture
Head!Tracker
Host !Processor
Graphics!renderer
Digital!Mixer Frame!
Buffer
head position/orientation
to network Display!Driver
Non see-thru!
Image source &
optics
Head-mounted camera
aligned to display optics
Video!Processor
Video image of real world
Virtual image inset into video of real world
Video see-through HMD
Video cameras
Monitors
Graphics
Combiner
Video
Video See-Through HMD
Vuzix Wrap 1200DXAR
▪ Stereo video see-through display ($1500) ■ Twin 852 x 480 LCD displays, 35 deg. FOV ■ Stereo VGA cameras ■ 3 DOF head tracking
View Through a Video See-Through HMD
Strengths of Video See-Through AR • True occlusion
• Kiyokawa optical display that supports occlusion
• Digitized image of real world • Flexibility in composition • Matchable time delays • More registration, calibration strategies
• Wide FOV is easier to support
Optical vs. Video AR Summary
• Both have proponents • Video is more popular today?
• Likely because lack of available optical products
• Depends on application? • Manufacturing: optical is cheaper • Medical: video for calibration strategies
Eye multiplexed AR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
Virtual Image inset into!real world scene
real world
Opaque!Image source
Virtual Image ‘inset’ into real
Google Glass
View Through Google Glass
Vuzix M-100
▪ Monocular multiplexed display ($1000) ■ 852 x 480 LCD display, 15 deg. FOV ■ 5 MP camera, HD video ■ GPS, gyro, accelerometer
Display Technology
• Curved Mirror • off-axis projection • curved mirrors in front of eye • high distortion, small eye-box
• Waveguide • use internal reflection • unobstructed view of world • large eye-box
See-through thin displays
• Waveguide techniques for thin see-through displays • Wider FOV, enable AR applications • Social acceptability
Opinvent Ora
Lumus DK40
SPATIAL AUGMENTED REALITY
Spatial Augmented Reality
• Project onto irregular surfaces • Geometric Registration • Projector blending, High dynamic range
• Book: Bimber, Rasker “Spatial Augmented Reality”
Projector-based AR
Examples: Raskar, MIT Media Lab Inami, Tachi Lab, U. Tokyo
Projector
Real objects with retroreflective covering
User (possibly head-tracked)
Example of projector-based AR
Ramesh Raskar, UNC, MERL
Example of projector-based AR
Ramesh Raskar, UNC Chapel Hill
Head Mounted Projector
• NVIS P-50 HMPD • 1280x1024/eye • Stereoscopic • 50 degree FOV • www.nvis.com
HMD vs. HMPD
Head Mounted Display Head Mounted Projected Display
CastAR - http://technicalillusions.com/
• Stereo head worn projectors • Interactive wand • Rollable retro-reflective sheet
• Designed for shared interaction
Demo: CastAR
https://www.youtube.com/watch?v=AOI5UW9khoQ#t=47
Pico Projectors
• Microvision - www.mvis.com • 3M, Samsung, Philips, etc
Demo: Pico Projector
https://www.youtube.com/watch?v=vtdH3CLKEuY
MIT Sixth Sense
• Body worn camera and projector • http://www.pranavmistry.com/projects/sixthsense/
Demo: Sixth Sense
https://www.youtube.com/watch?v=Q4Z9sOtiWUY
OTHER AR DISPLAYS
Video Monitor AR
Video cameras Monitor
Graphics Combiner
Video
Stereo glasses
Examples
Virtual Showcase
• Mirrors on a projection table • Head tracked stereo • Up to 4 users • Merges graphic and real objects • Exhibit/museum applications
• Fraunhofer Institute (2001) • Bimber, Frohlich
Demo: Virtual Showcase
https://www.youtube.com/watch?v=iXl4FolFUzc
Augmented Paleontology
Bimber et. al. IEEE Computer Sept. 2002
Alternate Displays
LCD Panel Laptop PDA
Handheld Displays
• Mobile Phones • Camera • Display • Input
Other Types of AR Display
• Audio • spatial sound • ambient audio
• Tactile • physical sensation
• Haptic • virtual touch
Haptic Input
• AR Haptic Workbench • CSIRO 2003 – Adcock et. al.
Phantom
• Sensable Technologies (www.sensable.com) • 6 DOF Force Feedback Device
AR Haptic Interface
• Phantom, ARToolKit, Magellan
