COMP 4010 Lecture9 AR Displays

LECTURE 9: AR TECHNOLOGY: DISPLAYS

AND TRACKING

COMP 4010 – Virtual Reality Semester 5 – 2016

Bruce Thomas, Mark Billinghurst University of South Australia

October 11th 2016

Augmented Reality Definition • Defining Characteristics [Azuma 97]

• Combines Real and Virtual Images • Both can be seen at the same time

• Interactive in real-time • The virtual content can be interacted with

• Registered in 3D • Virtual objects appear fixed in space

Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.

Augmented Reality Technology

! Combining Real and Virtual Images •  Display technologies

! Interactive in Real-Time •  Input and interactive technologies

! Registered in 3D •  Viewpoint tracking technologies

Display

Processing

Input Tracking

AR DISPLAYS

Display Technologies ! Types (Bimber/Raskar 2003)

! Head attached •  Head mounted display/projector

! Body attached •  Handheld display/projector

! Spatial •  Spatially aligned projector/monitor

Display Taxonomy

HEAD MOUNTED DISPLAYS

Head Mounted Displays (HMD) • Display and Optics mounted on Head • May or may not fully occlude real world •  Provide full-color images • Considerations

•  Cumbersome to wear •  Brightness •  Low power consumption •  Resolution limited •  Cost is high?

Types of Head Mounted Displays

Occluded See-thru

Multiplexed

Immersive VR Architecture

Head!Tracker

Host !Processor

Data Base!Model

Rendering!Engine Frame!

Buffer

head position/orientation

to network Display!Driver

Non see-thru!

Image source &

optics

virtual object

Virtual World

See-thru AR Architecture

Head!Tracker

Host !Processor

Data Base!Model

Rendering!Engine Frame!

Buffer

head position/orientation

to network Display!Driver

see-thru!combiner

Virtual Image superimposed!over real world object

real world

Image source

Optical see-through head-mounted display Virtual images from monitors

Real World

Optical Combiners

Optical See-Through HMD

Epson Moverio BT-200

▪ Stereo see-through display ($700) ▪  960 x 540 pixels, 23 degree FOV, 60Hz, 88g ▪ Android Powered, separate controller ▪ VGA camera, GPS, gyro, accelerometer

View Through Optical See-Through HMD

Strengths of optical see-through AR

• Simpler (cheaper) • Direct view of real world

• Full resolution, no time delay (for real world) • Safety • Lower distortion

• No eye displacement • but some video see-through displays avoid this

Video AR Architecture

Head!Tracker

Host !Processor

Graphics!renderer

Digital!Mixer Frame!

Buffer

head position/orientation

to network Display!Driver

Non see-thru!

Image source &

optics

Head-mounted camera

aligned to display optics

Video!Processor

Video image of real world

Virtual image inset into video of real world

Video see-through HMD

Video cameras

Monitors

Graphics

Combiner

Video

Video See-Through HMD

Vuzix Wrap 1200DXAR

▪ Stereo video see-through display ($1500) ■ Twin 852 x 480 LCD displays, 35 deg. FOV ■ Stereo VGA cameras ■ 3 DOF head tracking

View Through a Video See-Through HMD

Strengths of Video See-Through AR • True occlusion

• Kiyokawa optical display that supports occlusion

• Digitized image of real world • Flexibility in composition • Matchable time delays • More registration, calibration strategies

• Wide FOV is easier to support

Optical vs. Video AR Summary

• Both have proponents • Video is more popular today?

• Likely because lack of available optical products

• Depends on application? • Manufacturing: optical is cheaper • Medical: video for calibration strategies

Eye multiplexed AR Architecture

Head!Tracker

Host !Processor

Data Base!Model

Rendering!Engine Frame!

Buffer

head position/orientation

to network Display!Driver

Virtual Image inset into!real world scene

real world

Opaque!Image source

Virtual Image ‘inset’ into real

Google Glass

View Through Google Glass

Vuzix M-100

▪ Monocular multiplexed display ($1000) ■ 852 x 480 LCD display, 15 deg. FOV ■ 5 MP camera, HD video ■ GPS, gyro, accelerometer

Display Technology

•  Curved Mirror •  off-axis projection •  curved mirrors in front of eye •  high distortion, small eye-box

•  Waveguide •  use internal reflection •  unobstructed view of world •  large eye-box

See-through thin displays

•  Waveguide techniques for thin see-through displays •  Wider FOV, enable AR applications •  Social acceptability

Opinvent Ora

Lumus DK40

SPATIAL AUGMENTED REALITY

Spatial Augmented Reality

• Project onto irregular surfaces • Geometric Registration •  Projector blending, High dynamic range

• Book: Bimber, Rasker “Spatial Augmented Reality”

Projector-based AR

Examples: Raskar, MIT Media Lab Inami, Tachi Lab, U. Tokyo

Projector

Real objects with retroreflective covering

User (possibly head-tracked)

Example of projector-based AR

Ramesh Raskar, UNC, MERL

Example of projector-based AR

Ramesh Raskar, UNC Chapel Hill

Head Mounted Projector

• NVIS P-50 HMPD •  1280x1024/eye •  Stereoscopic •  50 degree FOV •  www.nvis.com

HMD vs. HMPD

Head Mounted Display Head Mounted Projected Display

CastAR - http://technicalillusions.com/

• Stereo head worn projectors •  Interactive wand • Rollable retro-reflective sheet

• Designed for shared interaction

Demo: CastAR

https://www.youtube.com/watch?v=AOI5UW9khoQ#t=47

Pico Projectors

• Microvision - www.mvis.com • 3M, Samsung, Philips, etc

Demo: Pico Projector

https://www.youtube.com/watch?v=vtdH3CLKEuY

MIT Sixth Sense

• Body worn camera and projector • http://www.pranavmistry.com/projects/sixthsense/

Demo: Sixth Sense

https://www.youtube.com/watch?v=Q4Z9sOtiWUY

OTHER AR DISPLAYS

Video Monitor AR

Video cameras Monitor

Graphics Combiner

Video

Stereo glasses

Examples

Virtual Showcase

• Mirrors on a projection table • Head tracked stereo • Up to 4 users • Merges graphic and real objects •  Exhibit/museum applications

• Fraunhofer Institute (2001) •  Bimber, Frohlich

Demo: Virtual Showcase

https://www.youtube.com/watch?v=iXl4FolFUzc

Augmented Paleontology

Bimber et. al. IEEE Computer Sept. 2002

Alternate Displays

LCD Panel Laptop PDA

Handheld Displays

• Mobile Phones • Camera • Display • Input

Other Types of AR Display

• Audio • spatial sound • ambient audio

• Tactile • physical sensation

• Haptic • virtual touch

Haptic Input

• AR Haptic Workbench •  CSIRO 2003 – Adcock et. al.

Phantom

• Sensable Technologies (www.sensable.com) • 6 DOF Force Feedback Device

AR Haptic Interface

•  Phantom, ARToolKit, Magellan

www.empathiccomputing.org

@marknb00

mark.billinghurst@unisa.edu.au