1@TheVRBookJason Jerald, Phd

VR InteractionsJason Jerald, PhD

2@TheVRBookJason Jerald, Phd

60+ VR Projects, 30+ Organizations

3@TheVRBookJason Jerald, Phd

The Psychology of Perception & Action• Objective vs subjective reality

• Distal and proximal stimuli

• Sensation vs perception

• Bottom-up and top-down processing

• Afference and efference

• Iterative perceptual processing

• The subconscious and conscious

• Mental models

• Visceral, behavioral, reflective, and emotional processes

• Neuro-linguistic programming

• Space, time, & motion

• Perceptual constancies

• Adaptation

• Attention

• Action

4@TheVRBookJason Jerald, Phd

VR Output = Distal stimuli

Image Courtesy of The VR Book(adapted from Goldstein [2007])

The Iterative Perceptual Process

Human action = VR input/tracking

With VR, we hijack the human perception-action system

5@TheVRBookJason Jerald, Phd

The Cycle of Interaction

Image Courtesy of The VR Book (adapted from Norman [2013])

6@TheVRBookJason Jerald, Phd

Input Device Classes

7@TheVRBookJason Jerald, Phd

The Most Important Input Device?

8@TheVRBookJason Jerald, Phd

The Most Important Input Device?The Human Hands!

– Large proportion of sensory and motor cortex devoted to the hands

– Hand tracking technology is simply the mediator that brings the hands into VR

9@TheVRBookJason Jerald, Phd

The First Hand Technique

Image courtesy of NextGen Interactions

10@TheVRBookJason Jerald, Phd

Interaction Patterns

• Hand

Selection

• Pointing

• Image-

Plane

• Volume

Based

Selection

• Walking

• Steering

• 3D Multi-

Touch

• Automated

Viewpoint

Control

• Direct Hand

• Proxy

• 3D Tool

Manipulation

• Widgets &

Panels

• Non Spatial

Indirect Control

• Pointing

Hand

• World-in-

Miniature

• MultiModal

Compound

11@TheVRBookJason Jerald, Phd

The Hand Selection PatternHands with

Arms

Video Courtesy of Digital ArtForms

12@TheVRBookJason Jerald, Phd

The Pointing Pattern

Image Courtesy of Digital ArtForms

13@TheVRBookJason Jerald, Phd

The Image-Plane Pattern

The Head-Crusher Technique

From Pierce et al. [1997]

14@TheVRBookJason Jerald, Phd

Widgets & Panels Pattern

• Hand-held panels

– Buttons

– Sliders

– Dials

– Color cubes

Images Courtesy of Sixense & Digital ArtForms

15@TheVRBookJason Jerald, Phd

World-In-Miniature PatternReferenced world view

within a box

Images Courtesy of Digital ArtForms

16@TheVRBookJason Jerald, Phd

3D Multitouch Pattern

• An extension of 2D Multitouch• Manipulate the world as an object

– Reduces sickness– Solves gorilla arm

• Low interaction fidelity– Appropriate for abstract non-realistic interactions

• Content/data independent– Emergency preparedness– Medical Visualization– MakeVR

17@TheVRBookJason Jerald, Phd

3D Multitouch—Medical Visualization

Video Courtesy of ACM SIGGRAPH

18@TheVRBookJason Jerald, Phd

3D Multitouch—Content Creation

MakeVR Collaborative

Video Courtesy of Sixense

19@TheVRBookJason Jerald, Phd

3D Tools PatternThe Jigs Technique

Images Courtesy of Digital ArtForms and Sixense

20@TheVRBookJason Jerald, Phd

What is Common to All VR Dev?

Iteration!

21@TheVRBookJason Jerald, Phd

Questions