Upload
ashlyn-jean-bond
View
242
Download
0
Tags:
Embed Size (px)
Citation preview
Interaction in the Virtual World: Overview• User interface• Navigation
– Wayfinding– Travel
• Interaction and communication with others– Sharing– Collaboration
• Manipulation– Methods– Properties– Selection– Operations
Broad Classification of Interactions
• Interactions– Reality-based– Imagination-based (magic)
Navigation
• Types of navigation goals: explore rate or position controlled), search (target) and maneuver (objects, environment)
• Methods:– Wayfinding– Travel
Navigation: Wayfinding (knowing where you are and where you’re going)
• Need a mental or cognitive map
• Relationship to the world, to the destination
• Situational awareness
• eg. of GPS system maps
• Great differences in people, sometimes gender related
• Problem of becoming lost, metaphors
Navigation: Styles
• Relative motion (steps)
• Absolute motion (point to a location and go)
• Teleport
• Move the world instead of the user
Navigation: Wayfinding (con’t)
• Mental map:– Divide and conquer: small regions, then piece
together– Global network: use landmarks– Memorize a map of the space– Remember a story about the space
Navigation: Wayfinding (con’t)
• Wayfinding aids– Path following: color, lines, labels– Maps (exo or ego): scale, icons, you-are-here– Landmarks: objects, audio– Placenames– Breadcrumbs (trail)– Compass– Instrument guidance: could be multimodal– Shift to exocentric view– Display of coordinates or grids, names– Constrain travel
Navigation: Travel Properties
• Manipulation method: physical (steering wheels, flight, dashboard) and virtual controls
• Constraints: terrain following, no z-axis, no lateral
• Frame of reference: relative motion
• Movement: gain, velocity, acc; voice
Navigation: Types of Travel
• Physical locomotion: direct tracking, issue of working volume
• Vehicle simulation
• Magic techniques, including steering, target-based travel
• Combination of natural and magic: manipulation-based travel
• Ride along or towrope: pre-set path
Navigation: Types of Travel (con’t)
• Fly-through, walkthrough, pilot-through: pointer directed, gaze-directed, torso-directed, dual-handed
• Move the world• Scale the world• Put me there, jump to destination• Time travel
Navigation and Travel
• Physical walking
• Vehicle simulation
• Magic techniques, including steering, target-based travel
• Combination of natural and magic: manipulation-based
Interaction: Communication
• Shared experience: same world, may be competing or co-existing
• Collaborative experience: need to work together in some way
Shared Experience
• Different ways of sharing in different mediums
• What can be shared: ideas, world, speech• Ways to share viewpoint
– Full multipresence: may have same equipment (such as HMDs), or be in same location (cockpit)
– Some viewers have partial presence: one immersed participant with onlookers, open display (possibly projection)
Shared Experience (con’t)
• Purpose of sharing: marketing (onlookers become interested), inc. throughput (onlookers get some experience even if not the full experience)
Collaborative Experience
• Communication: aural, visual, gestures, teleconferencing, marks or notes, physical proximity
• Synchronous communication: live conversations• Asynchronous communication: change a world
but then not be present, present in another part of the world – can leave marks or messages
• Can have both synchronous and asynchronous in the same world
Collaborative Experience (con’t)
• Messages or annotations: recipient, time (when needed), purpose– How do you know it’s there: voice, text,
gestures, pictures– Location– Point of view– Time– placeholders
Collaborative Experience (con’t)
• Who’s in charge (floor control and permissions): no one, moderated, permissions, formal, hierarchical, first come first served
Metacommands
• Manipulation of environment by someone other than participant (eg. of VR therapy)
• User can cause different worlds to load
Manipulation: methods
• Direct user control: gestures mimic real world
• Physical control devices: wands, buttons, joysticks, mouse
• Virtual controls: interfaces on the screen such as buttons, sliders, control panel, arrows, etc.
• Agent controls: intelligent intermediary
Manipulation: properties
• Feedback: user needs to know when something has happened- haptic (can feel the contact), aural (sound), visual (change color)
• Ratcheting: repeat input – eg. pick up mouse
• Constraints: restrict a DOF, snap to grid, lock to surface, keep on floor
Manipulation: properties (con’t)
• Distance: manipulate objects beyond reach
• Pointer beam scope: shape of beam; eg. laserbeam, spotlight, conelike
Manipulation: properties (con’t)
• Frame of reference: world representation, local representation– inside or outside the world- telepresence and teleoperation-egocentric or exocentric (god’s eye)-– Coordinate systems: world, local, parent-
vocabulary of DOF: vertical, lateral, longitudinal; Euler angles; yaw, pitch, roll
Manipulation: properties (con’t)
• Bimanual interface: sometimes with haptics, keypress with joystick (difficult), medical applications; must be well designed
• Control location: omnipresent or appear, placement (interfere with world or not): part of the world (door handle), attached to hand, front of the view (helmet, windshield), on the display, on a 2D panel of controls, physical devices (tablets)
Manipulation: properties (con’t)
• Control visibility: visible and invisible; how made visible
• Movement formula: gain (how much movement in one action), derivatives (velocity, acceleration)
Object Manipulation
• Manipulation of the environment: could also be considered viewpoint navigation
• Selection of objects: pointing, picking, grasping
• System control: usually through 2D interfaces, buttons
Manipulation: selection• Can be direction, item or value
• Direction– Pointer-directed: gesture– Gaze or eye tracking– Crosshair: pointer and gaze– Torso for direction of travel– Joystick, mouse, trackball- what about angles– Coordinates: possibly by voice– Landmarks
Manipulation: selection (con’t)• Item selection
– Contact between avatar and item – choose part of body; need feedback
– Point to select (prop, gesture)– 3D cursor select; may need button; question
of what’s in range and out of range; change in symbol
– Pinching or grasping– Naming– Menu; words or pictures– Select in a small world
Manipulation: selection (con’t)• Value selection: pen devices, keyboard,
tablet, physical sliders or controls, agents
Manipulation: operations
• Positioning and sizing objects
• Exerting force: pushing, hitting, supporting- does it move?
• Modifying attributes of objects: color, light, transparency, weight, shape, firmness, density, etc.
• Modifying global attributes: sounds, time of day, form of rendering
Interactions• Reality-based interaction:
– Head tracking, locomotion, 3D pointing– Direct manipulation of objects, gloves, haptics– Indirect manipulation through input devices– Objects for manipulation, tools (eg. wands), tangible
input devices, sliders, joysticks, touch sensors• Imagination-based interaction:
– Suspension of physics– Scaling of geometry, zooming, no clear standards yet– Scaling of motion– Automation of tasks, motion, etc.– Magic spells: dynamically assign meaning and tasks– Mode changes: eg from selecting to manipulation
SourcesBuilding on Realism and Magic for Designing 3D Interaction
Techniques by Kulik, IEEE Computer Graphics and Applications, Nov/Dec 2009
Questioning Naturalism in 3D User Interfaces, Bowman, McMahan, and Ragan, Comm. Of the ACM, 2012
Understanding Virtual Reality by Sherman & Craig, Morgan Kaufman, 2003
Computer Graphics and Virtual Environments by Slater et al