Virtual Reality

Lecture1. Introduction to Virtual Reality

고려대학교 그래픽스 연구실

Contents

• What is VR?• Conceptual Model of VR• VR Related Areas• History• State of the Art and R&D Issues• Some Application Examples

Science vs. Engineering

• What is not VR.

• high road - replication of reality low road - 3-D interface / interaction

• VR = studies on reality computational reality

• analogous to AI– lots of hypes– AI as a science / engineering

What is not VR.

• VR as computational reality– To seek for the computational model of reality.– To apply the model to the VR system.

• VR can be understood in the context of modeling efforts.– intelligence– linguistics– emotion– life– reality– compuational “X”

Definition : What is VR?

• Computer generated environment that is …– Immersive (like IMAX,dome projection),– Interactive (like computer game),– Multi-sensory,– Viewer-centered,– 3-D, and– The combination of technologies required to build such enviro

nments.

VR as a computer technology

• Technological trend– powerful– smart– physical

• computer - human interaction

• primary concern --- software other important issues --- hardware, human

factors, social issues,

infrastructure

Interacting with computers

ComputerWorld

New Paradigm for HCI

Conventional Computer

ComputerWorld

VR-based Computer

ComputerWorld

VR as a media

• All medium attempt to create the “virtual presence”.– Theater, poem, fine arts, novel, telephone, movies, TV, ...

• VR vs. existing media– immersive– interactive– 3-D– multimodal– mediated

• Information is not sent back and forth.• Mediated environments are created and then experience.

• effectiveness of communication depends on ... the sense of “being

there”.

• virtual presence

• virtual presence depends on ...– sensory breadth– sensory depth– interactivity

VR Application

Education/Training

DesignEngineering

Medicine

ScientificVisualization

Entertainment

Communication

ProductsEngineering

MuseumArts VR application

VR technologies

Character CRT

Graphic CRT

Virtual console

Virtual Reality

Tele-conference

TV phone

Telephone

Keyboard

Mouse tablet

3D mouse

Computersimulation/visualization

Tele-Existence

Tele-Operation

Computergraphics

RealtimeCG

Video arts

3D CAD

ComputerAided Design

Virtual products

design

All technologies meet together at VR !!

VR in Real World vs. Virtual World

Virtual Reality

Tele-Existence inReal World

Tele-Existence inVirtual World

• Physical World

• Quasi Physical World

• Non Physical World

• Standard Tele-Existence

• Augmented Tele-Existence

• Size• Sensation• Time

컴퓨터

로보트

인간에의 임장감 제시

시각 , 청각 , 촉각 , 미각 ,후각

체성감각 힘감각

인간의 상태추정외부상태(

비구속 ,

실시간)

내부상태(

비침투

실시간)

운동

음성

몸통

다리

팔

눈

머리

뇌파 , 심전 ,

근전 , 맥박 ,

혈압 , 발한 ,

etc

감각정보의

통합

인간에의

적합변환

인간의 행동음지추정

감각정보

감각정보

제어정보

제어정보

다른

로보트

상호작용

상호

작용 가상환경

실환경

가상인간

다른 가상인간

다른 Tele Existens system 으로

다른 Tele Existens system 으로

VR system 의 구성

Conceptual Model of VR

• 3D image Large-scale display, Head Mounted Display• Sound field by DSP • Force beedback mechanism• Tactile display

SimulationSystem

Computer

Display system

Sensing system• Non-contact type

magnetic field supersonic wave infrared light

• Contact type optical fiber strain gauge potentio-meter

Human

VR Related areas

1. Training simulation

2. Tele-operation

3. Computer graphics

4. Artificial intelligence

Training simulation

• Differences– reconfigurable by changing software– may include highly unnatural environment– highly interactive and adaptive– use of a wide varielty of human sensing modalities

and sensorimotor systems– highly immersive– near-field is synthetic; far-field is synthetic.

Tele-operation

• for at least 30 years.

• Tele-operator– directly (manually) controlled tele-operator– tele-robot

• Tele-operation vs. Virtual reality

• Tele-presence vs. Virtual presence

Computer graphics

• Modeling• Motion control (animation)• Rendering• User interface

Artificial intelligence

• Studies on perception and cognition

• Testbed for AI research

History

• 1’st stage: some visionaries– Morton Heilig : ExperienceTheatre(1962)– Ivan Surtherland : Sketchpad(1963), HMD(1966)– Myron Krueger : Artificial Reality(1972)– William Gibson : “Cyberspace” in Neuromancer(1984)

• 2’nd stage: technology development for specific purposes– training simulator : Earlier works– space exploration : NASA for astronaut simulation– tele-operation

History(con’t)

• 3’rd stage: VR as the general-purpose technology– Jaron Lanier : VPL(1987) – Dataglove,EyePhone,VR system– VR industry : Division Ltd. Sense8, WorldDesign(production h

ouse,W-Industry(game)– VR academia : MIT, UNC, UW,Tokyo U.

• Next stage: Toward a scientific discipline– computational reality– a new computing paradigm– a new media– a new art form– representation, creation and operation of virtual worlds

State of the Art & Issues

• Reference– Virtual Reality: Scientific and Technological

Challenges”, pp. 35-66, National Research Council, National Academic Press, 1995.

• Areas of the study– application domains– psychological issues– VR technologies– evaluation of VR systems

1. Application domains

• design, manufacturing & marketing• medicine, health care• hazardous operations• training• entertainment, military• experimental psychology• education• information visualization• tele-communication, tele-travel

2. Psychological topics

• human performance characteristics

• alteration of sensori-motor loops

• developing the cognitive model

• cognitive side-effect

3. VR technologies

• Gap between the current technology the required technology (exception -- entertainment, tele-operation)

(1) human-machine interface(2) computer generation of VE(3) tele-robotics(4) network

Human-machine interface (cont’)

• visual channelvisual channel• auditory channelauditory channel• haptic channelhaptic channel• motion interfacemotion interface• position trackingposition tracking• video camera• microphone• others

Visual channel

• visual display– HMD– OHD (off-head display)

• perceptual effects– mis-registration– sensori-motor alteration– distortion– time-delay– noise

• research issues– ergonomics– improvement of resolution and fov– wireless– integration of visual, auditory, position tracking– sun glass-like– see-through option– exploiting foveal and peripheral vision

Auditory channel

• Current hardware is adequate.

• Research issues– perceptual issues

• similar to the visual channel• use for sensory substitution (for visual, haptic)

– auditory scene analysis– hear-through display

Position tracking and mapping

• tracking = finding a pointmapping = finding a 3D surface

(e.g., environmental mapping)

• tracking mechanisms– mechanical linkage– magnetic– optical– acoustic– intertial

• eye tracking

• research issues– tracking– mapping

Haptic channel

• force, pressure, tactile feedback

• unique characteristics– Haptic interface requires manipulation and sensing

• mechanism– body-based -- glove, exoskeleton– ground-based -- joystick

• Research issues– haptic science = study on the human haptics

(bio-mechanical, psychophysical, cognitive)

– tool-hand system (which takes its metaphor from real tools.)

– creating the haptic illusion– the interaction effects of haptic and vision– texture, temperature devices

Motion interface

• motion– whole-body motion

• passive -- e.g., motion platform• active -- e.g., locomotion

– part-body motion• passive• active

• motion cues– vestibular system -- inertial– motor– visual– auditory– proprioceptive / kinesthetic -- muscle– tactile

• motion interface– inertial system

• moves the body (e.g., treadmill, motion platform)

– non-inertial system• simulates motion

Other types of interfaces

• olfactory (smell)• gustatory (taste)• heat, wind, humidity

• speech

• direct physiological sensing and control

VR technologies

(1) human-machine interface

(2) computer generation of VE

(3) tele-robotics

(4) network

(2) Generation of virtual environments

• 3D image Large-scale display, Head Mounted Display• Sound field by DSP • Force beedback mechanism• Tactile display

SimulationSystem

Computer

Display system

Sensing system• Non-contact type

magnetic field supersonic wave infrared light

• Contact type optical fiber strain gauge potentio-meter

Human

Generation of VE(cont’)

• the core issue

• general-purpose VR system?

• trade-off between realism and interactivity

• requirements– frame rate– response time– scene quality

• hardware• interaction and navigation• VE management

– simulation– rendering

• modeling• autonomous agent• hypermedia interaction• OS

Generation of VE(cont’)

VE management - simulation

• Task : simulating everyday world• Traditional simulation methods do not

work. (requires pre-processing)

• Need : “meta-modeling”

VE management - rendering

• Issue : load balancing– 1. partitioning VE– 2. LOD

• Much work has been done on static scene.

• Research issues– 1. dynamic scene– 2. parallel rendering

OS

• real-time, multi-modal requirements

• very high-resolution time slicing

• atomic, transparent distribution of tasks• large number of light-weighted processors,

communicating by means of shared memory

• support for time-critical computing:– negotiated, graceful degradation– guaranteed frame rate, lag time

VR technologies

(1) human-machine interface

(2) computer generation of VE

(3) tele-robotics

(4) network

(3) Tele-robotics

• tele-robotics and VR

• hardware• time-delay problem• distributed tele-robots

(4) Network

• The future is here!

• applications– distance learning– group entertainment– distributed training– distributed design

• current• future• What is needed

Research Organizations

International Efforts

U.S.A. defense, space, visualization, medicineU.K. education, training, entertainmentGermanyJapan VR as a logical extension of

robotics, automation, HDTV.

Academia

• HIT Lab, University of Washington• University of North Carolina• Media Lab, MIT• Georgia Institute of Technology• Naval Postgraduate School• University of Pennsylvania• University of California at Berkeley• University of Illinois - Chicago• Columbia University• University of Toronto

And, Some VR Example… (Video)