46
Defence Research and Development Canada Recherche et développement pour la défense Canada Canada Virtual Immersion Facility (VIF) A Training Perspective 28 February 2008

Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence Research andDevelopment Canada

Recherche et développementpour la défense Canada Canada

Virtual Immersion Facility (VIF)A Training Perspective

28 February 2008

Page 2: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

System of Systems (SoS) Section in DRDC - ValcartierSimulation and Comprehension of Complex Situation (SCSC) Group Team composed of 3 defense scientists and 1 computer support

Dr. Marielle MokhtariFrançois BernierÉric BoivinFrédéric Drolet

Who we are

Page 3: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

SCSC aims exploiting the Synthetic Environment (SE) for DND/CF(immersive environments, virtual reality, advanced visualization, simulation, serious gaming, haptic devices)We found out that there was no high-end general purpose immersive environment in the DND/CFThen we decided to create the Virtual Immersive Facility (VIF)

Rational

Page 4: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Virtual Immersive Facility

Supports experimentations (or training), allows testing of applications, and provides computer infrastructures (140 m²) to many projects simultaneouslyThe CAVE is the main component; the VIF also includes manyother advanced devicesOperational since August 2006Investment of 1.5 M$

Explores the potential of advanced visualization, interactive simulation & immersive capabilities for the DND/CF

Mission

Description

Page 5: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Developed in 1992 at the University of Illinois at ChicagoCave Automatic Virtual Environment (CAVE)Immersive virtual reality environment

What is a CAVE?

Room whose walls surround a viewer with projected images(Cruz-Neira et al., 1992)“In the CAVE, you are no longer on the outside looking in but on the inside looking out” (DeFanti)

Page 6: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

High degree of immersion by using advanced human machine interfaces (visual, haptic, tracking, odors, etc.)High degree of realismMore natural interaction with the virtual environmentHigh degree of presence (the sense of being in the virtual environment)

Main advantages

Page 7: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Visual Immersion - Full Field of View

Synthetic Environment 1

Page 8: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Synthetic Environment 1

Visual Immersion - Full Field of View

Synthetic Environment 2

Page 9: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Synthetic Environment 1

Visual Immersion – Full Field of View

Synthetic Environment 2

Page 10: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Synthetic Environment 1

Visual Immersion - Continuous display

Synthetic Environment 2

Page 11: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Synthetic Environment 1

Visual Immersion – Perspective Matters

Synthetic Environment 2

Synthetic Environment 1

Page 12: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Synthetic Environment 1

Visual Immersion – Perspective Matters

Synthetic Environment 2

Synthetic Environment 1

Synthetic Environment 2

Page 13: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Visual Immersion - Continuous display

Synthetic Environment 1

Synthetic Environment 2

Synthetic Environment 1

Synthetic Environment 2

Page 14: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Visual Immersion – Perspective matters

Immersed perspective

External perspective

Page 15: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Interaction - Simulated Weapon (1)

Page 16: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Interaction – Hands & Other Devices (2)

Page 17: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

The presence

Defined as the participant’s sense of “being there” in the virtual environment (Bartfield et al., 1995)Participants feel they are part of the simulated world because they forget that their perceptions are mediated by technologies (perceptual illusion of non-mediation)The most important characteristic of immersive systemsFacilitated by the immersion, the realism of the environment and natural interaction with the virtual environmentIt can be used to create stress, lost of equilibrium, fear, trigger reflex, increase concentration, motivation & interest, involvement in the situation, facilitate learning, better transmit the complexity of a situation, etc.Allows to integrate physical with cognitive aspects

Page 18: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Flex CAVE in Valcartier

"Holodeck" Immersive Mode

Immersive Theatre Mode

Flat Wall Mode

Reconfigurable

Virtual immersion system Flex™ by Fakespace: 3 walls and 1 floor of large size (2.4 m × 3.3 m × 3.3 m)

High intensity (8000 Lumens) and high resolution (1400×1050 pixels) stereoscopic projectors

3D realistic sound effectsReal-time user tracking

Characteristics

Page 19: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Approach

Focus on performance, flexibility & robustnessExploit high-end multi-screen visualization & immersion systemEmphasize on technologies integration: visualization, simulation& interactionExploit methodologies to improve application developmentRely on VR innovationsExploit gaming

Page 20: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Applications

Improve team-based activities by facilitating the collaboration & the development of a shared perspective (IMAGE project: team supported with interactive simulation & visualization)

Provide more realistic & flexible training capability at lower cost (e.g.: stress inoculation, sensitive site exploitation, tactical combat casualty care, foreign culture…)

Recreate the battlefield in a realistic way which can help in the treatment of post traumatic stress disorder treatment

Facilitate extraction of meaningful patterns from (massive) datasets, help to detect the expected & discover the unexpected

Improve situational awareness, mission planning / rehearsal & after action review

Facilitate engineering review

Page 21: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Existing training is mainly theoreticalTraining based on live simulation is very limited

Sensitive Site Exploitation

Who

ExplorationSince it is a highly spatial task, could the CAVE strengthen traditional Intelligence staff training by adding interactive experiments inside diversified, rich & realistic context-driven environments ?Focused on the technical feasibility of anticipated required functionalities : CAVE, serious gaming, scenariosDemonstrated: interaction metaphors, fast scenario development, task realism, etc.

SSE trainers

Problem

Page 22: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

SSE – Some examples

Page 23: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

SSE – Some examples

Object Move

Sound & scripts

Page 24: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

SSE – Some examples

Object Move

Sound & scripts

Page 25: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

SSE – Some examples

Object Move

Sound & scripts

Page 26: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Object Move

Sound & scripts

Page 27: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Difficulty to train medics in an operational contextLive simulation training is too constraining

Medic Training

Who

ExplorationCould we exploit the CAVE to supplement the traditional medic (instrumented dummy) training with a combat situation ?Tested with military and civilian scenarios

Centre Medical de Simulation des FC (CFB Valcartier)

Problem

Page 28: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Medic Training

Page 29: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Immersive vs. non immersive simulations

Capt. Mike Lakatos & John Benson form the tech staffWho

Detachment level training (30 part.): 1 (CAVE) + 4 (desktop)Reinforcement to platoon operation & vehicle checkpoint operation scenarios20 days of preparation + 4 days of training

Objective

Exploration

Assess the capabilities of immersive and non-immersive simulation platforms to assist training as outlined in the Training Needs Framework (TNF)

Page 30: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Immersive vs. non immersive simulations

Many Interesting lessons learned (observations)Familiarization faster than with desktop “Gaming” perception: lower than on desktopImprove motivationTrigger reflexes (e.g.: crouching when shoot)Increase fatigue level but also the “switch off” levelCan easily create information overloadIncrease threat level perceptionTraining – combination of cognitive & physical levels

Page 31: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Knowledge Wall

Page 32: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

VIF Points of ContactDr Marielle Mokhtari – Defence Scientist

Defence R&D Canada Valcartier(418) 844-4000 (4282)

[email protected]

François Bernier – Defence ScientistDefence R&D Canada Valcartier

(418) 844-4000 (4346) [email protected]

Éric Boivin – Defence ScientistDefence R&D Canada Valcartier

(418) 844-4000 (4394) [email protected]

Frédéric Drolet– Computer SupportDefence R&D Canada Valcartier

(418) 844-4000 (4351) [email protected]

Page 33: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Why not Head-Mounted Displays1 (HMD)?

Low-Cost <1k$ 25º 320x240 – 800x600

Category Price FOV Resolution Dequivalent

Professional 1-10k$ 45º 800x600 – 1280x1024

Expert 10-50k$ 60º 1024x768 – 1280x1024

50-135k$ 112º 1024x768 – 1280x1024

1. Laurendeau, D, Branzan-Albu, A., Boivin, E., Drouin, R., Martel, H., Ouellet, D., Schwartz, J.M. “Survey of the State-of-the-Art on Synthetic Environments, Sensori-Motor Activities in Synthetic Environments, Simulation Frameworks and Real-World Abstraction

Models”, Contract W7701-023981/QCA, Laval University, 2003.

30’

16’

11.5’

4.5’High Performance

Page 34: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

CAVE Equivalent to HMD

30’

16’11.5’

4.5’

<1k$ 1-10k$ 10-50k$ 50-135k$

Ex: 25º FOV is equivalent to a user standing at 30 feet from the CAVE front screen

HMD

CAVE

HMD≈ HMD≈ HMD≈

CAVE CAVE CAVE

Page 35: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Resolution

Contrast

FOV

Brightness

Latency

HMDMin

640x480

75:1

24º

50 cd/m2

10-30 ms

1280x1024

HMDMax

300:1

112º

171 cd/m2

10-30 ms

CAVEDLP

4200x1050

2000:1

270º

~400 cd/m2

0 ms

3840x1024

N/A

270º

~10 cd/m2

0 ms

CAVECRT

Comparison with HMD and CRT CAVE

Page 36: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Development,

Test &

Control Area

Virtual Immersive Facility - Drawing

Experimentation Area

ServerArea

Page 37: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Connection Schema

Crestron Control System

RS

232/

422/

485

RS

232/

422/

485

RS

232/

422/

485

RS

232/

422/

485

RS2

32/4

22/4

85

RS2

32/4

22/4

85

PC

Pro

gram

min

gPo

rt (R

S23

2)

A B C D E F

C2E

NET

-2

C2E

NET

-2

LAN_A LAN_B

Front right speaker

Rear left speaker

Rear right speaker

Front left speaker

Center speaker

Subwoofer

HUB

Eth

erne

t

Eth

er#1

Eth

er#2

Pow

er

Gra

sp

Seria

l

FCUEth

erne

t

USB

VG

AO

utpu

t

PC

Eth

erne

t

USBVG

AO

utpu

t

Ser

ial

Ser

ial

CGIU Left

Ser

ial

Inpu

t

Ser

ial

Inpu

t

CGIU Right

KVM

Vid

eo

out

Vide

o in

#1

USB

in #

1

Vide

o in

#2

USB

in #

2

US

B

mou

se

US

B

keyb

oard

PowerbarPower in

Lutron GRX-232

IS-900Wireless Receiver

RJ-

48

Pow

er

RJ-

48

Pow

er

2.4 Ghz 10 mW

Emitter Only

IS-900Wireless Receiver

2.4 Ghz 10 mW

IS-900 Right-Hand Tracker

Rac

k-M

ount

for I

mm

ersi

on e

quip

men

t

Crestron Display

Ethernet

RS-232 Aud

io

Cha

nnel

Input

Output

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Aud

io

Cha

nnel

Audio Amplifier

IS-900 Head-Tracker

IS-900 Left-Hand Tracker

Emitter Only

IS-900 Wand

Receiver Only

Receiver Only

Sonicstrip Sonicstrip

IS-900 Controler

Eth

erne

t

RS

-232

RJ-

48

RJ-

48

RJ-

48

RJ-

48

RJ-

48

RJ-

48

RJ-

48

RJ-

48

RS232MUX Link 1MUX Link 2MUX Link 3

Lutron Graphic Eye Main Unit

MUX Link 1 Zone 1

MUX Link 2

MUX Link 3

Zone 2

Zone 3RS232

Wall Control

Digital Audio ProcessorRS232

Dig

ital a

udio

S/P

DIF

Dig

ital a

udio

S/P

DIF

Dig

ital a

udio

Tosl

ink

Opt

ical

Dig

ital a

udio

Tosl

ink

Opt

ical

Dig

ital a

udio

RJ-

45 C

AT5

Ana

log

audi

oR

J-45

CA

T5

Input

Output

A.a

udio

RC

A

(Fro

nt le

ft)

A.a

udio

RC

A

(Fro

nt ri

ght)

A.a

udio

RC

A

(Rea

r lef

t)

A.a

udio

RC

A

(Rea

r rig

ht)

AI1 AI2 AI3 AI4A B

A.au

dio

RC

A

(Fro

nt le

ft)

A.au

dio

RC

A

(Fro

nt ri

ght)

A.au

dio

RC

A

(Rea

r lef

t)

A.au

dio

RC

A

(Rea

r rig

ht)

AO1 AO2 AO3 AO4

A.a

udio

RC

A

(Cen

ter)

AO5

A.a

udio

RC

A

(Sub

woo

fer)

AO6

Video Card 1 Sound Card 5.1

BOXX A

DV

I out

1A

DV

I out

1B

US

B in

1

Video Card 2

DV

I out

2A

DV

I out

2B

Seria

l

Fron

t L+R

Rea

r L+R

Cen

ter +

S

ubw

oofe

r

Mic

ro in

Ste

reo

sync

ou

t

Eth

erne

t

Video Card 1 Sound Card 5.1

BOXX B

DV

I out

1A

DV

I out

1B

US

B in

1

Video Card 2

DV

I out

2A

DV

I out

2B

Ser

ial

Fron

t L+R

Rea

r L+R

Cen

ter +

S

ubw

oofe

r

Mic

ro in

Ste

reo

sync

ou

t

Eth

erne

t

Video Card 1 Sound Card

SGI Onyx 4

DV

I out

1A

DV

I out

1B

US

B in

1

Video Card 2

DV

I out

2A

DV

I out

2B

Seria

l

Mic

ro in

Ste

reo

sync

ou

t

Eth

erne

t

Video Card 3

DV

I out

3A

DV

I out

3B

Video Card 4

DV

I out

4A

DV

I out

4B

Opt

ical

Fron

t L+R

Rea

r L+R

Cen

ter +

S

ubw

oofe

r

RJ-48

RJ-48

Main Projectors

Left ScreenProjector

Vid

eo

Middle ScreenProjector

Right ScreenProjector

Floor ScreenProjector

RS

-232

Eth

erne

t

Vid

eo

RS

-232

Eth

erne

t

Vid

eo

RS

-232

Eth

erne

t

Vid

eo

RS

-232

Eth

erne

t

Control Monitors

Left ScreenMonitor

Middle ScreenMonitor

Right ScreenMonitor

Floor ScreenMonitor

DV

I in

DV

I in

DV

I in

DV

I in

Stereo Signal SwitchboxStereo IR signal

generatorStereo sync in

LED

Mobile station

USB out 1

USB out 2

VGA in 1A

VGA in 1B

Control station

USB out 1

USB out 2

DV

I out

1

DV

I out

2 Ethernet

DVI in 1A

DVI in 2A

USB in 1

USB in 2

HD15 in 1B

HD15 in 2B

Development station #1

USB out 1

USB out 2

DV

I out

1

DV

I out

2 Ethernet

DVI in 1A

DVI in 2A

USB in 1

USB in 2

HD15 in 1B

HD15 in 2B

Development station #3

USB out 1

USB out 2

DV

I out

1

DV

I out

2 Ethernet

DVI in 1A

DVI in 2A

USB in 1

USB in 2

HD15 in 1B

HD15 in 2B

Development station #2

USB out 1

USB out 2

DV

I out

1

DV

I out

2 Ethernet

DVI in 1A

DVI in 2A

USB in 1

USB in 2

HD15 in 1B

HD15 in 2B

Development station #4

USB out 1

USB out 2

DV

I out

1

DV

I out

2 Ethernet

DVI in 1A

DVI in 2A

USB in 1

USB in 2

HD15 in 1B

HD15 in 2B

Home Theater Connect DTS-610

Analog Input

Digital Output

Opt

ical

S/P

DIF

Coa

xial

S/P

DIF

Fron

t L+R

Rea

r L+R

Cen

ter +

Su

bwoo

fer

Stereo sync out

Stereo sync in

Stereo sync in

RS-232

USB in 1A

USB in 1B

USB in 4A

USB in 4B

USB out 1

USB out 2

USB out 3

USB out 4

USB in 2A

USB in 2B

USB in 3A

USB in 3B

USB out 5

USB out 6

USB out 7

USB out 8

KVM 4x8(UNIMUX-4X8-U)

Floor Trap connectivity

USB in 1

VGA in 1

VGA in 2

VGA in 3

USB in 2

Ethernet 1

VGA out 1

Ethernet 2

Stereo in

Micro in

Power out 1

Power out 2

Phone

USB out 1

VGA out 1

VGA out 2

VGA out 3

USB out 2

Ethernet 1

VGA in 1

Ethernet 2

Stereo out

Micro out

Power in 1

Power in 2

Phone

Video Matrix 16x16RS232

RS422

Output

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

O1O2O3O4O5O6O7O8O9O10O11O12O13O14O15O16

Input

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

I1I2I3I4I5I6I7I8I9I10I11I12

RG

B V/H

RG

B V/H

RG

B V/H

RG

B V/H

I13I14I15I16

VGARGB V/HVideo Scaler

VGARGB V/HVideo Scaler

VGARGB V/HVideo Scaler

Home Theater Connect DTS-610

Analog Input

Digital Output

Opt

ical

S/P

DIF

Coa

xial

S/P

DIF

Fron

t L+R

Rea

r L+R

Cen

ter +

S

ubw

oofe

r

Mic

roph

one

Mic

roph

one

Page 38: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Future Battle Commanders

with Advanced Decision Making

Capabilities

Combination of advanced visualization & interaction techniques and simulation tools → To support commander’s need for mission planning/rehearsal, SA & AAR and to assist commanders in their

decision-making process.

Combination of advanced visualization & interaction techniques and simulation tools → To support commander’s need for mission planning/rehearsal, SA & AAR and to assist commanders in their

decision-making process.

“ What if a commander could virtually step into an actual operational environment before sending troops in? What if he could plan his entire mission by walking through each step & manoeuvre, pointing out the most probable point of retaliation? By bringing resources such as Google Earth & using the latest in gaming technology, it is possible to develop the future of battle command planning... ”

“ What if a commander could virtually step into an actual operational environment before sending troops in? What if he could plan his entire mission by walking through each step & manoeuvre, pointing out the most probable point of retaliation? By bringing resources such as Google Earth & using the latest in gaming technology, it is possible to develop the future of battle command planning... ”

Page 39: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Existing Limitations – (1) Applications can generate easily 3D models of the battlefield → …display on 2D monitors. (2) Huge of information to understand, manipulate & structure → …display on (small) 2D monitors. (3) No easy way to interact with a running simulation in order to change parameters value at one specific point to rapidly see the impact…→ Need of an “advanced” capability to “immerse” commanders in a SE supporting interaction – visualization - simulation (& modeling).

Idea - Simultaneously increase the overall comprehension of situation | information through simulation & intuitive visualization and improve the mechanisms for user perception | interaction.

Expected Outcome - Improve future commanders’ ability to make (C2) decisions by permitting interactive & dynamic virtual 3D course of action analysis, mission rehearsals, and mission planning so they may act decisively in the face of quickly changing circumstances.

Existing Limitations – (1) Applications can generate easily 3D models of the battlefield → …display on 2D monitors. (2) Huge of information to understand, manipulate & structure → …display on (small) 2D monitors. (3) No easy way to interact with a running simulation in order to change parameters value at one specific point to rapidly see the impact…→ Need of an “advanced” capability to “immerse” commanders in a SE supporting interaction – visualization - simulation (& modeling).

Idea - Simultaneously increase the overall comprehension of situation | information through simulation & intuitive visualization and improve the mechanisms for user perception | interaction.

Expected Outcome - Improve future commanders’ ability to make (C2) decisions by permitting interactive & dynamic virtual 3D course of action analysis, mission rehearsals, and mission planning so they may act decisively in the face of quickly changing circumstances.

Context

Page 40: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

(Large scale) immersive environments are considered beneficial for data analysis & visualization.→ Increase the volume of data available to commander but improve perception of these data.

(Large scale) immersive environments are considered beneficial for data analysis & visualization.→ Increase the volume of data available to commander but improve perception of these data.

“Interactive simulation” – Human-in-the-loop→ Improve the commander’s ability to control, to act upon the simulation, then to control, to act upon scenario & models.

“Interactive simulation” – Human-in-the-loop→ Improve the commander’s ability to control, to act upon the simulation, then to control, to act upon scenario & models.

Immersive 3D visualization stereoscopic technologies→ Improve the commander’s ability to comprehend this avalanche of information and act upon it quickly & decisively.

Immersive 3D visualization stereoscopic technologies→ Improve the commander’s ability to comprehend this avalanche of information and act upon it quickly & decisively.

Adequacy

Page 41: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Immersive view of a fictitious replica of a specific site enables commanders to comprehend complex environment that his troops face during operations (intuitive & efficient 3D exploration of, navigation in, learning about).

Using a combination of technological innovations in simulation, visualization & interaction, commanders can quickly gain an understanding of evolving situations, explore their options and communicate their plans.

Immersive view of a fictitious replica of a specific site enables commanders to comprehend complex environment that his troops face during operations (intuitive & efficient 3D exploration of, navigation in, learning about).

Using a combination of technological innovations in simulation, visualization & interaction, commanders can quickly gain an understanding of evolving situations, explore their options and communicate their plans.

3D virtual representations of battle scenarios provide commanders with a more reliable & accurate means of viewing information on the battlefield (in RT). Scenario simulation control adds a new dimension in the understanding of the situation.

3D virtual representations of battle scenarios provide commanders with a more reliable & accurate means of viewing information on the battlefield (in RT). Scenario simulation control adds a new dimension in the understanding of the situation.

Page 42: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

The U.S. Army Communications Electronics Research Development & Engineering Command (CERDEC) Command & Control Directorate (C2D) has developed immersive C2 applications to support the future commander’s ability to make decisions faster & more accurately.

The U.S. Army Communications Electronics Research Development & Engineering Command (CERDEC) Command & Control Directorate (C2D) has developed immersive C2 applications to support the future commander’s ability to make decisions faster & more accurately.

R&D in the application of immersive technologies & advanced visualization techniques → To assist commanders in their decision-making process

R&D in the application of immersive technologies & advanced visualization techniques → To assist commanders in their decision-making process

C2 Applications – Basic functionalities→ Terrain models that provide the 3D simulation of the modeled area;→ User interface that permits selection of the operating mode: live or playback;→ Event Data Manager which receives, stores & forwards tactical messages;→ Presentation software which displays the scenes & permits the user to interact with the immersive environment

C2 Applications – Basic functionalities→ Terrain models that provide the 3D simulation of the modeled area;→ User interface that permits selection of the operating mode: live or playback;→ Event Data Manager which receives, stores & forwards tactical messages;→ Presentation software which displays the scenes & permits the user to interact with the immersive environment

U.S Activities –CERDEC C2D CAVE Lab.

Page 43: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Mission Planning & Execution -Soldiers response to technologyMission Planning & Execution -Soldiers response to technology

→ On the morning of the MOUT mission, the Platoon Sergeant was confident in the locations of the buildings of interest in the area & knew where his soldiers & unmanned assets were to be placed. This technology provided an excellent tool for the soldiers to become familiar with the layout of the MOUT site.

→ On the morning of the MOUT mission, the Platoon Sergeant was confident in the locations of the buildings of interest in the area & knew where his soldiers & unmanned assets were to be placed. This technology provided an excellent tool for the soldiers to become familiar with the layout of the MOUT site.

→ Three leaders stated that the planning session was a great benefit to their planning. The Platoon Leader estimated that the technology improved his planning capabilities by 75%. The Platoon Sergeant stated that the technology is outstanding if it is accurate and up to date. He thought the colors & shapes of the buildings & the layout of the streets was very helpful. His estimate of how this improved their planning was 70%. The Robotics NCO estimated that the ROVR planning was 80% better than using a sand table.

→ Three leaders stated that the planning session was a great benefit to their planning. The Platoon Leader estimated that the technology improved his planning capabilities by 75%. The Platoon Sergeant stated that the technology is outstanding if it is accurate and up to date. He thought the colors & shapes of the buildings & the layout of the streets was very helpful. His estimate of how this improved their planning was 70%. The Robotics NCO estimated that the ROVR planning was 80% better than using a sand table.

U.S Activities –C4ISR On-The-Move 2006 Experiment

Page 44: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

Defence R&D Canada • R & D pour la défense Canada

Obtain a general view/orientation of the layout of the MOUT;Identify suspected enemy strongholds & safe houses;Determine the best locations to place UGS;Identify potential sniper positions;From an aerial view –

Identify routes leading into & out of the MOUT site;By freely navigating within the environment, use a bird-eye view of the

MOUT street layout for improved orientation;Based on intelligence reports, predict where enemy forces would enter the

site & able to determine approximate locations for UGS;Closer look at the terrain –

navigate down to a dismount or first person perspective street level for a closer inspection (buildings, streets, configuration of a square…)

walk through the streets to determine best routes, identify specific buildings & identify any obstacles that may impede | interrupt movements of troops

walk around the suspected enemy safe houses and strongholds to determine entrance ways, windows & learn characteristics of each building …

able to identify a foothold, to learn the general layout of this location and to identify routes of ingress and egress to/from the building….

(Hypothesis) Mission Planning Session - MOUT

Page 45: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

dcd03f ver.(06-1999)

SANS CLASSIFICATION COTE DE SÉCURITÉ DE LA FORMULE

(plus haut niveau du titre, du résumé ou des mots-clefs)

FICHE DE CONTRÔLE DU DOCUMENT

1. PROVENANCE (le nom et l’adresse) R&D pour le défense Canada - Valcartier

2. COTE DE SÉCURITÉ (y compris les notices d’avertissement, s’il y a lieu) Sans classification

3. TITRE (Indiquer la cote de sécurité au moyen de l’abréviation (S, C, R ou U) mise entre parenthèses, immédiatement après le titre.) Virtual Immersive Facility - A Training Perspective

4. AUTEURS (Nom de famille, prénom et initiales. Indiquer les grades militaires, ex.: Bleau, Maj. Louis E.) BERNIER, F., Boivin, E., Mokhtari, M.,

5. DATE DE PUBLICATION DU DOCUMENT (mois et année) 2008

6a. NOMBRE DE PAGES 44

6b. NOMBRE DE REFERENCES 0

7. DESCRIPTION DU DOCUMENT (La catégorie du document, par exemple rapport, note technique ou mémorandum. Indiquer les dates lorsque le rapport couvre une période définie.)

Présentation

8. PARRAIN (le nom et l’adresse)

9a. NUMÉRO DU PROJET OU DE LA SUBVENTION (Spécifier si c’est un projet ou une subvention)

9b. NUMÉRO DE CONTRAT

10a. NUMÉRO DU DOCUMENT DE L’ORGANISME EXPÉDITEUR

10b. AUTRES NUMÉROS DU DOCUMENT

N/A

11. ACCÈS AU DOCUMENT (Toutes les restrictions concernant une diffusion plus ample du document, autres que celles inhérentes à la cote de sécurité.)

Diffusion illimitée Diffusion limitée aux entrepreneurs des pays suivants (spécifier) Diffusion limitée aux entrepreneurs canadiens (avec une justification) Diffusion limitée aux organismes gouvernementaux (avec une justification) Diffusion limitée aux ministères de la Défense Autres

12. ANNONCE DU DOCUMENT (Toutes les restrictions à l’annonce bibliographique de ce document. Cela correspond, en principe, aux données d’accès au document (11). Lorsqu’une diffusion supplémentaire (à d’autres organismes que ceux précisés à la case 11) est possible, on pourra élargir le cercle de diffusion de l’annonce.)

SANS CLASSIFICATION

COTE DE LA SÉCURITÉ DE LA FORMULE (plus haut niveau du titre, du résumé ou des mots-clefs)

Page 46: Virtual Immersion Facility (VIF) A Training Perspective · Facilitate engineering review. ... Lutron Graphic Eye Main Unit MUX Link 1 Zone 1 MUX Link 2 MUX Link 3 Zone 2 RS232 Zone

dcd03f ver.(06-1999)

SANS CLASSIFICATION COTE DE LA SÉCURITÉ DE LA FORMULE

(plus haut niveau du titre, du résumé ou des mots-clefs)

13. SOMMAIRE (Un résumé clair et concis du document. Les renseignements peuvent aussi figurer ailleurs dans le document. Il est souhaitable que le sommaire des documents classifiés soit non classifié. Il faut inscrire au commencement de chaque paragraphe du sommaire la cote de sécurité applicable aux renseignements qui s’y trouvent, à moins que le document lui-même soit non classifié. Se servir des lettres suivantes: (S), (C), (R) ou (U). Il n’est pas nécessaire de fournir ici des sommaires dans les deux langues officielles à moins que le document soit bilingue.)

This document presents the Virtual Immersive Facility in DRDC Valcartier, some of its technologies and how it can improve the training in the CF. Some exploratory projects are also presented.

14. MOTS-CLÉS, DESCRIPTEURS OU RENSEIGNEMENTS SPÉCIAUX (Expressions ou mots significatifs du point de vue technique, qui caractérisent un document et peuvent aider à le cataloguer. Il faut choisir des termes qui n’exigent pas de cote de sécurité. Des renseignements tels que le modèle de l’équipement, la marque de fabrique, le nom de code du projet militaire, la situation géographique, peuvent servir de mots-clés. Si possible, on doit choisir des mots-clés d’un thésaurus, par exemple le “Thesaurus of Engineering and Scientific Terms (TESTS)”. Nommer ce thésaurus. Si l’on ne peut pas trouver de termes non classifiés, il faut indiquer la classification de chaque terme comme on le fait avec le titre.)

Virtual Immersive Facility, Presence, Immersion,Training

SANS CLASSIFICATION

COTE DE SÉCURITÉ DE LA FORMULE

(plus haut niveau du titre, du résumé ou des mots-clefs)