Hie
rarc
hica
l Mod
elin
g
•A
nod
e re
pres
ents
:–
rota
tion(
s)
–ge
omet
ric
prim
itive
(s)
–T
rans
form
atio
ns
The
roo
t can
be
anyw
here
(h
ip)
Con
trol
for
eac
h jo
int a
ngle
, pl
us g
loba
l pos
itio
n an
d or
ient
atio
n
hip
tors
o
head
l. ar
m2
l. ar
m1
r. a
rm1
r. a
rm2l.
leg1
l. le
g2
r. le
g1
r. le
g 2
shou
lder
neck
Rel
evan
t Ope
nGL
Rou
tine
s
•gl
Push
Mat
rix(
),gl
PopM
atri
x()
–pu
sh a
nd p
op th
e st
ack.
pus
h le
aves
a c
opy
of th
e cu
rren
t m
atri
x on
top
of th
e st
ack
•gl
Loa
dIde
ntity
(),g
lLoa
dMat
rixd
(M)
–lo
ad th
e Id
entit
y m
atri
x, o
r an
arb
itra
ry m
atri
x, o
nto
top
of th
e st
ack
•gl
Mul
tMat
rixd
(M)
–m
ulti
ply
the
mat
rix
C o
n to
p of
sta
ck b
y M
. C
= C
M
•gl
Ort
ho(x
0,y0
,x1,
y1,z
0,z1
)–
set u
p pa
ralle
l pro
ject
ion
mat
rix
•gl
Rot
atef
(the
ta,x
,y,z
),gl
Rot
ated
(…)
–ax
is/a
ngle
rot
ate.
“f”
and
“d”
take
flo
ats
and
doub
les,
res
pect
ivel
y
•gl
Tra
nsla
tef(
x,y,
z),g
lSca
lef(
x,y,
z)–
tran
slat
e, s
cale
. (al
so e
xist
in “
d” v
ersi
ons.
)
Bq
p
Ar
Tra
ns -
r
Rot
v
Tra
ns q
A
Tra
ns -
p
Rot
u
Tra
ns T
B
Ope
nGL
Exa
mpl
e
glL
oadI
dent
ity()
;gl
Ort
ho(…
);gl
Push
Mat
rix(
);gl
Tra
nsla
tef(
Tx,
Ty,
0);
glR
otat
ef(u
,0,0
,1);
glT
rans
late
f(-p
x,-p
y,0)
;gl
Push
Mat
rix(
);gl
Tra
nsla
tef(
qx,q
y,0)
;gl
Rot
atef
(v,0
,0,1
);gl
Tra
nsla
tef(
-rx,
-ry,
0);
Dra
w(A
);gl
PopM
atri
x();
Dra
w(B
);gl
PopM
atri
x();
Hie
rarc
hy m
etho
ds
•O
bjec
t Ori
ente
d
•Pu
sh m
atri
x st
ack
–Im
plie
s de
pth-
firs
t tra
vers
al
•D
o ty
pe-s
peci
fic
tran
sfor
m
•R
ecur
seon
its
child
ren,
and
pop
s.
Inte
ract
ive
App
lica
tion
s
•H
ow d
o w
e ad
d in
tera
ctiv
e co
ntro
l?•
Man
y di
ffer
ent p
arad
igm
s–
Exa
min
er =
> O
bjec
t in
hand
–Fl
y-th
ru =
> I
n a
virt
ual v
ehic
le p
od–
Wal
k-th
ru =
> C
onst
rain
ed to
sta
y on
gro
und.
–M
ove-
to /
re-c
ente
r =
> P
ick
a lo
catio
n to
fly
to.
•C
ollis
ion
dete
ctio
n?–
Can
we
pass
thru
obj
ects
like
gho
sts?
Inte
ract
ive
App
lica
tion
s
•W
hat d
o w
e us
e to
con
trol
the
mot
ion?
–M
ouse
•O
ne-b
utto
n, tw
o-bu
tton,
thre
e-bu
tton
•W
hat b
utto
n do
es w
hat?
•O
nly
whe
n m
ouse
is c
licke
d do
wn,
rel
ease
d up
, or
cont
inuo
usly
as
the
mou
se m
oves
?
–K
eybo
ard
•A
rrow
key
s?
Inpu
t Dev
ices
•In
tera
ctiv
e us
er c
ontr
ol d
evic
es–
Mou
se
–3D
poi
nter
-Po
lhem
us,M
icro
scri
be, …
–Sp
aceb
all
–H
and-
held
wan
d
–D
ata
Glo
ve
–G
estu
re
–C
usto
m
A V
irtu
al T
rack
ball
•A
rat
her
stan
dard
and
eas
y-to
-use
inte
rfac
e.
•E
xam
iner
type
of
inte
ract
ion.
•C
onsi
der
a he
mi-
sphe
re o
ver
the
imag
e-pl
ane.
•E
ach
poin
t in
the
imag
e is
pro
ject
edon
to th
e he
mi-
sphe
re.
A V
irtu
al T
rack
ball
•Po
ints
insi
de th
e pr
ojec
tion
of th
e he
mi-
sphe
re a
re
map
ped
up to
the
surf
ace.
–D
eter
min
e di
stan
ce f
rom
poi
nt (
mou
se p
ositi
on)
to th
e im
age-
plan
e ce
nter
.
–Sc
ale
such
that
poi
nts
on
the
silh
ouet
te o
f th
e sp
here
ha
ve u
nit l
engt
h.
–A
dd th
e z-
coor
dina
te to
norm
aliz
e th
e ve
ctor
.
A V
irtu
al T
rack
ball
•D
o th
is f
or a
ll po
ints
.
•K
eep
trac
k of
the
last
trac
kbal
l (m
ouse
) lo
catio
n an
d th
e cu
rren
t loc
atio
n.
•T
his
is th
e di
rect
ion
we
wan
t the
sce
ne to
mov
e in
.
•T
ake
the
dire
ctio
n pe
rpen
dicu
lar
to th
is a
nd u
se
it as
the
axis
of
rota
tion.
•U
se th
e di
stan
ce b
etw
een
the
two
poin
ts to
det
erm
ine
the
rota
tion
angl
e (o
r am
ount
).
A V
irtu
al T
rack
ball
•R
otat
ion
axis
:
Whe
re, v
1an
d v 2
are
the
mou
se p
oint
s m
appe
dto
the
sphe
re.
v 1
v 2
12
uv
v=
⊗�
��
A V
irtu
al T
rack
ball
•U
se g
lRot
atef
( an
gle,
u x, u
y, u z
)•
Slig
ht p
robl
em: W
e w
ant t
he r
otat
ion
to b
e th
e la
st o
pera
tion
perf
orm
ed.
•E
asily
fix
ed:
–R
ead
out t
he c
urre
nt G
L_M
OD
EL
VIE
Wm
atri
x–
Loa
d th
e id
entit
y m
atri
x–
Rot
ate
–M
ultip
ly b
y th
e sa
ved
GL
_MO
DE
LV
IEW
mat
rix
Vir
tual
Rea
lity
Rog
er C
raw
fis
Vir
tual
rea
lity
tech
nolo
gy
•m
any
defi
nitio
ns o
f vi
rtua
l rea
lity
(VR
), f
or e
xam
ple:
•"t
he c
reat
ion
of th
e ef
fect
of
imm
ersi
on in
a c
ompu
ter-
gene
rate
d th
ree-
dim
ensi
onal
env
iron
men
t in
whi
ch o
bjec
ts
have
spa
tial p
rese
nce"
[B
ryso
n &
Fei
ner,
199
4]
•"t
hing
s as
opp
osed
to p
ictu
res
of th
ings
”
•in
tera
ctio
n, n
ot c
onte
nt
•m
any
vari
atio
ns, d
eskt
op V
R, f
ish
tank
VR
, aug
men
ted
real
ity
Rel
ated
term
inol
ogy
•vi
rtua
l env
iron
men
t
•vi
rtua
l wor
ld
•ar
tific
ial r
ealit
y
•au
gmen
ted
real
ity
•te
lepr
esen
ce
•T
eleo
pera
tion
•C
olla
bora
tive
Spac
es
Per
form
ance
req
uire
men
ts
•w
ide-
fiel
d st
ereo
scop
ic d
ispl
ay f
ill's
the
user
's f
ield
of
view
•he
ad-t
rack
ing
supp
orts
the
illus
ion
that
the
user
is lo
okin
g ar
ound
in a
n en
viro
nmen
t
•3D
com
pute
r gr
aphi
cs f
ills
the
envi
ronm
ent w
ith o
bjec
ts
•3D
inte
ract
ion
give
s us
ers
the
feel
ing
that
they
are
in
tera
ctin
g w
ith r
eal o
bjec
ts
•ov
eral
l fra
me
rate
mus
t be
> 1
0 fr
ames
/sec
•en
d-to
-end
del
ays
mus
t be
< 0
.1 s
ec f
or in
tera
ctiv
e co
ntro
l
The
pro
blem
with
VR
is…
•th
at it
is a
ppar
ently
sim
ple
•N
OT
the
unus
ual h
ardw
are
•m
any
com
pone
nts
mus
t wor
k to
geth
er in
re
al-t
ime
•m
any
crite
ria
mus
t be
met
•un
clea
r ho
w to
use
the
inte
rfac
e
•hu
man
fac
tors
issu
es n
ot w
ell u
nder
stoo
d
The
evo
lutio
n of
VR
•19
60 M
orto
n H
eilig
file
s pa
tent
to th
e U
S Pa
tent
Off
ice
"Ste
reos
copi
c T
V A
ppar
atus
for
Ind
ivid
ual U
se"
My
inve
ntio
n ge
nera
lly s
peak
ing
com
pris
es th
e fo
llow
ing
elem
ents
: a
hollo
w c
asin
g, a
pai
r of
opt
ical
uni
ts, a
pai
r of
tele
visi
on tu
be u
nits
, a
pair
of
earp
hone
s an
d a
pair
of
air
disc
harg
e no
zzle
s, a
ll co
actin
gto
ca
use
the
user
to c
omfo
rtab
ly s
ee th
e im
ages
, hea
r th
e so
und
effe
cts
and
to b
e se
nsiti
ve to
the
air
disc
harg
e of
the
said
noz
zles
.
•19
60-7
0 Su
ther
land
's h
ead-
mou
nted
dis
play
•19
84 N
ASA
Am
es V
IVE
D p
roje
ct•
1986
-90
NA
SA A
mes
VIE
W la
b an
d V
PI•
1990
-onw
ards
VR
com
mun
ity f
ully
for
med
and
flo
uris
hing
…
Deg
rees
of i
mm
ersi
on
Hig
hfo
rce
trac
king
head
cou
pled
6D tr
acki
ng +
glo
ves
wid
e fi
eld
of v
iew
6D tr
acki
ng +
but
tons
head
trac
king
6D in
put d
evic
eSt
ereo
2D M
ouse
high
res
olut
ion
low
keyb
oard
colo
ur
IMM
ER
SIO
NIN
TE
RA
CT
ION
DIS
PL
AY
Vir
tual
Env
iron
men
ts
•Im
mer
sive
•In
tera
ctiv
e
•U
ser
Cen
tere
d��
����
�����
���
��
����
����
����
���
���
Typ
ical
con
figu
ratio
n
trac
ker
elec
tron
ics
glov
eel
ectr
onic
sgl
ove
mai
n co
mpu
ter
-co
mpu
tatio
n
spee
ch-
synt
hesi
s-
reco
gnit
ion
grap
hics
soun
d
trac
ker
sour
ce
mic
roph
one
head
phon
es
HM
D
��������
�����
Dis
play
s
•pr
imar
y te
chno
logy
und
erly
ing
imm
ersi
on
•m
any
aspe
cts:
col
our,
res
olut
ion,
fie
ld o
f vi
ew…
•di
spla
y pa
radi
gms:
–st
ereo
via
two
disp
lays
–st
ereo
via
one
dis
play
imag
es s
ynch
roni
sed
(eye
wea
r)
–C
AV
E: i
mm
ersi
on v
ia s
urro
undi
ng la
rge
scre
ens
–he
ad tr
acki
ng (
fish
tank
VR
)
–he
ad tr
acki
ng h
ead-
mou
nted
Vir
tual
Env
iron
men
ts
•D
ispl
ay T
echn
olog
ies
–H
MD
’s -
Hea
d M
ount
ed D
ispl
ays
–L
arge
thea
ter
-Im
ax, O
mni
max
–St
ereo
dis
play
s
–H
UD
’s -
Hea
d’s
Up
Dis
play
s•
win
dshi
elds
•go
ggle
s
–C
AV
E -
Sur
roun
d vi
deo
proj
ectio
ns
Tra
ckin
g pa
radi
gms
•us
ually
a s
enso
r de
term
ines
pos
ition
and
or
ient
atio
n re
lativ
e to
sou
rce
(cal
ibra
tion
rend
ers
posi
tion
of s
ourc
e ir
rele
vant
)•
sens
or d
etec
ts a
sig
nal f
rom
the
sour
ce in
suc
h a
way
that
the
posi
tion
and
orie
ntat
ion
can
be
dete
rmin
ed•
eith
er th
e so
urce
or
the
sens
or c
an b
e fi
xed
•nu
mer
ous
tech
nolo
gies
: ele
ctro
mag
netic
, ul
tras
onic
, mec
hani
cal,
vide
o, in
ertia
l
Wha
t to
trac
k?
•he
ad p
ositi
on a
nd o
rien
tatio
n•
any
sign
ific
ant b
ody
part
•an
y ar
ticul
atio
ns
Oth
er t
rack
ing
tech
nolo
gies
•pa
ssiv
e st
ereo
vis
ion
syst
ems
•m
arke
r sy
stem
s (u
sed
in m
otio
n ca
ptur
e)•
stru
ctur
ed li
ght m
etho
ds (
light
str
ipe)
•in
ertia
l tra
ckin
g (u
sing
acc
eler
omet
ers)
•ey
e tr
acki
ng (
com
mon
ly o
ptic
al -
corn
eal r
efle
ctio
n)
Vir
tual
Env
iron
men
ts
•In
tera
ctiv
e us
er n
avig
atio
n de
vice
s–
Hea
d tr
acke
r
–T
read
mill
–B
icyc
le
–W
heel
chai
r
–B
oom
–V
ideo
det
ectio
n
•A
nyon
e se
en th
e ne
wga
me
at G
ameW
orks
?
Vir
tual
Env
iron
men
ts
•In
tera
ctiv
e us
er c
ontr
ol d
evic
es–
Mou
se
–3D
poi
nter
-Po
lhem
us,M
icro
scri
be, …
–Sp
aceb
all
–H
and-
held
wan
d
–D
ata
Glo
ve
–G
estu
re
–C
usto
m
Fak
espa
ceB
OO
M 3
CV
ideo
Out
put
Ful
l Col
or S
tere
o -
orM
onos
copi
c.
Res
olu
tio
nU
p to
128
0 x
1024
pix
els
per
eye.
Op
tics
U
ser
inte
rcan
geab
lem
odul
es o
ffer
from
40
to 1
10 d
egre
es h
oriz
onta
l FO
VT
rack
ing
Opt
o-m
echa
nica
lA
ccu
racy
0.01
5" a
t 30"
Lat
ency
20
0ns
Sam
plin
g F
req
uen
cy>
70H
zR
ang
e6'
dia
met
er h
oriz
onta
l circ
le (
cent
er 1
fo
ot u
nava
ilabl
e) 2
.5' v
ertic
al.
Hum
an fa
ctor
s of
vir
tual
rea
lity
•L
imits
on
mot
ion
freq
uenc
ies:
–
head
(5
Hz)
–ha
nd (
10 H
z)
–fu
ll bo
dy (
5 H
z)
–ey
e (1
00 H
z)
Hum
an fa
ctor
s of
vir
tual
rea
lity
•L
imits
on
Vis
ion
(opt
ical
res
olut
ion)
:–
angu
lar
size
of
the
smal
lest
obj
ect t
hat c
an b
e re
solv
ed:
•es
sent
ially
the
angu
lar
size
of
a co
lour
pix
el
•m
easu
re a
s a
linea
r si
ze in
min
utes
of
arc
•fu
ll m
oon
is 3
0 m
inut
es o
f ar
c ac
ross
its
diam
eter
•hu
man
vis
ual s
yste
m c
an r
esol
ve 0
.5 m
inut
es o
f ar
c in
the
cent
ral v
isua
l fie
ld
•2-
3 m
inut
es o
f ar
c in
the
peri
pher
al v
isua
l fie
ld
Cue
s to
sup
port
the
sens
e of
im
mer
sion
•im
mer
sion
: wan
t to
be in
an
envi
ronm
ent t
hat c
onta
ins
“thi
ngs”
and
not l
ooki
ng a
t pic
ture
s•
spat
ial p
rese
nce
of v
irtu
al o
bjec
ts d
ue to
:–
spat
ial c
onst
ancy
•10
fra
mes
/sec
min
imum
req
uire
men
t•
if y
our
head
mov
es a
nd th
e sc
ene
does
n’t i
t isn
’t V
R•
obje
ct b
ehav
iour
(e.
g. a
pplic
atio
n of
con
sist
ent p
hysi
cal l
aws)
–de
pth
perc
epti
on•
ster
eo•
head
mot
ion
para
llax
•m
any
othe
r de
pth
cues
•w
ide
fiel
d of
vie
w–
envi
ronm
ent s
eem
s to
fill
fie
ld o
f vi
ew (
60ºm
inim
um th
resh
old)
Mot
ion
para
llax
Asp
ects
of h
ead-
mot
ion
para
llax
•du
e to
cha
nge
in v
isua
l sce
ne a
s th
e he
ad m
oves
•pe
rfor
med
in a
VR
sys
tem
by
trac
king
the
user
’s
head
and
ren
deri
ng th
e vi
rtua
l sce
ne f
rom
a
mov
ing
poin
t of
view
•he
ad-m
otio
n pa
ralla
x is
a m
onoc
ular
dep
th c
ue:
–be
yond
1m
mon
ocul
ar c
ues
dom
inat
e
–w
ithin
1m
bin
ocul
ar d
ispa
rity
and
mot
ion
para
llax
is
cruc
ial
–ne
ed 1
2 fr
ames
/sec
for
mot
ion
para
llax
Ster
eops
is
•fu
sion
of
imag
es f
rom
two
eyes
•pr
ojec
ted
rays
of
sam
e po
ints
in w
orld
dif
fere
nt f
or e
ach
eye
•po
ints
in th
e w
orld
are
vis
ible
to o
ne e
ye a
nd n
ot a
noth
er
Asp
ects
of s
tere
opsi
s
•Pe
ople
hav
e di
ffer
ent f
usio
n ca
pabi
litie
s (i
t is
bel
ieve
d th
at a
s m
any
as 2
0% h
ave
little
ca
pabi
lity)
•E
ffec
tive
out t
o 3-
6m b
ut c
ritic
al <
1m
•Fa
r-fi
eld,
not
that
cri
tical
.
Vir
tual
Env
iron
men
ts
•D
raw
at 1
20H
z
•T
rack
use
r po
sitio
n/or
ient
atio
n at
120
Hz
•Pr
ovid
e H
aptic
fee
dbac
k at
> 2
00H
z
•U
ser
trac
king
> 1
0Hz
Aug
men
ted
Rea
lity
•M
erge
d re
al im
ager
y an
d co
mpu
ter
gene
rate
d im
ager
y.–
Vid
eo c
aptu
re in
to v
isua
lizat
ion
syst
em
–Se
e-th
ru g
lass
es
�������������������������
�����������
Aug
men
ted
Rea
lity
������������������������������������
Aug
men
ted
Rea
lity
������������������������������������
Aug
men
ted
Rea
lity
•A
lso
usef
ul f
or n
on-m
edic
al–
Mec
hani
cs d
raw
ing
supe
r-im
pose
d ov
er th
e ac
tual
mac
hine
ry.
–G
uide
d to
urs.
Aug
men
ted
Rea
lity
•C
ompl
ex I
nstr
uctio
nal M
anua
ls
Hap
tics
•Fo
rce
feed
back
is n
eede
d at
ver
y fa
st r
ates
.
•G
love
s–
forc
e re
sist
ant
–ne
rve
stim
ulat
ed
Rap
id P
roto
typi
ng
•B
uild
rea
l mod
els
of th
e vi
sual
izat
ions
•St
ereo
Lith
ogra
phy
–L
aser
etc
hing
•L
amin
ated
Obj
ect M
anuf
actu
ring
–L
amin
ated
pap
er la
yer,
then
cut
with
lase
r
Lam
inat
ed O
bjec
t Man
ufac
turi
ng
Lam
inat
ed O
bjec
t Man
ufac
turi
ng
•M
olec
ular
Doc
king
NA
SA’s
Vir
tual
Win
d T
unne
l
The
CA
VE
Arc
hite
ctur
e
•Fo
ur p
roje
ctio
n sc
reen
s
•Fo
ur g
raph
ics
rend
erin
g en
gine
s
•St
ereo
gla
sses
•H
ead-
trac
king
of
one
user
•H
and
held
wan
d fo
r in
put
The
CA
VE
Arc
hite
ctur
e
���
�������
����
The
CA
VE
The
CA
VE
The
CA
VE
The
CA
VE
Arc
hite
ctur
e
•Se
vera
l peo
ple
can
view
at o
nce
•T
he p
roje
ctio
ns a
re o
nly
corr
ect f
or o
ne
pers
on.
•L
aser
’s s
ynch
the
ster
eo d
ispl
ays
with
L
iqui
d C
ryst
al s
hutte
r gl
asse
s on
eac
h vi
ewer
.
The
CA
VE
Arc
hite
ctur
e
•B
enef
its–
Eye
mov
emen
t pro
blem
s ar
e av
oide
d!!!
–U
ser’
s or
ient
atio
n do
es n
ot m
atte
r.
–C
an s
ee a
nd e
xam
ine
real
peo
ple
and
obje
cts
with
in th
e ro
om
The
CA
VE
Arc
hite
ctur
e
•Pr
oble
ms
–T
he li
ght i
nten
sity
on
each
pro
ject
or v
arie
s
–Pr
ecis
e al
ignm
ent o
f th
e pr
ojec
tors
is n
eces
sary
fo
r a
smoo
th s
eam
.
–V
iew
ing
does
not
cha
nge
for
the
othe
r vi
ewer
s.
–E
xpen
sive
.
Sing
le P
roje
ctor
Sys
tem
s
•Im
mer
saD
esk
•R
espo
nsiv
eW
orkb
ench
Res
pons
ive
Wor
kben
ch
Mak
ing
VR
Wor
k
•T
o en
sure
late
ncy,
man
y of
the
visu
aliz
atio
n te
chni
ques
nee
d to
be
stre
amlin
ed o
r pr
e-co
mpu
ted.
•E
xam
ples
, pre
-com
pute
d is
o-co
ntou
rs,
prec
ompu
ted
stre
am li
nes
and
part
icle
tr
aces
.
Recommended
