Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
How to (and not to) fool your brain to perceive 3D from planar displays
Stefan Seipel
2009-11-13
The perceptual basis for stereoscopic displays
IPD
Ver
genc
eD
ista
nce
1θ
2θ
When converging at objects at somecertain distance, a point at different distances in the scene will appear at the retina with some lateral disparity.
Binocular disparity
independent of vergence.
Lateral disparity of more than 10° cause diplopia.
Experiment: Thumb and Window
Retinal Disparity: Viewing in the real visual world
21 θθ −=Disp
The perceptual basis for stereoscopic displays
Parallax: Viewing on plano-stereoscopic displays
Parallax is lateral displacement of homologous points on the planar display.
Zero Parallax
Converging at a point on a display surface causes homologous points on display to have zero parallax.
The perceptual basis for stereoscopic displays
Positive Parallax
Converging at a point behind the display surface causes homologous points on display to have positive parallax.This point is said to be in screen space.
Parallax: Viewing on plano-stereoscopic displays
Parallax is lateral displacement of homologous points on the planar display.
The perceptual basis for stereoscopic displays
Negative Parallax
Converging at a point in front of the display surface causes homologous points on display to have negative parallax.This point is said to be in viewer space.
Parallax: Viewing on plano-stereoscopic displays
Parallax is lateral displacement of homologous points on the planar display.
The perceptual basis for stereoscopic displays
Converging the eye´s axes upon a real point in 3D space implies verging the eyes towards that point.The neuro-muscular functions to control vergence are also evaluated to assess distance.
The relationship between convergence and accommodation
The point is kept a focus by adjusting the lens (accommodation).
Under normal natural viewing conditions, accommodation and convergence correspond. The correspondence of convergence and accommodation is habitual an can be voluntarily put out of function (crossing the eyes).
The perceptual basis for stereoscopic displays
Converging the eye's axes upon a virtual point at distance Dc supportsfusion of the parallax image and stereopsis.
Keep the visual structure on-screen at focus, requires accommodation at screen distance Ds.
Usually, accommodation is dominant andfor unaided viewing, we see one planar double-image in focus.
With some training (crossing the eyes) wecan converge at Dc and see on fused 3D Image out of focus.
Convergence and accommodation in plano-stereoscopic displays
DsDc
The perceptual basis for stereoscopic displays
Converging the eye's axes upon a virtual point at distance Dc supportsfusion of the parallax image and stereopsis.
Keep the visual structure on-screen at focus, requires accommodation at screen distance Ds.
Usually, accommodation is dominant andfor un-aided viewing, we see one planar double-image in focus.
With some training (crossing the eyes) wecan converge at Dc and see on fused 3D Image out of focus.
Stereo-graphic devices (e.g. stereo glasses) greatly support image fusion and stereo-opsisat the cost of suppressed accommodation.
DsDc
The perceptual basis for stereoscopic displays
Summary:
• Parallax of regions in an image on a stereoscopic display is yielding to retinal disparity, which is one cue for perceiving depth in a scene.
• Parallax and retinal disparity are equivalent.
• The brain can fuse binocular retinal images into one spatial image (stereopsis)
• Stereopsis is limited to lateral disparities up to some certain amount (e.g. 10° Kalawasky[1992]), otherwise diplopia.
• Equivalently, there are limits to display parallax.(1,5° of visual angel Lipton[1991])
• Viewing non-zero parallax images on plano-stereoscopic displays infringes the accommodation/convergence relationship learned in natural viewing.
Practical issues in stereo graphics
Lipton [1991] – Parallax Control
Don’t exceed parallax values of more than 1.5° visual angle (rather than using explicit parallax values on screen).
R
L
1.5°
Viewing distance D
On-screen parallax osp
10,477,865,242,621,961,31osp [cm]
4003002001007550D [cm]
Practical examples for on-screen parallax values:
Practical issues in stereo graphics
Virtual spatial depth due to observer distance and limited parallax.
Negative parallax situation:Allowable spatial depth d towards viewer space
IPD
Vie
win
g D
ista
nce
D
1
...
+
=
−=
osp
IPD
Dd
dD
d
IPD
osp
d
osp
Example for IPD = 6,5 cm
0,620,550,450,290,230,17d/D
246,83164,1789,2428,7217,408,38d
10,477,865,242,621,961,31osp [cm]
400,00300,00200,00100,0075,0050,00D [cm]
Practical issues in stereo graphics
Virtual spatial depth due to observer distance and limited parallax.
Positive parallax situation:Allowable spatial depth d into screen space
IPD
Vie
win
g D
ista
nce
D
==
+=
− 1
...
osp
IPD
Dd
dD
d
IPD
osp d
osp
Example for IPD = 6,5 cm
75,977,794,150,670,430,25d/D
18612,491714,79829,4567,4732,4712,61d
6,425,765,242,621,961,31osp [cm]
245,00220,00200,00100,0075,0050,00D [cm]
IPDV
erge
nce
Dis
tanc
eD
1θ
2θ
21 θθ −=Disp
Practical issues in stereo graphics
Virtual spatial depth due to observer distance and limited disparity.
According to Kalawsky image fusion collapses at 10° disparity,
d( )
⎟⎠⎞
⎜⎝⎛
⋅=∧
⎟⎠⎞
⎜⎝⎛ +⋅
−=
⎟⎠⎞
⎜⎝⎛
⋅=∧⎟
⎠⎞
⎜⎝⎛ +=
−
−
−
D
IPDDisp
IPDDd
D
IPDDisp
dD
IPD
2tan
2tan2
...
2tan
2tan
2
1
1
αα
αα
0,920,890,840,730,670,58d/D
366,03266,98168,7272,9950,2528,81d [cm]
0,010,010,020,030,040,06α [rad]
400,00300,00200,00100,0075,0050,00D [cm]
Practical issues in stereo graphics
Worth mentioning:
Values for maximum allowable disparity in the literature are expressed in different terms (angular disparity, on-screen parallax etc…)
These values apply sometimes only as a rule of thumb.(Lipton’s appear to work for many VR applications)
Values do not generally apply for all viewing conditions
Once you know the basics, composing stereoscopic images is an art, not a science. (Lenny Lipton,1991)
Practical issues in stereo graphics
Does my desktop 3D application also work in the VR theatre?
Many visualization applications are parameterized to work in stereo on desktop systems.
Example:
17” screen approx. 34 cm wide.Application creates maximum parallax of 2 cm, which is comfortable when viewed from approx. 70-80 cm distance.
You bring your application to the computer in the VR theatre, without altering viewing parameters.The stereo-picture is now blown up to 3 meters horizontal size i.e. 850%.
Maximum parallax on the projection screen is now 17,14 cm.
According to Lipton’s recommendation for parallax of max. 1.5°, the observer should be seated at least 6,6 meters from the screen!
Note: Viewing parameters for stereo-viewing do NOT scale linearly!!!
A general problem in computer graphics
Traditional camera viewing metaphor not suitable for VR and stereo 3D!
Traditionally the field of computer graphics has adopted the metaphor of a synthetic camera
Projection math is then
based on central perspective projections
adapted to parameters such as horizontal and vertical field of view
… even worse: focal length and aspect ratio
This is manifested in industry standard APIs (e.g. OpenGL/GLUT, ”gluPerspective”)
A general problem in computer graphics
What is wrong with a camera model?
1. The scale level in recording/synthesizing an image is not related to the spatialrelations between observer and display at presentation.
This is usually not a problem in monoscopic graphics, but in stereo it is!
(parallax and convergence issues, discussed before)
Example: Hyper-stereo and excessive parallax when photographing a frog from 30 cm distance with a IPD of 6 cm and display on a 4m x 3m screen.
What focal length is applicable for visualizing atoms and molecules?
A general problem in computer graphics
What is wrong with a camera model?
2. Binocular viewing of (projected) plano-stereoscopic displays requires off-axis projected images
typical setup for camera based
stereo
Non co-planar projection planes Same display plane for both images
Left eye Right eye
A general problem in computer graphics
What is wrong with a camera model?
3. A central model with central perspective projection assumes theviewer to be located in front of the center of the image
Non co-planar projection planes Same display plane for both images
”Correct” viewing position”Incorrect” viewing position
What can we do about it?
Presentation of 3D objects on planar 2D displays requires generallyan arbitrary relationship between the projection (=presentation) planeand the center of projection (=observer's eye).
This viewing metaphor is called ”Window-on-world”, short WoW.
A WoW implies generally an off-axis projection
The result of off-axis projection is an anamorphic image which ,viewed from the CoP, appears geometrically undistorted.
Anamorphic images are not only compelling in computer graphics…
http://users.skynet.be/J.Beever/
Works of the famous artist Julian Beever
This is the center of projection
Works of the famous artist Julian Beever
http://users.skynet.be/J.Beever/
Here is what you see from the center of projection
How can I accomplish good off-axis projections?
A metrically correct Window-on-world projection requires
a) the relation between window and observer position to be modeled at the same level of scale (same WCS)
b) requires a parameterization of an appropriate off-axis projection matrix
c) zooming of objects in the scene is accomplished through manipulation of object parameters rather than viewing parameters
A) and b) usually require that you implement the projection pipeline into your Vis-App yourself to be sure that the job gets done right.
How to parameterize an off-axis projection matrix?
nearfar
nearfarD
nearfar
nearfarC
bottomtop
bottomtopB
leftright
leftrightA
−⋅⋅
−=
−+
−=
−+
=
−+
=
2
glFrustum(left,right,bottom,top,near,far);
Using built-in OpenGL frustum function
⎥⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢⎢
⎣
⎡
−
−⋅
−⋅
=
0100
000
02
0
002
C
Bbottomtop
near
Aleftright
near
P
(observer at origin, facing along –z, near defines projection plane)
left
righttop
bottom
near
Parameterize glFrustum for arbitrary WoW
Using built-in OpenGL frustum function
window position
width
height
up-vector
side-vector
viewer position
A window on world arbitrarily identified by:
Window position: wp
Orientation vectors (roll/tilt): sv, up
Width and height: w, h
And an arbitrary observer position: vp
Parameterize glFrustum for arbitrary WoW
Using built-in OpenGL frustum function
wp
w
h
up
sv
vp
A window on world arbitrarily identified by:
Window position: wpOrientation vectors (roll/tilt): sv, upWidth and height: w, hAnd an arbitrary observer position: vp
This WindowOnWorld is part of a central perspective projection along
(cp-vp)
Which is frustrated at
left, right, top, and bottom
cp
left
top
bottom
right
Parameterize glFrustum for arbitrary WoW
Using built-in OpenGL frustum function
wp
w
h
up
sv
vp
Calculating glFrustum parameters from WoW parameters:
cp
left
top
bottom
right
upsvwn ×= 0,10.1 =∧= upsv
wpvpvv −=
vv
wn
D
wnvvD •=
wnDvpcp ⋅−=
Parameterize glFrustum for arbitrary WoW
Using built-in OpenGL frustum function
wp
w
h
up
sv
vp
Calculating glFrustum parameters from WoW parameters:
cp
left
top
bottom
right
upsvwn ×= 0,10.1 =∧= upsv
wpvpvv −=
vv
wn
D
wnvvD •=
wnDvpcp ⋅−=bl tr
cpupsvwptr
cpupsvwpbl
−⋅+⋅+=−⋅−⋅−=
5.05.0
5.05.0
Parameterize glFrustum for arbitrary WoW
Using built-in OpenGL frustum function
wp
w
h
up
sv
Calculating glFrustum parameters from WoW parameters:
cp
left
top
bottom
right
upsvwn ×= 0,10.1 =∧= upsv
wpvpvv −= wn
wnvvD •=
wnDvpcp ⋅−=bl tr
uptrtop
svtrright
uplbbottom
svlbleft
•=•=•=
•=
Parameterize glFrustum for arbitrary WoW
Adjusting frustum parameters for clipping plane
viewer position
left
top
bottom’
right
cp
D
bottomright’
top’
left’
cd
glFrustum assumes clipping plane To co-incide with plane of projection
D
cdDtoppto
D
cdDrighttrigh
D
cdDbottommbotto
D
cdDlefttlef
−⋅=′
−⋅=′
−⋅=′
−⋅=′
cdDnearHence −=:
Parameterize glFrustum for arbitrary WoW
Pre-multiplying modelview matrix
viewer position
left
top
bottom’
right
cp
D
bottomright’
top’
left’
cd
glFrustum assumes further:vp is at origin (cp-vp) pointing toward -z
1
1000
−
⎥⎦
⎤⎢⎣
⎡=
vpwnupsvPM
Translating vp to O and frame alignmentBe pre-loading model-view matrix PM
glMatrixMode(GL_PROJECTION);glLoadIdentity(),glFrustum(left’,right’,bottom’,top’,near,far);
glMatrixMode(GL_MODELVIEW);glLoadIdentity();glMultMatrixd(PM);
Putting it together
In-proper interaction between 3Dcontent and screen surround destroys illusion:
Virtual 3D object intersects withscreen borders.
Contradictious 3D cues
Stereo-disparity says:“object is in front of screen”
Object occlusion by surround says:“object is behind the screen”
Sabotaging a good 3D illusion
Giving to much freedom to the user when interacting…
Sabotaging a good 3D illusion
…can cause hyper-parallax and diplopia
Example of dynamic perspective conditions ?
…technically speaking we get a whole lot of trouble
Requirements to visualization of 3D features
Eye’s positions needed(tracking)
Multiple individualized views
Dynamic rendering even for static content
Stereo cues (optionally)
This is what we can achieve…
Dynamic viewing conditions Individual content
Pettersson, Spak and Seipel, 2005 : “Collaborative 3D Visualizations of Geo-Spatial Informationfor Command and Control”, proceedings of Sigrad 2004
b) Collaborating in complex tasks
Experimental procedure:20 persons in 10 Dyads5 trial stimuli + 2 x 15 stimuli Blocks of 15 stimuli is randomized
Views of the actual experimental task – a 3D and logical/combinatorial task
Integral view (first person view) Partitioned view for user A
c) Understanding time and geography
Works submitted to Information Visualization Journal and Journal of Applied Perception
Series of experiments investigating the space-time cube:
The space-time cube uses elevation over ground to depict time
2D trajectory of moving targets 3D trajectory of moving targets
Summary: Cues for 3D image generation?
Correct stereo is only one of many spatial cuesCorrect parallax (software)Projector adjustment (hardware)Scaling conditions (sw + hw)Interference with surround
Utilize shadows, shading
Exploit motion parallax Object rotation creates strong optic flow
Keep noise or small spatial features/structures in e.g. 3D imagesobjects with lack of spatial freq. ? -> use textures
Try to use dynamic perspective conditions
3D Display Categories
True Volumetric Displays
Stereographic Displays(2 projected planar views)
Single Display Plane Multiplexed
Dual Display Immediate Mode
Temporal MUX Spatial MUX Chromatic MUX+
Spectral MUX
Polarization MUX
Time Multiplexed Stereo Image Pair
V-Sync at 60 Hz
Addidional V-Sync at 120 Hz(enforced with sync. doubler)
open
clos
e
clos
e
open
Active Shutter Glasses(LCD-Shutters)
Temporal Multiplexing and “Ghosting”
0%
100%
0%
100%
Shutter Opacity Left Eye
Shutter Opacity Right Eye
Pixel Intensity (Phosphor)
Open RightClose Left
Open RightClose Left
Open RightClose Left
Close RightOpen Left
Close RightOpen Left
Two Pixels are Drawn On-Screen:Purple Pixel for the Right EyeGreen Pixel for the Left Eye120Hz screen refresh rate
t
t
t8.3 ms
Left eye sees purple pixel due to after-glowingRight eye sees green pixel due to shutter response
Color Encoded Stereo Image Pair
Chromatic Multiplexing - Anaglyph Technique
Images curtsey: http://axon.physik.uni-bremen.de/research/stereo/color_anaglyph/index.html#ana
Encoding:
A stereo image pair is combined into one so called anaglyph.
The left eyes view is encoded with the red colour component.
The right eye view is encoded with the complementary colour i.e. greenAnd blue colour components.
Decoding:
The left eye uses a red colour filterthat passes through dominant red components.
The right eye uses a cyan colour filterthat passes through dominant blueAnd green components.
Chromatic Multiplexing - Anaglyph Technique
Conventional filter materialsare used to separate the colorspectra of a stereogram.
Image splitting by chromaticseparation.
spectrum of the red filter
spectrum of the cyan filter
Spatial Multiplexing
VR-3100
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
L RR L
LRLRLRLR
L R L R L R L R
The stereo image pair is presented on alternatingpixels, pixel columns, or scan-lines.
Optical arrangements are used to block-outthe the wrong image for the corresponding eye.
Effective resolution is reduced.
Spectral Filtering – Infitec Technology
More info at: http://www.infitec.net/infitec.pdf
Narrow band spectral filters allow differentsub-bands of the tri-stimulus spectra.
Advantage: Filter based optical multiplexingwhich provides ”natural color” stereo imageseparation.
Left eyeSpectral filter
Right eyeSpectral filter
Related literature and references
Additional reading for this lecture:http://www.cs.unc.edu/Research/stc/FAQs/Stereo/stereo-handbook.pdf
Research community in stereoscopic/3D displayshttp://www.stereoscopic.org/2009/program.html