Lecture 3 of 42: Realism in Computer Graphics · 1/28/2008 · 2 Math Review for CIS 636 • Overview: First Month (Weeks 2-5 of Course) – Review of mathematical foundations of

1

• These notes have been created and revised each year by many generations of CS123 TAs and by John Hughes and Andy van Dam

• Updated 2001-2005 by John Alex, other Brown TAs

• See also Chapter 14 in the book

Lecture 3 of 42: Realism in Computer Graphics

CIS 736 Computer Graphics

Realistic Rendering and Animation 1 of 2

Monday, 28 January 2008Reading: Shading

Adapted with Permission

W. H. Hsu

http://www.kddresearch.org

CIS 636 / 736 Computer Graphics Mon 23 Jan 2008 Lecture 3 of 42

Online Recorded Lectures for CIS 636Introduction to Computer Graphics

• Project Topics for CIS 636• Computer Graphics Basics (8)

– 1. Mathematical Foundations – Week 2– 2. Rasterizing and 2-D Clipping – Week 3– 3. OpenGL Primer 1 of 3 – Week 3– 4. Detailed Introduction to 3-D Viewing – Week 4– 5. OpenGL Primer 2 of 3 – Week 5– 6. Polygon Rendering – Week 6– 7. OpenGL Primer 3 of 3 – Week 8– 8. Visible Surface Determination – Week 9

• Recommended Background Reading for CIS 636• Shared Lectures with CIS 736 (Computer Graphics)

– Regular in-class lectures (35) and labs (7)– Guidelines for paper reviews – Week 7– Preparing term project presentations, demos for graphics – Week 11

2

Math Review for CIS 636

• Overview: First Month (Weeks 2-5 of Course)– Review of mathematical foundations of CG: analytic geometry, linear algebra– Line and polygon rendering– Matrix transformations– Graphical interfaces

• Line and Polygon Rendering (Week 3)– Basic line drawing and 2-D clipping– Bresenham’s algorithm– Follow-up: 3-D clipping, z-buffering (painter’s algorithm)

• Matrix Transformations (Week 4)– Application of linear transformations to rendering– Basic operations: translation, rotation, scaling, shearing– Follow-up: review of standard graphics libraries (e.g., OpenGL)

• Graphical Interfaces– Brief overview– Survey of windowing environments (MFC, Java AWT)

Textbook and Recommended Books

Required Textbook

Eberly, D. H. (2006). 3D Game Engine Design: A Practical Approach to Real-Time Computer Graphics, second edition. San Francisco, CA: Morgan Kauffman.

Recommended References

Angel, E. O. (2007). OpenGL: A Primer, third edition. Reading, MA: Addison-Wesley. [2nd edition on reserve]

Shreiner, D., Woo, M., Neider, J., & Davis, T. (2007). OpenGL® Programming Guide: The Official Guide to Learning OpenGL®, Version 2.1, sixth edition.[“The Red Book”: use 5th ed. or later]2nd edition (OK to use) 3rd edition

1st edition (outdated) 2nd edition

3

Syllabus [1]:First Half of Course

Syllabus [2]:Second Half of Course

4

Computer-Generated Animation (CGA)

Wall-E© 2008 Disney/Pixar

Kung-Fu Panda© 2008 DreamWorksAnimation SKG

Meet the Robinsons© 2007 Disney/Pixar

Shrek (2001)Shrek 2 (2004)

Shrek the Third (2007)© DreamWorks Animation SKG

Toy Story (1995)Toy Story 2 (1999)Toy Story 3 (2010)

© Disney/PixarHappy Feet© 2006Warner Brothers

Ratattouille© 2007 Disney/Pixar

Luxo Jr.© 1986 Pixar Animation Studios

Andy van Dam October 21, 2003 Realism 8/42

• Content– movie special-effects pack as much astonishment as possible into their budget:

use every trick in the book– conversely, CAD model rendering typically elides detail for clarity, and fancy

effects only interfere with communication– Scientific visualizations show artifacts and holes in the data, don’t smooth

them out. Also, don’t introduce artifacts due to geometric or rendering approximations (e.g., contouring)

• Resources– you settle for what you can get:– Intel 286 (1989): wireframe bounding boxes– Microsoft Xbox (Nov. 2001): complete computer with

graphics more powerful than a GeForce 3 forless than $200!

– nVidia GeForce FX5900 (2003) texture-mapped, environment-mapped, bump-mapped, shadow-mapped, high- polygon, articulated, physically-simulated,stencil-shadowed goodnessfor $450 fully loaded

Trade-offs (3/4)

5

Computing to a time budget (“time-critical” algos)• A vast array of techniques have been developed for generating “realistic”

geometry, behavior, rendering…• The “best” can often be traded for the “good” at a much lower computational price• We call bargain-basement deals “hacks”• Some techniques use progressive refinement (or its inverse, graceful degradation):

the more time we spend, the better output we get. Excellent for situations when we want the best quality output we can get for a fixed period of time, but we can’t overshoot our time limit (e.g., VR surgery!). Maintaining constant update rates is a form of guaranteed “Quality of Service” (a networking term).

– web image downloads– progressive refinement for extremely large meshes


Trade-offs (4/4)


• Texture-Maps: map an image onto surface geometry to create the appearance of fine surface detail. A high level of realism may require many layers of textures.

• Environment-Maps: multiple images (textures) which record the global reflection and lighting on a object. These images are resampled during rendering to extract view- specific information which is then applied as a texture to the object.

• Bump-Maps: fake surface normals by applying a height field (intensities in the map indicate height above surface). From height field calculate gradient across surface and use this to perturb the surface normal.

• Shadow-Maps: generate shadow texture by taking silhouettes of objects as seen from the light source. Project texture onto scene from light source. Note: must be recalculated for moving lights.

Digression - Definitions

6


• The Hacked– Texture mapping: excellent way to fake fine surface detail—more often used

to fake geometry than to add pretty colors– more complicated texture mapping strategies such as polynomial texture maps

use image-based rendering techniques for added realism

Techniques—Geometry (1/4)

• The Good– Polygonization: very finely

tessellated meshings of curved surfaces

– linear approximation– massively hardware-accelerated!


The Best• Splines

– no polygons at all! Continuous mathematical surface representations (polynomials)

– 2D and 3D curved surfaces: Non-Uniform Rational B-Splines (NURBS)

– control points– high order polynomials are hard to

work with

• Implicit Surfaces (blobbies)– F(x,y,z) = 0– add, subtract, blend– hard to render, manip


7


The Best• Subdivision Surfaces

– subdivide triangles into more triangles, moving to a continuous limit surface

– elegantly avoid gapping and tearing between features

– support creases– allow multi-resolution

deformations (editing of lower resolution representation of surface)



• The Gracefully Degraded– Level-of-Detail(LOD): as object gets farther away from viewer, replace it

with a lower-polygon version or lower quality texture map. Discontinuous jumps in model detail

– Mesh decimation: save polygons


Left: 30,392 trianglesRight: 3,774 triangles

8

Good Hacks• easily implemented in hardware: fast!• use polygons • only calculate lighting at polygon vertices, from point lights• for non-specular (i.e., not perfectly reflective), opaque objects, most light comes

directly from the lights (“locally”), and not “globally” from other surfaces in the scene

• local lighting approximations– diffuse Lambertian reflection: only accounts for angle between surface normal and

vectors to the light source. – fake specular spots on shiny surfaces: Phong lighting

• global lighting approximations– introduce a constant “ambient” lighting term to fake an overall global contribution– reflection: environment mapping– shadows: shadow mapping

• polygon interior pixels shaded by simple color interpolation: Gouraud shading– Phong shading: evaluate some lighting functions on a per-pixel basis, using interpolated

surface normal.

Techniques—Rendering (1/9)


An example: Quake III• Few polygons (i.e., low geometric complexity)• Purely local lighting calculations• Details created by texturing everything with precomputed texture maps

– surface detail – smoke, contrails, damage and debris– even the lighting and shadows are done with textures

• Bump mapping on some hardware



9


The Best• Global illumination: find out where all the light entering a scene comes from,

where and how much it is absorbed, reflected or refracted, and all the places it eventually winds up

• Three methods: Raytracing (specular), Radiosity (diffuse), IBR (avoid geometry)• Early method: Ray-tracing. Method to avoid forward tracing infinitely many light

rays from light sources to eye. Work backwards to do viewer/pixel-centric rendering: shoot viewing rays from viewer’s eyepoint through each pixel into scene, and see what objects they hit. Return color of object struck first. If object is transparent or reflective, recursively cast ray back into scene and add in reflected/refracted color

– Turner Whitted, 1980– moderately expensive to solve– “embarrassingly parallel”—can

use parallel computer ornetworked workstations

– models simple lighting equation (e.g., ambient, diffuse and specular) for direct illumination but only perfectly specular reflection for indirect (global) illumination



The Best: Ray Tracing (cont.)• Ray-tracing good for: shiny, reflective, transparent surfaces such as metal,

glass, linoleum. Can produce sharp shadows, lensed caustics (focusing of light due to interaction with curved specular surfaces). As these effects appear relatively infrequently in everyday life, grouped together they often look characteristically “computerish”

• Can do volumetric effects, caustics with straightforward extensions (such as “photon maps”)


10


The Best: Radiosity (Energy Transport) - Diffuse• Scene-centric rendering. Break scene up into small surface patches and

calculate how much light from each patch contributes to every other patch. Circular problem: some of patch A contributes to patch B, which contributes some back to A, which contributes back to B, etc. Very expensive to solve—iteratively solve system of simultaneous equations

– viewer-independent—batch preprocessing step followed by real-time, view-dependent display



The Best: Radiosity (cont.)• Good for: indirect (soft) lighting, color bleeding, soft shadows, indoor

scenes with matte surfaces. As we live most of our lives inside buildings with indirect lighting and matte surfaces, this technique looks remarkably convincing

• Even better results can be obtained by combining radiosity with ray-tracing

– Various methods for doing this. Looks great! Really expensive!


11


The Gracefully Degraded Best• Selectively ray-trace. Usually only a few shiny/transparent objects in a

given ray-traced scene. Can perform local lighting equations on matte objects, and only ray-trace the pixels that fall precisely upon the shiny/transparent objects

• Calculate radiosity at vertices of the scene once, and then use this data as the vertex colors for Gouraud shading (only works for diffuse colors in static scenes)



The Real Best: Sampling Realistically• The Kajiya rendering equation (covered in CS224 by Spike) describes

this in exacting detail – very expensive to compute!

• Previous techniques were different approximations to the full rendering equation

• Led to the development of path-tracing: point sampling the full rendering equation

• Eric Veach’s Metropolis Light Transport is a faster way of sampling the full rendering equation (CS224)


12


Techniques—Rendering (9/9)Side Note—Procedural Shading• Complicated lighting effects can be obtained through use of procedural

shading languages– provides nearly infinite lighting possibilities– global illumination can be faked with low computational overhead– but usually requires a skilled artist to get decent images

• Pixar’s Renderman• Procedural shading is now in hardware

– nVidia’s GeForce3+ has programmable vertex and pixel shaders– Cg – nVidia’s C-like language for coding vertex and pixel shaders – assembly

no longer required


A Different Approach:• Image-based rendering (IBR) is only a few years old. Instead of spending

a lot of time and money modeling every object in a complex scene, take a photo of it. You’ll capture both perfectly accurate geometry and lighting with very little overhead

• Analogous to image compositing in 3-D• Dilemma: how to generate views other than the one photo you took.

Various answers.

The Hacked• QuickTimeVR.

– Stitch together multiple photos taken from the same location at different orientations. Produces cylindrical or spherical map which allows generation of arbitrarily oriented views from that one position.

– generating multiple views: discontinuously jump from one precomputed viewpoint to the next. In other words, can’t reconstruct missing (obscured) information

Image-Based Rendering (1/2)

13


The Best• Plenoptic modeling: using multiple overlapping photos, calculate depth

information from image disparities. Combination of depth info and surface color allows on-the-fly reconstruction of “best guess”intermediary views between the original photo-positions

• Lightfield rendering: sample the path and color of many light rays within a volume (extremely time-consuming pre-processing step!). Then interpolate these sampled rays to place the camera plane anywhere within the volume and quickly generate a view.

– Treats images as 2d slices of a 5d function – position (xyz) and direction (theta, phi on sphere)

– Drawbacks: have to resample for any new geometry.

Image-Based Rendering (2/2)


Stills vs. Animation• At first, computer graphics researchers thought, “If we know how to

make one still frame, then we can make an animation by stringingtogether a sequence of stills”

• They were wrong. Long, slow process to learn what makes animations look acceptable

• One problem: reappearance of spatial aliasing• Individual stills may contain aliasing artifacts that aren’t immediately

apparent or irritating– impulse may be to ignore them

• Sequential stills may differ only slightly in camera or object position. However, these slight changes are often enough to displace aliasing artifacts by a distance of a pixel or two between frames

Temporal Aliasing (1/3)

14


Stills vs. Animation• Moving or flashing pixel artifacts are alarmingly noticeable in animations.

Called the “crawlies”. Edges and lines may ripple, but texture-mapped regions will scintillate like a tin-foil blizzard

• How to fix crawlies: use traditional filtering to get rid of spatial artifacts in individual stills



Motion Blur• Another unforeseen problem in animation: temporal aliasing• This is like spatial aliasing problem, only over time: if we sample a continuous

function (in this case, motion) in too few steps, we lose the continuity of the signal• Quickly moving objects seem to “jump” around if sampled too infrequently• One solution: motion blur. Turns out that cameras capture images over a relatively

short interval of time (function of shutter speed). For slow moving objects, the shutter interval is sufficiently fast to “freeze” the motion, but for quickly moving objects, the interval is long enough to “smear” the object across the film. This is, in effect, filtering the image over time instead of space

• Motion blur a very important cue to the eye for maintaining illusion of continuous motion

• We can simulate motion blur in rendering by taking the weighted average of series of samples over small time increments


15


Modeling the way the world moves• Cannot underestimate the importance of behavioral realism

– we are very distracted by unrealistic behavior even if the rendering is realistic– good behavior is very convincing even when the rendering is unrealistic (e.g.,

motion capture data animating a stick figure still looks very “real”)– most sensitive to human behavior – easier to get away with faking ants, toys,

monsters, fish etc.• Hand-made keyframe animations

– professional animators often develop an intuition for the behavior of physical forces that computers spend hours calculating

– “cartoon physics” sometimes more convincing or more appealing than exact, physically-based, computer calculated renderings

– vocabulary of cartoon effects: anticipation, squash, stretch, follow-through, etc.

Techniques—Behavior (1/4)


The Best

• Motion-capture– sample positions and orientations of motion-trackers over time. Trackers usually

attached to joints of human beings performing complex actions. Once captured, motion extremely cheap to play back: no more storage required than a keyframe animation. Irony: one of cheapest methods, but provides excellent results

– usually better than keyframe animations and can be used for a variety of characters with the same joint structure (e.g., Brown → CMU’s Nancy Pollard’s research)

– “motion synthesis”: a recent hot topic – how to make new animations out of the motion capture data that you have.


16


The Best (cont.)• Physics simulations (kinematics for rigid-body motion, dynamics for F =

ma)– Hugely complex modeling problem– expensive, using space-time constraints, inverse kinematics, Euler and

Runge-Kutta integration of forces, N2-body problems. These can take a long time to solve

– looks fairly convincing…but not quite real (yet)



Techniques—Behavior (4/4)The Gracefully Degraded• Break laws of physics (hopefully imperceptibly)

– Simplify numerical simulation: consider fewer forces, use bounding boxes instead of precise collision detection, etc.

– Decrease number of time steps used for Euler integration

17

Andries van Dam October 28, 2004 Illumination Models 1/18

Photograph

Scene rendered using a physically-based global illumination model

Scene rendered using direct lighting only

(Frederic Drago and Karol Myszkowski, Validation Proposal for Global Illumination and Rendering Techniques, http://www.mpi-sb.mpg.de/resources/atrium/)

Documents

Lecture 3 of 42: Realism in Computer Graphics · 1/28/2008 · 2 Math Review for CIS 636 • Overview: First Month (Weeks 2-5 of Course) – Review of mathematical foundations of