KAIST Computer Science

Global illumination for PIXAR® movie production

20100121 Jaehyun Jang

KAIST Computer Science

• Global illumination for movie production

• Current Lighting Computation Methods

• Point-based Global Illumination for Movie Production

• Physically-based Lighting at PIXAR

• Global illumination researches in PIXAR®

• Rendering Pipeline in Production (as a Rendering TD)

Contents

KAIST Computer Science

• The first 3D animation used global illumination - Shrek 2 (2004)

Global illumination for movie production

Fig 1. Shrek 2. [Photograph]. Retrieved from http://www.awn.com/vfxworld/illuminating-global-illumination

Courtesy of Paramount Pictures.

KAIST Computer Science

• Faking it : adding extra light sources

labour intensive

• Ray Tracing : requires many rays + shader evaluations

Too much slow (at 2010!)

• Radiosity

Requires entire scene data on memory

• Point-based

Little memory, no shader evaluations

Current Lighting Computation Method

KAIST Computer Science

Current : Physically-based Ray Tracing

Fig 2. Monster University [Photograph]. Retrieved from http://neogaf.com/forum/showthread.php?t=569821

Courtesy of Disney-PIXAR

• PIXAR has been rendering their movie using physically based ray-tracing after Monster University (2013)

KAIST Computer Science

Past : Point-based Global Illumination Techniques

Fig 3. Toy Story 3 [Photograph]. Retrieved from http://graphics.pixar.com/library/PointBasedGlobalIlluminationForMovieProduction/

Courtesy of Disney-PIXAR

• PIXAR uses point-based global illumination for rendering at 2010. At that time, ray-tracing cost is too high to render images.

KAIST Computer Science

• First introduced this technique in ‘Point-Based Approximate Color Bleeding’, (2008).

• Background : Ray-tracing based global illumination is still expensive.

• Advantages using point-based :

1. Fast Computation

2. Noisy-Free

• Disadvantages :

1.Multi-pass approach, it cannot guarantee results as precise as ray tracing.

Point-Based GI for Movie Production (2010)

KAIST Computer Science

Point-Based GI for Movie Production (2010)

• Point Cloud

Each point : position, normal, radius, color = a colored disk

Color : the direct illumination at that surface position

How to generate? : PRMan to tessellate the surfaces into small micropolygons, compute direct illumination from all light sources by using surface shader.

—> This approach is “Baking the direct illumination” (TD Terminology)

KAIST Computer Science

Point-Based GI for Movie Production (2010)

• Compute octree

A bottom-up computation

Consisting a node similar normals

Compute a spherical harmonic from leaf-node

Leaf-node : the sums of the coefficients for the surfels in the node.

Non-leaf-node : the sums of the coefficients of its child nodes.

KAIST Computer Science

Point-Based GI for Movie Production (2010)

Fig 4. Rasterization onto six raster cube faces [Photograph]. Retrieved from http://

graphics.pixar.com/library/PointBasedGlobalIlluminationForMovieProducti

on/ Courtesy of Disney-PIXAR

• Compute diffuse, and glossy global illumination

Traverse octree, visiting a node and rasterize the illumination from node.

Rasterize colors contributing to a point : world “as seen” by that point (a low resolution fish-eye image)

Convolution with the BRDF

1. loop over raster pixels and multiplying colors with the BRDF for the direction corresponding to that pixel.

2. Can deal with glossy global illumination

KAIST Computer Science

Point-Based GI for Movie Production (2010)

Fig 5. Rendering with PRMan, baked radiosity value of point cloud (700,000 points) (left), glossy reflection (right) [Photograph]. Retrieved from http://penguin.ewu.edu/RenderMan/RMS_2.0/pointbased.html

Courtesy of Disney-PIXAR

KAIST Computer Science

Point-Based GI for Movie Production (2010)

• Variation and extensions

Area light sources and soft shadows

Environment illumination

Multiple diffuse bounces

Final gathering for photon mapping

Ambient occlusion and reflection occlusion

etc.

KAIST Computer Science

Point-Based GI for Movie Production (2010)

Fig 6. Textured and displaced area light sources [Photograph]. Retrieved from http://penguin.ewu.edu/RenderMan/RMS_2.0/pointbased.html

Courtesy of Disney-PIXAR

KAIST Computer Science

Point-Based GI for Movie Production (2010)

Fig 7. Point cloud, Ambient occlusion, Reflection Occlusion, additional environment reflection [Photograph]. Retrieved from http://penguin.ewu.edu/RenderMan/RMS_2.0/pointbased.html

Courtesy of Disney-PIXAR

KAIST Computer Science

• The lighting pipeline at PIXAR was completely rewritten and switched to a physically based and ray-traced system.

• Rendering Equation

L : radiance, f : BRDF

2 parts working in tandem :

1.Physically correct lights emitting energy in the scene

2.Physically correct BRDFs bouncing energy in the scene

Physically based rendering pipeline (2013)

KAIST Computer Science

• Break down the rendering equation, each part will be solved by different coshaders(defined in RSL 2.0) called integrators (http://renderman.pixar.com/view/coshaders)

•

!

!

• direct lighting integrator + indirect diffuse integrator + indirect specular integrator

Physically based rendering pipeline (2013)

KAIST Computer Science

Fig 8. Light reflection [Photograph]. Retrieved from http://content.gpwiki.org/index.php/D3DBook:

(Lighting)_Foundation_and_theory Courtesy of Wikimedia

KAIST Computer Science

• Direct Lighting Integrator (Coshader)

1.Sample from lobes using BRDF coshaders

2.Sample from lights using light coshaders

3.Combine samples using Multiple Importance Sampling (Ryusuke Villemin, et al. 2013).

Physically based rendering pipeline (2013)

KAIST Computer Science

Fig 9. Structures communicates data between three coshaders consisting of Direct Light Integrator [Photograph]. Retrieved from http://graphics.pixar.com/library/PhysicallyBasedLighting/paper.pdf

Courtesy of Disney-PIXAR

KAIST Computer Science

Algorithm 1. (Light Integration) Several algorithms in the paper : http://graphics.pixar.com/library/PhysicallyBasedLighting/paper.pdf

KAIST Computer Science

Fig 10. Various luminaries in this shot, The Blue Umbrella [Photograph]. Retrieved from http://graphics.pixar.com/library/PhysicallyBasedLighting/paper.pdf

Courtesy of Disney-PIXAR

KAIST Computer Science

Fig 11. Physically-based lighting in Monster University [Photograph]. Retrieved from http://blog.selfshadow.com/publications/s2013-shading-course/pixar/

s2013_pbs_pixar_slides.pdf Courtesy of Disney-PIXAR

KAIST Computer Science

Fig 12. Physically-based lighting in Monster University [Photograph]. Retrieved from http://blog.selfshadow.com/publications/s2013-shading-course/pixar/

s2013_pbs_pixar_slides.pdf Courtesy of Disney-PIXAR

KAIST Computer Science

• Based on PIXAR online library (from 2011 - now)

• Sampling

Importance sampling

1. A statistical framework for comparing importance sampling methods, and an application to rectangular lights (Leonid Pekelis, et al. 2014).

2. Multiple Importance Sampling for Emissive Effects (Ryusuke Villemin, et al. 2013).

3. Importance Sampling of Reflections from Hair Fibers (Christophe Hery, et al. 2011).

Stratified Sampling

1. Correlated Multi-Jittered Sampling (Andrew Kensler, 2013).

Global Illumination Researches in PIXAR

KAIST Computer Science

• Caching

1. Multiresolution Radiosity Caching for Efficient Preview and Final Quality Global Illumination in Movies (Per H. Christensen, et al. 2012)

• Summary

PIXAR researches physically-based ray tracing, their rendering pipeline running on monte-carlo ray tracing, and it needs to develop proper sampling techniques for BRDF and etc.

Global Illumination Researches in PIXAR

KAIST Computer Science

• Physically-based rendering is production standard.

• Current PRMan® pipeline, RSL 2.0 supports physically plausible shaders.

• For rendering technical directors, these kind of techniques are tremendously demanded.

• — All is commented by technical director in PIXAR.

Rendering Pipeline in Production