The Ancient Temples, The Modern Cities, and the Stars: GPU ... · Foggy Lights in the Street •Used an approximation shader instead of an expensive volumetric technique •Simulate

The Ancient Temples, The Modern Cities, and the Stars: GPU Journeys with the ATI demo teamPart II: ToyShop Revealed

Natalya Tatarchuk3D Application Research GroupATI Research, Inc.

March 28, 2006 Game Developer Conference, San Jose, CA, March 2006 2

The Plan

•Overview of the ToyShop demo•Constraints and resources breakdown•High Dynamic Range Rendering and Lighting•Mist and secondary illumination effects•Shadows solution•Dynamic cubemaps rendering •Wet reflective objects •Effects medley•Conclusions


Imitating Reality: Background Research

• We wanted to create a moment in a dark city, downtown, during a rainy night

• Fortunately, having started on the concept in a middle of October in Boston, we had a lot of opportunities for research

• Gigabytes of videos of rain, droplets, streams, … and here we are:

Rainy night in Boston Rainy night in Toy Shop Town


Zen and The Art of Shader Writing

•This demo is a study in combination of complex computations with inexpensive yet effective shaders for a variety of effects

•Perspective-correct extruded surfaces with SM 3.0 parallax occlusion mapping (see “Practical Dynamic Parallax Occlusion Mapping” presentation)

•Water and raindrop simulation effects•A variety of post-processing effects for glow, blurry reflections and rain

•Many custom effects for rendering rain•HDR illumination with lightning and shadows•Many more…


Demo: The ToyShophttp://www.ati.com/developer/demos/rx1800.html


ToyShop Constraints

• Our goal was to fit this demo into 256MB of video memory• This was a challenge considering the rich complex environment• In the end we used 240Mb for the entire demo, including:

• 54Mb for backbuffers and offscreen buffers (many were 1010102)• 156Mb for texture memory• 28Mb for object vertex and index buffers

• 3Dc+ texture compression was crucial• We went from 478 MB texture memory total to 156Mb

• We used dec3n vertex data format to reduce memory • Gives 3:1 memory savings• For vertex normals / tangents / binormals / other data• Available on ATI Radeon line since Radeon 9700

• A lot of work went into optimization of draw calls: • Averaging 250


Rich Detailed Worlds Require Complex Shaders

•Everyone realizes the need to make immersive environments

•Doing so successfully requires many complex shaders with many artist parameters

•We created ~500 custom unique shaders for ToyShop

•>1/2 of those are rain / water / wet surfaces shaders•~1/3 draw depth only or render reflections •~1/6 relate to post-processing effects•Particle effect shaders•We used ~20 various shared include files (reflection / shadow mapping / HDR / math functions / lighting / lightning / etc)


What 3Dc Really Buys You

This is what our scene looks like with 3Dc and dec3n used:

With 3Dc

Without 3Dc and dec3n wewould be able to render just this much:

Without 3Dc


Dark City Night: The Lighting

• Even in the midst of a dark stormy night there is light…• Large number of perceived light sources

Bright street lights Bright street lights and building lampsand building lamps

Taxi cab lightsTaxi cab lights

Traffic lightsTraffic lights

LightningLightning

Neon sign lightsNeon sign lights


HDR Rendering on a Budget

•HDR is necessary, without it the scene will feel dull•But! HDR buffers and lightmaps can use a great deal of memory

•Every pixel on the screen is tone-mapped – can be a performance hit

•Because of the multitude of effects in this demo, memory requirements were strict

•But we still want to use HDR!

•Goal: Balance performance and memory usage•Optimal solution: 10-10-10-2 surfaces

•Gives good precision results at 2x less memory•Encode all scene lightmaps into this format•Use this for the backbuffers and other intermediate buffers (for blurring, etc)


Tone Mapping and HDR LightMaps

•RGBS approach for lightmap decoding to maximize the dynamic range

•10-10-10-X alone just gives you 10 bits (i.e. 0-4)•With extra 2 bits of alpha RGBS allows you to stretch the range to 0-16

•4-point artist-editable tone-mapping curve for more control of the lighting

•Integrating HDR into any engine / pipeline:•Start by applying a linear curve•Make sure the values you are getting from lightmaps are correlated to what’s in the lightmaps

•Lightmaps are time-consuming to render, so make sure your tone-mapping curves are working correctly


HDR Tone Mapping Examples

•Too dark

•Too bright

•Just right

•Remember…Tone mapping curve can be dynamically computed and animated

•Dynamic tone-mapping done in Day of Defeat by Valve



•Too dark

•Too bright

•Just right





•Too dark

•Too bright

•Just right




HDR Lightmap Authoring

•Authoring LightMaps is a time consuming step

•Should be the last step of development•Tip: Expensive – do it once and do it right

•Had to approximate Global Illumination•Render HDR lightmap (32-bit)•Render an Ambient Occlusion Map (32-bit)•Blend together to fake Global Illumination


HDR Authoring Examples

•32-bit lightmap•32-bit grey scale Ambient Occlusion map•Final 32-bit lightmap

+ =


Misty Glow in Bad Weather

•Water particles in the atmosphere during rain increase the amount of light scattering around objects

•Appearance of glow around light sources in stormy weather is due to multiple scatter

•We apply Kawase post-processing approach for rendering glows in our scene (4 feedback passes)

•Since we use 1010102 buffers for rendering, we only use 2 bits of alpha for glow amount

• This looks good in our demo

•Attenuate light in shaders to simulate fog


Kawase’s Bloom Filter

•Start by downsampling (¼ x ¼ = 1/16) •Apply small blur filters repeatedly•More iterations for more blurriness•Each iteration “Ping-Pongs” between two renderable textures used for storing intermediate results

Texture sampling pointsPixel being Rendered

1/16

1/16 1/16

1/16

2/16

2/16

2/16 4/16

2/16


Iterating Kawase’s Bloom Filter

•Each iteration takes the previous results as input and applies a new kernel to increase blurriness

2nd Iteration 3rd Iteration


Foggy Lights in the Street

• Our scene has many foggy volumetric lights with rain


Foggy Lights in the Street

• Used an approximation shader instead of an expensive volumetric technique

• Simulate lights as volumetric light cones • Render scrolling noisy cloud-like fog through the light cone: Cheap!• Uses tangent space technique to control lit fog fading out towards the

edges of the light frustum• To render rain falling through the light frustum:

• Render an inserted plane in the middle of the light cone rendering scrolling rain texture

• Must match the fade out toward the edges of the light frustum: use a map to match the cone attenuation

• This plane is oriented around up-axis in world-space for angled lamps so that the rain appears to fall vertical to the viewer


Traffic Light Illumination

•Traffic lamps illuminate the traffic light signal object•Properly computing global illumination to simulate color bleeding and inter-reflection is expensive

•Instead, use another, special “traffic light” lightmap, animated in accordance with the blinking traffic light

• Bake light with color bleeding for the object

• Inexpensive and small map

•Rendering the traffic lamp: • We simulate both internal

reflection through colored glassand reflection from the outside


Shadows Rule the Darkness

•Used shadow mapping as shadowing solution •One shadowing light

•To save memory, our shadow maps use a new Radeon series feature: Depth textures

•Efficient and flexible surfaces for shadow mapping and other algorithms

•Saves memory and bandwidth with good precision balance•Depth only rendering•Create the depth buffer of any size, and bind a tiny color buffer (say, 1x1)

•We used DF16 surfaces


Combined Approaches for High Quality

•Randomized PCF offset sampling with custom filtering kernels for better quality

•Bright regions used 8-tap Poisson disk kernel with random rotation

•Dark and noisy surfaces used regular Poisson disk kernel sampling

•Used excellent dynamic branching performance of X1K line to optimize

•Only perform expensive shadow map filtering when actually in shadowing light frustum!

•GDC session “Shadow Mapping Tricks and Techniques” Friday 12-1pm by John Isidoro with more details


Shadow Details

•Random rotation was needed for bright surfaces with smooth albedo.

•Shadow frustum test and N·L computation culling•Darker surfaces with a lot of texture detail just uses a fixed disc.


Dynamic CubeMap for Reflective Taxi

•Taxi cab reflections•Rendered a dynamic cube map for reflection lookup

•Only rendered on the frames where the cab was actually moving

•Built a low resolution “billboard”version of the ToyShop world

•This gets rendered 6 times into the dynamic cubemap!

•Saves a great deal on performance•Using lower LOD if available is another good approach


Demo: Dynamic Cube MapDemo: Dynamic Cube MapDemo: Dynamic Cube Map


Reflections and Cube Map Filtering

•Distinct reflections require cube maps

•Standard cube map mipchain generation techniques create seams

•Because each cube face is filtered independently

•Trash area was particularly problematic

•Lots of thin polygons didn’t help


ATI CubeMapGen Tool

• Cube Map Filtering and Mip Chain Generation• Angular Extent Filtering• 2x2 mip level could be used for diffuse lighting or a 4x4 could be

used for burnished metal• Cube map import or interactive

assembly• Command-line mode/scriptability• Fully supports 32 bit HDR formats•• http://http://www.ati.com/developer/cubemapgenwww.ati.com/developer/cubemapgen//


Rendering Misty Rain Halos on Objects

•Misty rain halos – fins with scrolling rain •Shells – texture-based ripple effect (expanding rings)•Used this effect on the taxi and on the awning

• Requires extra geometry (fins) placed

Misty objects in the rain


Taxi Windshield Wipers

•Want a cheap effect to rendertaxi’s windshield with raindrops

•The windshield wipers need to dynamically affect the dropletsas they move

•Computing collision with the wipersand affecting droplet movement wasn’t practical

• The windshield wasn’t prominentin the demo


Wiping the Raindrops Away

•Solution: Wiper maps to control “clearing” of the droplets

•Wiper parameter is passed into the shader•Wiper maps are used to determine what regions have been recently swept clean

•Two separate wiper maps so wiped regions can overlap

Left wiper map Right wiper map


Other Cool Effects to See!

•Swinging telephone lines• Artist controllable vertex shader

animations




animations

•The Toy Shop awning blowing in the wind

• Vertex shader based animation




animations

•The Toy Shop awning blowing in the wind

• Vertex shader based animation

•Toy Shop Neon Sign• Scrolling texture along linear UVs to

animate bulbs


ToyShop Revealed: Conclusions

•We presented you some of the technology that was developed for the ToyShop demos

•Rich, complex environments demand convincing details•Lots and lots of custom shaders allowed us to create a thorough illusion of a dark, rainy night in the city

• A variety of rain effects• Different approaches for reflection

rendering• Per-pixel displacement mapping for

surface details• Water and raindrop movement

simulation effects•We hope to see even better environments in next gen games

• The secrets are out: Re-use all of our technology to your benefit!


The ToyShop TeamThe ToyShop Team

Lead ArtistLead Artist Lead ProgrammerLead ProgrammerDan Roeger Natalya TatarchukDan Roeger Natalya Tatarchuk

David GosselinDavid Gosselin

ArtistsArtistsDaniel Szecket, Eli Turner, and Abe WileyDaniel Szecket, Eli Turner, and Abe Wiley

Engine / Shader ProgrammingEngine / Shader ProgrammingJohn Isidoro, Dan Ginsburg, Thorsten Scheuermann and Chris OatJohn Isidoro, Dan Ginsburg, Thorsten Scheuermann and Chris Oat

ProducerProducer ManagerManagerLisa CloseLisa Close Callan McInallyCallan McInally


Other Interesting Sessions by ATI

Shadow Mapping Tricks and Techniques Shadow Mapping Tricks and Techniques ((Programming TrackProgramming Track))12:00 12:00 –– 1:00 1:00 PMPM

Practical Parallax Occlusion Mapping for Highly Detailed SurfacePractical Parallax Occlusion Mapping for Highly Detailed SurfaceRendering Rendering ((Programming TrackProgramming Track) ) (ToyShop)(ToyShop)9:00 9:00 –– 10:00 10:00 AMAM

The Ancient Temples, the Modern Cities, and the Stars: GPU The Ancient Temples, the Modern Cities, and the Stars: GPU Journeys with the ATI Demo Team Journeys with the ATI Demo Team ((Programming TrackProgramming Track) ) (ToyShop)(ToyShop)2:30 2:30 –– 5:005:00PMPM

FridayFriday

WednesdayWednesday

Efficient Tricks That Will Impress Your Friends! Efficient Tricks That Will Impress Your Friends! ((Programming Programming TrackTrack))12:00 12:00 –– 1:001:00PMPM

ToyShop ToyShop –– The Devil is in the Details (The Devil is in the Details (Art Track)Art Track)10:30 10:30 ––11:3011:30AMAM

Squeezing Performance out of your Game with ATI Developer Squeezing Performance out of your Game with ATI Developer Performance Tools and Optimization Techniques (Performance Tools and Optimization Techniques (Programming Programming TrackTrack))9:00 9:00 –– 10:0010:00AMAM


Questions?

• [email protected]

• Download the ToyShop demo here:

http://www.ati.com/developer/demos/rx1800.html


We’re hiring!

3D Applications Research GroupDemo Team

Research TeamTools team

Visit our web site:http://www.ati.com/companyinfo/careers/

Mention 3DARG and the team you’re interested in joining!

Documents

The Ancient Temples, The Modern Cities, and the Stars: GPU ... · Foggy Lights in the Street •Used an approximation shader instead of an expensive volumetric technique •Simulate