Upload
wolfgang-engel
View
378
Download
2
Embed Size (px)
Citation preview
A new Post-Processing Pipeline
Wolfgang EngelConfetti
09/26/2013
Confetti
• Think-Tank for game and movie related industries• Middleware Provider
– Aura – Dynamic Global Illumination System– PixelPuzzle – PostFX pipeline– Ephemeris – Dynamic Skydome system
-> License comes with full source-code• Services:
– Hardware vendors– many game developers (“Engine Tuner”)
• Provides software solutions for games, movies and tools for GPU manufacturers
• http://www.conffx.com
Confetti
Tomb-Raider – TressFX / Hair Rendering
Confetti
Confetti
Confetti
Confetti
Confetti
Confetti
Dynamic Global Illumination system / Skydome System with volumetric Clouds
Agenda Post-Processing
• History • Call for a new Post-Processing Pipeline• Yxy Color Space• Dynamic Local Gamma• Depth of Field
History
• The first HDR rendering pipeline appeared in a DirectX SDK in 2004
• From there on we called a collection of image space effects at the end of the rendering pipeline Post-Processing Pipeline
History
• The idea was to re-use resources == render targets and data in the Post-Processing Pipeline to apply effects like – Tone-mapping + HDR rendering == dynamic contrast
operator– Camera effects like Depth of Field, Motion Blur, lens flare– Color filters like contrast, saturation, color additions and
multiplications• One of the first coverages of a collection of effects in a
Post-Processing Pipelines happened at GDC 2007 [Engel2007]
History
• Since then – Numerous new tone mapping operators were
introduced [Day2012]– New more advanced Depth of Field algorithms
with shaped bokeh were introduced• … but nothing changed fundamentally
Call for a new Post-Processing Pipeline
• RGB is not a color space for PostFX-> we should use a color space that uses luminance
• Global tone mapping operators didn’t work out well in practice, only on paper-> your artists probably limit the luminance values and therefore the tone mapping operator because the textures “blow out”
Call for a new Post-Processing Pipeline
• A fixed global gamma adjustment at the end of the pipeline is a waste of cycles – because it does a “similar thing” as the tone
mapper … why not just make it dynamic and part of the tone mapper
– … we can also make it more local and not just globally adjusting gamma
Call for a new Post-Processing Pipeline
• We can also add more stages to the Post-Processing Pipeline
• Adding Screen-space– Ambient occlusion by occupying the fourth channel of the
PostFX render targets or the luminance channel– Skin– Reflections by utilizing the “same” blur kernels
• Any new Post-Processing Pipeline needs to be written in compute-> substantial bandwidth savings and speed increases
Simplified Pipeline Stages from 2007
Depth of FieldTone MappingColor Filters
Gamma Control
Frame-Buffer
16x16 RTLuminance
4x4 RTLuminance
1x1 RTLuminance
64x64 RTLuminance
1x1 RTAdapt Lum.
Measure Luminance
Adapt Luminance
Primary Render Target
BrightPass Filter
Z-Buffer
Downsample½ Size
Downsample¼ Size
Gauss Filter
Gauss Filter II
Yxy Color Space
• Instead of running the Post-Processing Pipeline in RGB we can run it in Yxy
• Y holds then the luminance
Yxy Color Space
• To apply any tone mapping operation, we have to convert RGB into a space that let’s us easily separate luminance-> tone mapping is applied to luminance*
* … don’t apply it to a RGB value … an artist will notice
Yxy Color Space
• If we use Bloom, we have to apply tone mapping twice in our pipeline-> once in the Bright-Pass filter and once in the final pass-> that means we convert from RGB to luminance and from luminance to RGB twice
Yxy Color Space
• If we run the whole pipeline in a color space that holds luminance in a dedicated channel, we only covert once into this color space and at the end of the pipeline back
Yxy Color Space
• For Bloom we can run the bright-pass filter in one channel-> modern scalar GPU hardware -> speed-up
Yxy Color Space
• To apply SSAO … just “mix” it into your Y channel
• … and re-use a blur kernel you use anyway to blur it
Yxy Color Space
• Why Yxy color space?
*based on Greg Wards LogLuv model
High-Dynamic Range Rendering
Color Space
# of cycles (encoding)
Bilinear Filtering
Blur Filter Alpha Blending
RGB - Yes Yes Yes
HSV ~34 Yes No No
CIE Yxy ~19 Yes Yes No
L16uv* ~19 Yes Yes No
RGBE ~13 No No No
Dynamic Local Gamma
• From linear gamma to sRGB
float3 Color; Color = (Color <= 0.00304) ? Color * 12.92 : (1.055 * pow(Color, 1.0/2.4) - 0.055);
Dynamic Local Gamma
• It seems everyone is using a global Gamma setting that is the same for every pixel on screen
• Propose to change Gamma per-pixel-> in fact Gamma correction is considered different depending on brightness-> we just didn’t implement it this way …
Dynamic Local Gamma
• The human eye’s visual gamma changes the perceived luminance for various adaptation conditions [Bartleson 1967] [Kwon 2011]
• If the eye’s adaptation level is low, the exponent for Gamma increases
Dynamic Local Gamma
Changes in relative brightness contrast as a function of adaptation of relative luminance and adaptation luminance levels according to the result [Bartleson 1967]
Dynamic Local Gamma
• Local Gamma varies with luminance [Kwon 2011]*
* … build your own equation, depending on the type of game
Dynamic Local Gamma
• is changed based on the luminance value of the current pixel -> that means each pixels luminance value might be gamma corrected with a different exponent-> with the equation above, the exponent == gamma value is in the range of 0.421 to 0.465
Dynamic Local Gamma
Applied Gamma Curve per-pixel based on luminance of pixel• Eye’s adaptation == low -> blue curve• Eye’s adaptation value == high -> green curve
Dynamic Local Gamma
• works with any tone mapping operator• Example: [Reinhard]
• Artistically desirable to burn out bright areas• Source art not always HDR
• Leaves 0..1
– White = 6.0
•What is the visual difference?• Dynamic light & shadow information is considered for
gamma-> new information introduced in the pipeline is used to modify the gamma correction-> changes “naturally” depending on how bright or dark a scene is
• Light looks better / shadows look better
Dynamic Local Gamma
Depth of Field
• When using cameras, this refers to the distance in a scene that appears in focus
• To add artistic effect and/or focus attention• Physically correct parameters and calculations
needed for a proper looking depth of field– Tradeoff: parameters are not artist friendly
• Need a fast and efficient way to generate a high quality Depth of Field effect
Depth of Field
• Circle of Confusion(CoC)• When using a lens to produce an image, the size of
the resulting “spot” produced by a point in the scene
• The Depth of Field is the region where the CoC is less than the resolution of the human eye (or of the display medium)
Depth of Field
• Circle of Confusion(CoC)– Affected by:
• F-stop - ratio of focal length to aperture size• Focal length – distance from lens to image in focus• Focus distance – distance to plane in focus
Depth of Field
• Calculating Circle of Confusion(CoC)[Potmesil1981]
• CoC is negative for far field, positive for near field• Convert CoC from meters to pixel units to find the
effective CoC on screen
Depth of Field
• Basic depth of field effect:– Calculate CoC for each pixel– Use CoC to generate separate results for near field
and far field• Flat, shaped kernel useful for producing Bokeh effect
– Combine far field and focus field based on CoC– Combine with the near field based on CoC and
near field coverage
Depth of Field
Red = max CoC (near field CoC)Green = min CoC (far field CoC)
Depth of Field
Far field result in Yxy
Depth of Field
Near field result in Yxy
Depth of Field
References• [Bartleson 1967] C. J. Bartleson and E. J. Breneman, “Brightness function: Effects of
adaptation,” J. Opt. Soc. Am., vol. 57, pp. 953-957, 1967. • [Day2012] Mike Day, “An efficient and user-friendly tone mapping operator”,
http://www.insomniacgames.com/mike-day-an-efficient-and-user-friendly-tone-mapping-operator/
• [Engel2007] Wolfgang Engel, “Post-Processing Pipeline”, GDC 2007 http://www.coretechniques.info/index_2007.html
• [Kwon 2011] Hyuk-Ju Kwon, Sung-Hak Lee, Seok-Min Chae, Kyu-Ik Sohng, “Tone Mapping Algorithm for Luminance Separated HDR Rendering Based on Visual Brightness Function”, online at http://world-comp.org/p2012/IPC3874.pdf
• [Reinhard] Erik Reinhard, Michael Stark, Peter Shirley, James Ferwerda, "Photographic Tone Reproduction for Digital Images", http://www.cs.utah.edu/~reinhard/cdrom/