Wilf LaLonde ©2012 Comp 4501 95.4501 Ambient Occlusion

Wilf LaLonde ©2012Comp 4501

95.450195.4501

Ambient Occlusion


• Ambient occlusion (AO): a “smart” ambient term that adds shadowing to diffuse objects lit with environment lighting; i.e., shadows in corners and creases in room without lights.

• Screen space ambient occlusion (SSAO), a variation computed directly from information on the screen; e.g., the depth buffer or camera space positional information containing z…

Ambient OcclusionAmbient Occlusion

Corners have more occlusion than flat walls.


Ambient Occlusion (AO)Ambient Occlusion (AO)

Diagram from Hoang and Low (Multi-Resolution SSAO)

AO is the lighting phenomenon under the direct illumination of a diffuse, uniform, spherical light

source surrounding the scene.


• How much does the environment lighting contribute at a point? It depends how much occlusion there is in the volume visible from the sample point...

The IssueThe Issue

Diagram from Loos and Sloan(Volumetric Obscurance)


Technically It’s A Volume ContributionTechnically It’s A Volume Contribution

Occlusion is cosine-weighted fraction of a tangent hemisphere

Minimal occlusion = 0 (no contribution, denoted ) for sample at or outside radius of influence or on negative side of normal)

[1 - (di / dmax)] cos i

Maximal occlusion 1 occurs when occlusion occurs immediately at

p; i.e., di is 0

The AO Definition


VariationsVariations

Obscurance SSAO

Horizon-Based SSAO (HSSAO)

Z-Buffer Based SSAO

Multi-Resolution SSAO (MSSAO)

Volumetric Obscurance (VO)


95.450195.4501

Brief Overview

Of a Number

of Techniques


obscurance

• From a point ALONG THE BASE POINT NORMAL, find how many samples in 3D sphere are obscured.

Technique 1: ObscuranceTechnique 1: Obscurance

The Alchemy Screen-Space Ambient Obscurance Algorithm, McGuire, Osman, Bukowski, Hennessay, High Performance Graphics 2011.

Paper Available

base point in green

COMPLEX (LIKE BUMPMAPPING)


• Use 3D sphere of points to sample the z buffer in the neighbourhood of the point (approximate occlusion by ratio of occluded to total points).

Technique 2: CrisisTechnique 2: Crisis

Screen Space Ambient Occlusion, Kajalin (Crysis), pp 413-424, ShaderX7, 2009.

wall 0.5 inside corner 0.75 outside corner 0.25

Z-buffer View 3D View

base point in green

works with z-buffer

SIMPLE


Actual occlusion is ignored but you have access to all unoccluded points.

Technique 3: MSSAO (AO Definition in Eye Space)Technique 3: MSSAO (AO Definition in Eye Space)

Uses a position bufferinstead of just z-buffer

sampling via 2D sphere

We ignore the factor 1/.

Distance based Occlusion

Multi-resolution screen-space ambient occlusion,, pp. 101-102, ACM Symposium on VR Software and Technology 2010.

MORE COMPLEX + DEMO LATER

Additionally focuses on how a hierarchy can be used to improve AO for both near

and far...

We will use it as a basis for a reimplementation


Technique 4: Volumetric ObscuranceTechnique 4: Volumetric Obscurance

Say it’s fully occluded (1) if on negative side of normal;

otherwise not (0).

We will implement this as a MOD of MSSAO just to see how good/bad it is.

This is a simplistic modification of #3 with only 1 pass (though it

probably uses much larger filters).


horizon-based

• How far toward horizon you can rotate before hitting… Quantizes horizon angle + quantizes rotation angle around normal... Requires many samples in 3D sphere...

Technique 5: Horizon-Based SSAOTechnique 5: Horizon-Based SSAO

MORE COMPLEX + DEMOS

NVIDiA SDK 10

NVIDiA SDK 11

Image-Space Horizon-Based Ambient Occlusion (NVIDIA demo), Bavoil and Sainz, pp 425-444, ShaderX7, 2009.


95.450195.4501

Preparing To

Implement: Some

Ideas


• Standard ratio increases linearly from 0 to 1.

Ratios: We’ll want Ratios: We’ll want

dmax

d

• Squared ratio increases more slowly.dmax

2

d2

makes occlusion nearby (more crispy)

Recall: occlusion 1 (at radius 0), occlusion 0 (at radius dmax).

dmax is radius of influence


From Bavoil and Sainz (Horizon based…)and Hoang and Low (Multi-Resolution SSAO)



• Use per pixel normals (more accurately computed from derivatives)…

Details That Make it BetterDetails That Make it Better




• Blurring the result helps… even more if the blurring preserves edges.


From Kajalin (Crysis)…From Kajalin (Crysis)…


• Sample in direction of normal to eliminate those that are on the negative side (so fewer samples can be used). Applies mostly in 3D.


From Briney et al, Fast Fake Global Illumination, ShaderX7, 2009

From Briney et al, Fast Fake Global Illumination, ShaderX7, 2009


95.450195.4501

Hoang and Low

multi-resolution screen-

space ambient occlusion

(MSSAO)


• Far (low frequency detail) and near (high frequency detail) integrated together...

• Can be done with smaller filters without resorting to random sampling for lower resolutions.

• Results do not suffer from noise and excessive blur (even though some blurring is used).

AdvantagesAdvantages

Multi-resolution screen-space ambient occlusion, Hoang and Low, pp. 101-102, ACM Symposium on

Virtual Reality Software and Technology 2010

Multi-resolution screen-space ambient occlusion, Hoang and Low, pp. 101-102, ACM Symposium on

Virtual Reality Software and Technology 2010


Ambient Occlusion (AO)Ambient Occlusion (AO)

𝐩

𝐪

Diagram from Hoang and Low

Look at Demo


• It’s in OpenGL, so a switch from DirectX.• Some simple demo changes to allow experiments

(including bug fixes) and changing the number of hierarchical steps. Original demo needs at least 3; we want 1 or more...

• First use of poisson disk filter. • First use of downsampling median filter.• First use of gaussian blur filter.• First use of bilateral upsampling filter (what’s that).• Minor use of temporal coherence (what’s that).• Experiment with volumetric obscurance when done to

see how good/bad that is...

What We Will Learn By ReImplementing the MSSAO AlgorithmWhat We Will Learn By ReImplementing the MSSAO Algorithm


To approximate integration, we sum the sample contributions and divide by their number.

Remember: We Sample in Eye SpaceRemember: We Sample in Eye Space

So all samples are on the surface

We ignore the factor 1/.

We’ll revisit the full AO definition later.


• The idea will be presented in 3 ways:

• In words: a bit verbose but accurate...

• Diagrammatically: via diagrams based on (but not the same as) Hoang and Low’s

slides.

• In code: a piece of code with much less

detail that might give you a better perspective...

How Its DoneHow Its Done


• The standard drawing pass uses MRT to create 2 render targets; a position buffer in camera coordinates and a normal buffer... (a buffer of attributes is referred to as a g-buffer; short for geometry buffer – predates the notion of geometry shaders which isn’t used here).

• Draw successive full screen quads to ping pong the position/normal textures into lower and lower resolution textures again using MRT (a simple “median downsample” shader).

• On your way back up from lower resolution levels, compute an AO texture using a diamond grid filter and combine with a lower level AO texture (if there is one) with an additional blur pass. Note to Wilf... Make blur optional.

• For the last pass, use poisson filter and no blur...

How Its Done: In WordsHow Its Done: In Words


How it’s Done: Diagrammatically using 3 levels...How it’s Done: Diagrammatically using 3 levels...

Initial Datag-buffer (1024x1024)

Downsampleg-buffer (512x512)

g-buffer (256x256)

Compute AO #1

Blur

g-buffer (1024x1024)

AO buffer (256x256)

AO buffer (512x512)

AO buffer (1024x1024)

Blur AO buffer (256x256)

Blur AO buffer (512x512)

Compute A0 #2 Combined with Blurred AO #1

Blur

Compute A0 #3 2 Combined with Blurred AO #2

poisson filter

diamond grid filter

diamond grid filter

No Blur in last stage


#define DownsampleGBuffer DownsampleRender2GBuffer (); //Hoang and Low’s name...

for (int index = 1, size = RESOLUTION / 2; index < LEVEL_COUNT; ++index, size /= 2) {DownsampleGBuffer (size, index); //Each step decreases resolution...

} for (int index = LEVEL_COUNT - 1, size = minResolution; index >= 0; --index, size *= 2) {

RenderAO (size, index); //Each step increases resolution.}

How Its Done: In CodeHow Its Done: In Code

Index (LEVEL_COUNT) strictly decrementing for downsampling g-buffer and strictly incrementing for building AO buffer bottom up...,


Combining 5 Levels Into ONE FINAL RESULTCombining 5 Levels Into ONE FINAL RESULT

Each one ½ the resolution of previous one


Blizzard (noisier) MSSAO

Comparison With Other TechniquesComparison With Other TechniquesComparison With Other TechniquesComparison With Other Techniques


Horizon-based AO (noisier) MSSAO

Comparison With Other TechniquesComparison With Other Techniques


Volumetric Obscurance MSSAO

Comparison With Other TechniquesComparison With Other Techniques


Ground Truth Comparison With Non-Real-time TechniqueGround Truth Comparison With Non-Real-time Technique

Blender MSSAO


Blizzard HBAO MSSAO

Summarizing: Less noise and blur and better high-frequency detailsSummarizing: Less noise and blur and better high-frequency details


Performance ResultsPerformance Results

MSSAO VAO Blizzard HBAO

Sibenik Cathedral

21.9 ms 22.9 25.7 50.1

Conference Room

24.0 ms 24.8 24.9 49.5

Sponza Atrium

22.2 ms 24.0 28.9 54.3

• Scenes rendered at 1024x1024 on GeForce GTX 460M

• Exclusive of geometry pass


Let’s Attack The Problem in StagesLet’s Attack The Problem in Stages

• Start with a brief look at the code (including simple extensions I made).

• Have a look at the framebuffer creation code including the initial compiling of the shaders (and how we’ll simplify that).

• Deal with the simpler aspects first; namely the downsampling code and shader (DownsampleGBuffer) and the blur code and shader (Blur).

• Deals with the more complex code dealing with AO processing (RenderAO).

• Temporal coherance (a minor aspect).Main Focus


95.450195.4501

Looking at the Code


inline void detectOpenGLError () {//WILF ADDITION...GLenum code = glGetError ();if (code != GL_NO_ERROR) {

printf ("\nEncountered error %x \"%s\".", code, gluErrorString (code));

}}

Addition To Find Out Why Original Demo Wouldn’t RunAddition To Find Out Why Original Demo Wouldn’t Run

• Implemented detectOpenGLError () and peppered code with it.

//varying out vec4 pos; //wilfout vec4 pos;

• Shader “geometry.vert” wasn’t compiling.


//WILF VERSION EXPERIMENTS...#define ORIGINAL_VERSION 0#define WILF_VERSION 1

//#define WILF_version ORIGINAL_VERSION#define WILF_version WILF_VERSION#define LEVEL_COUNT 5 // number of mip-map levels.

//NOTE: ORIGINAL_VERSION needs to be 3 or more...

Additions For Switching Between ImplementationsAdditions For Switching Between Implementations


• Shader #include is NOT part of this OpenGL. Added includePreprocessor to “shader.cpp”.

Additions To Support One Shader Include FileAdditions To Support One Shader Include File

GLuint LoadShader(EShaderType type, char *file_name) { printf ("Load shader \"%s\".\n", file_name); char *temp = ReadFile(file_name); temp = includePreprocessor (temp); //WILF const char *shader_text = temp; “Code to compile shader and return handle to it.”}


• Looks Like it Used to Run from command line via “xxx.bat”. Took that out and instrumented it so one of 3 supplied models could be run.

Modification To MainModification To Main

enum ModelType {UseSponza, UseSibenik, UseConference};

ModelType modelType = UseSponza; //WILF


95.450195.4501

Framebuffer

Creation Code


// buffers and textures ([0] = finest resolution)GLuint frameBufs [LEVEL_COUNT];GLuint depthBufs [LEVEL_COUNT];GLuint posTex [LEVEL_COUNT];GLuint normTex [LEVEL_COUNT];GLuint aoTex [LEVEL_COUNT];GLuint aoTexBlur [LEVEL_COUNT]; GLuint lastFrameAOTex;GLuint lastFramePosTex;

DeclarationsDeclarations

All handles

Took out “randRotTex” since not used by any shader.


void SetupFBOs () {//WILF MADE SHORTER... #define initializeColorTexture(name,size) \

glBindTexture (GL_TEXTURE_RECTANGLE, name); \glTexImage2D (GL_TEXTURE_RECTANGLE, 0, GL_RGBA32F, size, size, 0, GL_RGBA, GL_FLOAT, NULL); \glTexParameterf (GL_TEXTURE_RECTANGLE, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); \glTexParameterf (GL_TEXTURE_RECTANGLE, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); \glTexParameteri (GL_TEXTURE_RECTANGLE, GL_TEXTURE_MAG_FILTER, GL_NEAREST); \glTexParameteri (GL_TEXTURE_RECTANGLE, GL_TEXTURE_MIN_FILTER, GL_NEAREST);

#define initializeDepthTexture(name,size) \glBindRenderbuffer (GL_RENDERBUFFER, name); \glRenderbufferStorage (GL_RENDERBUFFER, GL_DEPTH_COMPONENT, size, size);

#define attachColorTexture(which,name) \glFramebufferTexture2D (GL_FRAMEBUFFER, which, GL_TEXTURE_RECTANGLE, name, 0);

#define attachDepthTexture(name) \glFramebufferRenderbuffer (GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, name);

#define errorCheck(name) \if (glCheckFramebufferStatus (GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE) \ printf ("Framebuffer created successfully.\n"); \else \ printf ("Framebuffer created unsuccessfully. Error code: %d.\n", glCheckFramebufferStatus (name));

#undef initializeColorTexture#undef initializeDepthTexture#undef attachDepthTexture#undef attachColorTexture#undef errorCheck(name

}

Frame Buffers SetupFrame Buffers Setup

code


void SetupFBOs () {//WILF MADE SHORTER...

glGenFramebuffers (LEVEL_COUNT, frameBufs); glGenRenderbuffers (LEVEL_COUNT, depthBufs);

glGenTextures (LEVEL_COUNT, posTex); glGenTextures (LEVEL_COUNT, normTex);glGenTextures (LEVEL_COUNT, aoTex); glGenTextures (LEVEL_COUNT, aoTexBlur);glGenTextures (1, &lastFrameAOTex); glGenTextures (1, &lastFramePosTex);

initializeColorTexture (lastFrameAOTex, RESOLUTION); initializeColorTexture (lastFramePosTex, RESOLUTION);

for (int i = 0, size = RESOLUTION; i < LEVEL_COUNT; ++i, size /= 2) { initializeColorTexture (posTex [i], size); initializeColorTexture (normTex [i], size);initializeColorTexture (aoTex [i], size); initializeColorTexture (aoTexBlur [i], size);

initializeDepthTexture (depthBufs [i], size);

glBindFramebuffer (GL_FRAMEBUFFER, frameBufs [i]);attachDepthTexture (depthBufs [i]);attachColorTexture (GL_COLOR_ATTACHMENT0, posTex [i]);attachColorTexture (GL_COLOR_ATTACHMENT1, normTex [i]);attachColorTexture (GL_COLOR_ATTACHMENT2, aoTex [i]);attachColorTexture (GL_COLOR_ATTACHMENT3, aoTexBlur [i]);if (i == 0) { attachColorTexture (GL_COLOR_ATTACHMENT4, lastFrameAOTex); attachColorTexture (GL_COLOR_ATTACHMENT5, lastFramePosTex);}errorCheck (frameBufs [i]);

}}

Frame Buffers SetupFrame Buffers Setup


glPushMatrix(); camera.ApplyCameraTransform ();

Render2GBuffer (); DownsampleGBuffer LOOP in decreasing resolutionRenderAO LOOP in increasing resolution

glPopMatrix();

if (oddFrame)glBindTexture(GL_TEXTURE_RECTANGLE, aoTex[0]);

elseglBindTexture(GL_TEXTURE_RECTANGLE, lastFrameAOTex);

oddFrame = !oddFrame;glCopyTexSubImage2D (GL_TEXTURE_RECTANGLE, 0, 0, 0, 0, 0,

RESOLUTION, RESOLUTION);

glutSwapBuffers();

Main Rendering TaskMain Rendering Task

Optional

temporal c

ohesion


95.450195.4501

Setting Up The

Shaders


3 Groups of Shaders3 Groups of Shaders

SetupGeometryProgram (initial MRT textures)

geometry.vert geometry.frag

ortho.vertdownsample.frag

SetupDownsamplePrograms (lower resolution MRT textures)

SetupAOPrograms (upsampling into AO texture)ortho.vertaoWilf.frag MRT has a position texture

+ normal texture

Render2GBuffer

DownsampleGBuffer

RenderAO


• Easier to look at actual code BUT we made one important design change.

Original version: Used 3 different AO shaders

aoLast.frag ao.frag aoFirst.frag

Setting Up The AO (Upsampling) ShadersSetting Up The AO (Upsampling) Shaders

lowest resolutionmiddle resolutionshighest resolution

• As mentioned on previous slide, we use one

aoWilf.frag


• Added 3 shader variables to aoWilf.frag

To Support the Use of JUST ONE SHADERTo Support the Use of JUST ONE SHADER

struct ShaderSetting {bool usePoisson; bool useUpsampling; bool useTemporalSmoothing;

};

ShaderSetting computeShaderSetting (long levelCount) {ShaderSetting setting;setting.usePoisson = levelCount == 0; //highest is 0setting.useUpsampling = levelCount < (LEVEL_COUNT - 1) //lowest

&& LEVEL_COUNT > 1;setting.useTemporalSmoothing = true; //or falsereturn setting;

}

• Created a struct in .cpp code

uniform bool usePoisson; uniform bool useUpsampling; uniform bool useTemporalSmoothing;

Now look at actual codeNote to Wilf... Make blur optional.


95.450195.4501

Drawing the INITIAL

MRT Textures

(Render2GBuffer)


GLenum bufs01[2] = {GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1};GLenum bufs51[2] = {GL_COLOR_ATTACHMENT5, GL_COLOR_ATTACHMENT1};

void Render2GBuffer () { glUseProgram (geometryProg); glBindFramebuffer (GL_FRAMEBUFFER, frameBufs[0]); glDrawBuffers (2, oddFrame ? Bufs01 : bufs51); glPushAttrib (GL_VIEWPORT_BIT); glViewport (0, 0, RESOLUTION, RESOLUTION); glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); DrawModel (); glPopAttrib (); }

Gbuffer Rendering (INITIAL MRT Textures)Gbuffer Rendering (INITIAL MRT Textures)

0: posTex [i]1: normTex [i])5: lastFramePosTex

All we draw

Switching back and forth is for optional temporal cohesion

USE MRT

first shader (geometry.vert + ...frag”


• Geometry.vert

G-Buffer Drawing ShaderG-Buffer Drawing Shader

#version 140out vec4 pos; //varying out vec4 pos; //wilf

void main() {pos = gl_ModelViewMatrix * gl_Vertex;gl_Position = ftransform (); }

• Geometry.frag

#version 140#extension GL_ARB_texture_rectangle : require

in vec4 pos; out vec4 Pos; out vec4 Norm;

void main() {Pos = pos / pos.w;//WILF BUG FIX: On my ATI card, one of the differentials has the //wrong sign (needs a card independent solution to fix this properly), vec3 n = -cross (dFdx(pos.xyz), dFdy(pos.xyz));Norm = vec4(normalize(n), sign(abs(pos.z))); }

Differentials Need An Explanation

0 in 4th component if z is 0

MRT


• dFdx gives the change of the function with respect to x... (the slope)

DifferentialsDifferentials

f f2xyz – f1xyz fx, fy, fz

x x x x xf1

f2= =

• The direction of greatest x change can be interpreted as a good direction for an x-axis.

• Similarly, dFdy is a good direction for a y-axis.


• Consequently, with dFdx and dFdy both on the surface, their cross product is perpendicular to the surface

DifferentialsDifferentials

is normal to the surface

fx

fy

fx

fy


95.450195.4501

Drawing the LOWER

RESOLUTION MRT Textures

(DownsampleGBuffer)


• Hoang and Low claim to be using the median of the position and normals.

Downsizing to a Lower Resolution G-Buffer Downsizing to a Lower Resolution G-Buffer

• Definition (modified from Wikipedia version): A median of n numbers is a middle value (IF THEY WERE SORTED). When n is odd, the median is unique; otherwise, there are 2 candidates (often, the average is picked to resolve the issue).

• Example:

Median of 1,3,7,100,101 is 7.

Median of 1,2,3,4 is 2 or 3 (or 2.5).


Downsampling; e.g. 1024x 1024 to 512x512Downsampling; e.g. 1024x 1024 to 512x512

𝑝1 𝑝2

𝑝3 𝑝4

𝑝

Given p1.z p2.z p3.z p4.z

If (p4.z – p1.z dmax)p (p2+p3)/2; n (n2+n3)/2;

elsep p2; n n2;

After sorting by deph, pick the median (AVERAGE ONLY IF the Z-SPAN is within the radius of influence dmax).


void Downsample (int size, int index) {//Note that index is 1 or more; index – 1 is previous one).glUseProgram (downsampleProg [index]);

glActiveTexture (GL_TEXTURE0);if (index == 1 && !oddFrame)

glBindTexture (GL_TEXTURE_RECTANGLE, lastFramePosTex); else

glBindTexture (GL_TEXTURE_RECTANGLE, posTex [index - 1]);

glActiveTexture (GL_TEXTURE1);glBindTexture( GL_TEXTURE_RECTANGLE, normTex [index - 1]);

glBindFramebuffer(GL_FRAMEBUFFER, frameBufs [index]);glDrawBuffers (2, bufs01);

}

The Downsizing Code for the Lower Res G-BufferThe Downsizing Code for the Lower Res G-Buffer

Optional

temporal c

ohesion

Inputs: position + normal

Code to draw a quad (see next slide)outputs in the frame buffer (2 outputs)


glPushAttrib (GL_VIEWPORT_BIT);glViewport (0, 0, size, size);glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);glBegin (GL_QUADS); //WILF FIX: Need vertices counter-clockwise

glVertex2d (0, 0); //start at bottom leftglVertex2d (size, 0); //go rightglVertex2d (size, size); //go upglVertex2d (0, size); //go left

glEnd ();glPopAttrib ();

The Downsizing CodeThe Downsizing Code

This is the code to draw a quad...

Note that original code was clockwise which works ONLY if face culling is turned off…


The Downsizing ShaderThe Downsizing Shader

[i,j]

0 1 2 30 1 2 3 4 5 6 7

[2i, 2j] Note that red pixels are

+1 or –1 away from center


• Recall: AO G-Buffer uses eye space position and pixel normal (one hint for better behavior).

• Hoang and Low uses the median (another hint) of the position and normals (averages only if within radius of influence).

• When averaging normals, does not normalize.

Summarizing Downsizing DetailsSummarizing Downsizing Details

• Claim: using the this median method helps reduce self-occlusion artifacts and provides better temporal coherence.

Let’s look at the shader code


• ortho.vert

G-Buffer Downsizing ShaderG-Buffer Downsizing Shader

#version 140

uniform mat4 gluOrtho; //wilf: orthographic

void main() {gl_Position = gluOrtho * gl_Vertex;}

• downsample.frag

#version 140

#extension GL_ARB_texture_rectangle: enable

uniform sampler2DRect hiResNormTex;uniform sampler2DRect hiResPosTex;

out vec4 Norm; out vec4 Pos;

Higher resolution textures

main (next slide)


G-Buffer Downsizing ShaderG-Buffer Downsizing Shader

void main() {

vec2 xy2 = gl_FragCoord.xy * 2.0; vec4 pos[4]; vec4 norm[4];

pos[0] = texture2DRect (hiResPosTex, xy2 + vec2(-1.0, 1.0)); //WILF: Change each 0.5 to 1.0. pos[1] = texture2DRect (hiResPosTex, xy2 + vec2(1.0, 1.0));pos[2] = texture2DRect (hiResPosTex, xy2 + vec2(1.0, -1.0));pos[3] = texture2DRect (hiResPosTex, xy2 + vec2(-1.0, -1.0));

norm[0] = texture2DRect (hiResNormTex, xy2 + vec2(-1.0, 1.0));norm[1] = texture2DRect (hiResNormTex, xy2 + vec2(1.0, 1.0));norm[2] = texture2DRect (hiResNormTex, xy2 + vec2(1.0, -1.0));norm[3] = texture2DRect (hiResNormTex, xy2 + vec2(-1.0, -1.0));

float maxZ = max (max (pos[0].z, pos[1].z), max (pos[2].z, pos[3].z));float minZ = min (min (pos[0].z, pos[1].z), min (pos[2].z, pos[3].z));

int minPos, maxPos;

for (int i = 0; i < 4; ++i) {if (pos[i].z == minZ) minPos = i; if (pos[i].z == maxZ) maxPos = i;

}

float d = distance(pos[minPos].xyz, pos[maxPos].xyz); ivec2 median = ivec2 (0, 0); int index = 0;for (int i = 0; i < 4 && index < 2; ++i)

if (i != minPos && i != maxPos) median [index++] = i;

if (d < 1.0) {Pos = (pos [median.x] + pos [median.y]) / 2.0; Norm = (norm [median.x] + norm [median.y]) / 2.0;

} else {Pos = pos [median.x]; Norm = norm [median.x];

}}


95.450195.4501

Blurring


• Uses a gaussian blur... with weights computed at setup time...

• This shader (ortho.vert + blur.frag) has been untouched…

BlurBlur


• Uses a VARIATION OF A gaussian blur... with weights computed at setup time...

• This shader (ortho.vert + blur.frag) has been untouched…

BlurBlur

This has a special name which we will reveal shortly


The Blur ShaderThe Blur Shader#version 140#extension GL_ARB_texture_rectangle: enable#extension GL_EXT_gpu_shader4: enable

uniform sampler2DRect aoTex; uniform sampler2DRect normTex; uniform sampler2DRect posTex;out vec3 AO;

void main () {vec3 n = texture2DRect (normTex, gl_FragCoord.xy).xyz;vec3 p = texture2DRect (posTex, gl_FragCoord.xy).xyz;

vec3 ss = vec3 (0.0); float weight = 0.0;

for (float i = -1.0; i <= 1.0; i += 1.0) {for (float j = -1.0; j <= 1.0; j += 1.0) {

vec2 ij = vec2(i, j);vec3 t = texture2DRect(aoTex, gl_FragCoord.xy + ij).xyz;vec3 norm = texture2DRect(normTex, gl_FragCoord.xy + ij).xyz;float depth = texture2DRect(posTex, gl_FragCoord.xy + ij).z;

float normWeight = (dot (norm, n) + 1.2) / 2.2;normWeight = pow (normWeight, 8.0);

float depthWeight = 1.0 / (1.0 + abs(p.z - depth) * 0.2);depthWeight = pow(depthWeight, 16.0);

float gaussianWeight = 1.0 / ((abs(i) + 1.0) * (abs(j) + 1.0));

weight += normWeight * depthWeight * gaussianWeight;ss += t * normWeight * depthWeight * gaussianWeight;

}}

AO = vec3 (ss / weight);}

1 when equal, less otherwise

1 when equal, less otherwise

1 when i=j=0; else ½, ¼.


• It’s called a bilinear filter (although it might also make sense to call it a trilinear filter)…

• A bit about the history of bilinear filters coming up.

What Kind Of Filter Was That What Kind Of Filter Was That


95.450195.4501

Bilinear Filters


• Upsamples source using context around it...

Bilateral Upsampling Introduced in 2007Bilateral Upsampling Introduced in 2007

Joint Bilateral Upsampling, Kopf et al, SIGGRAPH 2007.

Joint Bilateral Upsampling, Kopf et al, SIGGRAPH 2007.

Nearest neighbor upsampling Bicubic upsampling

Upsampled result Gaussian upsampling Joint bilateral upsampling


source context

Intuition: Two Filters (weights) Instead of OneIntuition: Two Filters (weights) Instead of One

Sample (p) = 1

kp q

∑ Sq f(||p-q||) g(||Ip-Iq||)

Normal filtering 1

kp q

∑ Sq W(||p-q||)

Sample at q

Filter kernel support (filter coordinates)

Weight depends on distance to reference

Normalizing factor (sum of weights)

Essentially multiply the 2 weights together


High resolution low resolution

This Special ApplicationThis Special Application

• Combining a high res image with a low res version that contains more distant information in the same size filter.

• It is a “natural” edge preserving filter... (one of our hints for obtaining a better result).

Like using two 3x3 filters instead of one 6x6 filter


The Formalism looks like this (which we’ll ignore)The Formalism looks like this (which we’ll ignore)

Upsamplep = 1

kp q

∑ Lq f(||p-q||) g(|Hp-Hq||)

Low resolution L

Let p and q denote integer coordinate of

pixels in H

Let p and q denote corresponding fractional coordinate of pixels in L

High resolution HUpsampled

version


Bilateral Upsampling in MSSAO (Inventor’s Slides)Bilateral Upsampling in MSSAO (Inventor’s Slides)

• Filter weights wf (pi)

• Depth weights wz

wz(pi) = ( )• Normal weights wn

wn(pi) = ( )

tz1

1 + |zi – z|

tn n.ni + 1

2

Low def p1, p2, p3, p4 for high def p tz = 16, tn = 8

Basic idea: make weights go to 0 as

samples differ more (fraction < 1)

AO (p) = wf (pi) wz(pi) wn(pi) AO (pi) ∑ l = 1

4


• We’re done with bilinear filters but not done with filters (we need yet another type).

• Why not have a quick look at some well known filters.

Since We Need Another Filter Type… Since We Need Another Filter Type…

Switch to slides “#02bFilters.ppt”


Back to Our 3 Shader GroupsBack to Our 3 Shader Groups

SetupGeometryProgram (initial MRT textures)

geometry.vert geometry.frag

ortho.vertdownsample.frag

SetupDownsamplePrograms (lower resolution MRT textures)

SetupAOPrograms (upsampling into AO texture)ortho.vertaoWilf.frag

MRT has a position texture + normal texture

Seen this one

Seen this one

Let’s look at this one

Render2GBuffer

DownsampleGBuffer

RenderAO


95.450195.4501

Upsampling into

an AO texture(RenderAO)


#version 140#extension GL_ARB_texture_rectangle: enable#extension GL_EXT_gpu_shader4: enable

#include "wilfAmbientOcclusionLibrary.all"

uniform sampler2DRect normTex;uniform sampler2DRect posTex;//And a host of other variables (not shown)…

uniform bool usePoisson; uniform bool useUpsampling; uniform bool useTemporalSmoothing;

out vec4 AO;

//Local variables accessed by 3 functions highResolutionOcclusion,//lowResolutionOcclusion, temporallyBlendedOcclusion...vec2 uv; vec4 basePosition; vec3 baseNormal; //To be filled in...

//3 functions and main (shown next)…

Initial RenderAOInitial RenderAO

We use one shader aoWilf.frag rather than 3: aoLast.frag, ao.frag, aoFirst.frag


//It's convenient for 2 occlusion functions below to return the result in pieces;//namely vec3 (occlusion / occlusionCount, occlusion, occlusionCount)...

vec3 highResolutionOcclusion () {return usePoisson //Fills in basePosition, baseNormal…

? poissonFilterAmbientOcclusion (...): gridFilterAmbientOcclusion (...);

}

vec3 lowResolutionOcclusion () {return (useUpsampling)

? vec3 (0, 0, 0); //edgePreserveUpsample (...): vec3 (0, 0, 0); //None...

}

float temporallyBlendedOcclusion (float occlusion) {return useTemporalSmoothing

? occlusion //temporalBlend (...): occlusion;

}


We will shortly override to INITIALLY use simpler

routines

Effectively “do nothing”for now.

Effectively “do nothing”for now.


void main () { uv = gl_FragCoord.xy; //Initialize the variable containing the texture

coordinates...

//Sample format: vec3 (occlusion / occlusionCount, occlusion, occlusionCount)... vec3 sample1 = highResolutionOcclusion (); vec3 sample2 = lowResolutionOcclusion ();

float maximum = max (sample1.x, sample2.x); float sum = sample1.y + sample2.y;float count = sample1.z + sample2.z;

if (usePoisson) {//Top levelfloat weightedAverage = sum / count;float ambientOcclusion = (1.0 - maximum) * (1.0 - weightedAverage); AO = vec4 (temporallyBlendedOcclusion (ambientOcclusion));

} else {AO = vec4 (maximum, sum, count, 0.0);

}}


Since we are drawing; high occlusion 1 is meant to be black (0) and low occlusion 0 is meant to be white (1) (flip via 1 - occlusion).

AO here is really AL (ambient lighting)

Convert to gray scale (all 4 components the same)

Flip


The LibraryThe Library

//Wilf's library for ambient occlusion experiments...

//interpolate (a, b, t) gives a for t = 0 and b for t = 1 and blends otherwise...

#define interpolate mix

#define clamp01(x) clamp (x, 0.0, 1.0)

Let’s look at the first routine for computing ambient occlusion


Compute ambient occlusion as “high occlusion if far, low occlusion if near”.

To Get Started With The LibraryTo Get Started With The Library

//Return the result in pieces (occlusion / occlusionCount, occlusion, occlusionCount)...

vec3 HACKambientOcclusion (vec2 uv, out vec4 basePosition, out vec3 baseNormal, sampler2DRect positionTexture, sampler2DRect normalTexture) {

basePosition = texture2DRect (positionTexture, uv);baseNormal = texture2DRect (normalTexture, uv).xyz;

//Compute t = 0 => close when z = 0; t = 1 => far when z = 100.float t = clamp01 (abs (basePosition.z) / 100.0);

float sampleOcclusion = t; //0 => close; 1 => farfloat sampleCount = 1.0;return vec3 (sampleOcclusion / sampleCount, sampleOcclusion, sampleCount);

}

Back to Visual Studio to invoke it in highResolutionOcclusion


ResultsResults


• Recall: As part of the AO definition which we’ll review next, we’ll compute

Adding More Correct Ambient Occlusion FunctionAdding More Correct Ambient Occlusion Function

float squaredLength (vec3 vector) {return dot (vector, vector);}

float squaredDistance (vec3 point1, vec3 point2) {return squaredLength (point1 - point2);}

float linearDistanceRatio (vec3 distanceVector, float dMaximum) {

float d = length (distanceVector);

return min (d / dMaximum, 1.0); //Clamp at 1 if bigger than dMaximum...

}

float squaredDistanceRatio (vec3 distanceVector, float dMaximum) {

float dSquared = squaredLength (distanceVector);

return min (dSquared / (dMaximum * dMaximum), 1.0); //Clamp at 1 if past dMaximum...

}



Technically It’s A Volume ContributionTechnically It’s A Volume Contribution

Occlusion is cosine-weighted fraction of a tangent hemisphere

Minimal occlusion = 0 (no contribution, denoted ) for sample at or outside radius of influence or on negative side of normal)


At base point p, di is 0.So we’ll get 0 occlusionif we use the base point

The AO Definition


The Definition Based AO FunctionThe Definition Based AO Function

float ambientOcclusion (vec4 basePosition, vec3 baseNormal, vec4 position, vec4 normal, float cameraSpaceRadius) {//AO = ambientOcclusion

//Compute AO of {position, normal} relative to {basePosition, baseNormal}.//Note that normal is a vec4 (4th component 0 means distance to eye is 0)...

//Compute the direction vector from the base point to the other point.vec3 distanceVector = position.xyz - basePosition.xyz;

//Compute the distance ratio clamped at dMaximum...float distanceRatio = linearDistanceRatio (distanceVector, cameraSpaceRadius);

//As the ratio goes from 0 to 1, go from 1 to 0 instead.float occlusionFactor = 1.0 - distanceRatio;

//Need POSITIVE ONLY cosine contribution. Note A.B = |A| |B| cos angle if |A| = |B| = 1...float cosine = max (dot (baseNormal, normalize (distanceVector)), 0.0);return occlusionFactor * cosine * normal.w; //Use 0 for a point at the near plane...

}


Recall: LAST LINE OF Geometry.frag for creating MRT texture.… out vec4 Pos; out vec4 Norm; …vec3 n = -cross (dFdx(pos.xyz), dFdy(pos.xyz));Norm = vec4(normalize(n), sign(abs(pos.z))); } 0 in 4th component if z

is 0; 1 otherwise


Version with Samplers Instead of SamplesVersion with Samplers Instead of Samples

float ambientOcclusion (vec2 uv, vec4 basePosition, vec3 baseNormal, sampler2DRect positionTexture, sampler2DRect normalTexture, float cameraSpaceRadius) {

//Version with samplers instead of {position, normal}...

vec4 position = texture2DRect (positionTexture, uv);vec4 normal = texture2DRect (normalTexture, uv);

return ambientOcclusion (basePosition, baseNormal, position, normal, cameraSpaceRadius);

}

Back to Visual Studio

Note: with 1 probe, if position == basePosition, di will be 0

So occlusion will be 1 which will draw as 1-1 (black)



Result: As Expected, Occlusion is 0Result: As Expected, Occlusion is 0


Making a Small ChangeMaking a Small Change

uv += 1; //So sample point is right + up from base uv...


• One probe is not enough... But there are 2 issues to resolve.

• How many probes to take. Hoang and Low take 16 probes per sample.

• At the sample point, the probe in camera space has a fixed radius of influence called “cameraSpaceRadius”. We probe a full screen texture and so we need the corresponding “pixelSpaceRadius”.

But We Want A Poisson FilterBut We Want A Poisson Filter

Note: By sampling at the same resolution far away as close by, discontinuities are sharp at all distances.


const float dMax = 2.5; //Radius of influence in camera space (meters).

const float rMax = 7.0; //Radius of influence in pixel space (pixels).

const float fovFactor; //To be derived (and to show why needed).

Variables Actually Initialized At Shader Setup TimeVariables Actually Initialized At Shader Setup Time

dMax (meters)rMax (pixels)


float pixelFromCameraSpace (float cameraSpaceRadius, float resolution, float fovFactor, float depth) {return (resolution * fovFactor / depth) * cameraSpaceRadius;

}

All We Want Is A Simple Conversion RoutineAll We Want Is A Simple Conversion Routine

Let fovFactor = 0.5 / tan (/2) where is field of view

Slightly different from original version which plugged the resolution INTO the factor and called it

resolutionDependentFactor


3 Spaces Involved for RADIUS: Camera/Screen/Pixel Spaces3 Spaces Involved for RADIUS: Camera/Screen/Pixel Spaces

rscreenSpacercameraSpace

n = focal length (near)

= fov (field of view)

h

CAMERA SPACE: in meters

Consider radius relative to center point (more exact result not useful).

SCREEN SPACE: 0 to 1 units

PIXEL SPACE: 0 to 1024 units(related to half RESOLUTION

and therefore )


Differences between Camera/Screen/Pixel SpacesDifferences between Camera/Screen/Pixel Spaces




hrpixelSpace

How are rpixelSpace, rscreenSpace and rcameraSpace related?

dMax(meters)rMax(pixels)

If sample is very close to screen rpixekSpace can exceed a rMax.


Step 1: Camera/Screen SpacesStep 1: Camera/Screen Spaces




hrpixelSpace

By similar triangles, rscreenSpace/n = rcameraSpace/|z|.

|z|

rscreenSpace = (n/|z|) * rcameraSpace


Derivation: h screen space units = R/2 pixels

1 screen space unit = (R/2)/h pixels

But h/n = tan (/2)

So 1 screen space unit = (R/2)/(n tan (/2)) pixels

So r screen space unit = (R/2)/(n tan (/2)) * r pixels

Step 2: Screen/Pixel SpacesStep 2: Screen/Pixel Spaces


n

h

Let R be resolution, h be half screen size.

rpixelSpace = (R/2)/(n tan (/2)) * rscreenSpace


Step 3: Everything put togetherStep 3: Everything put together

rscreenSpace = (n/|z|) * rcameraSpace

rpixelSpace = (R/2)/(n tan (/2)) * rscreenSpace

rpixelSpace = (R/2)/(n tan (/2)) * (n/|z|) * rcameraSpace

Let fovFactor = 0.5 / tan (/2)

rpixelSpace = (R fovFactor / |z|) * rcameraSpace

float pixelFromCameraSpace (float cameraSpaceRadius, float resolution, float fovFactor, float depth) {return (resolution * fovFactor / depth) * cameraSpaceRadius;

}


The Poisson FilterThe Poisson Filtervec3 poissonFilterAmbientOcclusion (vec2 uv, sampler2DRect positionTexture, sampler2DRect normalTexture, float maximumPixelSpaceRadius,

float cameraSpaceRadius, float fovFactor, float resolution, out vec4 basePosition, out vec3 baseNormal) {

//Filter via a poisson disk and return the result in pieces //i.e., (occlusion / occlusionCount, occlusion, occlusionCount)...

//We need the maximum pixel space radius for the poisson disk and the camera space//radius for the occlusion sphere's radius of influence...

const vec2 poissonDisk [16] = …

basePosition = texture2DRect (positionTexture, uv); baseNormal = texture2DRect (normalTexture, uv).xyz;

float pixelSpaceRadius = pixelFromCameraSpace (cameraSpaceRadius, resolution, fovFactor, abs (basePosition.z));

float radius = min (pixelSpaceRadius, maximumPixelSpaceRadius);

float occlusion = 0.0; for (int i = 0; i < 16; i++) {

vec2 randomUV = uv + poissonDisk [i] * radius;occlusion += ambientOcclusion (randomUV, basePosition, baseNormal,

positionTexture, normalTexture, cameraSpaceRadius);}return vec3 (occlusion * (1.0 / 16.0), occlusion, 16.0);

}

Next slide for details

Note: By sampling at the same resolution far away as close by, discontinuities are sharp at all distances.

Is this a trick?.


The Poisson FilterThe Poisson Filter

const vec2 poissonDisk [16] = vec2 [16] (

vec2 (-0.6116678, 0.04548655), vec2 (-0.26605980, -0.6445347),vec2 (-0.4798763, 0.78557830), vec2 (-0.19723210, -0.1348270),vec2 (-0.7351842, -0.58396650), vec2 (-0.35353550, 0.3798947),vec2 ( 0.1423388, 0.39469180), vec2 (-0.01819171, 0.8008046),vec2 ( 0.3313283, -0.04656135), vec2 ( 0.58593510, 0.4467109),vec2 ( 0.8577477, 0.11188750), vec2 ( 0.03690137, -0.9906120),vec2 ( 0.4768903, -0.84335800), vec2 ( 0.13749180, -0.4746810),vec2 ( 0.7814927, -0.48938420), vec2 ( 0.38269190, 0.8695006));


We now have enough to draw at the high resolution level


Using the Poisson FilterUsing the Poisson Filter


• We use a diamond grid filter (right and up for simplicity) at odd points 1, 3, 5, ... up to radius of influence.

We Need A Much Simpler Filter To Probe AO at Lower ResolutionsWe Need A Much Simpler Filter To Probe AO at Lower Resolutions

Half the probes of the poisson filter...


The Diamond Grid FilterThe Diamond Grid Filtervec3 gridFilterAmbientOcclusion (vec2 uv, sampler2DRect positionTexture,

sampler2DRect normalTexture, float maximumPixelSpaceRadius, float cameraSpaceRadius, float fovFactor, float resolution,out vec4 basePosition, out vec3 baseNormal) {

//Sampling right and up at all odd pixel distances; i.e., 1, 3, 5, ... to the limit allowed//and return the pieces (occlusion / occlusionCount, occlusion, occlusionCount)...

basePosition = texture2DRect (positionTexture, uv); baseNormal = texture2DRect (normalTexture, uv).xyz;

float pixelSpaceRadius = pixelFromCameraSpace (cameraSpaceRadius, resolution, fovFactor, abs (basePosition.z));

float radius = min (pixelSpaceRadius, maximumPixelSpaceRadius);

So far, exactly the same as the poisson filter


The Diamond Grid FilterThe Diamond Grid Filter

int occlusionCount = 0; float occlusion = 0.00001; //Prevent divide by 0 below...

for (float x = 1.0; x <= radius; x += 2.0) {for (float y = 1.0; y <= radius; y += 2.0) {

occlusion += ambientOcclusion (uv + vec2 (x, y), basePosition, baseNormal, positionTexture, normalTexture, cameraSpaceRadius);

occlusion += ambientOcclusion (uv + vec2 (-x, y), basePosition, baseNormal, positionTexture, normalTexture, cameraSpaceRadius);

occlusion += ambientOcclusion (uv + vec2 (-x, -y), basePosition, baseNormal, positionTexture, normalTexture, cameraSpaceRadius);

occlusion += ambientOcclusion (uv + vec2 (x, -y), basePosition, baseNormal, positionTexture, normalTexture, cameraSpaceRadius);

occlusionCount += 4;}

}return vec3 (occlusion / occlusionCount, occlusion, occlusionCount);

}

Recall: rMax = 7.0;So x can be 1, 3, 5, 7


Need an Upsample HACK To Test This OutNeed an Upsample HACK To Test This Out

vec3 HACKedgePreserveUpsample (vec2 highRESuv, vec4 basePosition, vec3 baseNormal, sampler2DRect lowResNormTex, sampler2DRect lowResPosTex, sampler2DRect lowResAOTex) {

//lowResAOTex is stored as (occlusion / occlusionCount, occlusion, occlusionCount)...

vec2 lowResUV = floor ((highRESuv) * 0.5) + vec2 (0.5, 0.5);vec3 lowResAO = texture2DRect (lowResAOTex, loResUV).xyz;return lowResAO;

}


Divide high res by 2 + half a pixel to get low res

Note to wilf: Need to adjust “#define LEVEL_COUNT 1”.


HACK Upsampling with Different LEVEL_COUNT SettingsHACK Upsampling with Different LEVEL_COUNT Settings

1 2 3 4 5

Very small improvements (need to replace HACK…upsample by better version)


• The code is a rewrite with no changes except for merging 4 loops into one... It’s computing a weighted sum of the vec3 lower resolution AO samples; i.e.,

A Better Edge Preserving BILATERAL Upsampler A Better Edge Preserving BILATERAL Upsampler

Vec3 C = vec3 (0);C += sample1 * W1

C += sample2 * W2

...Return C / W

A sample is a triple{occlusion/count, occlusion, count}

float weight = normalWeight * depthWeight * (9.0 / 16.0) /

(abs((highRESuv.x - loResUV.x * 2.0) * (highRESuv.y - loResUV.y * 2.0)) * 4.0);

Each low resolution sample provided it own normalWeight

and depthWeight


It’s Based On The Following BUT uses 9/16 everywhereIt’s Based On The Following BUT uses 9/16 everywhere

• Filter weights wf (pi)

• Depth weights wz

wz(pi) = ( )• Normal weights wn

wn(pi) = ( )

tz1

1 + |zi – z|

tn n.ni + 1

2

Low def p1, p2, p3, p4 for high def p tz = 16, tn = 8

Basic idea: make weights go to 0 as

samples differ more (fraction < 1)

AO (p) = wf (pi) wz(pi) wn(pi) AO (pi) ∑ l = 1

4


The Edge Preserving UpsamplerThe Edge Preserving Upsamplervec3 edgePreserveUpsample (vec2 highRESuv, vec4 basePosition, vec3 baseNormal,

sampler2DRect loResNormTex, sampler2DRect loResPosTex, sampler2DRect loResAOTex) {

//Downsampled values should be almost equal to basePosition and baseNormal...//So weigths will be approximately 1 most of the time and 0 when they are different...

const vec2 offsets [4] = vec2 [4] (vec2 (-1.0, 1.0), vec2 ( 1.0, 1.0), vec2 (-1.0, -1.0), vec2 ( 1.0, -1.0));

float totalWeight = 0.0; vec3 combinedAO = vec3 (0.0);

for (int i = 0; i < 4; ++i) {vec2 loResUV = floor ((highRESuv + offsets [i]) * 0.5) + vec2 (0.5, 0.5);

vec3 loRESNormal = texture2DRect (loResNormTex, loResUV).xyz;float loResDepth = texture2DRect (loResPosTex, loResUV).z;vec3 loResAO = texture2DRect (loResAOTex, loResUV).xyz;

//COMPUTE normalWeight and depthWeight (see next slide).

float weight = normalWeight * depthWeight * (9.0 / 16.0) /

(abs((highRESuv.x - loResUV.x * 2.0) * (highRESuv.y - loResUV.y * 2.0)) * 4.0);

totalWeight += weight; combinedAO += loResAO * weight; }

return combinedAO / totalWeight;}


The Edge Preserving UpsamplerThe Edge Preserving Upsampler

//Compute 1 if almost equal, approaches 0 otherwise...

float normalWeight = (dot (loRESNormal, baseNormal) + 1.1) / 2.1;

normalWeight = pow (normalWeight, 8.0);

//Compute 1 if almost equal, approaches 0 otherwise...

float depthWeight = 1.0 / (1.0 + abs (basePosition.z - loResDepth) * 0.2);

depthWeight = pow (depthWeight, 16.0);



author version

Result Are Different But Hard To See HereResult Are Different But Hard To See Here

hack version


author version

Zooming InZooming In

hack version


• Now’s the time to try the simplistic volumetric occlusion technique...

Now Everything Should WorkNow Everything Should Work

float simplisticAmbientOcclusion (vec4 basePosition, vec3 baseNormal, vec4 position, vec4 normal, float cameraSpaceRadius) {

//Computes the ambient occlusion of {position, normal} //relative to {basePosition, baseNormal}.

//Note that z is negative in OpenGL.

bool furtherOut = position.z < basePosition.z;

return furtherOut ? 0.0 /*no occlusion */ : 1.0; /* full occlusion */

}

no occlusion full occlusion


One Pass Simplistic AlgorithmOne Pass Simplistic Algorithm


One Pass Sophisticated AlgorithmOne Pass Sophisticated Algorithm


5-Pass Simplistic Algorithm5-Pass Simplistic Algorithm


5-Pass Sophisticated Algorithm5-Pass Sophisticated Algorithm


• The observation that very little changes from frame to frame is called temporal coherence (TC).

• Rapid changes imply rapid movement... This can be exploited in 2 ways:

Avoid recomputing what never changes to achieve a speedup...

Use previous information to provide temporal blurring...

Temporal CoherenceTemporal Coherence


panning left

Temporal Coherence ExamplesTemporal Coherence Examples

A Survey on Temporal Coherence Methods in Real-Time Rendering, Scherzer et al, Eurographics 2011


AI walking left

green is visible in previous scene; red is new


Applications of Temporal CoherenceApplications of Temporal Coherence

• Bypass all or part of computation of complex shader whenever previous results available.


Applications of Temporal CoherenceApplications of Temporal Coherence

• Eliminate flickering artifacts by interpolating old with new in non-photorealistic rendering (which created many moving lines).


• An important decision when utilizing TC is how the previously computed data is stored, tracked, retrieved and reused.

• Given a current pixel location, determining the previous location associated with it is termed reverse reprojection.

Temporal CoherenceTemporal Coherence


Reverse ReprojectionReverse Reprojection



reverseReprojected (P1) at time t

gives P1 at time t-1 denoted t-1(P1).

inconsistent depths or offscreen indicates no solution


Sample = SampleNew (PNew).interpolate (SampleOld (POld), t)

where confidence t (how confident you are in PNew) is between 0 and 1; e.g, 0.6

Performing Reverse ReprojectionPerforming Reverse Reprojection

inverse

Use weighting t to determine how much of the new sample to use compared with the old...

at time t-1 at time t

POld = MVMatrixt-1 (MVMatrixt (PNew))


float temporalBlend (float confidence, float resolution, float ambientOcclusion, vec4 position, sampler2DRect oldPositionTexture, sampler2DRect oldAOTexture, mat4 projection, mat4 modelViewInverseNow, mat4 modelViewOld) {

//Blend ambient occlusion at position with version in old position/AO //textures using the transformations to perform the reverse reprojection... //Note that position in camera (view) space...

//Compute the old position from the model view transformations.vec4 oldPosition = modelViewOld * (modelViewInverseNow * position);

//To get screen space UV, we need to convert to perspective space which //gives coordinate in range -1 to +1...vec4 oldScreenSpacePosition = projection * oldPosition;vec2 oldUV = oldScreenSpacePosition.xy / oldScreenSpacePosition.w;

Temporal CoheranceTemporal Coherance


//float temporalBlend (...) CONTINUEDif (oldUV.x < -1.0 || oldUV.x > +1.0 || oldUV.y < -1.0 || oldUV.y > +1.0)

return ambientOcclusion; //If offscreen, return the new value...//These coordinates are from -1 to +1 (not 0 to R-1) WHERE R = resolution...oldUV += 1.0; //Now they range from +0 to +2.oldUV *= (resolution - 1.0) * 0.5; //Now from +0 to +2*(R-1)*0.5; i.e., 0 to R-1.oldUV += 0.5; //Add half a pixel to be at the center of the pixel.

//Use stored position.z which may be more accurate than oldPosition.z.float oldPositionZ = texture2DRect (oldPositionTexture, oldUV).z;if (abs (1.0 - oldPositionZ / position.z) > 0.01)

return ambientOcclusion; //Too different to use...

//Use the confidence t value to determine the blend.. ;1 is all new, 0 all old.float oldAmbientOcclusion = texture2DRect (oldAOTexture, oldUV).x;return interpolate (ambientOcclusion, oldAmbientOcclusion, confidence);

}

Temporal CoheranceTemporal Coherance


• The Alchemy Screen-Space Ambient Obscurance Algorithm, McGuire, Osman, Bukowski, Hennessay, High Performance Graphics 2011.

• Temporal Screen-Space Ambient Occlusion, Mattausch, Schernzer, and Wimmer, pp 123-141, GPU Pro 2, 2011.

• Screen Space Ambient Occlusion, Kajalin (Crysis), pp 413-424, ShaderX7, 2009.

• Image-Space Horizon-Based Ambient Occlusion (NVIDIA demo), Bavoil and Sainz, pp 425-444, ShaderX7, 2009.

ReferencesReferences


• Volumetric Obscurance, Loos and Sloan, Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, 1010.

• Multi-resolution screen-space ambient occlusion, Hoang and Low, pp. 101-102, ACM Symposium on Virtual Reality Software and Technology 2010.

• Joint Bilateral Upsampling, Kopf et al, SIGGRAPH 2007.

ReferencesReferences

Documents

Wilf LaLonde ©2012 Comp 4501 95.4501 Ambient Occlusion