Upload
signa
View
50
Download
1
Embed Size (px)
DESCRIPTION
Illumination and Shading. Lights Diffuse and Specular Illumination BasicEffect Setting and Animating Lights Shaders and HLSL Lambertian Illumination Pixel Shaders Textures. The XNA graphics pipeline. Multiply by World Matrix. Effect. Multiply by View Matrix. Vertex Shader. - PowerPoint PPT Presentation
Citation preview
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics1
Illumination and Shading
LightsDiffuse and Specular IlluminationBasicEffectSetting and Animating LightsShaders and HLSLLambertian IlluminationPixel ShadersTextures
LightsDiffuse and Specular IlluminationBasicEffectSetting and Animating LightsShaders and HLSLLambertian IlluminationPixel ShadersTextures
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics2
The XNA graphics pipelineMultiply by
World Matrix
Multiply by View Matrix
Homogenize and Clip
Multiply byProjection Matrix
Compute Color
Shade
Effect
Vertex Shader
Pixel Shader
Vertex positionVertex normalTexture coordinateVertex colorother stuff sometimes…
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics3
Some Terms
Illumination – What gives a surface its color. This is what our effect will compute.
Material – Description of the surface. Will include one or more colors. These are parameters set in the effect.
Reflection – The reflection of light from a surface. This is what we will simulate to compute the illumination.
Shading – Setting the pixels to the illumination
We’ll often use reflection and illumination just about interchangeably.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics4
Effects/Shaders
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics5
My First Effect: FirstEffect.fxfloat4x4 World;float4x4 View;float4x4 Projection;
struct VertexShaderInput{ float4 Position : POSITION0;};
struct VertexShaderOutput{ float4 Position : POSITION0;};
VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output;
float4 worldPosition = mul(input.Position, World); float4 viewPosition = mul(worldPosition, View); output.Position = mul(viewPosition, Projection);
return output;}
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return float4(1, 0, 0, 1);}
technique FirstShader{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderFunction(); }}
This is the default effect Visual Studio will create when you do New/Effect. It does nothing except set pixels to red.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics6
What this does
Anywhere there is something to draw, it draws red
Always something to draw other than through the window.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics7
How to use this… firstEffect = Content.Load<Effect>("FirstEffect");
foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (ModelMeshPart part in mesh.MeshParts) { part.Effect = firstEffect; } }
protected void DrawModel(GraphicsDevice graphics, Camera camera, Model model, Matrix world) { Matrix[] transforms = new Matrix[model.Bones.Count]; model.CopyAbsoluteBoneTransformsTo(transforms);
foreach (ModelMesh mesh in model.Meshes) { foreach (Effect effect in mesh.Effects) { effect.Parameters["World"].SetValue(transforms[mesh.ParentBone.Index] * world); effect.Parameters["View"].SetValue(camera.View); effect.Parameters["Projection"].SetValue(camera.Projection); } mesh.Draw(); } }
Loading/Installingin LoadContent()
Drawing
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics8
What does what…
float4x4 World;float4x4 View;float4x4 Projection;
effect.Parameters["World"].SetValue(transforms[mesh.ParentBone.Index] * world);effect.Parameters["View"].SetValue(camera.View);effect.Parameters["Projection"].SetValue(camera.Projection);
Setting the effect parameter sets the equivalent value inside the effect.
Sets
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics9
The process
The Vertex Shader runs firstIt converts object vertex coordinates into projected coordinates.
The Pixel shader runs next. It computes the actual pixel
color
VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output;
float4 worldPosition = mul(input.Position, World); float4 viewPosition = mul(worldPosition, View); output.Position = mul(viewPosition, Projection);
return output;}
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return float4(1, 0, 0, 1);}
Once per pixel
Once per vertex
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics10
Adding a Material Property
I added this line to the veretx shader
And changed the pixel shader to this
// This is the surface diffuse colorfloat3 DiffuseColor = float3(0, 0, 0);
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return float4(DiffuseColor, 1);}
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics11
Setting this Effect private void SetDiffuseColorEffect() { foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (ModelMeshPart part in mesh.MeshParts) { BasicEffect bEffect = part.Effect as BasicEffect; part.Effect = diffuseColorEffect.Clone(part.Effect.GraphicsDevice);
part.Effect.Parameters["DiffuseColor"].SetValue(bEffect.DiffuseColor); } } }
The default content processing supplied a BasicEffect object with the color set in it. We’re creating our own effect and setting the color to match what was loaded.
Note the “Clone”. This makes a copy of the effect. Since we’re putting a material property into it, we need a unique copy here.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics12
What this does
Each pixel is simply set to the material diffuse color. No lighting is included, yet.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics13
Now let’s do real lightingfloat4x4 World;float4x4 View;float4x4 Projection;
// This is the surface diffuse colorfloat3 DiffuseColor = float3(0, 0, 0);
// Light definitionfloat3 LightAmbient = float3(0.07, 0.1, 0.1);
struct VertexShaderInput{ float4 Position : POSITION0;};
struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0;};
VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output;
float4 worldPosition = mul(input.Position, World); float4 viewPosition = mul(worldPosition, View); output.Position = mul(viewPosition, Projection); float3 color = LightAmbient * DiffuseColor; output.Color = float4(color, 1);
return output;}
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color;}
technique FirstShader{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderFunction(); }}
I’ve moved our computation to the vertex shader (why?). Ambient illumination is simply the light ambient color times the surface diffuse color.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics14
Some Things
float4x4 World;float4x4 View;float4x4 Projection;
// This is the surface diffuse colorfloat3 DiffuseColor = float3(0, 0, 0);
// Light definitionfloat3 LightAmbient = float3(0.07, 0.1, 0.1);
struct VertexShaderInput{ float4 Position : POSITION0;};
struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0;};
VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output;
float4 worldPosition = mul(input.Position, World); float4 viewPosition = mul(worldPosition, View); output.Position = mul(viewPosition, Projection); float3 color = LightAmbient * DiffuseColor; output.Color = float4(color, 1);
return output;}
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color;}
technique FirstShader{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderFunction(); }}
Note that the VertexShaderOutput now has a color.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics15
What this looks like
Inset has brightness artificially increased in Photoshop
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics16
Now for Diffuse Illumination
What we need to know Light location in space Light color
float3 Light1Location = float3(5, 221, -19);float3 Light1Color = float3(1, 1, 1);
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics17
HLSL code
// Light definitionfloat3 LightAmbient = float3(0.07, 0.1, 0.1);float3 Light1Location = float3(5, 221, -19);float3 Light1Color = float3(1, 1, 1);
struct VertexShaderInput{ float4 Position : POSITION0; float3 Normal : NORMAL0;};
struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0;};
We need to know the normal
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics18
Vertex ShaderVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output;
// We need the position and normal in world coordinates float4 position = mul(input.Position, World); float3 normal = normalize(mul(input.Normal, World)); // Ambient lighting hitting the location float3 color = LightAmbient; // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position); // Add contribution due to this light color += saturate(dot(Light1Direction, normal)) * Light1Color; // Multiply by material color color *= DiffuseColor; output.Color = float4(color.x, color.y, color.z, 1);
float4 viewPosition = mul(position, View); output.Position = mul(viewPosition, Projection); return output;}
position
Light1Location
Ligh
t1Lo
catio
n - p
ositi
on
Ligh
t1Di
recti
on
normal
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics19
What this looks like
What is missing?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics20
Texture Mapping
Mapping a picture onto the surface of a triangle.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics21
HLSL – Adding a texture variable
// The texture we usetexture Texture;
Not everything in our scene is texture mapped. If it is texture mapped, we use the texture color as the diffuse color. If not, we use the set diffuse color. We have two different ways we will compute the color.
I’ll move the multiplication by the Diffuse Color to the pixel shader.
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor, 1);;}
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics22
Slightly modified version, now.
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor, 1);}
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics23
We need something called a “sampler”
sampler Sampler = sampler_state{ Texture = <Texture>;
MinFilter = LINEAR; MagFilter = LINEAR; AddressU = Wrap; AddressV = Wrap; AddressW = Wrap;};
This just parameterizes how we get pixels from our texture. Wrap means tiling will be used. LINEAR means linear interpolation. We’ll do some other options later.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics24
And, we need to have the texture coordinatesstruct VertexShaderInput{ float4 Position : POSITION0; float3 Normal : NORMAL0; float2 TexCoord : TEXCOORD0;};
struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0; float2 TexCoord : TEXCOORD0;};
output.TexCoord = input.TexCoord;
And add this to the vertex shader function:
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics25
TechniquesWe have two ways things will be treated. We could create two different effects. But, it’s easier to create one effect with two different “techniques”.
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor.x, DiffuseColor.y, DiffuseColor.z, 1);}
technique NoTexture{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderFunction(); }}
Here is the technique we had before.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics26
Techniques
float4 PixelShaderTexturedFunction(VertexShaderOutput input) : COLOR0{ return input.Color * tex2D(Sampler, input.TexCoord);}
technique Textured{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderTexturedFunction(); }}
This is the other technique
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics27
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor.x, DiffuseColor.y, DiffuseColor.z, 1);}
float4 PixelShaderTexturedFunction(VertexShaderOutput input) : COLOR0{ return input.Color * tex2D(Sampler, input.TexCoord);}
technique NoTexture{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderFunction(); }}
technique Textured{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderTexturedFunction(); }}
All in one place so you can see it.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics28
Loading This Effect private void SetLambertianTextureEffect() { foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (ModelMeshPart part in mesh.MeshParts) { part.Effect = effectsOrig[part];
BasicEffect bEffect = part.Effect as BasicEffect; part.Effect = lambertianTextureEffect.Clone(part.Effect.GraphicsDevice);
part.Effect.Parameters["DiffuseColor"].SetValue(bEffect.DiffuseColor); part.Effect.Parameters["Texture"].SetValue(bEffect.Texture);
if (bEffect.Texture != null) { part.Effect.CurrentTechnique = part.Effect.Techniques["Textured"]; } else { part.Effect.CurrentTechnique = part.Effect.Techniques["NoTexture"]; } } } }
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics29
What we get
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics30
The complete effect – Variables and typesfloat4x4 World;float4x4 View;float4x4 Projection;
// This is the surface diffuse colorfloat3 DiffuseColor = float3(0, 0, 0);
texture Texture;
// Light definitionfloat3 LightAmbient = float3(0.07, 0.1, 0.1);float3 Light1Location = float3(5, 221, -19);float3 Light1Color = float3(1, 1, 1);
struct VertexShaderInput{ float4 Position : POSITION0; float3 Normal : NORMAL0; float2 TexCoord : TEXCOORD0;};
struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0; float2 TexCoord : TEXCOORD0;};
sampler Sampler = sampler_state{ Texture = <Texture>;
MinFilter = LINEAR; MagFilter = LINEAR; AddressU = Wrap; AddressV = Wrap; AddressW = Wrap;};
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics31
The complete effect – Vertex shaderVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.TexCoord = input.TexCoord;
// We need the position and normal in world coordinates float4 position = mul(input.Position, World); float3 normal = normalize(mul(input.Normal, World)); // Ambient lighting hitting the location float3 color = LightAmbient; // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position); // Add contribution due to this light color += max(dot(Light1Direction, normal), 0) * Light1Color;
output.Color = float4(color.x, color.y, color.z, 1);
float4 viewPosition = mul(position, View); output.Position = mul(viewPosition, Projection); return output;}
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics32
The complete effect = Pixel Shader and Techniquesfloat4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor.x, DiffuseColor.y, DiffuseColor.z, 1);}
float4 PixelShaderTexturedFunction(VertexShaderOutput input) : COLOR0{ return input.Color * tex2D(Sampler, input.TexCoord);}
technique NoTexture{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderFunction(); }}
technique Textured{ pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_1_1 PixelShaderTexturedFunction(); }}
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics33
Specular Illumination
Specular ReflectionsReflections from the surface of the material
Generally a different color than the underlying material
diffuse color
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics34
More Specular Examples
Specular Reflection makes things appear shiny
The left Dalek has no specular illumination, the right does
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics35
Previous lighting components
AmbientIa=Ambient part of illuminationMd=Diffuse material colorCa=Light ambient color
Diffuse IlluminationId=Diffuse part of illuminationMd=Diffuse material colorCi=Color of light ILi=Vector pointing at light IN=Surface normal
ada CMI float3 color = LightAmbient;
return input.Color * float4(DiffuseColor, 1);
)0,max(1
m
iiidd NLCMI
color += max(dot(Light1Direction, normal), 0) * Light1Color;
return input.Color * float4(DiffuseColor, 1);
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics36
Specular Component
Specular ComponentIs=Specular part of illuminationMs=Specular material colorCi=Color of light IN=Surface normalH=“Half vector”n=Shininess or SpecularPower
Basic HLSL code
nm
iiss HNCMI )0,max(
1
VL
VLH
// Add specular contribution due to this light float3 V = normalize(Eye - position); float3 H = normalize(Light1Direction + V); scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color;
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics37
Specular Reflection Highlight Coefficient
• The term n is called the specular reflection highlight coefficient or “Shininess” or “Specular Power”
• This effects how large the spectral highlight is. A larger value makes the highlight smaller and sharper. – This is the “shininess” factor in OpenGL, SpecularPower in
XNA– Matte surfaces has smaller n. – Very shiny surfaces have large n. – A perfect mirror would have infinite n.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics38
Shininess Examples
n=1 n=10 n=35
n=65 n=100
nm
iiss HNCMI )0,max(
1
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics39
HLSL// This is the surface diffuse colorfloat3 DiffuseColor = float3(0, 0, 0);float3 SpecularColor = float3(0, 0, 0);float Shininess = 0;
texture Texture;
// Light definitionfloat3 LightAmbient = float3(0.07, 0.1, 0.1);float3 Light1Location = float3(5, 221, -19);float3 Light1Color = float3(1, 1, 1);
float3 Eye = float3(0, 0, 0);
struct VertexShaderInput{ float4 Position : POSITION0; float3 Normal : NORMAL0; float2 TexCoord : TEXCOORD0;};
struct VertexShaderOutput{ float4 Position : POSITION0; float4 Color : COLOR0; float4 SColor : COLOR1; float2 TexCoord : TEXCOORD0;};
New shader variables
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics40
Vertex ShaderVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.TexCoord = input.TexCoord;
float4 position = mul(input.Position, World); float3 normal = normalize(mul(input.Normal, World)); // Ambient lighting hitting the location float3 color = LightAmbient; float3 scolor = 0; // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position); // Add diffuse contribution due to this light color += max(dot(Light1Direction, normal), 0) * Light1Color; // Add specular contribution due to this light float3 V = normalize(Eye - position); float3 H = normalize(Light1Direction + V); scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color; output.Color = float4(color, 1); output.SColor = float4(scolor, 1);
float4 viewPosition = mul(position, View); output.Position = mul(viewPosition, Projection); return output;}
Note that we send the specular illumination separate from the diffuse illumination. Any ideas why?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics41
Pixel Shaders
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor, 1) + input.SColor * float4(SpecularColor, 1);}
float4 PixelShaderTexturedFunction(VertexShaderOutput input) : COLOR0{ return input.Color * tex2D(Sampler, input.TexCoord) + input.SColor * float4(SpecularColor, 1);}
Important: We only multiply the texture color by the diffuse illumination, not the specular illuminations. We can have another texture map for specular illumination sometimes.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics42
Those extra parameters
BasicEffect bEffect = part.Effect as BasicEffect; part.Effect = diffuseSpecularEffect.Clone(part.Effect.GraphicsDevice);
part.Effect.Parameters["DiffuseColor"].SetValue(bEffect.DiffuseColor); part.Effect.Parameters["SpecularColor"].SetValue(bEffect.SpecularColor); part.Effect.Parameters["Shininess"].SetValue(bEffect.SpecularPower);
effect.Parameters["Eye"].SetValue(camera.Eye);
Setting up the effect:
When you draw:
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics43
Phong Shading or Per-Pixel Lighting
All of the methods we have used computed the lighting at the vertex and interpolated the color between the vertices.
Phong Shading interpolates the normal over the surface and computes the color at every pixel.
Will always look better and the only way to do some effects light spotlights, but can be costly!
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics44
HLSL
struct VertexShaderOutput{ float4 Position : POSITION0; float2 TexCoord : TEXCOORD0; float4 WorldPosition : TEXCOORD1; float3 Normal : TEXCOORD2;};
We put the world position and the normal into “texture coordinates” because these are interpolated for the pixel shader.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics45
Vertex ShaderVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output;
// We need the position and normal in world coordinates output.TexCoord = input.TexCoord;
output.WorldPosition = mul(input.Position, World);
output.Normal = normalize(mul(input.Normal, World));
output.Position = mul(mul(output.WorldPosition, View), Projection);
return output;}
This does surprisingly little!
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics46
Pixel Shader (no texture version)float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ float3 normal = input.Normal; float4 position = input.WorldPosition; // Ambient lighting hitting the location float3 color = LightAmbient; float3 scolor = 0; // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position); // Add diffuse contribution due to this light color += max(dot(Light1Direction, normal), 0) * Light1Color; // Add specular contribution due to this light float3 V = normalize(Eye - position); float3 H = normalize(Light1Direction + V); scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color; return float4(color * DiffuseColor + scolor * SpecularColor, 1);}
Identical to Vertex Shader Version, just moved!
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics47
Shader Models
technique NoTexture{ pass Pass1 { VertexShader = compile vs_2_0 VertexShaderFunction(); PixelShader = compile ps_2_0 PixelShaderFunction(); }}
A “shader model” specifies the capabilities we require. 2.0 is required to support using the texture coordinates this way. You also needed 2.0 for the many matrices in the skinned model.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics48
Other extremes of efficiency
Graphics systems such as Maya and 3DS Max can precompute the color for every vertex in advance and save it with the vertex data. We call this Vertex Lighting.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics49
Vertex LightingVertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.Position = mul(input.Position, Matrix); output.Color = input.Color; output.TexCoord = input.TexCoord;
return output;}
VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.Position = mul(input.Position, Matrix); output.Color = input.Color; output.TexCoord = input.TexCoord;
return output;} float4 PixelShaderTexture(PixelShaderInput input) : COLOR0
{ float4 color = input.Color; float4 texColor = tex2D(Sampler, input.TexCoord); color.rgba *= texColor; return color;}
float4 PixelShader(PixelShaderInput input) : COLOR0{ return input.Color;}
float4 PixelShaderTexture(PixelShaderInput input) : COLOR0{ float4 color = input.Color; float4 texColor = tex2D(Sampler, input.TexCoord); color.rgba *= texColor; return color;}
float4 PixelShader(PixelShaderInput input) : COLOR0{ return input.Color;}
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics50
The Ultimate Extreme
float4 PixelShaderTexture(PixelShaderInput input) : COLOR0{ float4 color = input.Color; float4 texColor = tex2D(Sampler, input.TexCoord); color.rgba *= texColor; return color;}
float4 PixelShader(PixelShaderInput input) : COLOR0{ return input.Color;}
float4 PixelShaderTexture(PixelShaderInput input) : COLOR0{ float4 color = input.Color; float4 texColor = tex2D(Sampler, input.TexCoord); color.rgba *= texColor; return color;}
float4 PixelShader(PixelShaderInput input) : COLOR0{ return input.Color;}
It’s possible to even avoid this bit of work and keeping the colors around. Instead, the graphics system creates a version of the texture with the lighting pre-multiplied into it. We call this Baked Textures.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics51
Baked Textures Example
float4 PixelShader (PixelShaderInput input) : COLOR0{ return tex2D(Sampler, input.TexCoord);}
float4 PixelShader (PixelShaderInput input) : COLOR0{ return tex2D(Sampler, input.TexCoord);}
VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.Position = mul(input.Position, Matrix); output.TexCoord = input.TexCoord;
return output;}
VertexShaderOutput VertexShaderFunction(VertexShaderInput input){ VertexShaderOutput output; output.Position = mul(input.Position, Matrix); output.TexCoord = input.TexCoord;
return output;}
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics52
Baked Lighting
We refer to vertex lighting or baked textures as “baked lighting”, meaning the lighting is precomputed and built into the model as vertex colors or into the textures directly. Most all games use baked lighting extensively.
Advantages of baked lightingLighting model can be much more complex including as many lights as we want, complex light falloffs, radiosity, ray-tracing, shadows, anything we like! Baked lighting is as fast as is possible.
Disadvantages of baked lightingOnly works for things that don’t move relative to lights.Diffuse and ambient illumination only, no specular illumination (sometimes we bake the diffuse and ambient, then add the specular at runtime, though).Can’t change the illumination at runtime (no sunsets, etc.)
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics53
Incandescence
Incandescence is simply an added term for the color that represents light generated by the surface.
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ return input.Color * float4(DiffuseColor, 1) + float4(Incandescence, 1);}
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics54
Render to Texture
Instead of rendering to the screen, we can render to a texture image.
Why would we want to?
private RenderTarget2D renderTarget = null;
if (renderTarget == null) { renderTarget = new RenderTarget2D(graphics, 512, 512, 1, graphics.DisplayMode.Format); }
graphics.SetRenderTarget(0, renderTarget); DrawActual(gameTime); graphics.SetRenderTarget(0, null);
Texture2D rendering = renderTarget.GetTexture();
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics55
How about something in this scene?
Anything missing?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics56
How about something in this scene?
How would you do something like this?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics57
Mirrors
Eye
Center
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics58
Notes
This example is simplified by putting the mirror parallel to the x/y plane (a single value of z).
We’ll assume an upright rectangular mirror.
We’ll assume the mirror is flat.
The first two assumptions can be overcome by just using a bit of math. The last requires a completely different method.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics59
Eye Reflection
Eye
Center
Mirrored Eye
Z=-248
Suppose Eye=(-100, 204, 0). What would be the coordinates of the mirrored eye?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics60
Mirroring the eye around z=-248
// How far are we from the mirror in the Z direction?float zDist = camera.Eye.Z - MirrorPlaneZ;
// Create a mirrored cameraCamera mirrorCamera = new Camera();mirrorCamera.Eye = new Vector3(camera.Eye.X, camera.Eye.Y, MirrorPlaneZ - zDist);
x,y are the same, only z changes. It was zDist from the mirror, it is zDist away in the other direction, now.
Suppose Eye=(-100, 204, 0). What would be the coordinates of the mirrored eye?
Mirrored eye = (-100, 204, -496)
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics61
Which way will the camera face?
Eye
Center
Mirrored Eye
Z=-248
Seems like it would be facing a mirrored direction, right? Sorry, it’s not that simple…
?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics62
Which way will the camera face?
Eye
Center
Mirrored Eye
Z=-248
Would not project onto the mirror, but onto a plane at an angle to the mirror!
?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics63
What the mirror looks like
-125 -39
74
192
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics64
If we look at it from an angle…
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics65
Remember Frustums?
Eye
Center
Mirrored Eye
Z=-248
Make the camera point directly at the wall, then use CreatePerspectiveOffCenter to create a custom camera frustum.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics66
Creating a custom frustum
Front view (from back)
Top view
left right
bottom
top
zNear
public static Matrix CreateOrthographicOffCenter ( float left, float right, float bottom, float top, float zNearPlane, float zFarPlane )
Important, left, right, bottom, top MUST be in view coordinates, not world coordinates.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics67
Putting it all together
// Determine the camera view direction Vector3 cameraView = camera.Center - camera.Eye;
// Are we looking towards the mirror? if (cameraView.Z >= 0) return;
// How far are we from the mirror in the Z direction? float zDist = camera.Eye.Z - MirrorPlaneZ;
// Create a mirror camera Camera mirrorCamera = new Camera(); mirrorCamera.Eye = new Vector3(camera.Eye.X, camera.Eye.Y, MirrorPlaneZ - zDist); mirrorCamera.Center = new Vector3(camera.Eye.X, camera.Eye.Y, MirrorPlaneZ); mirrorCamera.Up = new Vector3(0, 1, 0);
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics68
The Custom Projection Matrix // Compute mirror corners in the view coordinate system Vector3 corner1 = new Vector3(-125, 74, -248); Vector3 corner2 = new Vector3(-39, 192, -248);
corner1 = Vector3.Transform(corner1, mirrorCamera.View); corner2 = Vector3.Transform(corner2, mirrorCamera.View);
// Create a projection matrix Matrix projection = Matrix.CreatePerspectiveOffCenter(corner2.X, corner1.X, corner1.Y, corner2.Y, zDist, 10000);
Front view (from back)
left right
bottom
top
corner1
corner2
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics69
Rendering and using it if (renderTarget == null) { renderTarget = new RenderTarget2D(graphics, 512, 512, 1, graphics.DisplayMode.Format); }
graphics.SetRenderTarget(0, renderTarget); graphics.Clear(Color.Black); graphics.RenderState.DepthBufferEnable = true; graphics.RenderState.DepthBufferWriteEnable = true;
DrawModels(graphics, mirrorCamera, mirrorCamera.View, projection, gameTime); graphics.SetRenderTarget(0, null);
Texture2D rendering = renderTarget.GetTexture();
foreach (ModelMeshPart part in mirrorMesh.MeshParts) { part.Effect.Parameters["Texture"].SetValue(rendering); part.Effect.CurrentTechnique = part.Effect.Techniques["Textured"]; }
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics70
Other Notes
The image will be as viewed from the back of the mirror. You need to “mirror” it to see it from the front of the mirror.
Just ask your artist to create u,v coordinates that will mirror the texture.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics71
Getting Really Fancy…
Javier Cantón Ferrerohttp://www.codeplex.com/XNACommunity/Wiki/View.aspx?title=Reflection
Any Ideas?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics72
See any difference?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics73
ShadowsShadows are a nice effect, plus, the provide an important depth cue.
Are you sure this chair is sitting on the floor?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics74
Shadow Map
Image from light 0 viewpoint
Depth map from light 0 viewpoint
The map tells how far the “lit” point is from the light.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics75
Shadow Map Depth Image
Hard to see, but all we are doing is saving the depth for each pixel.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics76
Creating a depth map
We render the scene from the viewpoint of the light into a depth texture.
1.Create buffers to write into.2.Set up to write to the buffers (writing to a texture).3.Save off the buffers for later use.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics77
Creating a depth texture private RenderTarget2D shadowRenderTarget ; private DepthStencilBuffer shadowDepthBuffer; private Texture2D shadowMap;
SurfaceFormat shadowMapFormat = SurfaceFormat.Unknown;
// Check to see if the device supports a 32 or 16 bit floating point render targetif (GraphicsAdapter.DefaultAdapter.CheckDeviceFormat(DeviceType.Hardware, GraphicsAdapter.DefaultAdapter.CurrentDisplayMode.Format, TextureUsage.Linear, QueryUsages.None, ResourceType.RenderTarget, SurfaceFormat.Single) == true){ shadowMapFormat = SurfaceFormat.Single;}else if (GraphicsAdapter.DefaultAdapter.CheckDeviceFormat( DeviceType.Hardware, GraphicsAdapter.DefaultAdapter.CurrentDisplayMode.Format, TextureUsage.Linear, QueryUsages.None, ResourceType.RenderTarget, SurfaceFormat.HalfSingle) == true){ shadowMapFormat = SurfaceFormat.HalfSingle;}
// Create new floating point render targetshadowRenderTarget = new RenderTarget2D(graphics, shadowMapWidthHeight, shadowMapWidthHeight, 1, shadowMapFormat);
// Create depth buffer to use when rendering to the shadow mapshadowDepthBuffer = new DepthStencilBuffer(graphics, shadowMapWidthHeight, shadowMapWidthHeight, DepthFormat.Depth24);
You usually do this in LoadContent or Activate.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics78
Creating the Shadow Map: Setting up to draw /// <summary> /// Renders the scene to the floating point render target then /// sets the texture for use when drawing the scene. /// </summary> void CreateShadowMap(GraphicsDevice graphics, Camera camera) { // We need a view * projection matrix for the light Matrix lightViewProjection = CreateLightViewProjectionMatrix(camera);
// Set our render target to our floating point render target graphics.SetRenderTarget(0, shadowRenderTarget);
// Save the current stencil buffer DepthStencilBuffer oldDepthBuffer = graphics.DepthStencilBuffer;
// Set the graphics device to use the shadow depth stencil buffer graphics.DepthStencilBuffer = shadowDepthBuffer;
// Clear the render target to white or all 1's // We set the clear to white since that represents the // furthest the object could be away graphics.Clear(Color.White); graphics.RenderState.DepthBufferEnable = true; graphics.RenderState.DepthBufferWriteEnable = true;
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics79
Creating the Shadow Map: Drawing foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (Effect effect in mesh.Effects) { effect.CurrentTechnique = effect.Techniques["CreateShadowMap"]; effect.Parameters["LightViewProj"].SetValue(lightViewProjection); } }
// Draw any occluders DrawModel(graphics, camera, Bedroom, Matrix.Identity);
// Set render target back to the back buffer graphics.SetRenderTarget(0, null);
// Reset the depth buffer graphics.DepthStencilBuffer = oldDepthBuffer;
// Return the shadow map as a texture shadowMap = shadowRenderTarget.GetTexture();
Set technique and light matrix.
Obtaining the result.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics80
Setting the effect for regular drawing
foreach (ModelMesh mesh in Bedroom.Meshes) { foreach (Effect effect in mesh.Effects) { Texture2D texture = effect.Parameters["Texture"].GetValueTexture2D(); if (texture != null) { effect.CurrentTechnique = effect.Techniques["Textured"]; } else { effect.CurrentTechnique = effect.Techniques["NoTexture"]; }
effect.Parameters["ShadowMap"].SetValue(shadowMap); } }
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics81
Shader Additionsstruct ShadowVertexShaderOutput{ float4 Position : POSITION0; float4 WorldPosition : TEXCOORD1;};
// Transforms the model into light space an renders out the depth of the objectShadowVertexShaderOutput CreateShadowMap_VertexShader(float4 Position: POSITION){ ShadowVertexShaderOutput Out; Out.WorldPosition = mul(Position, World); Out.Position = mul(Out.WorldPosition, LightViewProj); return Out;}
// Saves the depth value out to the 32bit floating point texturefloat4 CreateShadowMap_PixelShader(ShadowVertexShaderOutput input) : COLOR{
float4 pos = mul(input.WorldPosition, LightViewProj); return float4(pos.z / pos.w, 0, 0, 1);}
// Technique for creating the shadow maptechnique CreateShadowMap{ pass Pass1 { VertexShader = compile vs_2_0 CreateShadowMap_VertexShader(); PixelShader = compile ps_2_0 CreateShadowMap_PixelShader(); }}
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics82
How do we use this?
Light
Eye
a
b
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics83
How do we use this?
Light
Eye
13.4
10.0
14.5
Point a is shadowed, point b is not.Point a is 13.4 from the light, but the depth map says the visible point is 10.0 away, so point a is not seen by the light. Point b is 14.5 from the light and the depth buffer says 14.5, so the point is lit.
a
b
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics84
The shadow algorithm
• Determine location of vertex on the shadow map
• d1 the depth stored in the shadow map at that location
• d2 the depth the vertex is relative to the light
• If d1 ≥ d2 the light hits the object
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics85
float4 PixelShaderTexturedFunction(VertexShaderOutput input) : COLOR0{ float3 normal = input.Normal; float4 position = input.WorldPosition; // Ambient lighting hitting the location float3 color = LightAmbient; float3 scolor = 0; // Find the position of this pixel in the light space float4 lightingPosition = mul(input.WorldPosition, LightViewProj);
// Find position in the shadow map float2 ShadowTexCoord = 0.5 * lightingPosition.xy / lightingPosition.w + float2(0.5, 0.5); ShadowTexCoord.y = 1.0 - ShadowTexCoord.y;
// Get the depth stored in the shadow map float shadowdepth = tex2D(ShadowMapSampler, ShadowTexCoord).r;
// Calculate the pixel dpeth float ourdepth = (lightingPosition.z / lightingPosition.w) - 0.001f;
if(shadowdepth >= ourdepth) { // Compute direction to the light float3 Light1Direction = normalize(Light1Location - position);
// Add diffuse contribution due to this light color += max(dot(Light1Direction, normal), 0) * Light1Color;
// Add specular contribution due to this light float3 V = normalize(Eye - position); float3 H = normalize(Light1Direction + V);
scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color; } return float4(color * tex2D(Sampler, input.TexCoord) + scolor * SpecularColor, 1);}
Pixel shader – only works for per-pixel lighting
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics86
Only piece left…
How do we figure out where to point the camera when taking a picture from the light’s viewpoint?
Any ideas?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics87
We want to enclose the visible camera frustum
Light
Eye
There are 8 points in a camera frustum. Be sure all 8 are visible.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics88
Creating the view matrix
Matrix CreateLightViewProjectionMatrix(Camera camera) { // Create a bounding frustum for our camera BoundingFrustum frustum = new BoundingFrustum(camera.View * camera.Projection);
Vector3[] frustumCorners = frustum.GetCorners(); Vector3 frustumCenter = frustumCorners[0]; for (int i = 1; i < frustumCorners.Length; i++) { frustumCenter += frustumCorners[i]; }
frustumCenter /= frustumCorners.Length;
Vector3 Light1Location = new Vector3(5, 221, -19);
// Create the view and projection matrix for the light Matrix lightView = Matrix.CreateLookAt(Light1Location, frustumCenter, new Vector3(0, 1, 0));
This just means the center of the shadow map will be the center of the view frustum.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics89
Creating the projection matrix float zNear = 50;
// Determine maximum extents in each direction float left = 0, right = 0, top = 0, bottom = 0; foreach (Vector3 corner in frustumCorners) { // Transform to view coordinate system Vector3 v = Vector3.Transform(corner, lightView);
// Project to the near clipping plane v.X = -v.X / v.Z * zNear; v.Y = -v.Y / v.Z * zNear;
if (v.X < left) left = v.X; if (v.X > right) right = v.X;
if (v.Y < bottom) bottom = v.Y; if (v.Y > top) top = v.Y; }
Matrix lightProjection = Matrix.CreatePerspectiveOffCenter(left, right, bottom, top, zNear, 1000);
return lightView * lightProjection; }
This projects the frustum points onto a view plane, then ensures the frustum for the light is big enough.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics90
The net effect
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics91
More Touching…
How can we determine if we have clicked on something?
Especially, how could we determine if we have clicked on Victoria’s hand?
Think about this one, particularly the second point!
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics92
Item Buffers
Have your artist create extra meshes with a fixed color. These don’t show normally. But, if you render these to a texture, you can tell what is where.
An image where the pixels tell what the object is (often by color) rather than the color of the image is called an item buffer.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics93
Item Buffers
Rendered image
Item buffer
Create a shader that just sets the pixel to the diffuse color; no lighting or anything. Then the diffuse color tells you what is at the pixel.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics94
Optimizations
Often I’m only interested in what’s at one locationWhere the mouse is right now.
Render a small image (4x4 for example) only around the mouse coordinates.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics95
Doing this…
int wid = device.Viewport.Width; int hit = device.Viewport.Height;
float scaleX = 1 / rangeX; float scaleY = 1 / rangeY;
// Where is the mouse on the screen? float rx = ((float)x / (float)wid - 0.5f) * 2; float ry = -((float)y / (float)hit - 0.5f) * 2; mapViewport = device.Viewport;
projection = camera.Projection * Matrix.CreateTranslation(-rx, -ry, 0) * Matrix.CreateScale(scaleX, scaleY, 1);
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics96
Spotlightsfloat4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ float3 color = float3(0, 0, 0); // Compute direction to the light float3 Light1Direction = normalize(Light1Location - input.PositionWorld); float3 Light1Pointing = normalize(Light1Location - Light1Target); if(dot(Light1Pointing, Light1Direction) > 0.98) { // Add contribution due to this light color = max(dot(Light1Direction, input.NormalWorld), 0) * Light1Color; }
return ((input.Color + float4(color, 1)) * float4(DiffuseColor, 1)) ;}
0.98 is cos(11.4o). So, our cone will have that spread.
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics97
Any ideas?
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics98
Fog
Two things at play here:
1.The farther the point the dimmer it gets.
2.The farther the point the more we see the fog color.
We need a weighted sum of the fog color and the computed color.
)1( dfog
dcomputedresult FCFCC
Distance to pointFog density constantFog colorComputed ColorResult Color
CSE 872 Dr. Charles B. OwenAdvanced Computer Graphics99
HLSLfloat4 fog = 0.992;float4 fogColor = float4(0.4, 0.5, 0.4, 1);
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0{ … // Add specular contribution due to this light float3 V = Eye - position; float d = length(V); V /= d; float3 H = normalize(Light1Direction + V); scolor += pow(saturate(dot(normal, H)), Shininess) * Light1Color; float4 thecolor = float4(color * DiffuseColor + scolor * SpecularColor, 1); float fogW = pow(fog, d); thecolor = thecolor * fogW + fogColor * (1 - fogW); return thecolor;}