J. Krüger R. Westermann computer graphics & visualization Technical University Munich Acceleration Techniques for GPU-based Volume Rendering

J. KrügerJ. KrügerR. WestermannR. Westermann

computer graphics & computer graphics & visualizationvisualization

Technical University MunichTechnical University Munich

Acceleration Techniques for Acceleration Techniques for GPU-based Volume RenderingGPU-based Volume Rendering


MotivationMotivation

Properties of real data setsProperties of real data sets

Surface like Surface like structuresstructuresSurface like Surface like structuresstructures

Images courtesy of Joe Kniss and volren.orgImages courtesy of Joe Kniss and volren.org

OcclusionOcclusionssOcclusionOcclusionss

Surface illuminationSurface illuminationSurface illuminationSurface illumination


MotivationMotivation

Texture based volume rendering today:Texture based volume rendering today:

Hardware accelerated rendering

of totally invisible structures

Hardware accelerated rendering

of totally invisible structures

only 2% of all fragments visibleonly 2% of all fragments visible


Our ContributionOur Contribution

We use new GPU Features to:We use new GPU Features to:

• Discard invisible fragmentsDiscard invisible fragments• early ray terminationearly ray termination• empty space skippingempty space skipping

• Implement a high quality GPU RaycasterImplement a high quality GPU Raycaster

• Accelerate the RaycasterAccelerate the Raycaster


Our ToolkitOur Toolkit

Programmable graphics hardwareProgrammable graphics hardware

• ““Arbitrarily” programmable shadersArbitrarily” programmable shaders• Assembly languageAssembly language• High level languages High level languages

• 32 Bit-Floating-point processing32 Bit-Floating-point processing• Input:Input: fp32 texturesfp32 textures• Arithmetic:Arithmetic: fp24-fp32fp24-fp32• Output:Output: fp32 render targetsfp32 render targets

• Early Z-TestEarly Z-Test• Multiple dependent texture fetchesMultiple dependent texture fetches


Stream ArchitectureStream Architecture

Our ToolkitOur Toolkit

RasterizerRasterizer

Early Z-TestEarly Z-Test

GeometryGeometry

FragmentsFragments

Fragment ShaderFragment Shader

Render TextureRender Texture

User TexturesUser Textures

……

……

……


RaycasterRaycaster

Slicing (SBVR) vs. GPU-Raycaster (RC)Slicing (SBVR) vs. GPU-Raycaster (RC)

SBVR SBVR render proxy geometry (slices) render proxy geometry (slices)

RC RC render proxy geometry (faces) render proxy geometry (faces)


RaycastingRaycasting

Using only Using only oneone main pass for Raymarching main pass for Raymarching


Basic IdeaBasic Idea

Multi-pass approachMulti-pass approach

• Generate entry pointGenerate entry point• first hit in local texture coordinatesfirst hit in local texture coordinates

• Compute ray directionCompute ray direction• ray direction in local texture coordinatesray direction in local texture coordinates

• Ray Marcher (Ray Marcher (simplifiedsimplified))• Main pass (Ray traversal)Main pass (Ray traversal)• Intermediate pass (Stopping criterion)Intermediate pass (Stopping criterion)


Entry PointEntry Point• Render into 2D textureRender into 2D texture

• Front facesFront faces of the volume bounding box of the volume bounding box• Texture coordinates as color componentsTexture coordinates as color components

(1,1,1)

(0,1,0)

(0,0,1)

(1,0,1)

(1,0,0)


Ray DirectionRay Direction• render into second texturerender into second texture

• back facesback faces of the bounding box of the bounding box• subtract from entry-point texture to get ray directionsubtract from entry-point texture to get ray direction

(0,1,0)

(1,0,1)

(0,0,1)

(0,0,0)

(1,0,0)

(1,1,1)

(0,1,0)

(0,0,1)

(1,0,1)

(1,0,0)

subtractsubtract


Basic IdeaBasic Idea

Multi-pass approachMulti-pass approach

• Generate entry pointGenerate entry point• first hit in local texture coordinatesfirst hit in local texture coordinates

• Compute ray directionCompute ray direction• ray direction in local texture coordinatesray direction in local texture coordinates

• Ray MarcherRay Marcher• Main passes (Ray traversal)Main passes (Ray traversal)• Intermediate pass (Stopping criterion)Intermediate pass (Stopping criterion)



Using multiple Raymarching passesUsing multiple Raymarching passes


Ray Marcher (Main passes)Ray Marcher (Main passes)

We know: We know: Entry pointEntry point and and ray directionray direction

float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;

float3 fFrontVal = v.Color.rgb;float3 fFrontVal = v.Color.rgb;

float3 fVolCoord = fFrontVal+fDepth*fDirection;float3 fVolCoord = fFrontVal+fDepth*fDirection;

float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;

float3 fFrontVal = v.Color.rgb;float3 fFrontVal = v.Color.rgb;

float3 fVolCoord = fFrontVal+fDepth*fDirection;float3 fVolCoord = fFrontVal+fDepth*fDirection;

Render front-facesRender front-faces• set Color like before set Color like before CC• set Texture coords to NDCset Texture coords to NDC (x,y)(x,y)• activate direction textureactivate direction texture DIRDIR• a global counter a global counter tt

compute ray as: r(t)=compute ray as: r(t)=CC+t*+t*DIR(x,y)DIR(x,y)


Ray Marcher (Main passes)Ray Marcher (Main passes)

[..][..]

for (int i=0;i<N;i++) {for (int i=0;i<N;i++) {

fResultVal = under(tex3D(sVolume, fVolCoord),fResultVal);fResultVal = under(tex3D(sVolume, fVolCoord),fResultVal);

fVolCoord += fStepSize*fDirection;fVolCoord += fStepSize*fDirection;

}}

return fResultVal; return fResultVal;

[..][..]

for (int i=0;i<N;i++) {for (int i=0;i<N;i++) {

fResultVal = under(tex3D(sVolume, fVolCoord),fResultVal);fResultVal = under(tex3D(sVolume, fVolCoord),fResultVal);

fVolCoord += fStepSize*fDirection;fVolCoord += fStepSize*fDirection;

}}

return fResultVal; return fResultVal;

Integration along the rayIntegration along the ray

• use use r(t)r(t) to access the volume to access the volume• integrate over multiple steps (N)integrate over multiple steps (N)• combine with frame buffer contentscombine with frame buffer contents



RaycastingRaycastingEarly Ray TerminationEarly Ray TerminationEmpty Space SkippingEmpty Space Skipping

Acceleration techniquesAcceleration techniques


Acceleration TechniquesAcceleration Techniques

Terminate Rays on one of the following conditions:Terminate Rays on one of the following conditions:• it has left the volumeit has left the volume• it has reached a certain opacityit has reached a certain opacity

Suspend a ray if:Suspend a ray if:• it‘s traversing an empty regionit‘s traversing an empty region

this needs to be done on a per-fragment basisthis needs to be done on a per-fragment basis

early Z-Test early Z-Test


Acceleration techniquesAcceleration techniques

Early-Z Restrictions:Early-Z Restrictions:

• on current GPUs early Z-Test only works ifon current GPUs early Z-Test only works if• no no clipclip or or texkilltexkill operation is executed operation is executed• the Z-value is not changed in the shaderthe Z-value is not changed in the shader

the raymarching shader can not terminate himselfthe raymarching shader can not terminate himself

• Solution: use intermediate pass to do the ray-Solution: use intermediate pass to do the ray-suspending/terminatingsuspending/terminating


Acceleration techniquesAcceleration techniques• Execute intermediate pass after every main passExecute intermediate pass after every main pass

(= N volume samples)(= N volume samples)• Access z-value only in this passAccess z-value only in this pass• If ray is terminated or suspended set z-value If ray is terminated or suspended set z-value

beforebefore main-pass-geometry main-pass-geometry• Reset depth Reset depth behindbehind it if ray is to be resumed it if ray is to be resumed


Ray Marcher (intermediate pass)Ray Marcher (intermediate pass)

for early ray terminationfor early ray termination : : check accumulated check accumulated alphaalpha

for empty space skippingfor empty space skipping : : check “skip-volume”check “skip-volume”

Skip volume:Skip volume:• volume sampled down to 1/8 in every dimensionvolume sampled down to 1/8 in every dimension• every sample contains the minimum and maximum every sample contains the minimum and maximum

of the 512 corresponding entriesof the 512 corresponding entries

Check TextureCheck Texture::• 2D lookup texture2D lookup texture• lookup(x,y)=1 iff all values between x and y are 0 lookup(x,y)=1 iff all values between x and y are 0

under the current transfer functionunder the current transfer function


Ray Marcher (intermediate pass)Ray Marcher (intermediate pass)

float4 fLastValfloat4 fLastVal = tex2D(sLastFrame, v.TexCoords.rg);= tex2D(sLastFrame, v.TexCoords.rg);

float3 fDirection float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;= tex2D(sDirection, v.TexCoords.rg).rgb;

float3 fFrontVal float3 fFrontVal = v.Color.rgb;= v.Color.rgb;

float3 fVolCoord float3 fVolCoord = fFrontVal+fDepth*fDirection;= fFrontVal+fDepth*fDirection;

float2 fMaxMinfloat2 fMaxMin = tex3D(sEmptySpace, fVolCoord).rg;= tex3D(sEmptySpace, fVolCoord).rg;

float fEmptyfloat fEmpty = tex2D(sEmptyLookUp, fMaxMin);= tex2D(sEmptyLookUp, fMaxMin);

OUT.depthOUT.depth = ((fLastVal.a < 0.9999) && (fEmpty<1)) ? 0 : 100;= ((fLastVal.a < 0.9999) && (fEmpty<1)) ? 0 : 100;

OUT.col OUT.col = fLastVal;= fLastVal;

return OUT;return OUT;

float4 fLastValfloat4 fLastVal = tex2D(sLastFrame, v.TexCoords.rg);= tex2D(sLastFrame, v.TexCoords.rg);

float3 fDirection float3 fDirection = tex2D(sDirection, v.TexCoords.rg).rgb;= tex2D(sDirection, v.TexCoords.rg).rgb;

float3 fFrontVal float3 fFrontVal = v.Color.rgb;= v.Color.rgb;

float3 fVolCoord float3 fVolCoord = fFrontVal+fDepth*fDirection;= fFrontVal+fDepth*fDirection;

float2 fMaxMinfloat2 fMaxMin = tex3D(sEmptySpace, fVolCoord).rg;= tex3D(sEmptySpace, fVolCoord).rg;

float fEmptyfloat fEmpty = tex2D(sEmptyLookUp, fMaxMin);= tex2D(sEmptyLookUp, fMaxMin);

OUT.depthOUT.depth = ((fLastVal.a < 0.9999) && (fEmpty<1)) ? 0 : 100;= ((fLastVal.a < 0.9999) && (fEmpty<1)) ? 0 : 100;

OUT.col OUT.col = fLastVal;= fLastVal;

return OUT;return OUT;

for early ray terminationfor early ray termination : : check accumulated check accumulated alphaalpha

for empty space skippingfor empty space skipping : : check “skip-volume”check “skip-volume”


DEMOSDEMOS

Run onRun on

•Intel Pentium IV 2.4GHz

•ATI Radeon 9800 Pro.

•Microsoft Windows XP

•DirectX 9

•Pixel/Vertex Shader 2.0


Iso-Surface RaycastingIso-Surface Raycasting• first 2 passes and intermediate passes remain first 2 passes and intermediate passes remain

unchangedunchanged• global traversal order still front to backglobal traversal order still front to back• order within the main shader is back to frontorder within the main shader is back to front• keep only last isovaluekeep only last isovalue• if isovalue was found computeif isovalue was found compute

illumination with 3D gradientillumination with 3D gradienttexture and write it to thetexture and write it to thefinal imagefinal image


ConclusionConclusion

Using current GPU features we showed how toUsing current GPU features we showed how to

• Discard invisible fragmentsDiscard invisible fragments• early ray terminationearly ray termination• empty space skippingempty space skipping

• Implement a high quality GPU RaycasterImplement a high quality GPU Raycaster

• Accelerate that RaycasterAccelerate that Raycaster


Thank you!Thank you!

Questions?Questions?

Download Slides at: Download Slides at: http://wwwcg.in.tum.de/Research/Publications/Raycast/http://wwwcg.in.tum.de/Research/Publications/Raycast/

The EndThe End

Documents

J. Krüger R. Westermann computer graphics & visualization Technical University Munich Acceleration Techniques for GPU-based Volume Rendering