Originally presented at: ACM SIGGRAPH 2003 Symposium on Interactive 3D Graphics Interruptible Rendering Joint work with: David Luebke * Benjamin Watson†

Originally presented at:ACM SIGGRAPH 2003Symposium on Interactive 3D Graphics

Interruptible Rendering

Joint work with:David Luebke * Benjamin Watson† Abhinav Dayal†

* University of Virginia† Northwestern University

Master’s Project Presentation Cliff Woolley

University of Virginia

4/30/2003 Interruptible Rendering 2

Motivation


Balance complexity with interactivity

Age-old tradeoff: detail vs. frame rate

Typical approach: Maintain a high, constant frame rate Create a mesh that can always be rendered in

time


Improve on traditional LOD

Is a high, constant frame rate good enough?

Is fidelity ever more important than frame rate?

How can this be decided at runtime while still guaranteeing interactivity?


Error metric unification

Inspired by [Meruvia 2000]

Combine notions of temporal and spatial error (i.e. “lateness” vs. “coarseness”)

Use to drive progressive refinement


Unified error metrics

With interruptible rendering, we address this key question:

At what point does further refinement of the current frame become pointless?


Methodology


Methodology

Refine a stream of continuous LODs

Monitor input frequently

Minimize dynamic visual error


Refine a stream of continuous LODs

Render progressive refinements on top of each other until “out of time”

Ensure that we can stop refining at any time and move on to the next frame


Refinement

Three refinement schemes implemented: Splatting (Tech sketch, SIGGRAPH 2002)

Progressive polygonal hulls Progressive ray casting (Abhinav Dayal, I3D 2003)

Will work with any progressive rendering scheme


Start with a very coarse rendering

Progressively render refinement stream over top of the coarse image

Progressive hull refinement


Progressive hulls

Simplification method by [Sander et al. 2000]

Record a sequence of constrained edge collapses and play it back in reverse

Guarantees containment

V1

Vn

V4

V3V2


V1

Vn

V4

V3V2

Progressive hulls


Progressive hulls


Monitor input frequently

Ideally: input monitored continuously

Realistically: check every x ms

Allows quick reaction when sudden changes in input occur

Allows system to be self-tuning


Minimize dynamic visual error

Always display image with least error

Sometimes: Further refinement is pointless

Temporal error (lateness) exceeds spatial error (coarseness) for the current frame

Front buffer “out of date” Dynamic visual error (spatial + temporal) in

front is greater than in the back


Calculating dynamic visual error

Spatial error Screen-space size of

coarsest geometry visible

Temporal error Project bounding box of

object in screen-space Find the maximum

screen-space distance any corner has moved since frame began

clear front bufferIfront = Icurrent

start

Refine the current imagein the front buffer.

improve ifront

tfront = Icurrent – Ifront

sfront = Ifront – ifront

Refine the current image in theback buffer.

improve iback

tback = Icurrent – Iback

sback = Iback – iback

Start refining a new image towardmost current input in the back buffer.

clear back bufferIback = Icurrent

Back buffer now closer to Icurrent than front is.

swap buffersIfront = Iback

It is pointless tocontinue refining.

swap buffersIfront = Iback

sfront = sback

efront >= eback?no yesyesno

tfront > sfront?

yesnotback > sback?

Rendering to Back BufferRendering to Front Buffer

time

tfront = Icurrent – Ifront

efront = sfront + tfront

eback = sback + tback

Clear thefront buffer

start

Refine thefront buffer

Refine the back buffer

Clear the back buffer

Swap buffers

Swap buffers

ef >= eb?N YYN tf > sf?

YN tb > sb?

Rendering to Back BufferRendering to Front Buffer

time

Compute dynamic

visual error


Evaluation


“Gold Standard” evaluation

Compares an ideal rendering to interactive approximations

Ideal rendering: full detail, zero delay

Interactive approximations are Unmanaged Constant fidelity (in pixels) Constant frame rate (fixed Hz) Interruptible rendering


Frame generation

Record what the user sees and when Generate each approximation offline Record actual frames displayed over time

Account for: Render time Double buffering Frame locking Rendering into front buffer (interruptible)


Comparing frames

Error measures RMS – Root Mean Square error Lindstrom's perceptually based error

Difft = Idealt - Renderedt


Two test input streams

Rotation Model-centered Fixed angular velocity

User interaction Includes both view translation and rotation Includes both static and dynamic segments

Both input streams recorded to files, with timestamps

Ideal

Inter-ruptible

Un-managed

Frames:

Interaction sequence: ray caster

ConstantFidelity

33 5754514845423936 60


Video

Interactive input stream Rotation input streamR

ay

ca

stin

gP

rog

ress

ive

hu

llser

ror(

rms)

erro

r(rm

s)

erro

r(rm

s)er

ror(

rms)

time(secs) time(secs)

time(secs) time(secs)

constant fidelityunmanaged

constant frame rateinterruptible


Demo


Conclusions


Benefits

Accuracy Balancing spatial and temporal error

Interactivity Slow renderers, like ray caster Large models

Self-tuning feedback system Adapts to hardware


Limitations

Overdraw in progressive renderer Progressive ray caster better here

Cost of monitoring temporal error

Rendering immersive models Requires reversing the containment criteria for

polygon rendering Not a problem for ray caster


Future work

Temporal & spatial error Better estimates of each (e.g. color) Better comparisons of both

Textures, normal maps, etc.

Reducing overdraw

View-dependent refinement

Ray tracing: a promising candidate


Acknowledgements

Peter Lindstrom for ltdiff

OpenRT Interactive Raytracing Project

Stanford 3D Scanning Repository

National Science Foundation Awards 0092973, 0093172, and 0112937


Questions?


Progressive ray casting

Progressive refinement Coarse to fine sampling/reconstruction

Sampling Where to shoot the ray

Adaptive Non-adaptive

Reconstruction Producing imagery from samples


Sampling

Coarse to fine - quadtree approach Sample every quadtree node's center Breadth first traversal Predetermined random traversal per level

Ray casting – using OpenRT Per ray API ~ 600,000 rays per sec. Simple phong shading No shadows or specular highlights


Reconstruction Placement

Place splat at center of each quadtree node

Shading Flat shaded quad

covering the node's screen space

Alpha-textured quads (smooth reconstruction) Quad size = twice of

node's screen space Texture: Gaussian blob

Transparent at edges Opaque at center


Flat shaded Alpha textured


Calculating dynamic visual error

Temporal error Computed as in case of polygonal hulls

Spatial error Diagonal length of the largest quadtree node

displayed on the screen

Spatial error

Documents

Originally presented at: ACM SIGGRAPH 2003 Symposium on Interactive 3D Graphics Interruptible Rendering Joint work with: David Luebke * Benjamin Watson†