High-Quality Surface Splatting on Today’s

GPUsM. Botsch, A. Hornung, M. Zwicker, L. Kobbelt

Presented byJulian Yu-Chung Chen

CS594 GPU Programming2006-03-30

Outline

• Point Based Rendering

• Surface splatting

• This paper’s unique contributions

• Multi-Pass deferred shading

• Elliptical Weighted Average approximation

Point Based Rendering

• Laser scanning range data, huge

• Mesh: need connectivity, triangulation

• PBR: Rendering directly with point set

Courtesy of Stanford “The Digital Michelangelo Project”

Point-based surface representation

No connectivity

To appear in the Proceedings of SIGGRAPH 2001

Surface Splatting

Matthias Zwicker ! Hanspeter Pfister † Jeroen van Baar† Markus Gross!

Figure 1: Surface splatting of a scan of a human face, textured terrain, and a complex point-sampled object with semi-transparent surfaces.

AbstractModern laser range and optical scanners need rendering techniquesthat can handle millions of points with high resolution textures.This paper describes a point rendering and texture filtering tech-nique called surface splatting which directly renders opaque andtransparent surfaces from point clouds without connectivity. Itis based on a novel screen space formulation of the EllipticalWeighted Average (EWA) filter. Our rigorous mathematical anal-ysis extends the texture resampling framework of Heckbert to ir-regularly spaced point samples. To render the points, we develop asurface splat primitive that implements the screen space EWA filter.Moreover, we show how to optimally sample image and proceduraltextures to irregular point data during pre-processing. We also com-pare the optimal algorithm with a more efficient view-independentEWA pre-filter. Surface splatting makes the benefits of EWA tex-ture filtering available to point-based rendering. It provides highquality anisotropic texture filtering, hidden surface removal, edgeanti-aliasing, and order-independent transparency.

Keywords: Rendering Systems, Texture Mapping, Antialiasing,Image-Based Rendering, Frame Buffer Algorithms.

1 IntroductionLaser range and image-based scanning techniques have producedsome of the most complex and visually stunning models to date [9].One of the challenges with these techniques is the huge volume of

!ETH Zurich, Switzerland. Email: [zwicker,grossm]@inf.ethz.ch†MERL, Cambridge, MA. Email: [pfister,jeroen]@merl.com

point samples they generate. A commonly used approach is gener-ating triangle meshes from the point data and using mesh reductiontechniques to render them [7, 2]. However, some scanned meshesare too large to be rendered interactively [9], and some applicationscannot tolerate the inherent loss in geometric accuracy and texturefidelity that comes from polygon reduction.

Recent efforts have focused on direct rendering techniques forpoint samples without connectivity [16, 4, 15]. These techniquesuse hierarchical data structures and forward warping to store andrender the point data efficiently. One important challenge for pointrendering techniques is to reconstruct continuous surfaces from theirregularly spaced point samples while maintaining the high texturefidelity of the scanned data. In addition, the point rendering shouldcorrectly handle hidden surface removal and transparency.

In this paper we propose a new point rendering technique calledsurface splatting, focusing on high quality texture filtering. In con-trast to previous point rendering approaches, surface splatting usesa novel screen space formulation of the EllipticalWeighted Average(EWA) filter [3], the best anisotropic texture filtering algorithm forinteractive systems. Extending the framework of Heckbert [6], wederive a screen space form of the EWA filter for irregularly spacedpoint samples without global texture parameterization. This makessurface splatting applicable to high-resolution laser range scans, ter-rain with high texture detail, or point-sampled geometric objects(see Figure 1). A modified A-buffer [1] provides hidden surfaceremoval, edge anti-aliasing, and order-independent transparency ata modest increase in computation efforts.

The main contribution of this paper is a rigorous mathemati-cal formulation of screen space EWA texture filtering for irregularpoint data, presented in Section 3. We show how the screen spaceEWA filter can be efficiently implemented using surface splattingin Section 4. If points are used as rendering primitives for complexgeometry, we want to apply regular image textures to point sam-ples during conversion from geometric models. Hence, Section 5introduces an optimal texture sampling and pre-filtering method forirregular point samples. Sections 6 and 7 present our modified A-buffer method for order-independent transparency and edge anti-aliasing, respectively. Finally, we discuss implementation, timings,and image quality issues in Section 8.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copiesare not made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. To copyotherwise, to republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.

ACM SIGGRAPH 2001, 12-17 August 2001, Los Angeles, CA, USA© 2001 ACM 1-58113-374-X/01/08...$5.00

371

Surface Splatting

Volume and Surface

PBR - Splat

• Volume splatting to Surface splatting

• Interactive Application, trade-offs

• High performance

• High visual quality

Computer Graphics Group

Mario Botsch

Comparison

Persp. Correct

Phong Shading

Anti-Aliasing

Splats/sec

EWA

Splatting ! ! ! 1M

NV30 Splatting

PG ‘03 ! ! ! 27M

Persp. Accurate

GI ‘04 (!) ! ! 5M

Phong Splatting

PBG ‘04 ! ! ! 6M

NV40 Splatting

PBG ‘05 !

Previous Work

• Well designed data structure: QSplat

• Pre-processing time: 1 hour/ 10M points

• Represent each splat by an alpha-textured quad or triangle

• Expensive, 3~4 times in size

• Representing each splat by one OpenGL vertex + pixel shader for rasterization

• Correct depth values for correcting blending result: depth map

Procedures

• Splat rasterization

• Hardware deferred shading, using MRT

• Visibility pass - depth map

• Attribute pass - color value & normal vector

• Shading pass - final color with MRTs

Splat Rasterization(*BSK04)

To appear in the Proceedings of SIGGRAPH 2001

2 Previous WorkTexture mapping increases the visual complexity of objects by map-ping functions for color, normals, or other material properties ontothe surfaces [5]. If these texture functions are inappropriately band-limited, texture aliasing may occur during projection to raster im-ages. For a general discussion of this problem see [21]. Althoughwe develop our contributions along similar lines to the seminalwork of Heckbert [6], our approach is fundamentally different fromconventional texture mapping. We present the first systematic anal-ysis for representing and rendering texture functions on irregularlypoint-sampled surfaces.

The concept of representing objects as a set of points and usingthese as rendering primitives has been introduced in a pioneering re-port by Levoy and Whitted [10]. Due to the continuing increase ingeometric complexity, their idea has recently gained more interest.QSplat [16] is a point rendering system that was designed to interac-tively render large data sets produced by modern scanning devices.Other researchers demonstrated the efficiency of point-based meth-ods for rendering geometrically complex objects [4, 15]. In somesystems, point-based representations are temporarily stored in therendering pipeline to accelerate rendering [11, 17]. Surprisingly,nobody has systematically addressed the problem of representingtexture functions on point-sampled objects and avoiding aliasingduring rendering. We present a surface splatting technique that canreplace the heuristics used in previous methods and provide supe-rior texture quality.

Volume splatting [19] is closely related to point rendering andsurface splatting. A spherical 3D reconstruction kernel centered ateach voxel is integrated along one dimension into a 2D “footprintfunction.” As each voxel is projected onto the screen, the 2D foot-prints are accumulated directly into the image buffer or into image-aligned sheet buffers. Some papers [18, 14] address aliasing causedby insufficient resampling rates during perspective projections. Toprevent aliasing, the 3D reconstruction kernels are scaled using aheuristic. In contrast, surface splatting models both reconstruct-ing and band-limiting the texture function in a unified framework.Moreover, instead of pre-integrating isotropic 3D kernels, it usesoriented 2D kernels, providing anisotropic filtering for surface tex-tures.

3 The Surface Splatting FrameworkThe basis of our surface splatting method is a model for the rep-resentation of continuous texture functions on the surface of point-based graphics objects, which is introduced in Section 3.1. Sincethe 3D points are usually positioned irregularly, we use a weightedsum of radially symmetric basis functions. With this model at hand,we look at the task of rendering point-based objects as a concatena-tion of warping, filtering, and sampling the continuous texture func-tion. In Section 3.2 we extend Heckbert’s resampling theory [6]to process point-based objects and develop a mathematical frame-work of the rendering procedure. In Section 3.3 we derive an al-ternative formulation of the EWA texture filter that we call screenspace EWA, leading to the surface splatting algorithm discussed inSection 4. In Section 5, we describe how to acquire the texturefunctions, which can be regarded as a scattered data approximationproblem. A continuous approximation of the unknown original tex-ture function needs to be computed from an irregular set of sam-ples. We distinguish between scanned objects with color per pointand regular textures that are explicitly applied to point-sampled ge-ometry.

3.1 Texture Functions on Point-Based Objects

In conventional polygonal rendering, texture coordinates are usu-ally stored per vertex. This enables the graphics engine to combinethe mappings from 2D texture space to 3D object space and fromthere to 2D screen space into a compound 2D to 2D mapping be-

tween texture and screen space. Using this mapping, pixel colorsare computed by looking up and filtering texture samples in 2Dtexture space at rendering time. There is no need for a sampledrepresentation of the texture in 3D object space. By contrast, thecompound mapping function is not available with point-based ob-jects at rendering time. Consequently, we must store an explicittexture representation in object space.We represent point-based objects as a set of irregularly spaced

points {Pk} in three dimensional object space without connectiv-ity. A point Pk has a position and a normal. It is associated witha radially symmetric basis function rk and coefficients wr

k, wgk, wb

k

that represent continuous functions for red, green, and blue colorcomponents. Without loss of generality, we perform all further cal-culations with scalar coefficients wk. Note that the basis functionsrk and coefficients wk are determined in a pre-processing step, de-scribed in Section 5.We define a continuous function on the surface represented by

the set of points as illustrated in Figure 2. Given a point Q any-

3D object space

Pk

Q

u0

u1

2D parameterizationlocal parameterization

basis function rk(u-uk)ksmall neighborhoodaround Q

P3P1

P2

1

Figure 2: Defining a texture function on the surface of a point-basedobject.

where on the surface, shown left, we construct a local parameteri-zation of the surface in a small neighborhood of Q, illustrated onthe right. The points Q and Pk have local coordinates u and uk,respectively. We define the continuous surface function fc(u) asthe weighted sum:

fc(u) =k!

wkrk(u ! uk). (1)

We choose basis functions rk that have local support or that are ap-propriately truncated. Then u lies in the support of a small numberof basis functions. Note that in order to evaluate (1), the local pa-rameterization has to be established in the union of these support ar-eas only, which is very small. Furthermore, we will compute theselocal parameterizations on the fly during rendering as described inSection 4.

3.2 Rendering

Heckbert introduced a general resampling framework for texturemapping and the EWA texture filter in [6]. His method takes a reg-ularly sampled input function in source space, reconstructs a con-tinuous function, warps it to destination space, and computes theproperly sampled function in destination space. Properly sampledmeans that the Nyquist criterion is met. We will use the term screenspace instead of destination space.We extend this framework towards a more general class of in-

put functions as given by Equation (1) and describe our renderingprocess as a resampling problem. In contrast to Heckbert’s regularsetting, in our representation the basis functions rk are irregularlyspaced. In the following derivation, we adopt Heckbert’s notation.Given an input function as in Equation (1) and a mapping x =

m(u) : 2 " 2 from source to screen space, rendering involvesthe three steps illustrated in Figure 3:

2372

Splat RasterizationTo appear in the Proceedings of SIGGRAPH 2001

xu

texture function discrete output

warp sample

x x

filter

fc(u)rk(u-uk)wk

g'c(x)!k(x)

g(x)

gc(x)=fc(m-1(x))rk(m-1(x)-uk)wk

warped texture function band limited texture function

Figure 3: Warping, filtering, and sampling the texture function.

1. Warp fc(u) to screen space, yielding the warped, continuousscreen space signal gc(x):

gc(x) = (fc !m!1)(x) = fc(m!1(x)),

where ! denotes function concatenation.

2. Band-limit the screen space signal using a prefilter h, result-ing in the continuous output function g"c(x):

g"c(x) = gc(x) " h(x) =

2gc(!)h(x# !)d!,

where " denotes convolution.

3. Sample the continuous output function by multiplying it withan impulse train i to produce the discrete output g(x):

g(x) = g"c(x)i(x).

An explicit expression for the warped continuous output functioncan be derived by expanding the above relations in reverse order:

g"c(x) =

2h(x # !)

k#

wkrk(m!1(!) # uk)d!

=k#

wk"k(x), (2)

where "k(x) =2h(x # !)rk(m!1(!) # uk)d!. (3)

We call a warped and filtered basis function "k(x) a resamplingkernel, which is expressed here as a screen space integral. Equa-tion (2) states that we can first warp and filter each basis functionrk individually to construct the resampling kernels "k and then sumup the contributions of these kernels in screen space. We call thisapproach surface splatting, as illustrated in Figure 4. In contrastto Heckbert, who transformed the screen space integral of Equa-tion (2) to a source space integral and formulated a source spaceresampling kernel, we proceed with (3) to derive a screen spaceresampling kernel.In order to simplify the integral for "k(x) in (3), we replace a

general mapping m(u) by its local affine approximation muk at apoint uk ,

muk(u) = xk + Juk · (u # uk), (4)

where xk = m(uk) and the Jacobian Juk = !m!u (uk).

screen space object space

resampling kernel !k(x)

Pk

discrete output g(x)mapping x=m(u)

m(uk)

Figure 4: Rendering by surface splatting, resampling kernels areaccumulated in screen space.

Heckbert relied on the same approximation in his derivation [6].Since the basis functions rk have local support, muk is used onlyin a small neighborhood around uk in (3). Moreover, the approx-imation is most accurate in the neighborhood of uk and so it doesnot cause visual artifacts. We use it to rearrange Equation (3), andafter a few steps we find:

"k(x) =2h(x # muk(uk) # !)r"k(!)d!

= (r"k " h)(x #muk (uk)), (5)

where r"k(x) = rk(J!1uk

x) denotes a warped basis function. Thus,although the texture function is defined on an irregular grid, Equa-tion (5) states that the resampling kernel in screen space, "k(x),can be written as a convolution of a warped basis function r"k andthe low-pass filter kernel h. This is essential for the derivation ofscreen space EWA in the next section. Note that from now on weare omitting the subscript uk form and J.

3.3 Screen Space EWA

Like Greene and Heckbert [3], we choose elliptical Gaussians bothfor the basis functions and the low-pass filter, since they are closedunder affine mappings and convolution. In the following derivationwe apply these mathematical properties to the results of the previ-ous section. This enables us to express the resampling kernel as asingle Gaussian, facilitating efficient evaluation during rendering.An elliptical Gaussian GV(x) with variance matrixV is defined

as:

GV(x) =1

2#|V| 12e!

12xT V!1x

,

where |V| is the determinant ofV. We denote the variance matricesof the basis functions rk and the low-pass filter h withVr

k andVh,respectively. The warped basis function and the low-pass filter are:

r"k(x) = r(J!1x) = GVrk(J!1x) =

1|J!1|GJVr

kJT (x)

h(x) = GVh(x).

The resampling kernel "k of (5) can be written as a single Gaussianwith a variance matrix that combines the warped basis function andthe low-pass filter. TypicallyVh = I, yielding:

"k(x) = (r"k " h)(x # m(uk))

=1

|J!1| (GJVrkJT " GI)(x #m(uk))

=1

|J!1|GJVrkJT +I(x # m(uk)). (6)

We will show how to determine J!1 in Section 4, and how to com-puteVr

k in Section 5. Substituting the Gaussian resampling kernel

3373

Splat Rasterization

• A OpenGL vertex c -> d*d image space square (vert.s)

• Pixel shader test pixel(x,y) in d*d in or out of projected elliptical splat contour, discard outsiders

• Local raycast to get point on splat surface(with local parameter (u, v)), test against

•

• Calculate and accumulate weighting factor using

• Pixel depth, normal, color determined in each pass

M. Botsch, A. Hornung, M. Zwicker, L. Kobbelt / High-Quality Surface Splatting

timings and splat rates given in this paper were measuredusing this configuration.

Most of the above methods neglect the screen space fil-ter of the EWA framework and restrict to the Gaussianreconstruction filter in object space. While this leads tosufficient anti-aliasing in magnified regions, it cannot pre-vent aliasing artifacts in minified areas. In contrast, themethod of [ZRB!04] implements the full EWA splatting ap-proach on the GPU, but due to the complex computationsits performance is limited to about 4M splats/sec using theabove mentioned hardware configuration. Finally, the useof deferred shading techniques has shown to allow for effi-cient high-quality per-pixel shading for surface splatting ap-proaches such as EWA splatting as well as splat-based vol-ume rendering [MMC99].

Using per-pixel Phong shading and a simple but effec-tive approximation to the screen space filter, the approachpresented in this paper provides results comparable to theoriginal EWA splatting. By exploiting deferred shading tech-niques we achieve this superior visual quality at a rate ofabout 23M splats/sec, such that our method is close toEWA splatting in terms of quality and close to the fast butlow-quality renderer of [BK03], which achieves about 27Msplats/sec.

3. Splat RasterizationIn this section we shortly describe the perspectively cor-rect rasterization of elliptical splats, which was introduced in[BSK04]. This method will then be used in the next sectionto accumulate normal and color contributions of individualsplats in the rendering buffer.

Following the notation of [BSK04], a splat S j is definedby its center c j and two orthogonal tangent directions u j andv j. These tangent vectors are scaled according to the princi-pal radii of the elliptical splat, such that an arbitrary point qin the splat’s embedding plane lies in the interior of the splatif its local parameter values u and v satisfy the condition

u2 + v2 =!

uTj (q! c j)

"2+

!vT

j (q! c j)"2" 1 . (1)

The rasterization of a splat S j is performed by sendingits center c j, tangent axes

#u j,v j

$, and optional material

properties to OpenGL, which are then processed by customshaders for both the vertex and the pixel stage. The vertexshader conservatively estimates the size d of the projectedsplat based on a perspective division of the larger of the el-lipse radii r by the eye-space depth value cz of the splat cen-ter, followed by a window-to-viewport scaling as describedin [BSK04].

This causes the single OpenGL vertex c to be rasterizedas a d # d image space square, each pixel (x,y) of whichis then tested by a pixel shader to lie either inside or out-side of the projected elliptical splat contour. Local ray cast-ing through the corresponding projected point qn on the near

plane yields the eye-space point q on the splat’s supportingplane. From this projectively exact 3D position the local pa-rameter values (u,v) can be determined and tested as shownin (1). While pixels corresponding to points outside the splatare discarded, pixels belonging to the splat are accepted andprocessed further. If a pixel (x,y) is accepted, its weightingfactor is determined as

w(x,y) = h!%

u2 + v2"

, (2)

where h(·) is typically chosen as a Gaussian. To allow forexact blending and occlusion, the pixel’s depth value has tobe adjusted as described in [BSK04] in order to correspondto the computed 3D position q. This finally results in a per-pixel projectively correct rasterization of elliptical splats.

The output of the rasterization pixel shader are depth val-ues only for the visibility pass, and additionally weightedsplat attributes, such as normal vectors or color values, in thesecond pass, which are then accumulated in the render targetby additive alpha blending. The final normalization/shadingpass then processes each pixel in order to compute its finalcolor, as described in the next section.

4. Hardware-Accelerated Deferred Shading

For both point-based models and polygonal meshes, one ma-jor requirement for high-quality visualization is the use ofper-pixel Phong shading based on interpolated normal vec-tors, instead of Gouraud shading, which blends color con-tributions resulting from lighting each splat or mesh ver-tex, respectively. In contrast to polygonal meshes, point-based models do not store any neighborhood relation be-tween splats, therefore an equivalent interpolation of neigh-boring splats’ normal vectors is not possible.

In order to still be able to generate smoothly interpolatedper-pixel normal vectors, two basic approaches are possible.The first is to associate with each splat a pre-computed linearnormal field, as proposed in the Phong splatting approach[BSK04]. However, while leading to a high-quality shad-ing, this method is limited to static geometries and boundto about 6M splats/sec, as mentioned in Section 2.

The second approach for normal interpolation wasproposed in the software-based EWA splatting approach[ZPvBG01]. Instead of splatting color values into the frame-buffer, they use multiple buffers into which they splat normalvectors and material properties. As a consequence, normalsand colors of overlapping splats are smoothly interpolatedand averaged into the pixels they cover, with weights de-pending on the respective EWA filter kernels evaluated atthat pixel. In a final pass over each image pixel, lightingcomputations are performed based on the pixel’s accumu-lated normal vector and surface material.

c! The Eurographics Association 2005.

M. Botsch, A. Hornung, M. Zwicker, L. Kobbelt / High-Quality Surface Splatting

timings and splat rates given in this paper were measuredusing this configuration.

Most of the above methods neglect the screen space fil-ter of the EWA framework and restrict to the Gaussianreconstruction filter in object space. While this leads tosufficient anti-aliasing in magnified regions, it cannot pre-vent aliasing artifacts in minified areas. In contrast, themethod of [ZRB!04] implements the full EWA splatting ap-proach on the GPU, but due to the complex computationsits performance is limited to about 4M splats/sec using theabove mentioned hardware configuration. Finally, the useof deferred shading techniques has shown to allow for effi-cient high-quality per-pixel shading for surface splatting ap-proaches such as EWA splatting as well as splat-based vol-ume rendering [MMC99].

Using per-pixel Phong shading and a simple but effec-tive approximation to the screen space filter, the approachpresented in this paper provides results comparable to theoriginal EWA splatting. By exploiting deferred shading tech-niques we achieve this superior visual quality at a rate ofabout 23M splats/sec, such that our method is close toEWA splatting in terms of quality and close to the fast butlow-quality renderer of [BK03], which achieves about 27Msplats/sec.

3. Splat RasterizationIn this section we shortly describe the perspectively cor-rect rasterization of elliptical splats, which was introduced in[BSK04]. This method will then be used in the next sectionto accumulate normal and color contributions of individualsplats in the rendering buffer.

Following the notation of [BSK04], a splat S j is definedby its center c j and two orthogonal tangent directions u j andv j. These tangent vectors are scaled according to the princi-pal radii of the elliptical splat, such that an arbitrary point qin the splat’s embedding plane lies in the interior of the splatif its local parameter values u and v satisfy the condition

u2 + v2 =!

uTj (q! c j)

"2+

!vT

j (q! c j)"2" 1 . (1)

The rasterization of a splat S j is performed by sendingits center c j, tangent axes

#u j,v j

$, and optional material

properties to OpenGL, which are then processed by customshaders for both the vertex and the pixel stage. The vertexshader conservatively estimates the size d of the projectedsplat based on a perspective division of the larger of the el-lipse radii r by the eye-space depth value cz of the splat cen-ter, followed by a window-to-viewport scaling as describedin [BSK04].

This causes the single OpenGL vertex c to be rasterizedas a d # d image space square, each pixel (x,y) of whichis then tested by a pixel shader to lie either inside or out-side of the projected elliptical splat contour. Local ray cast-ing through the corresponding projected point qn on the near

plane yields the eye-space point q on the splat’s supportingplane. From this projectively exact 3D position the local pa-rameter values (u,v) can be determined and tested as shownin (1). While pixels corresponding to points outside the splatare discarded, pixels belonging to the splat are accepted andprocessed further. If a pixel (x,y) is accepted, its weightingfactor is determined as

w(x,y) = h!%

u2 + v2"

, (2)

where h(·) is typically chosen as a Gaussian. To allow forexact blending and occlusion, the pixel’s depth value has tobe adjusted as described in [BSK04] in order to correspondto the computed 3D position q. This finally results in a per-pixel projectively correct rasterization of elliptical splats.

The output of the rasterization pixel shader are depth val-ues only for the visibility pass, and additionally weightedsplat attributes, such as normal vectors or color values, in thesecond pass, which are then accumulated in the render targetby additive alpha blending. The final normalization/shadingpass then processes each pixel in order to compute its finalcolor, as described in the next section.

4. Hardware-Accelerated Deferred Shading

For both point-based models and polygonal meshes, one ma-jor requirement for high-quality visualization is the use ofper-pixel Phong shading based on interpolated normal vec-tors, instead of Gouraud shading, which blends color con-tributions resulting from lighting each splat or mesh ver-tex, respectively. In contrast to polygonal meshes, point-based models do not store any neighborhood relation be-tween splats, therefore an equivalent interpolation of neigh-boring splats’ normal vectors is not possible.

In order to still be able to generate smoothly interpolatedper-pixel normal vectors, two basic approaches are possible.The first is to associate with each splat a pre-computed linearnormal field, as proposed in the Phong splatting approach[BSK04]. However, while leading to a high-quality shad-ing, this method is limited to static geometries and boundto about 6M splats/sec, as mentioned in Section 2.

The second approach for normal interpolation wasproposed in the software-based EWA splatting approach[ZPvBG01]. Instead of splatting color values into the frame-buffer, they use multiple buffers into which they splat normalvectors and material properties. As a consequence, normalsand colors of overlapping splats are smoothly interpolatedand averaged into the pixels they cover, with weights de-pending on the respective EWA filter kernels evaluated atthat pixel. In a final pass over each image pixel, lightingcomputations are performed based on the pixel’s accumu-lated normal vector and surface material.

c! The Eurographics Association 2005.

Computer Graphics Group

Mario Botsch

3-Pass Deferred Shading

VisibilitySplatting

AttributeSplatting

NormalizationShading

HW-Accelerated Deferred Shading

Generate Normal map

Avoid holes

Normals

• Interpolate normal vectors

• No connectivity like for meshes

• Assign linear normal field

• Static geometries

• Splat wrt. normals, generate normal map in multi-pass rendering

M. Botsch, A. Hornung, M. Zwicker, L. Kobbelt / High-Quality Surface Splatting

Another important point to be considered is the precisionof the render targets. The standard framebuffer used in pre-vious approaches offers 8 bits for each of the four RGBAcomponents. As an additional constraint, these color channelalso have to be clamped to [0,1]. This leads to the frequentlyobserved shading artifacts due to color buffer overflows orinsufficient precision for the sum of weights stored in the al-pha channel. The NV40 GPU generation now allows to useun-clamped floating point values for render targets, whicheffectively avoids these problems (cf. Fig. 2). This is espe-cially important as in addition to colors we also accumulatenormal vectors, where noise due to discretization would im-mediately lead to shading artifacts.

Figure 2: Standard framebuffers provide 8 bit precision foreach channel and clamp color values to [0,1]. Due to largeoverlaps of individual splats, these buffers may overflow dur-ing accumulation, resulting in the too bright and sparkledleft image. Using floating point render targets (on the sameillumination conditions) effectively avoids these problems(right).

5. EWA Approximation

In the original EWA surface splatting, two components areresponsible for high visual quality: per-pixel Phong shad-ing, which can be mapped to the GPU as shown in the lastsection, and anisotropic anti-aliasing provided by the EWAfilter.

The complete EWA filter is composed of an object-spacereconstruction kernel (the weight function of Eq. (2)) and aband-limiting screen-space pre-filter. As the required com-putations are quite involved, many rendering approachessimply omit the screen-space filter and use the reconstruc-tion kernel only. However, in the case of extreme minifica-tion, when the size of projected splats falls below one pixel,the signal corresponding to the accumulated projected splatsmay have frequencies higher than the Nyquist frequency ofthe pixel sampling grid, resulting in the alias artifacts shownin the top image of Fig. 3.

Figure 3: The object-space reconstruction filter alone can-not avoid aliasing in minification regions (top). Full-screenanti-aliasing removes aliasing to some degree, but the super-sampled image can still contain sampling artifacts (center).Our approximation to the EWA filter band-limits the signalbefore it is sampled on the pixel grid and hence successfullyremoves the aliasing problems (bottom).

An appealing idea might be to diminish these aliasing arti-facts by full-screen anti-aliasing (FSAA), which is supportedby any modern graphics hardware. In general, FSAA redi-rects the rendering to a higher resolution framebuffer in or-der to achieve a (typically 2! 2 or 3! 3) super-samplingof the image signal. This buffer is then scaled down to theactual framebuffer resolution using linear or Gaussian filter-ing. The problem with that approach is that even the highresolution super-sampling buffer might suffer from aliasing,in which case a high resolution aliased image will be down-scaled to the framebuffer. The resulting image will still con-tain alias artifacts (cf. Fig. 3, center).

We propose a simple — and hence efficient — heuristicfor approximating the EWA screen-space filter. By restrict-ing the size of projected splats to be at least 2! 2 pixels itis guaranteed that enough fragments are generated for anti-aliasing purposes, even for splats projecting to sub-pixel ar-

c! The Eurographics Association 2005.

Final normalization step to fix saturated lighting

Normalization Shading

EWA approximation on screen space

• A heuristic approximating EWA screen-space filter

• Generate enough fragments for anti-aliasing purpose, done in vertex shader

• Limit projected splat to be at least (2*rho)*(2rho) pixels

• Combine minimum, done in fragment shader

• #frag/#pixel increases to about 30, need DS

Approximate EWA Filtering

Computer Graphics Group

Mario Botsch

Approximate EWA Filtering

• Reconstruction filter radius

• Screen-space filter radius

• Combined filter

r2D := d (x, y) /!

r3D :=

!

u2 + v

2

r := min {r3D, r2D}

r3D ! 1

r2D ! 1

r ! 1w := Gauss (r)

Computer Graphics Group

Mario Botsch

Approximate EWA Filtering

No filtering Object-SpaceObject-SpaceScreen-Space

No filter Object space Object and screen space

Some Results

Computer Graphics Group

Mario Botsch

Phong ShadingPhong shading

M. Botsch, A. Hornung, M. Zwicker, L. Kobbelt / High-Quality Surface Splatting

Visibility Pass Attribute Pass Shading Pass

Figure 1: The deferred shading pipeline for GPU-based splatting. The visibility pass fills the z-buffer, such that the attributepass can correctly accumulate surface attributes, like color values and normal vectors, in separate render targets. The finalshading pass computes the actual color value for each image pixel based on the information stored in these render targets.

Figure 5: From left to right: The Phong-shaded octopus model and NPR-shaded renderings of the dinosaur model, the Igeaartifact, and the massive Lucy dataset. All models are rendered with shadow mapping enabled and hence require one additionalvisibility rendering pass for the shadow map generation.

c! The Eurographics Association 2005.

SpeedM. Botsch, A. Hornung, M. Zwicker, L. Kobbelt / High-Quality Surface Splatting

Model #splats Overdraw Phong-SM NPR-SM Phong+SM NPR+SM [BK03] [BSK04]

Balljoint 137k 5.9 / 7.2 20.1 15.5 13.3 11.9 24.8 4.5Max 655k 7.9 / 15.4 22.0 20.4 14.9 14.5 27.0 5.9David Head 1.1M 6.4 / 14.4 23.9 22.6 16.3 15.8 27.1 5.6David Head 4.0M 6.9 / 37.7 26.0 25.3 17.5 17.2 31.3 5.5David 8.3M 7.0 / 202 22.6 22.0 16.2 15.5 19.6 4.6Lucy 14M 19.3 / 242 22.6 22.1 15.9 15.2 20.2 5.0

Table 1: This table shows the performance of our rendering approach in million splats per second for a 512! 512 window,using a GeForce 6800 Ultra GPU. We give timings for several different shaders (Phong shading, NPR shading, with and withoutshadow mapping) and compare to the fast but low-quality splatting of [BK03], and the high-quality but expensive Phongsplatting [BSK04]. Due to per-pixel Phong shading and anti-aliasing, the quality of our method is superior even to [BSK04],while the rendering performance is still comparable to [BK03]. The third column shows the average number of fragmentscontributing to resulting image pixels, without and with our anti-aliasing technique. Since the latter generates significantlymore fragments for complex models, the acceleration provided by the deferred shading approach is even more important.

References[ABCO!01] ALEXA M., BEHR J., COHEN-OR D., FLEISHMAN

S., LEVIN D., SILVA C. T.: Point set surfaces. InProc. of IEEE Visualization 01 (2001), pp. 21–28.

[BK03] BOTSCH M., KOBBELT L.: High-quality point-basedrendering on modern GPUs. In Proc. of PacificGraphics 03 (2003), pp. 335–343.

[BK05] BOTSCH M., KOBBELT L.: Real-time shape editingusing radial basis functions. In Proc. of Eurographics05 (2005).

[BSK04] BOTSCH M., SPERNAT M., KOBBELT L.: Phongsplatting. In Proc. of symposium on Point-BasedGraphics 04 (2004).

[DWS!88] DEERING M., WINNER S., SCHEDIWY B., DUFFYC., HUNT N.: The triangle processor and normal vec-tor shader: a vlsi system for high performance graph-ics. In Proc. of ACM SIGGRAPH 88 (1988), pp. 21–30.

[GGC98] GOOCH A., GOOCH B., COHEN E.: A non-photorealistic lighting model for automatic technicalillustration. In Proc. of ACM SIGGRAPH 98 (1998),pp. 447–452.

[GP03] GUENNEBAUD G., PAULIN M.: Efficient screenspace approach for hardware accelerated surfel ren-dering. In Proc. of Vision, Modeling, and Visualiza-tion 03 (2003).

[KB04] KOBBELT L., BOTSCH M.: A survey of point-based techniques in computer graphics. Computers& Graphics 28, 6 (2004), 801–814.

[KV01] KALAIAH A., VARSHNEY A.: Differential point ren-dering. In Proc. of Eurographics Workshop on Ren-dering Techniques 2001 (2001).

[MMC99] MUELLER K., MÖLLER T., CRAWFIS R.: Splat-ting without the blur. In Proc. of IEEE Visualization(1999), pp. 363–370.

[PKKG03] PAULY M., KEISER R., KOBBELT L., GROSS M.:Shape modeling with point-sampled geometry. InProc. of ACM SIGGRAPH 03 (2003), pp. 641–650.

[PSG04] PAJAROLA R., SAINZ M., GUIDOTTI P.: Confetti:Object-space point blending and splatting. IEEETransactions on Visualization and Computer Graph-ics 10, 5 (2004), 598–608.

[PZvBG00] PFISTER H., ZWICKER M., VAN BAAR J., GROSSM.: Surfels: Surface elements as rendering primitives.In Proc. of ACM SIGGRAPH 00 (2000), pp. 335–342.

[RL00] RUSINKIEWICZ S., LEVOY M.: QSplat: a multires-olution point rendering system for large meshes. InProc. of ACM SIGGRAPH 00 (2000), pp. 343–352.

[RPZ02] REN L., PFISTER H., ZWICKER M.: Object spaceewa surface splatting: A hardware accelerated ap-proach to high quality point rendering. In Proc. ofEurographics 02 (2002), pp. 461–470.

[SP04] SAINZ M., PAJAROLA R.: Point-based renderingtechniques. Computers & Graphics 28, 6 (2004), 869–879.

[WK04] WU J., KOBBELT L.: Optimized subsampling ofpoint sets for surface splatting. In Proc. of Eurograph-ics 04 (2004), pp. 643–652.

[WTG04] WICKE M., TESCHNER M., GROSS M.: CSG treerendering for point-sampled objects. In Proc. of Pa-cific Graphics 04 (2004), pp. 160–168.

[ZPvBG01] ZWICKER M., PFISTER H., VAN BAAR J., GROSSM.: Surface splatting. In Proc. of ACM SIGGRAPH01 (2001), pp. 371–378.

[ZRB!04] ZWICKER M., RÄSÄNEN J., BOTSCH M., DACHS-BACHER C., PAULY M.: Perspective accurate splat-ting. In Proc. of Graphics Interface 04 (2004).

c" The Eurographics Association 2005.

Fragments/Pixel with/out AA Low

QualityHigh

Quality

Quality

Computer Graphics Group

Mario Botsch

Comparison

Persp. Correct

Phong Shading

Anti-Aliasing

Splats/sec

EWA

Splatting ! ! ! 1M

NV30 Splatting

PG ‘03 ! ! ! 27M

Persp. Accurate

GI ‘04 (!) ! ! 5M

Phong Splatting

PBG ‘04 ! ! ! 6M

NV40 Splatting

PBG ‘05 ! ! (!) 23M

Drawbacks

• Flexibility of z-buffering

• Need expensive two render passes for visibility splatting and attribute blending

Conclusion

• New GPU features

• MRT - Multiple Render Target

• High precision floating point pipe

• Arithmetic, buffers, textures, blending

• High performance and high quality splatting

• Deferred shading

• EWA approximation

Questions?

References

• http://people.csail.mit.edu/matthias/

• http://graphics.cs.cmu.edu/projects/adpewa/

• http://www-i8.informatik.rwth-aachen.de/publications/publications.html