Vis Comput (2009) 25: 549–557DOI 10.1007/s00371-009-0335-3

O R I G I NA L A RT I C L E

Towards multi-perspective rasterization

Xuan Yu · Jingyi Yu · Leonard McMillan

Published online: 19 March 2009© Springer-Verlag 2009

Abstract We present a novel framework for real-timemulti-perspective rendering. While most existing approachesare based on ray-tracing, we present an alternative approachby emulating multi-perspective rasterization on the classicalperspective graphics pipeline. To render a general multi-perspective camera, we first decompose the camera intopiecewise linear primitive cameras called the general lin-ear cameras or GLCs. We derive the closed-form projec-tion equations for GLCs and show how to rasterize trianglesonto GLCs via a two-pass rendering algorithm. In the firstpass, we compute the GLC projection coefficients of eachscene triangle using a vertex shader. The linear raster onthe graphics hardware then interpolates these coefficients ateach pixel. Finally, we use these interpolated coefficients tocompute the projected pixel coordinates using a fragmentshader. In the second pass, we move the pixels to their ac-tual projected positions. To avoid holes, we treat neighbor-ing pixels as triangles and re-render them onto the GLC im-age plane. We demonstrate our real-time multi-perspectiverendering framework in a wide range of applications in-cluding synthesizing panoramic and omnidirectional views,rendering reflections on curved mirrors, and creating multi-perspective faux animations. Compared with the GPU-based

Electronic supplementary material The online version of this article(http://dx.doi.org/10.1007/s00371-009-0335-3) containssupplementary material, which is available to authorized users.

X. Yu · J. Yu (�)University of Delaware, Newark, USAe-mail: jingyiyu@udel.edu

X. Yue-mail: xuanyu@udel.edu

L. McMillanUniversity of North Carolina at Chapel Hill, Chapel Hill, USA

ray tracing methods, our rasterization approach scales betterwith scene complexity and it can render scenes with a largenumber of triangles at interactive frame rates.

Keywords Multi-perspective rendering · Graphicshardware · Real-time rendering · Rasterization

1 Introduction

A perspective camera captures the spatial relationships ofobjects in a scene as they appear from a single viewpoint.In art, the use of perspective cameras is surprisingly rare:artists, architects, and engineers regularly combine what isseen from several viewpoints into a single image. Despitetheir incongruity of view, these multi-perspective images areable to preserve spatial coherence. They can depict, withina single context, details of a scene that are simultaneouslyinaccessible from a single view.

In Computer Graphics, image-based approaches are com-monly used to render multi-perspective images [3, 27, 30].Their rendering quality depends on the sampling of theplenoptic function [1] and aliasing artifacts can be intro-duced during initial sampling and final reconstruction. Alter-natively, ray tracing can be used to render high quality multi-perspective cameras. However, ray tracing is often too slowfor interactive rendering. Recently, GPU-based ray tracingtechniques [4, 19] have been developed to efficiently de-termine ray–object intersections. However, a fundamentalproblem in GPU ray tracing is that it requires random ac-cess to a database of scene triangles, which does not fit wellwith the SIMD feed-forward graphics pipeline [11].

In this paper, we present a novel rasterization frame-work for rendering multi-perspective images in real-time. To

550 X. Yu et al.

Fig. 1 Our framework interactively renders a complex city scene of 15k triangles at 70 fps. A cross-slit camera is used to render the scene at an1024 × 320 resolution

render a general multi-perspective camera, we first decom-pose the camera into piecewise primitive multi-perspectivecameras. We use the recently proposed general linear cam-eras (GLC) model that describes typical pinhole and ortho-graphic cameras, as well as many commonly studied multi-perspective cameras. We derive the closed-form projectionequations for GLCs and show how to rasterize triangles ontothe GPU via a two-pass rendering algorithm.

In the first pass, we compute the GLC projection coeffi-cients of each scene triangle using a vertex shader. The lin-ear raster on the graphics hardware then interpolates thesecoefficients at each pixel. Finally, we use these interpolatedcoefficients to compute the projected pixel coordinates us-ing a fragment shader. In the second pass, we move thesepixels to their actual projected positions. To avoid holes, wetreat neighboring pixels as triangles and re-render them ontothe GLC image plane. We demonstrate our real-time multi-perspective rendering framework in a wide range of applica-tions including synthesizing panoramic and omnidirectionalviews, rendering reflections on curved mirrors, and creat-ing multi-perspective faux animations. Compared to GPU-based ray tracing methods, our rasterization approach scalesbetter with scene complexity and can render highly complexscenes with a large number of triangles at interactive framerates.

2 Previous work

Historically, multi-perspective images have been frequentlyemployed by pre-Renaissance and post-impressionist artiststo depict more than can be seen from any specific point[29]. Escher uses highly curved projection models to gener-ate “impossible” perspectives of a scene. Picasso and otherCubism pioneers made effective use of rearranging differ-ent parts of the depicted scene while maintaining their localspatial relationships, which results in an incongruous spatialsystem [8].

In Computer Graphics, multi-perspective images havebeen widely used in cel-animations [27] and omnidirectional

visualizations [15, 20]. A commonly used technique for cre-ating these images is to combine strips from different pin-hole images. This approach, often called a strip camera, hasappeared quite often in Graphics literature. For example,computer generated multi-perspective panoramas [27] com-bined elements of multiple pinhole strips into a single imageusing a semi-automatic image registration process. The con-centric mosaics of [24] and [17] are another type of multi-perspective image that is useful for exploring captured envi-ronments.

Durand [6] suggests that specifying multi-perspectivecameras can also be an interactive process and uses themas an example to distinguish between picture generation anduser interaction. Examples of such approaches include the3D-based interactive rendering systems by Agrawala et al.[3] and Hanson and Wernert [10]. Roman et al. [21, 22]provide a semi-interactive system that uses a linear cam-era to combine photographs into panoramas of street scenes.Agrawala et al. [2] proposed to composite large regions ofordinary perspective images. They reduce the degree of userinteraction by identifying the dominant plane and then usegraph cuts to minimize multi-perspective distortions.

Recently, real-time multi-perspective rendering tech-niques have been developed based on polygonal graphicshardware. These include techniques for supporting multiplecenters of projection in VR applications [12, 25], render-ing general curved reflections or refractions [16, 28], andsynthesizing special panoramic effects [7, 29]. The work byHou et al. [11] decomposes an arbitrary multi-perspectiveimage into piecewise-linear multi-perspective primitives.They then render each primitive camera by implementing anon-linear beam-tracing using a pair of vertex and fragmentprograms.

Finally, multi-perspective images have received attentionfrom the computer vision community for analyzing struc-ture revealed via motion [17, 23] and generating panoramicimages with a wide field-of-view using mirrors [26]. Sev-eral researchers have proposed alternative multi-perspectivecamera models which capture rays from different points inspace. These multi-perspective cameras include pushbroomcameras [9], which collect rays along parallel planes from

Towards multi-perspective rasterization 551

Fig. 2 A diagram showing the pipeline of our rendering algorithm

Fig. 3 (a) A General Linear Camera is described by three generatorrays parametrized under two parallel planes (2PP). (b) We can decom-pose a reflection image of a curved mirror into piecewise GLCs

points swept along a linear trajectory, and two-slit cameras[31], which collect all rays passing through two lines.

3 Multi-perspective decomposition

To render an arbitrary multi-perspective camera, we first de-compose the camera into piecewise linear primitive multi-perspective cameras whose projection models can be easilycharacterized.

3.1 General linear cameras

Our decomposition is based on the recently proposed gen-eral linear camera or GLCs [30]. GLCs unify traditional per-spective, orthographic, and multi-perspective cameras mod-els such as the pushbroom and the cross-slit cameras. Inthe GLC framework, every ray is parameterized by its in-tersections with the two parallel planes, where [s, t] is theintersection with the first and [u,v] the second, as shown inFig. 3(a). This parametrization is often called a two-planeparametrization (2PP) [13]. In this paper, we further sim-plify the analysis of [30] by substituting σ = s − u and

τ = t − v and use [σ, τ,u, v] to represent rays. We also as-sume the default uv plane is the default image plane and isat z = 0 while st plane is at z = 1.

A GLC is defined as the affine combination of three rays:

GLC = {r : r = α · [σ1, τ1, u1, v1] + β · [σ2, τ2, u2, v2]

+ (1 − α − β) · [σ3, τ3, u3, v3],∀α,β}. (1)

We treat the problem of multi-perspective rendering asone of specifying the set of rays collected by the camera.Yu and McMillan [29] have shown that these GLC cam-eras, when constrained by an appropriate set of rules, canbe laid out on the image plane, thereby generating arbitrarymulti-perspective renderings. In fact, to a first order, theGLC-based framework can describe any multi-perspectiveimage that is consistent with a smoothly varying set ofrays. In Sect. 5.1, we demonstrate rendering 180 or 360degree panoramas by stitching piecewise cross-slit GLCs.In Sect. 5.3, we show, by dynamically composing a multi-perspective camera from smoothly varying specific GLCs,that it is also possible to create faux animations from staticmodels.

GLCs can also be used to render reflections on arbitrar-ily curved mirrors (Sect. 5.2). Given a mirror surface rep-resented as a triangle mesh, we can associate the reflectionray with each vertex so that each triangle corresponds to aray triplet, as shown in Fig. 3(b). We then specify the uv

parametrization plane for each ray triplet as the plane of thetriangle itself and render each GLC individually. Finally, wecan compose multiple GLC images into a single reflectionimage.

Next, we derive the GLC projection equation. To furthersimplify our analysis, we assume the uv plane is the im-age plane of the GLC. We also assume the three generatorshave the form [σ1, τ1,0,0], [σ2, τ2,1,0], and [σ3, τ3,0,1],

552 X. Yu et al.

Fig. 4 (a) Projecting a point P to a ray in the GLC. (b) The projectionof P can be computed using the affine coordinate on the sweeping planeΠz

i.e., these three generator rays originate from pixels [0,0],[1,0], and [0,1], respectively. Under these three generatorrays, every ray r in the GLC can be written as the followingaffine combination:

r[σ, τ,u, v] = (1 − α − β) · [σ1, τ1,0,0]+ α · [σ2, τ2,1,0] + β · [σ3, τ3,0,1]. (2)

It is easy to see that α = u and β = v under this simplifi-cation. Therefore, computing the affine coefficients of r isequivalent to projecting r onto the image plane.

To compute the projection of a 3D point P(x, y, z) inthe GLC, we sweep a plane Πz parallel to the uv plane andpassing through P . The three generator rays will intersectΠz at T1, T2, T3, where

T1 = (0,0,0) + z · (σ1, τ1,1) = (σ1z, τ1z, z),

T2 = (1,0,0) + z · (σ2, τ2,1) = (σ2z + 1, τ2z, z), (3)

T3 = (0,1,0) + z · (σ3, τ3,1) = (σ3z, τ3z + 1, z).

Our goal is to compute the affine combination [α,β] ofthe three generator rays that pass through P . Notice, thesame affine coefficients should apply to point P with respectto triangle �T1T2T3, i.e.,

P = (1 − α − β) · T1 + α · T2 + β · T3. (4)

Recall that [α,β] can be computed using the ratio ofthe signed areas formed by triangle �T1P T3, �T1T2P over�T1T2T3 as shown in Fig. 4(b); we have

u = �T1P T3

�T1T2T3=

∣∣∣∣∣∣

zσ1 zτ1 1x y 1

zσ3 1 + zτ3 1

∣∣∣∣∣∣

∣∣∣∣∣∣

zσ1 zτ1 11 + zσ2 0 + zτ2 1

zσ3 1 + zτ3 1

∣∣∣∣∣∣

,

v = �T1T2P

�T1T2T3=

∣∣∣∣∣∣

zσ1 zτ1 11 + zσ2 zτ2 1

x y 1

∣∣∣∣∣∣

∣∣∣∣∣∣

zσ1 zτ1 11 + zσ2 0 + zτ2 1

zσ3 1 + zτ3 1

∣∣∣∣∣∣

. (5)

For GLC-based decompositions, the smoothness is guar-anteed if all cameras are parameterized under the same2PP. However, when GLCs are used to model more com-plicated phenomenons such as reflections, it is commonpractice to use different parametrization planes (e.g., localtangent planes), for minimizing the approximation errors.Therefore, the projection continuity for two adjacent GLCsis not guaranteed in a general configuration [18]. In [11],Hou et al. proposed a similar multi-perspective camera de-composition scheme. Their approach used the normalizedvector [nx,ny, nz] to represent the ray direction instead ofthe 2PP. Their parametrization naturally maintains smooth-ness across the camera boundaries. However, their projec-tion equation is more complicated and it may have multiple(up to 4) solutions, and hence, additional steps are needed todetermine the actual solution. Popescu et al. [18] extendedGLCs to a continuous form, but at the cost of a cubic projec-tion equation. For multi-perspective rasterization, the linearprojection not only has the advantage of simpler computa-tion, but also the more important advantage of single projec-tion.

4 Multi-perspective rasterization

Our goal is to emulate GLC-based triangle rasterization onthe perspective camera graphics pipeline. In this section, wefirst briefly review the perspective-camera based linear ras-terization and then identify the difficulty in implementingGLC rasterization. Finally, we present a simple and efficienttechnique for emulating GLC rasterization on the GPU.

4.1 Linear rasterization

In a perspective camera, 3D lines project to 2D lines viaperspective projection. Therefore, to render a triangle, wecan simply first project the vertices of the triangle onto theimage plane of the camera and then linearly interpolate thepixels between the projected vertices. Therefore, the rasteron the graphics hardware serves simply as a linear interpola-tion engine. In contrast, a triangle maps to a curved trianglein a multi-perspective camera. Thus, the linear raster cannotbe directly used to rasterize the triangles.

Modern graphics hardware has been focused on addingprogrammable shading to their capabilities, to allow eachvertex or pixel be processed by a short program [5, 14]. For

Towards multi-perspective rasterization 553

example, it is easy to write simple vertex and fragment pro-grams to support Phong shading using per-pixel-based light-ing, surface normal and positions. However, very little efforthas been made to change the linear raster. In fact, rasteri-zation of curved triangle still remains an open problem incomputer graphics.

4.2 Emulating GLC rasterization

We present a simple and efficient GLC rasterization schemeon the GPU. Our goal is to use the raster to interpolate thelinear components in the GLC projection equation and relyon the fragment shader to conduct the nonlinear operations.

Recall that both the numerator and the denominator in theGLC projection equation (5) can be written as the determi-nant of matrices. Furthermore, all entries in the two matricesare linear in the x, y, and z coordinates of 3D points. There-fore, we can linearly interpolate these projection coordinateswithin the triangle. Specifically, to render each scene trian-gle, we first compute the per-vertex-based projection coef-ficients (i.e., each entry in the numerator and denominatormatrices) using a vertex shader. The raster then interpolatesthese coefficients. On the fragment shader, we then com-pute the projected pixel coordinates using the GLC projec-tion equation and store them in a source texture S. Next, weneed to move these pixels from S to their actual projectedpositions in the GLC image T . A simple solution is to mapeach pixel in S to a pixel in T . However, this simple map-ping will introduce holes in the target GLC image, i.e., somepixels in T may not correspond to any pixels in S due todiscretization. To resolve this problem, we treat every threeneighboring pixels in S as a triangle and re-render these tri-angles onto T . Similar to frustum culling for perspectivecameras, we also discard triangles that lie completely out-side the image plane in the second pass. Fig. 2 shows thecomplete pipeline of our approach.

To maintain correct depth ordering, we also compute per-pixel ray depth. Specifically, once we compute the projectedpixel coordinate [u,v] of a 3D point P(x, y, z) in the GLC,we calculate the ray depth dP as ‖(x − u)2 + (y − v)2 +z2‖1/2. This step can be efficiently combined with the GLCprojection at the end of the first pass in the same fragmentshader. When we re-rasterize the triangles from S to T in thesecond pass, we simply use z-buffer to resolve the visibilityproblem.

4.3 GLC projection equation

To render the complete 3D scene, it is important that werasterize each scene triangle without overlapping in the firstpass. We benefit from the large texture memory size oncommodity graphics card. In our experiment, we use upto 1K × 1K textures and each scene triangle is set to have10×10 image resolution. This allows us to interactively ren-der complex scenes with over 10K triangles.

4.4 Results

We have implemented our algorithm using DirectX 9.0cwith Shader model 3.0. All experiments are recorded on aPC with 2.13 GHz intel Core2 Duo CPU, 2 GB memory,and an NVidia 8800 GTX graphics card. Figure 10 summa-rizes the performance data. The computational overhead inour GLC rasterization pipeline is comparable to the classicalperspective rendering. This is because to render a pinholecamera, all scene vertices need to be transformed via per-spective transformation and tested for view frusturm cullingwhereas our GLC rasterization scheme separates the pro-jection and culling in two separate passes. Our approach,however, will incur larger overhead due to context switch-ing between the two passes. When rendering cameras usinga single or a small number of GLCs, the overhead causedby context switching is negligible and our algorithm scaleslinearly with the resolution used to render each scene trian-gle. However, when rendering a large number of GLCs, thisoverhead can severely affect the performance of our algo-rithm (Sect. 6).

In Fig. 5, we render a cross-slit camera and comparethe rendering result with ray tracing. We have modified thePovRay ray-tracer to support all types of GLCs. Our testscene consists of geometric primitives like spheres, cylin-ders, and boxes. We discretize the sphere by 260 triangles,the cylinder by 40 triangles, and the box by 12 triangles. InFig. 5(b), we set the scene triangle resolution to be 5 × 5pixels and we observe slight polygonization artifacts on theground plane. However, since we re-render the projectedpixels as triangles in the second pass, the final renderingdoes not contain holes. Furthermore, as we increase the reso-lution for rasterizing the scene triangles, the polygonizationartifacts disappear.

5 Applications

5.1 Multi-perspective panoramas

Multi-perspective panoramas attempt to combine a series ofviews into a single image that is viewed in pieces. One wayto construct multi-perspective panoramas is to stitch GLCstogether. In Fig. 7(a), we approximate a circular cross-slitpanorama [31] by stitching piecewise linear cross-slit GLCs.Specifically, all these GLCs share a common slit that passesthrough the center of the circle. We then discretize the circleusing a sequence of slit segments. In Fig. 6(b), we render a360 degree view of a cow model of 5800 triangles. We areable to fuse both the head and the tail of the cow into a singlemulti-perspective image. Our system renders at 47 fps andwe can interactively rotate the cow as shown in the supple-mentary video.

554 X. Yu et al.

Fig. 5 (a) Rendering a cross-slit camera using ray tracing. (b)–(d)show the rendering result of our approach using 5 × 5 (b), 10 × 10 (c),and 20 × 20 (d) rasterization resolution for scene triangles. Notice at

a low rasterization resolution, polygonization artifacts can occur in thefinal rendering (e.g., the ground plane in (b))

Fig. 6 Real-time rendering of 360 degree panoramas using GLC rasterization. (a) A perspective image of a cow model that consists of 5800triangles. (b) Our panoramic rendering of the cow model at 47 fps at an 800 × 600 resolution

In Fig. 7, we render an omnidirectional view of a teapotmodel that consists of 14k triangles at 32 fps at a 800 × 600resolution, reminiscent of an MCOP image [20]. The classi-cal pinhole camera 7(b) cannot simultaneously illustrate thenose and the handle of the teapot. Using multi-perspectiverendering, we can fuse both the nose and the handle in asingle image 7(c) while maintaining low image distortions.Figure 7(d) shows the top view of the teapot.

5.2 Real-time reflection

A special class of multi-perspective images are reflectionson curved mirrors. To render reflections, we reuse the de-fault triangulation of the reflector and compute per-vertexreflection rays. We then treat each individual triangle as aGLC and use the triangle plane as the parametrization planeof the GLC. Finally, we render each GLC separately andstitch them to generate the final reflection image.

In Fig. 8, we render reflections from a mirror sphere.The butterfly scene contains 1100 triangles. We tessellatethe sphere with 960 GLC triangles and render each GLC at

a 256 × 256 resolution. We set each scene triangle to be ras-terized at a 6 × 6 resolution. Our approach is able to renderat 10 fps and the reflections appear highly smooth.

Since our algorithm uses two passes to render each GLC,context switching between each pass can introduce addi-tional overhead. In particular, with very fine triangulationof the reflector (∼1K triangles), our algorithm is slower thanthe GPU ray-tracing approach [11] where scene triangle pro-jection and ray tracing are combined in a single pass. Themain advantage of our approach over ray tracing is that itscales well with scene complexity whereas the performanceof GPU ray-tracing can significantly decrease when render-ing thousands of triangles.

5.3 Multi-perspective faux-animation

Finally, it is possible to use multi-perspective renderingto create fake or faux-animations from static models bydynamically constructing the multi-perspective images. InFig. 9, we show three frames from a synthesized animation.

Towards multi-perspective rasterization 555

Fig. 7 (a) We construct the multi-perspective panorama by stitching asequence of cross-slit cameras. (b) shows a perspective view of theteapot. (b) shows a 270 degree view of the teapot. Notice the CGI

textures near the handle and side can be simultaneously seen in thepanorama. (d) shows a top-down view of the teapot. Our system ren-ders at 32 fps at an 800 × 600 resolution

Fig. 8 Rendering reflections using the GLC rasterization framework. We decompose the mirror sphere into 960 triangles, each corresponding toa GLC. (a) We render the sphere in a butterfly scene at 10 fps. The rendering results (b) are highly smooth. (c) shows a close-up view of (b)

Fig. 9 (a) shows a perspective view of the cow model. (b), (c) Multi-perspective renderings were used to turn the head quarters of the cow in afake animation

We first construct a multi-perspective camera composed oftwo GLCs. The first GLC, a cross-slit camera, captures thetorso and the tail of the cow model. The second cross-slitGLC hinges on the first one and can rotate towards or awayfrom the head of the cow. As we rotate the second camera,

we are able to synthesize realistic head rotations of the cowmodel while maintaining continuity across the two GLCcameras. Zomet [31] used similar approach by using singlecross-slit camera to achieve rotation effects from a sequenceof images.

556 X. Yu et al.

Fig. 10 The performance of our algorithm in reflection rendering.(a) We fix the number of the scene triangles and each triangle’s ras-terization resolution while changing the number of GLCs used to ap-

proximate the mirror sphere and each GLC’s image resolution. (b) Wefix the number of GLCs and the GLC image resolution while changingthe scene complexity and each scene triangle’s rasterization resolution

6 Discussions and future work

Our GLC-rasterization framework has several advantages.First, our method is very easy to implement: the mappingfrom a 3D point to its image in the GLC is unique andcan be directly computed on the GPU whereas other multi-perspective cameras such as the ones used in [11] requiresolving higher-order polynomial equations and selecting theproper solution. Second, since we use rasterization in placeof ray tracing, the performance is expected to ride in pro-portion with advances in commodity graphics hardware. Fi-nally, our framework will naturally support fully dynamicscenes since it does not rely on acceleration data structurescommonly used for ray tracing.

The main limitation of our framework is that it does notscale well with the number of the GLCs. In our experiments,we find that this is caused by context switching betweenthe two passes. When rendering multi-perspective camerascomposed of a large number of GLCs, the overhead dueto context switching can significantly decrease the overallperformance of our algorithm. Although the ray-tracing ap-proach [11] also used multi-perspective decomposition andstitching, it is not an issue for their method as the authorscombined the bounding triangle computation and ray trac-ing in a single pass. In the future, we plan to investigatehow to combine the ray-tracing and rasterization methodsinto a single framework to handle both complex scenes andcomplex camera compositions. It may also be possible tocombine our technique with existing GPU-based algorithmsfor rendering more sophisticated visual phenomenons suchas glossy reflections, refractions and caustics under subscat-tering. Finally, we will explore efficient methods to extendthe GLCs to support both linear projection and projectioncontinuity.

We envision our proposed multi-perspective renderingframework will serve as conceptual inspiration for designingnew graphics hardware. Specifically, we anticipate that ourframework would motivate the hardware community to de-velop nonlinear rasterization units to support general multi-perspective rendering. Notice that any multi-perspective

camera can be locally approximated by the GLCs. There-fore, if the rasterization unit can support the GLC projectionmodel (i.e., quadratic rational functions), it may be used todirectly render arbitrary multi-perspective effects.

Acknowledgements This work has been supported by the NationalScience Foundation under grant MSPA-MCS-0625931.

References

1. Adelson, E.H., Bergen, J.R.: The plenoptic function and theelements of early vision. In: Computational Models of VisualProcessing, pp. 3–20 (1991)

2. Agrawala, M., Zorin, D., Munzner, T.: Artistic multiprojectionrendering. In: Proceedings of the Eurographics Workshop on Ren-dering Techniques 2000, pp. 125–136 (2000)

3. Aseem, A., Agrawala, M., Cohen, M., Salesin, D., Szeliski, R.:Photographing long scenes with multi-viewpoint panoramas. In:SIGGRAPH ’06: ACM SIGGRAPH 2006 Papers, pp. 853–861(2006)

4. Carr, N.A., Hall, J.D., Hart, J.C.: The ray engine. In: Proceed-ings of the ACM SIGGRAPH/EUROGRAPHICS Conference onGraphics Hardware (2002)

5. Diefenbach, P.J., Badler, N.I.: Multi-pass pipeline rendering: real-ism for dynamic environments. In: Proceedings of the Symposiumon Interactive 3D Graphics, pp. 59–ff. (1997)

6. Durand, F.: An invitation to discuss computer depiction. In: NPAR’02: Proceedings of the 2nd International Symposium on Non-photorealistic Animation and Rendering, pp. 111–124 (2002)

7. Gascuel, J.D., Holzschuch, N., Fournier, G., Péroche, B.: Fastnon-linear projections using graphics hardware. In: SI3D ’08: Pro-ceedings of the 2008 Symposium on Interactive 3D Graphics andGames, pp. 107–114 (2008)

8. Glassner, A.S.: Cubism and cameras: Free-form optics for com-puter graphics. Tech. Rep. MSR-TR-2000-05, January (2000)

9. Gupta, R., Hartley, R.I.: Linear pushbroom cameras. IEEE Trans.Pattern Anal. Mach. Intell. 19(9), 963–975 (1997)

10. Hanson, A.J., Wernert, E.A.: Image-based rendering with occlu-sions via cubist images. In: VIS ’98: Proceedings of the Confer-ence on Visualization ’98, pp. 327–334 (1998)

11. Hou, X., Wei, L.Y., Shum, H.Y., Guo, B.: Real-time multi-perspective rendering on graphics hardware. In: SIGGRAPH ’06:ACM SIGGRAPH 2006 Sketches, p. 79 (2006)

12. Kitamura, Y., Konishi, T., Yamamoto, S., Kishino, F.: Interactivestereoscopic display for three or more users. In: SIGGRAPH ’01:Proceedings of the 28th Annual Conference on Computer Graph-ics and Interactive Techniques, pp. 231–240 (2001)

Towards multi-perspective rasterization 557

13. Levoy, M., Hanrahan, P.: Light field rendering. In: SIGGRAPH’96: Proceedings of the 23rd Annual Conference on ComputerGraphics and Interactive Techniques, pp. 31–42 (1996)

14. Lindholm, E., Kligard, M.J., Moreton, H.: A user-programmablevertex engine. In: SIGGRAPH ’01: Proceedings of the 28th An-nual Conference on Computer Graphics and Interactive Tech-niques, pp. 149–158 (2001)

15. Mei, C., Popescu, V., Sacks, E.: The occlusion camera, In: Proc.of Eurographics 2005. Comput. Graph. Forum. 24(3), pp. 335–342(Sept. 2005)

16. Ofek, E., Rappoport, A.: Interactive reflections on curved objects.In: SIGGRAPH ’98: the 25th Annual Conference on ComputerGraphics and Interactive Techniques, pp. 333–342 (1998)

17. Peleg, S., Ben-ezra, M., Pritch, Y.: Omnistereo: Panoramic stereoimaging. IEEE Trans. Pattern Anal. Mach. Intell. 23, 279–290(2001)

18. Popescu, V., Dauble, J., Mei, C., Sacks, E.: An efficient error-bounded general camera model. In: 3DPVT ’06: Proceedings ofthe Third International Symposium on 3D Data Processing, Vi-sualization, and Transmission (3DPVT’06), pp. 121–128. IEEEComputer Society, Washington (2006). DOI: 10.1109/3DPVT.2006.26

19. Purcell, T.J., Donner, C., Cammarano, M., Jensen, H.W., Hanra-han, P.: Photon mapping on programmable graphics hardware. In:ACM SIGGRAPH 2005 Courses, p. 258 (2005)

20. Rademacher, P., Bishop, G.: Multiple-center-of-projection im-ages. Comput. Graph. 32, 199–206 (1998) (Annual ConferenceSeries)

21. Roman, A., Lensch, H.: Automatic multiperspective images. IEEETrans. Pattern Anal. Mach. Intell. 25, 741–754 (2003)

22. Roman, A., Garg, G., Levoy, M.: Interactive design of multi-perspective images for visualizing urban landscapes. In: VIS ’04:Proceedings of the Conference on Visualization ’04, pp. 537–544.IEEE Computer Society, Washington (2004)

23. Seitz, S.M., Kim, J.: Multiperspective imaging. IEEE Comput.Graph. Appl. 23(6), 16–19 (2003)

24. Shum, H.Y., He, L.W.: Rendering with concentric mosaics. In:SIGGRAPH ’99: the 26th Annual Conference on ComputerGraphics and Interactive Techniques, pp. 299–306 (1999)

25. Simon, A., Smith, R.C., Pawlicki, R.R.: Omnistereo for panoramicvirtual environment display systems. In: VR ’04: Proceedings ofthe IEEE Virtual Reality 2004, p. 67 (2004)

26. Swaminathan, R., Grossberg, M.D., Nayar, S.K.: Non-singleviewpoint catadioptric cameras: Geometry and analysis. Int. J.Comput. Vis. 66(3), 211–229 (2006)

27. Wood, D.N., Finkelstein, A., Hughes, J.F., Thayer, C.E., Salesin,D.H.: Multiperspective panoramas for cel animation. In: SIG-GRAPH ’97: Proceedings of the 24th Annual Conference on Com-puter Graphics and Interactive Techniques, pp. 243–250 (1997)

28. Wyman, C., Davis, S.: Interactive image-space techniques for ap-proximating caustics. In: Proceedings of the 2006 Symposium onInteractive 3D Graphics and Games, pp. 153–160 (2006)

29. Yu, J., McMillan, L.: A framework for multiperspective rendering.In: Rendering Techniques (2004)

30. Yu, J., McMillan, L.: General linear cameras. In: the 8th EuropeanConference on Computer Vision (ECCV) (2004)

31. Zomet, A., Feldman, D., Peleg, S., Weinshall, D., Society, I.C.:Mosaicing new views: The crossed-slits projection. IEEE Trans.Pattern Anal. Mach. Intell. 25, 741–754 (2003)

Xuan Yu is a graduate student inComputer and Information SciencesDepartment at the University ofDelaware. He received his B.Sc.from Shanghai Jiaotong Universityand is currently pursuing his Ph.D.with Dr. Jingyi Yu at the Universityof Delaware.

Jingyi Yu is Assistant Professorin Computer and Information Sci-ences Department at the Universityof Delaware. He received his B.Sc.from Caltech in 2000, and M.Sc.and Ph.D. degrees in EECS fromMIT in 2005. His research interestsspan a range of topics in computergraphics, computer vision, and im-age processing, including computa-tional photography, medical imag-ing, non-conventional optics andcamera design, tracking and surveil-lance, and graphics hardware.

Leonard McMillan is AssociateProfessor of Computer Science atthe University of North Carolinain Chapel Hill. Leonard receivedhis Bachelor’s (1983) and Master’s(1984) degrees in Electrical Engi-neering from Georgia Institute ofTechnology. Leonard received hisPh.D. in Computer Science fromthe University of North Carolinaat Chapel Hill (1997). Leonard hasbeen a Member of Technical Staffat AT&T Bell Laboratories wherehe worked in the Digital SignalProcessing Architecture Group and

was a coarchitect of the AT&T Pixel Machine. Leonard has alsoworked as a Senior Staff Engineer at Sun Microsystems where hehelped developing several visualization and multimedia products.Leonard is a pioneer in the field of image-based rendering. Image-based rendering is a new approach to computer graphics where scenesare rendered directly from a collection of reference images rather thana geometric model. Leonard is also interested in a wide range of re-lated topics, including computer graphics rendering, imaging methodsand technologies, three-dimensional display technologies, computergraphics hardware, and the fusion of image processing, multimedia,and computer graphics.