11
COMPUTER ANIMATION AND VIRTUAL WORLDS Comp. Anim. Virtual Worlds 2009; 20: 321–331 Published online 11 June 2009 in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/cav.308 ........................................................................................... Patches: character skinning with local deformation layer By Jieun Lee, Myung-Soo Kim and Seung-Hyun Yoon * .......................................................................... We present a layered geometric approach to the skinning of character animation. On top of a global shape deformation, B-spline surface patches are attached to various body parts for local control. The patches are directly manipulated so as to generate example local shapes at some important poses. During character animation, the B-spline control points move continuously by kinematically interpolating a set of example patches, and at the same time they can also move dynamically by elastic simulation. The movement of control points changes the shape of a local patch, and consequently skin vertices move according to the patch deformation, finally generating the desired local deformation. We demonstrate the effectiveness of our approach by generating natural local details at various poses including muscular effects and elastic deformation. Moreover, dealing with only a small number of control points, the proposed method is very efficient, while generating hundreds to thousands frames per second. Copyright © 2009 John Wiley & Sons, Ltd. Received: 25 March 2009; Accepted: 26 March 2009 KEY WORDS: Skinning; shape deformation; local shape control; surface-oriented deformation; muscle deformation; elastic deformation; direct manipulation Introduction Character animation is an important subject of research for computer graphics and animation with diverse applications. Digital humans and creatures have been widely used in computer games, film productions, and virtual reality applications. 1 Many effective tools are currently available for skin deformation, also known as skinning, in character animation. 2–7 There are also many approaches suggested for character animation including human characters and other creatures, and for different body parts of a character including face, hand, and body. 8–10 Nevertheless, each individual method has its own strength and weakness in certain details of the shape control. In particular, a realistic shape control over various local body parts is still a challenging task. In this paper, we propose a simple yet very effective local shape control scheme that is based on B-spline surface patches. Our approach combines dynamic as well as kinematic local shape controls in the same patch-based paradigm. *Correspondence to: S.-H. Yoon, Dongguk University, Seoul 100-715, South Korea. E-mail: [email protected] Skeleton-driven deformation is an indispensable tool for character skinning. Skeletal subspace deformation (SSD) 2 is a very popular skeletal deformation method. New positions of skin vertices are determined by a weighted sum of deformations from all influencing bones. SSD is very intuitive and easy to use; however, it is yet rather inconvenient to control the weights assigned to different local deformations. The user has difficulty obtaining intended results as the deformation is limited to the skeletal subspace. Problems often occur, in partic- ular, near the joints; typical problems include those such as buckling and collapsing. Many modifications have been suggested to alleviate the limitations of SSD. 3–6,11–13 More- over, various physically based techniques 14–16 have been developed to provide elastic deformation effects that can be combined with SSD or freeform deformations (FFD). However, these techniques mainly deal with global elas- tic deformation. Efficient techniques tailored for local elastic deformations are also needed, which is an impor- tant subject of research we attack in the current work. Shape deformation techniques usually deal with the whole body of a character. The control for a global shape is; however, considerably different from those for local parts with many shape details. It is thus quite important to develop simple control schemes that can effectively ............................................................................................ Copyright © 2009 John Wiley & Sons, Ltd.

Patches: character skinning with local deformation layer

Embed Size (px)

Citation preview

Page 1: Patches: character skinning with local deformation layer

COMPUTER ANIMATION AND VIRTUAL WORLDSComp. Anim. Virtual Worlds 2009; 20: 321–331Published online 11 June 2009 in Wiley InterScience(www.interscience.wiley.com) DOI: 10.1002/cav.308...........................................................................................Patches: character skinning with localdeformation layer

By Jieun Lee, Myung-Soo Kim and Seung-Hyun Yoon*

..........................................................................

We present a layered geometric approach to the skinning of character animation. On top ofa global shape deformation, B-spline surface patches are attached to various body parts forlocal control. The patches are directly manipulated so as to generate example local shapesat some important poses. During character animation, the B-spline control points movecontinuously by kinematically interpolating a set of example patches, and at the same timethey can also move dynamically by elastic simulation. The movement of control pointschanges the shape of a local patch, and consequently skin vertices move according to thepatch deformation, finally generating the desired local deformation. We demonstrate theeffectiveness of our approach by generating natural local details at various poses includingmuscular effects and elastic deformation. Moreover, dealing with only a small number ofcontrol points, the proposed method is very efficient, while generating hundreds tothousands frames per second. Copyright © 2009 John Wiley & Sons, Ltd.

Received: 25 March 2009; Accepted: 26 March 2009

KEY WORDS: Skinning; shape deformation; local shape control; surface-oriented deformation;muscle deformation; elastic deformation; direct manipulation

Introduction

Character animation is an important subject of researchfor computer graphics and animation with diverseapplications. Digital humans and creatures have beenwidely used in computer games, film productions, andvirtual reality applications.1 Many effective tools arecurrently available for skin deformation, also knownas skinning, in character animation.2–7 There are alsomany approaches suggested for character animationincluding human characters and other creatures, and fordifferent body parts of a character including face, hand,and body.8–10 Nevertheless, each individual method hasits own strength and weakness in certain details of theshape control. In particular, a realistic shape control overvarious local body parts is still a challenging task. In thispaper, we propose a simple yet very effective local shapecontrol scheme that is based on B-spline surface patches.Our approach combines dynamic as well as kinematiclocal shape controls in the same patch-based paradigm.

*Correspondence to: S.-H. Yoon, Dongguk University, Seoul100-715, South Korea. E-mail: [email protected]

Skeleton-driven deformation is an indispensable toolfor character skinning. Skeletal subspace deformation(SSD)2 is a very popular skeletal deformation method.New positions of skin vertices are determined by aweighted sum of deformations from all influencingbones. SSD is very intuitive and easy to use; however, it isyet rather inconvenient to control the weights assignedto different local deformations. The user has difficultyobtaining intended results as the deformation is limitedto the skeletal subspace. Problems often occur, in partic-ular, near the joints; typical problems include those suchas buckling and collapsing. Many modifications have beensuggested to alleviate the limitations of SSD.3–6,11–13 More-over, various physically based techniques14–16 have beendeveloped to provide elastic deformation effects that canbe combined with SSD or freeform deformations (FFD).However, these techniques mainly deal with global elas-tic deformation. Efficient techniques tailored for localelastic deformations are also needed, which is an impor-tant subject of research we attack in the current work.

Shape deformation techniques usually deal with thewhole body of a character. The control for a global shapeis; however, considerably different from those for localparts with many shape details. It is thus quite importantto develop simple control schemes that can effectively

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd.

Page 2: Patches: character skinning with local deformation layer

J. LEE, M.-S. KIM AND S.-H. YOON...........................................................................................

Figure 1. Examples of local shape improvements using patches.

control local shape details in addition to the global shapeof a character. Such a local shape control method needsto satisfy the following requirements:

� The control scheme supports arbitrary deformations.(Freeform surfaces are a good candidate for thisgeneral shape design capability.)

� The local deformation works in a control frameworksimilar to that for the global deformation. (The major-ity of global deformations basically work for meshdeformation; thus we propose a local method thatprovides local mesh editing capabilities.)

� The control scheme supports dynamic as well as kine-matic deformations in the same paradigm. (The controlpoints of a B-spline surface are bound to a kinematicstructure, and at the same time the network structureof B-spline control points can easily be adapted to amass-spring system for elastic simulation.)

� The system supports a natural user interface for thedesired local deformation. (Direct manipulation is anideal choice for this purpose.)

� The local deformation should be processed efficientlyso as to impose no computational burden to theoverall shape control system. (Our method generateshundreds to thousands of frames per second withouthardware acceleration.)

In this paper, we propose a patch-based local defor-mation technique that satisfies the above requirements.A patch is a B-spline surface attached to the global shapecontrol layer by interpolating a set of sampled vertices.We extract the control points of a B-spline surface, andbind them to the global shape control mechanism, which

makes the patch automatically updated according tothe global deformation. To provide desired local detailsover the global deformation, some control points movecontinuously by kinematically interpolating a set ofexamples, and at the same time their positions arealso dynamically updated by physically based elasticsimulation, during character animation. The movementof control points changes the patch shape, and thenskin vertices move according to the patch deformation,finally generating the desired local deformation.

Figure 1 shows a brief overview of our proposedapproach. Figure 1(a) shows an arm deformed by theSSD method only. There is a collapsing problem nearthe elbow, and no muscle effect in the upper arm.By attaching a patch to the arm, we can resolve theproblems such as buckling and collapsing as mentionedabove (Figure 1(b)). Figure 1(c) and (d) show the resultgenerated by our patch-based local deformation; thecollapsing problem has been resolved and the biceps ofthe arm bulge as the result of the elbow bending.

Using the patch-based local deformation, we can alsohandle elastic deformation effects for local body partsin the same shape control paradigm. Figure 1(e) showsan animal character where a patch is attached to itsabdomen for elastic effect as in Figure 1(f). Figure 1(g)and (h) demonstrate elastic deformation results incharacter motion.

Examples of desired local deformation at variousimportant poses can easily be generated by deformingthe patch. Direct manipulation of the patch providesan intuitive user interface for the shape control overexamples. A shape example at a pose determines the

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd. 322 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav

Page 3: Patches: character skinning with local deformation layer

GEOMETRIC APPROACH TO CHARACTER SKINNING...........................................................................................

B-spline control points of the deformed patch. Multipleexamples are then blended together to generate acontinuous deformation of the patch. All skin verticesin a local region are bound to the patch and they arecontinuously updated according to the smooth patchdeformation that generates desired local shape details.

We demonstrate the effectiveness of the proposedpatch-based approach using several test examples ofhuman body deformation in various body parts such aselbow, knee, shoulder, hip, and abdomen. Our methodgenerates hundreds to thousands of frames per secondwithout hardware acceleration.

Previous Work

Magnenat-Thalmann et al.2 are the first who introducedthe idea of SSD for hand mesh models. The deformationof each vertex is determined as a weighted sum of alltransformations of influencing bones to the vertex. SSDis a very popular technique for character animationdue to its simplicity and efficiency. It is, however, oftendifficult to produce intended results as the deformationis restricted to the skeletal subspace. Joint collapsingand candy wrapping are typical artifacts of SSD. (SeeLewis et al.3 for detailed discussions.) There are manyother methods3–6,11,12 developed for overcoming earlylimitations of SSD.

One approach proposes a modification of SSD usingdisplacements obtained from examples at severalposes.3,10–12 The displacements are interpolated amongdifferent poses to generate a continuously changing nat-ural deformation. A conventional strategy for extendingSSD is to employ multiple weights instead a singleweight. Wang et al.4 assign a different weight to eachcomponent of the bone transformation matrix. On theother hand, Merry et al.6 assign a different weight to eachcomponent of a vertex; as a result, the deformed positionof a vertex may be located outside the skeletal subspace.The multiple-weight methods have more flexibility inshape deformation. To determine a large number ofweights, the weights are trained on a set of exampledeformations using least-squares methods. However,the results may sometimes have an overfitting problem.6

There are also other approaches that change thecombination of skeletal transformations to producesmooth blending results around joints. Magnenat-Thalmann et al.17 and Kavan et al.18,19 blend transfor-mation matrices instead of blending vertices. To avoiddegenerate cases that may arise from interpolatingtransformation matrices, Magnenat-Thalmann et al.17

adapted the linear combination of transformationsproposed by Alexa,20 and Kavan et al.18,19 employed aquaternion representation. Extra joints can be insertedto a skeleton using the methods of Weber,21 Mohr andGleicher,22 and Hyun et al.5

Sweep-based approach smoothly parametrizes thecombination of bone transformations.5,13,23 The methodinterpolates a sequence of bone transformations usinga continuously moving frame, and binds skin verticesto sweep surfaces. The skin vertices then continuouslydeform as a sweep-based skinning. The weight assign-ment is greatly simplified. In fact, weights need to be con-trolled only in the regions where different sweeps meet.

Lee et al.13 present a human hand deformation methodthat combines a sweep-based global method witha surface-based local method. They also resolve theproblems in controlling local details such as crushesand bulges using a local surface. In their work, the localsurface is determined by a certain fixed set of featurevertices. On the other hand, in the current work wepropose a more general framework for surface-basedlocal deformation. We show that various global defor-mation methods can easily be combined to our localshape deformation method. We also provide a naturaluser interface based on direct manipulation to surfacepatches and surface fitting to selected mesh vertices.

Many physically based techniques8,16,24–26 have beenproposed for modeling elastic skin deformations.Mass-spring systems are well-established methods witha simple structure. Platt and Badler24 employed staticmass-spring systems for animating facial expressionsand Waters25 extended this method to a more sophis-ticated model. Terzopoulos and Waters8 developed a3D hierarchical model for a human face using dynamicmass-spring systems and extended this technique to fa-cial models of particular individuals using laser-scannedimage data.26 Chadwick et al.16 simulated muscles inhuman character animation by combining mass-springsystems with FFD,27 whereas other techniques8,24–26

modeled deformable objects explicitly. An importantadvantage of our surface-based approach is that elasticdeformation of local body parts can easily be combinedwith our patch-based local shape control in the sameshape control paradigm.

Patch-Based Local Deformation

Conventional global shape deformation techniqueshave limitations in controlling local details of shapedeformation. In this section, we propose a layered

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd. 323 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav

Page 4: Patches: character skinning with local deformation layer

J. LEE, M.-S. KIM AND S.-H. YOON...........................................................................................

approach for the local deformation. B-spline surfacepatches are layered on top of a global shape deformationmodule and they control the local details of variousbody parts under deformation.

Local Shape Deformation UsingSurface Patches

A B-spline surface s(u, v) is defined as

s(u, v) =m∑

i=0

n∑j=0

Np

i (u)Nq

j (v)qij (1)

where qij are the control points and Np

i (u) and Nq

j (v) arethe B-spline basis functions defined on non-uniform knotsequences respectively. The set of control points deter-mines the overall shape of a B-spline surface.

In this paper, we slightly modify the control point termqij so as to include two intuitive shape control facilities:

s(u, v) =m∑

i=0

n∑j=0

Np

i (u)Nq

j (v)[qij + dS

ij(�θ) + dEij (�f )

](2)

where the additional terms d∗ij(·) are vector-valued

displacement functions defined for each control pointqij . The displacement function dS

ij(�θ) represents the localshape deformation based on example shapes and itscontrol parameter �θ is a vector concatenating the Eulerangles of all joints influencing to the control point qij .The other function dE

ij (�f ) represents elastic displacementgenerated by a physically based elastic simulation on adamped mass-spring system for the network of controlpoints. Joint angle changes and external forces producenew displacements which can also be applied to thecorresponding control points and finally to the shapechange of a surface patch.

Example Interpolation for dSij(�θ). The displace-

ment function dSij(�θ) is constructed by interpolating

example shapes of the surface patch generated at someimportant skeletal poses �θ. We deform the surface patchby adjusting the position of qij at each of these exampleposes. The corresponding difference vector between theinitial position and the adjusted one is the displacementat that pose. Displacements are interpolated by dS

ij(�θ)at example poses. Figure 2(a)–(d) show the exampleshapes of a surface patch generated by adjusting thecontrol point q11 at four different example poses. Tointerpolate the displacements at discrete example poses

by a continuous displacement function dSij(�θ), we employ

a scattered data interpolation technique that is basedon radial-basis functions (RBFs).28 Figure 2(e)–(g) showthe surface patches generated by applying interpolateddisplacements to q11 at arbitrary skeletal poses. Thecapability of surface patch deformation provides the userwith a great facility for controlling local shape detailsand continuous shape deformations in the pose space.

Elastic Simulation for dEij(�f ). The elastic displace-

ments are obtained from the simulation applied to thedamped mass-spring system of control points as shownin Figure 2(h) and (i). Although we show the controlpoints in a global coordinate system for the purpose ofillustrating their topological connectivity, each controlpoint is actually located at the origin of a local coordinatesince only its relative displacement is needed for itsupdate. External force is applied to initiate the elasticsimulation of control points and then the simulateddisplacements of corresponding control points generateelastic changes of the surface patch. For a stable restitu-tion of a surface patch, we fix the boundary control pointsto their initial positions and add additional dampedsprings that connect the interior control points to theorigins of their local coordinates as shown in Figure 2(i).

Patch Generation and VertexBinding

To control the local details of a shape we construct asurface patch that approximates a deformable part. Forthis purpose we first determine a local region of a bodypart by selecting a set of vertices on the skin mesh. Weemploy an intuitive sketch-based scheme for the vertexselection procedure. When the user draws a rectangularregion on the display screen, the corresponding area isautomatically selected by projecting the region on theboundary surface. Figure 3(a) shows the procedure ofselecting a local region (around a knee area) using asketch-based interface.

After the selection of a region, a set V of interpolationvertices is constructed. A surface patch is then generatedby interpolating these vertices. To determine the verticesto be contained in V , we uniformly sample the selectedregion using a 2D domain [0, 1] × [0, 1] and generate aset of uniform parameters (ui, vj), i = 0, . . . , s − 1 andj = 0, . . . , t − 1. The sampled vertices of V are thenselected as the closest vertices in the selected region tothe corresponding surface points s(ui, vj). Depending onspecific applications, instead of the above set of uniform

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd. 324 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav

Page 5: Patches: character skinning with local deformation layer

GEOMETRIC APPROACH TO CHARACTER SKINNING...........................................................................................

Figure 2. Patch deformation: (a)–(d) example patches at four different example skeletal poses and (e)–(g) continuously interpolatedpatches at arbitrary skeletal poses; (h) topological connectivity for a damped mass-spring of 3 × 3 control points, and (i) additional

damped springs (in red) for stable restitution.

Figure 3. Patch generation: (a) selected local region, (b) 2D parametrization and interpolation parameters (s, t = 6), (c) interpo-lation vertices (in green), and (d) surface patch generated.

parameters, we may use non-uniform parameters, or thesampled vertices may even be selected manually by theuser. Finally, the surface patch is generated by interpo-lating the sampled vertices of V . Figure 3(b) shows theparametrization of the selected local region in the surfaceparameter domain [0, 1] × [0, 1], and Figure 3(c) and (d)

respectively show the interpolation vertices of V and thegenerated surface patch by interpolating these vertices.

Once the surface patch has been generated, all themesh vertices of the character model in the selectedregion are bound to the patch. Let p be a mesh vertex inthe region. The orthogonal projection parameters (u, v)

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd. 325 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav

Page 6: Patches: character skinning with local deformation layer

J. LEE, M.-S. KIM AND S.-H. YOON...........................................................................................

Figure 4. Direct manipulation of a patch; all bound vertices move simultaneously as the user deforms a patch interactively.

are computed by solving the following equation:

{〈p − s(u, v), su(u, v)〉 = 0

〈p − s(u, v), sv(u, v)〉 = 0

where 〈·, ·〉 is an inner product and su(u, v) and sv(u,v) represent the partial derivatives of s(u, v) in u and v

directions respectively. The scalar displacement d is setto the signed distance of p from s(u, v) in the unit normaldirection n(u, v). Using these binding parameters (u, v,d), all vertices of the selected region can be reconstructedfrom the surface patch as follows:

p = s(u, v) + d × n(u, v) (3)

where n(u, v) is the unit normal vector of s(u, v).

Construction of Example Shapes

To produce natural and continuous deformations, it isquite important to construct a deformable shape modelthat has a sufficient degree of shape flexibility. For thispurpose we employ B-spline surfaces that can representvarious examples with detailed local shapes. Mostexisting methods3,4,7,11,12,20,22 obtain these exampleshapes from 3D-scanner or by manually adjusting

blending weights, which are highly time-consumingprocesses.

Our patch-based approach, however, provides anintuitive and efficient mechanism for the constructionof examples by employing the standard techniquesdeveloped for editing B-spline surfaces. At an examplepose, we can easily generate an example shape with localdetails by manipulating the position of arbitrary skinvertex p. Since the vertex p is bound to a surface points(u, v), we can apply to s(u, v) a direct manipulationtechnique for B-spline surfaces.29 Manipulating thevertex p changes the positions of some relevant controlpoints qij in Equation (2) and the relative position changeof qij is then stored as a displacement vector at thatpose. (Note that each pose is parameterized by a vector �θconcatenating the Euler angles of the influencing joints.)Figure 4 shows the process of making an example shapeby directly manipulating skin vertices (in red) on theshoulder part of a human character.

Using this control mechanism we can easily con-struct a set of examples of the surface patch and thedisplacement functions dS

ij(�θ) are then constructed byinterpolating the displacements obtained from the exam-ple shapes. In addition, other geometric techniques suchas knot insertion and refinement to surface patches canalso be applied to the creation of local details of exampleshapes.

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd. 326 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav

Page 7: Patches: character skinning with local deformation layer

GEOMETRIC APPROACH TO CHARACTER SKINNING...........................................................................................

Layered Shape DeformationMechanism

In our approach, a global shape deformation techniquesuch as SSD2 or a sweep-based shape deformation(SWD)5 can be applied to update the positions of theB-spline control points. The displacement functions d∗

ij(·)are then added to the control points qij and the changeof these control points then triggers a variety of effectssuch as local shape changes, muscle bulge and elasticdeformation. All skin vertices in the local region followthe deformed surface patch by Equation (3) for thebinding parameters (u, v, d) of the vertex p. As a result,continuous shape deformations are realized over thepose space.

Experimental Results

Implementation

We have implemented the proposed algorithm in C++on an Intel Core 2 Quad 2.4 GHz CPUs with a 2 GB ofRAM, and double NVIDIA GeForce 8800 GTX GPUs inSLI mode.

The whole processing time for the patch-based localdeformation can roughly be classified into three majorcomponents: the B-spline surface patch generation, localpatch deformation, and skin vertex reconstruction. Themost time-consuming part is the skin vertex reconstruc-tion using the deformed patch. To reduce computingtime, we have precomputed the values of B-spline basisfunctions N3

i (u) and N3j (v) for the surface parameters (u,

v) of each skin vertex. Eight values are needed for eachvertex as we have employed bi-cubic B-spline surfaces.We simply need to multiply the basis function values tothe control points of the patch at run time to reconstructa vertex from the patch.

Skeletal Character Deformation

We applied our method to human character animation.For a global shape deformation, we have employed SSD2

and sweep-based deformation (SWD).5 Other globalshape control schemes can also work in a similar fash-ion. Control patches are then attached to local body partssuch as shoulders, hips, elbows, and knees for the controlof local details.

Figure 5(a) shows the hip patch and comparison resultsbetween a global shape control (SWD) and our patch-based local shape control at a pose where the left leg

stretches backward. There occurs a crush in the centerarea of the hip using the global shape control only. Tofix this problem, we pull out the hip patch by a directmanipulation, and produce a more natural deformationof the hip.

Elbows and knees are locally deformable in a similarfashion. In the elbow deformation, we enforce theprotrusion of the ulna bone by pulling out the center ofthe elbow patch (Figure 5(b)). In Figure 5(c), the kneeshape generated from a global deformation of SWDlooks rather sharper than a natural spherical shape at abending pose. We produce a rounder knee shape usingthe proposed local control.

Figure 5(d) and (e) show the shoulder deformationof a male character. SSD is used for a global shapedeformation and five example shapes are constructed byadjusting the shoulder patch at primary poses (Figure5(d)). The patch-based shoulder deformation results atarbitrary poses are shown in Figure 5(e).

Elastic Character Deformation

An important advantage of our surface-based approachis its seamless combination of dynamic as well as kine-matic shape changes in local body parts. To demonstratethe effectiveness of our elastic skin deformation, we haveapplied the proposed patch-based approach to variouscharacters. The motion dynamics of these characterswere extracted from their motion capture data. Figure6(a) shows surface patches attached to various bodyparts of each model. For numerical integration, weemployed a fourth order Runge-Kutta method30 with afixed time step. Although the simulations of the surfacepatch are performed only for a small number of B-splinecontrol points, the surface patch changes its shaperealistically while generating continuous elastic skindeformations. Figure 6(b) and (c) show the sequences ofelastic skin deformations of each model.

Performance

We report in Table 1 the performance of our system interms of the frame-rates obtained for the number of ver-tices bound to each patch. We measure only the com-putation time for the local patch deformation and thevertex reconstruction in the local region. (We have notincluded the time for a global deformation and the ren-dering time.) In a patch-based deformation, the mosttime-consuming step is in the reconstruction of skinvertices from patches (see the case of a male model).

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd. 327 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav

Page 8: Patches: character skinning with local deformation layer

J. LEE, M.-S. KIM AND S.-H. YOON..........................................................................

Figure 5. Patch-based local shape deformation of human body parts: (a), (b), and (c) patches and comparison results between aglobal shape control and our patch-based local shape control for a hip, an elbow, and a knee; (d) example shoulder shapes at five

different poses and (e) shoulder deformation results at arbitrary poses.

Vertex reconstruction time has been considerablyreduced by precomputing B-spline basis function values.On the other hand, in an elastic deformation, the physi-cal simulation of B-spline control points is the most time-consuming step (see the cases of dog and gorilla models).

The usage of memory space is proportional to the num-ber of vertices bound to a local patch. Each vertex uses44 Bytes for 11 floating-point values. Three values arethe surface binding parameters, and the others are theB-spline basis function values.

...........................................................................Copyright © 2009 John Wiley & Sons, Ltd. 328 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav

Page 9: Patches: character skinning with local deformation layer

GEOMETRIC APPROACH TO CHARACTER SKINNING...........................................................................................

Figure 6. Patch-based elastic skin deformations: (a) surface patches, (b) and (c) elastic skin deformation results.

Updating patches (FPS1)Control Skin Updating vertices Total (FPS)Model Part points vertices RBF Physical sim. (FPS2) (FPS1 +FPS2)

Male Elbow + Biceps 9 × 10 1149 3194.9 — 688 581.4Shoulder 9 × 6 1346 5347.6 — 660 531.9

Gorilla Abdomen 5 × 5 55 — 1122 10 638 1075Dog Face 7 × 7 247 — 400 3773 376

Table 1. Performance of the local patch-based deformation; FPS1 is the frame rate for updating onlythe patch and FPS2 is the frame rate for updating all bound vertices

ConclusionWe have presented an effective surface-oriented methodfor controlling the local shape deformation of variousbody parts in character animation. Freeform surfacepatches are employed and the patches change theirshapes following the position changes of their control

points. Various methods can easily be combined andapplied in the same paradigm for updating the controlpoints of local surface patches; for example, the shapeinterpolation to a set of examples can be seamlesslycombined with an elastic shape deformation module thatis based on a physical simulation. Moreover, the patch-based approach can also be combined with any global

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd. 329 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav

Page 10: Patches: character skinning with local deformation layer

J. LEE, M.-S. KIM AND S.-H. YOON...........................................................................................

shape deformation schemes that smoothly change thevertex positions of a mesh. In the future work, we planto explore the possibility of applying the proposed localdeformation approach to complex facial animations.

ACKNOWLEDGEMENT

This work was supported by research grants from ChosunUniversity, 2008.

References

1. Magnenat-Thalmann N, Thalmann D. Handbook of VirtualHumans. John Wiley & Sons, Chichester, UK, 2004.

2. Magnenat-Thalmann N, Lemperriere R, Thalmann D. Joint-dependent local deformations for hand animation and objectgrasping. In Proceedings of Graphics Interface 1988, 26–33.

3. Lewis JP, Cordner M, Fong N. Pose space deformation: aunified approach to shape interpolation and skeleton-drivendeformation. In Proceedings of ACM SIGGRAPH 2000, 165–172.

4. Wang XC, Phillips C. Multi-weight enveloping: least-squares approximation techniques for skin animation. InProceedings of the 2002 ACM SIGGRAPH/Eurographics Sym-posium on Computer Animation, 129–138.

5. Hyun D-E, Yoon S-H, Chang J-W, Seong J-K, Kim M-S, JuttlerB. Sweep-based human deformation. The Visual Computer2005; 21(8–10): 542–550.

6. Merry B, Marais P, Gain JE. Animation space: a truly lin-ear framework for character animation. ACM Transactionson Graphics 2006; 25(4): 1400–1423.

7. Wang RY, Pulli K, Popovic J. Real-time enveloping with rota-tional regression. ACM Transactions on Graphics 2007; 26(3):731–739.

8. Terzopoulos D, Waters K. Physically-based facial modeling,analysis and animation. Journal of Visualization and ComputerAnimation 1990; 1(2): 73–80.

9. Albrecht I, Haber J, Seidel H-P. Construction and animationof anatomically based human hand models. In Proceedings ofthe 2003 ACM SIGGRAPH/Eurographics Symposium on Com-puter Animation, 2003, 98–109.

10. Allen B, Curless B, Popovic Z. Articulated body deformationfrom range scan data. ACM Transactions on Graphics 2002;21(3): 612–619.

11. Sloan P-PJ, Rose CF, III, Cohen MF. Shape by example. InProceedings of the 2001 Symposium on Interactive 3D Graphicsand Games, 2001, 135–143.

12. Kry PG, James DL, Pai DK. Eigenskin: real time large defor-mation character skinning in hardware. In Proceedings of the2002 ACM SIGGRAPH/Eurographics Symposium on ComputerAnimation, 2002, 153–159.

13. Lee J, Yoon S-H, Kim M-S. Realistic human hand deforma-tion. Computer Animation and Virtual Worlds 2006; 17(3–4):479–489.

14. Capell S, Green S, Curless B, Duchamp T, Popovic Z. Inter-active skeleton-driven dynamic deformations. ACM Trans-actions on Graphics 2002; 21(3): 586–593.

15. Choi M-G, Ko H-S. Modal warping: real-time simulation oflarge rotational deformation and manipulation. IEEE Trans-actions on Visualization and Computer Graphics 2005; 11(1): 91–101.

16. Chadwick J, Haumann D, Parent R. Layered constructionfor deformable animated characters. In Proceedings of ACMSIGGRAPH, 1989, 243–252.

17. Magnenat-Thalmann N, Cordier F, Seo H, Papagiannakis G.Modeling of bodies and clothes for virtual environments. InProceedings of the 2004 International Conference on Cyberworlds,2004, 201–208.

18. Kavan L, Zara J. Spherical blend skinning: a real-time de-formation of articulated models. In I3D’05: Proceedings of the2005 Symposium on Interactive 3D Graphics and Games, 2005,9–16.

19. Kavan L, Collins S, Zara J, O’Sullivan C. Geometric skinningwith approximate dual quaternion blending. ACM Transac-tions on Graphics 2008; 27(3): 1–23.

20. Alexa M. Linear combination of transformations. ACMTransactions on Graphics 2002; 21(3): 380–387.

21. Weber J. Run-time skin deformation. In Proceedings ofGame Developers Conference 2000, http://www.gamasutra.com/features/gdcarchive/2000/weber.doc.

22. Mohr A, Gleicher M. Building efficient, accurate characterskins from examples. ACM Transactions on Graphics 2003;22(3): 562–568.

23. Yoon S-H, Kim M-S. Sweep-based freeform deformations.Computer Graphics Forum 2006; 25(3): 487–496.

24. Platt S, Badler N. Animating facial expressions. ComputerGraphics 1981; 15(3): 245–252.

25. Waters K. A muscle model for animating three dimensionalfacial expression. Computer Graphics 1987; 21(4): 17–24.

26. Waters K, Terzopolulos D. Modeling and animating facesusing scanned data. Journal of Visualization and Computer An-imation 1991; 2(4): 123–128.

27. Sederberg TW, Parry SR. Free-form deformation of solid ge-ometric models. In Proceedings of ACM SIGGRAPH, 1986,151–160.

28. Powell MJD. Radial basis functions for multivariable inter-polation: a review. Algorithms for Approximation JC Mason,MG Cox (eds). Clarendon Press: New York, NY, USA, 1987;143–167.

29. Piegl L, Tiller W. The NURBS Book. Springer-Verlag: Berlin,Germany, 1995.

30. Press W-H, Teukolsky S-A, Vetterling W-T, Flannery B-P. Nu-merical Recipes in C (2nd edn). Cambridge University Press:New York, NY, USA, 1992.

Authors’ biographies:

Jieun Lee received the BS degree in computer scienceand engineering from Ewha Womans University in

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd. 330 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav

Page 11: Patches: character skinning with local deformation layer

GEOMETRIC APPROACH TO CHARACTER SKINNING...........................................................................................

1997, the MS degree from POSTECH in 1999, and thePhD degree from Seoul National University in 2007.She is currently an assistant professor of the Division ofComputer Engineering, Chosun University, Korea. Herresearch interests are in geometric modeling, computergraphics, and multimedia information processing. From1999 to 2002, she worked at LG Electronics Instituteof Technology as a research engineer and mainlyparticipated in the MPEG-7 standardization activity.

Myung-Soo Kim received the BS and MS degrees fromSeoul National University in 1980 and 1982, respectively,and the MS degree in applied mathematics and theMS and PhD degrees in computer science from PurdueUniversity in 1985, 1987, and 1988, respectively. He iscurrently a professor of the School of Computer Scienceand Engineering, Seoul National University. Until re-cently, he was also the CIO of Seoul National University.His research interests are in computer graphics andgeometric modeling. From 1988 to 1998, he was with the

Department of Computer Science, POSTECH, Korea. Heserves on the editorial boards of the journals Computer-Aided Design, Computer Aided Geometric Design,and the International Journal of Shape Modeling. Healso edited several special issues of journals includingComputer-Aided Design, Computer Aided GeometricDesign, Graphical Models, the Journal of Visualizationand Computer Animation, The Visual Computer, andthe International Journal of Shape Modeling. Togetherwith G. Farin and J. Hoschek, he edited the Handbookof Computer Aided Geometric Design (Amsterdam:North-Holland, 2002).

Seung-Hyun Yoon received the BS degree in mathe-matics from Hanyang University in 2001 and the PhDdegree in computer science and engineering from SeoulNational University in 2007. He is currently an assistantprofessor of the Department of Game and MultimediaEngineering, Dongguk University. His research interestsare in computer graphics and geometric modeling.

............................................................................................Copyright © 2009 John Wiley & Sons, Ltd. 331 Comp. Anim. Virtual Worlds 2009; 20: 321–331

DOI: 10.1002/cav