Cs665 writeup

Cornell University

CS 6650: Computational Motion Semester Project

Rigid Body Sound Synthesis

Authors:Charles MoyesShentong Wang

Supervisor:Dr. Doug L. James

May 16, 2011

Contents

1 Abstract 2

2 Math Library 32.1 Dense Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Armadillo Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.3 Sparse Matrix Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.4 SVDLIBC C Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Sound Synthesis 53.1 Finite Element Model Matrix Formation . . . . . . . . . . . . . . . . . . . . 53.2 Material Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.3 Modal Synthesis Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . 63.4 Audio Playback Using SDL . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4 Rigid Body Dynamics 84.1 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84.2 Resting Contact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4.2.1 Analytical Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84.2.2 Projected Relaxed Gauss-Seidel Solver . . . . . . . . . . . . . . . . . 8

4.3 Collision Detection and Response . . . . . . . . . . . . . . . . . . . . . . . . 94.3.1 Collision Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.3.2 Newton’s Law of Restitution . . . . . . . . . . . . . . . . . . . . . . . 94.3.3 Tangential Coulomb Friction . . . . . . . . . . . . . . . . . . . . . . . 9

4.4 Open Dynamics Engine (ODE) Integration . . . . . . . . . . . . . . . . . . . 10

5 Graphics Engine 115.1 Test GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5.1.1 Keyboard Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.2 OpenGL Renderer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5.2.1 Keyboard Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

6 Future Plans 15

7 Conclusion 16

1

Chapter 1

Abstract

The basic idea of the project is to implement a rigid body sound synthesis engine in C++using OpenGL and a rigid body dynamics simulation. Modal synthesis in the frequency-domain will be used to generate rigid body contact sounds based on the contact forces andpositions applied to objects in the simulation. The Armadillo linear algebra library will beused to perform routine matrix decompositions such as Cholesky factorizations and singularvalue decompositions (SVDs). The SDL library will be used for audio output and graphicalrendering. Another goal is to build an interesting simulation to show off the applications ofthe engine.

Figure 1.1: Proof-of-concept image from [?].

2

Chapter 2

Math Library

The project required linear algebra routines for forming the mass and stiffness matrices,along with solving the eigenvalue problem associated with modal analysis. For this project,we implemented our own dense and sparse matrix classes, along with using the Armadillolinear algebra library (found at http://arma.sourceforge.net/) and the SVDLIBC sparsesingular value decomposition library (found at http://tedlab.mit.edu/~dr/SVDLIBC/).We have implemented both a sparse and a dense matrix solver. We found that the densesolver works best with smaller tetrahedral meshes.

2.1 Dense Solver

The dense solver was implemented first using the Armadillo library to have a metric fortesting the sparse matrix routines. Meanwhile, the sparse solver was developed concurrently.

2.2 Armadillo Integration

Armadillo was used for the dense solver for computing the Cholesky decomposition of themass matrix. For the sparse matrix solver, we generate a diagonal “lumped” mass ma-trix, avoiding the need to perform an explicit Cholesky decomposition. In this case, theCholesky decomposition is trivially the square root of the diagonal values of the matrix.The Eigen linear algebra library (http://eigen.tuxfamily.org/) was also investigatedfor this project, but we found that the Armadillo API was easier to use with our code.

2.3 Sparse Matrix Library

Our sparse matrix library initially handled both static and dynamic matrices. We used astorage format very similar to the Harwell-Boeing format. Column indices separate an arrayof non-zero values into partitions, where each group resides in a different column. Withineach column, each non-zero value in the sparse matrix has a row index associated with it.Very thorough test batteries were written to ensure the correctness of this code in order toprevent tedious debugging further down the road.

2.4 SVDLIBC C Library

At first, we tried developing our own sparse matrix eigenvalue decomposition code, butafter consulting with Charles Van Loan and David Bindel, we decided that this endeavorwould be a separate project in itself. The SVDLIBC library was used for performingsparse eigenvalue decompositions. It uses the Lanczos algorithm for computing the singular

3

Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project

value decomposition, and it handles both dense and sparse matrices. Code was writtento generate sparse matrices in the Harwell-Boeing binary format for use with this library.Several Fortran libraries were investigated, including PROPACK, TRLan, and LAPACK.We found that SVDLIBC was the easiest to use, so we went with it instead.

4

Chapter 3

Sound Synthesis

3.1 Finite Element Model Matrix Formation

We used [?] as a reference for constructing the matrices for tetrahedral finite element models.We discretize our mesh into tetrahedron in order to perform modal analysis. The mesh ver-tices and tetrahedron indices were loaded directly from binary files generated using ChangxiZheng’s IsoStuffer utility [?]. We used linear basis functions as described in the paper.We wrote code to assemble the mass and stiffness matrices as suggested. The element massmatrix M was computed as:

m[ij]ab =∂2κ

∂p[i]ap[j]b=ρ vol

20(1 + δij)δab (3.1)

and the “lumped” mass matrix was formed as follows:

mlumped[ij]ab =

ρ vol

4δijδab (3.2)

The stiffness matrix K was formed using:

k[ij]ab =∂f[i]a∂p[j]b

∣∣∣p=prest

= −vol

2(λβiaβjb + µβibβja + µ

3∑k=1

βikβjkδab) (3.3)

where β represents the Barycentric coordinates within a tetrahedral element (defining alinear shape function for that element), and the volume of a tetrahedral was computed usingthe triple scalar product:

vol =1

6[(m[2] −m[1])× (m[3] −m[1])] · (m[4] −m[1]) (3.4)

where m[i] is the position in “material” (or object) coordinates of element node i.The damping matrix C is defined as a linear combination of the stiffness and mass

matrices (so-called Rayleigh damping):

C = α1K + α2M (3.5)

3.2 Material Parameters

The so-called Lame parameters λ and µ define the properties of the material in its elasticstress tensor (assuming the material is isotropic). Our simulation code allows one to tunethese parameters, along with the material density and its Rayleigh damping α coefficients.Example material parameters for various objects can be found in the following table:

5


Material λ (Pa) µ (Pa) α1 α2 ρ (Kg/m3)Ceramic 3.99× 109 2.05× 109 1× 10−6 10 2700Plastic 2.49× 1010 1.28× 1010 1× 10−6 50 2700

Aluminum 4.98× 1010 2.56× 1010 1× 10−7 0 2700Wood 5.00× 108 1.00× 108 8× 10−6 50 750Metal 4.99× 1010 2.56× 1010 1× 10−7 0 2700

Table 3.1: Example Material Parameters

3.3 Modal Synthesis Calculations

Modal synthesis is performed by computing the oscillation modes of the finite element mesh.A brief overview of the modal analysis calculations involved follows. Starting with a non-linear mass/spring/damping force system:

K(d) + C(d, d) +M(d) = f (3.6)

The system is linearized (with a loss in accuracy when modelling thin shells withoutmodal coupling) into the following form:

Kd + Cd + Md = f (3.7)

By letting M = LLT (Cholesky factorization) and L−1KL−T = VΛV−T (Singular ValueDecomposition):

Λz + (α1Λ + α2I)z + z = g (3.8)

where g = VTL−1f .The Rayleigh damping assumption allows the system to be diagonalized into a set of

second-order ordinary differential equations:

λizi + (α1λi + α2)zi + zi = gi (3.9)

The analytical solution for each mode was computed to determine its frequency (|Im(ωi)|)and decay rate (Re(ωi)):

zi = c1etω+

i + c2etω−

i ⇒ ω±i =−(α1λi + α2)±

√(α1λi + α2)2 − 4λi2

(3.10)

Finally, given the c constants for mode reponses to projected impulses:

c1 =2∆tgi

ω+i − ω−i

(3.11)

c2 =2∆tgi

ω−i − ω+i

(3.12)

These c constants can be used to find a time-varying oscillator equation for the audio wave-form:

zi =2∆tgi|Im(ωi)|

etRe(ωi) sin(t|Im(ωi)|) (3.13)

6


Thus, an eigenvalue decomposition can be used to find the modal frequency and decayrates of the mass-spring-damper system. Contact forces from the rigid body simulation canbe projected onto the vibration modes (the columns of L−TV). As an optimization, onlythe modes with frequencies within the audible frequency range of the human ear (20 Hz to20 kHz) are used for sound synthesis computations.

3.4 Audio Playback Using SDL

The SDL audio library was used for cross-platform audio playback. The summed contributionof each mode’s oscillator equation is used to fill up the audio buffer, which is invoked usinga mix audio() callback function. A queue of modes contributing to the resultant waveformare stored in the STL vector, modeQueue. A mutex lock sound lock protects this queue fromrace conditions. Moreover, a sampling rate of 44,100 Hz was used, and the audio data isstored internally using floating-point numbers until the final mixdown where it is discretizedinto 16-bit signed integers. Special care was taken to avoid clipping by scaling the resultantwaveform so that it lies within the dynamic range of the audio buffer. This approach hasthe weakness that an unappropriately chosen scaling value will still allow clipping to occur,trading off dynamic range for safety against clipping. A smarter approach could be todynamically scale the resultant waveform (like audio compressors in sound engineering).This resultant waveform represents the sound you hear.

7

Chapter 4

Rigid Body Dynamics

A rigid body dynamics simulation was implemented mostly using the results from [?]. Therigid body dynamics simulation component of this project provided support for Newtonianphysics simulation. The physics engine uses a projected, relaxed Gauss-Seidel solver toanalytically determine contact forces:

~b = J(~ut + ∆tM−1 ~fext) (4.1)

A = JM−1JT (4.2)

~λ = ncp(A,~b) (4.3)

~ut+1 = ~ut + M−1JT~λ+ ∆tM−1 ~fext (4.4)

~st+1 = ~st + ∆tS~ut+1 (4.5)

4.1 Numerical Integration

The Euler method of numerical integration is used to timestep the simulation by computingupdated velocities and positions for each body. It is represented in matrix-form as discussedin the Erleben paper. We needed a stable integrator to solve the dynamics equations. Prof.James suggested using a reverse Euler method. This worked fine for our needs. The Time-Corrected Verlet method was also considered for use, along with the fourth-order Runge-Kutta method.

xn+1 = xn + f(tn, vn)∆t (4.6)

vn+1 = vn + g(tn, xn)∆t (4.7)

4.2 Resting Contact

4.2.1 Analytical Method

Contact forces were computed analytically using a relaxed Gauss-Seidel solver. To improvesimulation stability, I also manually resolved penetration at each time step.

4.2.2 Projected Relaxed Gauss-Seidel Solver

The projected Gauss-Seidel solver was implemented using the optimizations suggested in theErleben paper.

~λk+1i =

−∑i−1

j=0 Li,j~λk+1j −

∑n−1j=i+1 Ui,j

~λkj −~biDi,i

(4.8)

8


The projection step enforces the friction cone and the linear complementarity problemconstraints:

~λk+1i = min(max(~λloi ,

~λk+1i , ~λhii) (4.9)

We added a relaxation term ω = 0.25 to the iteration step given in the paper.

4.3 Collision Detection and Response

4.3.1 Collision Detection

Initially, the FreeSOLID library (found at http://www.win.tue.nl/~gino/solid/) wasused for collision detection. However, the Gauss-Seidel solver often exhibited divergence,necessitating the use of Velocity Shock Propagation (VSP) as described in [?]. Unfortunately,the implementation of this algorithm requires knowledge of the penetration depth. For simplecases like ground-sphere and ground-box collisions, this could be determined without any pre-determined knowledge from the FreeSOLID library, but in cases like stacking (where VSP isespecially needed to maintain stability and improve convergence), the FreeSOLID library’slack of a penetration depth result prevented the development of a VSP implementation.Unfortunately, this left us with either the option of implementing our own collision detectionlibrary or using the Open Dynamics Engine (ODE) for our rigid body calculations. We optedfor the latter option.

4.3.2 Newton’s Law of Restitution

Newton’s Law of Restitution is used to compute the collision response impulses:

j =−(1 + e)vAB

1 · n

n · n(

1MA

+ 1MB

)+[(

I−1A (rAP × n))× rAP +

(I−1B (rBP × n)

)× rBP

]· n

(4.10)

It is represented in Matrix form as part of the ~b vector used in the rigid body systemof equations discussed in the Erleben paper. This is an elegant way to add bouncing as acollision response to the simulation.

4.3.3 Tangential Coulomb Friction

Coulomb friction was also incorporate in the rigid body solver using extra constraints on λenforced during the projection step as discussed in the Erleben paper:

Ff ≤ µFn (4.11)

9


4.4 Open Dynamics Engine (ODE) Integration

The Open Dynamics Engine was integrated using a wrapper for rigid objects (with tetra-hedral meshes for sound generation). The contact forces, normals, and positions are firstaccumulated in the nearCallback() function. This function creates contact joints as spec-ified in the ODE documentation. Then in the updatePhysicsState() function, the worldis time stepped, and the joint feedback functionality of ODE is used to retrieve the forcesstored in the callback function. The relative velocity and acceleration for the contactingbodies are computed, and if the relative velocity is non-zero, then the bodies are not inresting contact. This means that they are “bouncing,” and so a rigid body contact soundshould be triggered. To prevent artifacts from contact sounds that are triggered too quicklyin succession, a grace time of MIN SOUND REPEAT TIME (set to 0.25 seconds by default) isused.

Following the procedure outlined in [?], the contact force is projected onto the modes bytransforming the world space contact position back to object space and finding the nearestvertex in the tetrahedral mesh. This vertex is the excited node, and so the force is appliedto that node in the f vector. From there, the g vector is computed as outlined in section§ 3.3. The mode frequencies and rates are then determined, and if the frequency lies withinthe audible hearing range, a new mode is added to the mode queue for the sound synthesiscode.

An interesting curiousity in the Open Dynamics Engine is that its integrator is onlystable with constant time steps. However, one can update the error reduction parameter(ERP) and constraint force mixing (CFM) values as functions of the current time step size∆t, as outlined in [?]:

ERP =∆t · k

∆t · k + c(4.12)

CFM =1

∆t · k + c(4.13)

These updates allow ODE’s numerical integrator to remain stable, even with variably-sized time steps.

10

Chapter 5

Graphics Engine

Multiple user interfaces were developed for this project. Initially, a testing interface thatused SDL’s 2D drawing capabilities was used to ensure that the modal synthesis code wasworking properly. This test interface featured an on-screen oscilloscope and a visualizationof the excitation of each modal amplitude. Later, OpenGL 3D graphics and rigid bodydynamics were added to visualize the actual rigid body contact sounds being generated.

5.1 Test GUI

The test GUI has several features, including the features to generate test modes, produce anA 440 Hz test tone, project test forces onto the tetrahedral mesh, and hear all of the modesat once. Moreover, some of the parameters can be changed on-the-fly without re-evaluatingthe system matrices. A screenshot of the test GUI follows. The red bars represent therelative amplitudes of the excited modes:

Figure 5.1: Sound Synthesis GUI

5.1.1 Keyboard Controls

A table outlining the basic keyboard controls of the test GUI follows:

5.2 OpenGL Renderer

The OpenGL renderer allows the user to spawn falling spheres and boxes to demo the rigidbody contact sound functionality of the sound synthesis engine. The 3D camera view can be

11


Functionality KeyExit [ESCAPE]

Play 20 Highest Frequency Modes [Q]A 440 Hz Test Tone [W]Clear Mode Queue [E]Project Test Force [R]

Play All Modes At Once [T]Increase Number of Modes [A]Decrease Number of Modes [S]

Increase α1 [Z]Decrease α1 [X]Increase α2 [C]Decrease α2 [V]

Increase ρ Scale Factor [B]Decrease ρ Scale Factor [N]

Increase Lame Scale Factor [M]Decrease Lame Scale Factor [,]Increase Excited Node Index [.]Decrease Excited Node Index [/]

Toggle Full-Screen [TAB]

Table 5.1: Test GUI Keyboard Controls

adjusted using keyboard and mouse controls. Moreover, the parameters can be tuned usingthe keyboard controls, much like with the Test GUI. A screenshot of the OpenGL rendererdemonstrating a sample physics environment follows:

5.2.1 Keyboard Controls

A table outlining the basic keyboard controls of the OpenGL 3D renderer follows. Note thatthe parameter tuning keyboard controls are identical to the Test GUI interface. Additionally,by dragging with the left mouse button, the viewing direction can be manipulated.

12


Figure 5.2: OpenGL 3D Graphics Engine

13


Functionality KeyExit [ESCAPE]

Camera Forward [W]Camera Backward [S]

Camera Left [A]Camera Right [D]

Apply Test Force [SPACE]Create Rigid Box [H]

Create Rigid Sphere [J]Clear Mode Queue [Q]

Increase α1 [Z]Decrease α1 [X]Increase α2 [C]Decrease α2 [V]

Increase ρ Scale Factor [B]Decrease ρ Scale Factor [N]

Increase Lame Scale Factor [M]Decrease Lame Scale Factor [,]Increase Excited Node Index [.]Decrease Excited Node Index [/]

Toggle Full-Screen [TAB]

Table 5.2: OpenGL 3D Renderer Keyboard Controls

14

Chapter 6

Future Plans

Several future plans are in place for expanding this project:

• Investigate and implement a far-field acoustic radiation model. [?]

• Add support for non-linear mode coupling (as in harmonic shells). [?]

• Parallelize the sound synthesis engine using GPGPU programming (CUDA or GLSL)or multi-threading.

• Investigate and implement fracture sound synthesis. [?]

• Implement recent advances in modal synthesis such as contact damping and modalre-distribution. [?]

15

Chapter 7

Conclusion

The project was successful in that rigid body contact sounds were able to be generated usingthe model presented in [?]. A flexible user interface was developed, and the results are thatobjects within the virtual 3D environment are able to generate contact sounds accordingly.One can envision applications for real-time rigid body sound synthesis in videogames andanimated movies. The audio in videogames is often limited to playing back a pre-recorded setof sound samples when game events take place. The end result is that the same “glass shat-tering” sample is played regardless of the fracture pattern or the actual in-game shatteringbehavior. Furthermore, sound artists have to resort to manually recording and overdubbingthe audio tracks in films. These processes could be automated if a sufficiently realistic modelfor sound generation is used.

16

Bibliography

[1] David Baraff. Fast contact force computation for nonpenetrating rigid bodies. In Pro-ceedings of the 21st annual conference on Computer graphics and interactive techniques,SIGGRAPH ’94, pages 23–34, New York, NY, USA, 1994. ACM.

[2] Barringer. Tweaking ode for variable time steps, 2007.

[3] Nicolas Bonneel, George Drettakis, Nicolas Tsingos, and Doug James. Fast modalsounds with scalable frequency-domain synthesis, 2008.

[4] Jeffrey N. Chadwick, Steven S. An, and Doug L. James. Harmonic shells: a practicalnonlinear sound model for near-rigid thin shells. ACM Transactions on Graphics, 28,2009.

[5] Kees Van Den Doel, Paul G. Kry, and Dinesh K. Pai. Foleyautomatic: Physically-basedsound effects for interactive simulation and animation. In in Computer Graphics (ACMSIGGRAPH 01 Conference Proceedings, pages 537–544. ACM Press, 2001.

[6] Kenny Erleben. Velocity-based shock propagation for multibody dynamics animation.ACM Trans. Graph., 26, June 2007.

[7] Gene H. Golub and Charles F. Van Loan. Matrix Computations. The Johns HopkinsUniversity Press, 3rd edition, 1996.

[8] Doug L. James, Jernej Barbic, and Dinesh K. Pai. Precomputed acoustic trans-fer: Output-sensitive, accurate sound generation for geometrically complex vibrationsources. ACM Transactions on Graphics (SIGGRAPH 2006), 25(3), August 2006.

[9] Danny M. Kaufman. Staggered projections for frictional contact in multibody systems,2008.

[10] James F. O’Brien, Perry R. Cook, and Georg Essl. Synthesizing sounds from physicallybased motion. In Proceedings of ACM SIGGRAPH 2001, pages 529–536, August 2001.

[11] James F. O’Brien and Jessica K. Hodgins. Graphical modeling and animation of brittlefracture. In Proceedings of the 26th annual conference on Computer graphics and in-teractive techniques, SIGGRAPH ’99, pages 137–146, New York, NY, USA, 1999. ACMPress/Addison-Wesley Publishing Co.

[12] James F. O’Brien, Chen Shen, and Christine M. Gatchalian. Synthesizing sounds fromrigid-body simulations. In Proceedings of the 2002 ACM SIGGRAPH/Eurographicssymposium on Computer animation, SCA ’02, pages 175–181, New York, NY, USA,2002. ACM.

[13] Cecile Picard, Christian Frisson, Francois Faure, George Drettakis, and Paul G. Kry.Advances in modal analysis using a robust and multiscale method. EURASIP J. Adv.Signal Process, 2010:7:1–7:12, February 2010.

17


[14] David E. Stewart. Rigid-body dynamics with friction and impact. SIAM Rev., 42:3–39,March 2000.

[15] Changxi Zheng. Tetrahedra generation.

[16] Changxi Zheng and Doug L. James. Toward high-quality modal contact sound, 2001.

[17] Changxi Zheng and Doug L. James. Rigid-body fracture sound with precomputedsoundbanks. ACM Trans. Graph., 29:69:1–69:13, July 2010.

18

Documents

Cs665 writeup