Space-time multigrid method for molecular

dynamics simulations

Haim Waisman and Jacob Fish

Department of Civil, Mechanical and Aerospace Engineering, RensselaerPolytechnic Institute, Troy, NY 12180-3590

Abstract

We present a novel multiscale approach for molecular dynamics simulations. It isaimed at bridging discrete scales with either coarse grained discrete or continuumscales. The method consists of the waveform relaxation scheme aimed at capturingthe high frequency response of atomistic vibrations and the coarse scale solution inspace and time aimed at resolving smooth features (in both space and time domains)of the discrete medium. The method is implicit in space and time, possesses superiorstability properties and consequently enables larger time steps governed by theaccuracy considerations of coarse scale quantities of interest. Performance studieson polymer melts have shown significant speed-up compared to the classical explicitmethods, in particular on parallel machines.

Key words: Waveform relaxation, Space-Time Multigrid, Molecular Dynamicsintegration

1 Introduction

Molecular dynamics (MD) simulations provide the methodology for detailedfine scale modeling on the molecular level. MD can be viewed as a processby which one generates atomic trajectories of a system of particles by directnumerical integration of Newton’s equations of motion with the appropriateinitial and boundary conditions. The system can be formulated either by dy-namic equilibrium consideration or by means of variational principle and is

∗ Corresponding author: fishj@rpi.edu (Jacob Fish)

Preprint submitted to Elsevier Science 11 September 2005

written as

Md = F int (d)− F ext (d)

d(0) = d0

d(0) = v0

(1)

where d is a vector of atom positions, M the mass matrix and F int = −∇Φ(d)the force field defined as the gradient of the potential energy.Current MD algorithms severely restrict the modeling efforts to relativelysmall systems and/or short time intervals. The algorithmic challenges fac-ing MD simulations stem from the difficulty of designing methods which areinsensitive in terms of the integration time step to rapid fluctuations in chem-ical bonds and short ranged forces. Most widely used algorithms in MD areexplicit methods, such as Verlet and Gear’s predictor-corrector methods. Asevere limitation in the ability of the explicit methods to propagate numericaltrajectories stems from a wide range of time scales spanning many orders ofmagnitude. For instance, in polymer chains, bond-stretching vibrations arethe fastest atomic motions in a molecule, typically in the order of femtosec-onds, whereas the relaxation of polymers in the form of segmental motions orterminal relaxations of chains is spanning the time scales in the range between10−2s to 104s The maximum time step is limited by the smallest oscillationperiod that can be found in the simulated system. This time step is necessaryto maintain stability of explicit numerical integration schemes [1].Considerable efforts have been devoted to increasing the critical time step.Many insightful observations on the numerical properties of the algorithmswere reported, and numerous improvements have been proposed. Neverthe-less, the increases in the integration time step have been quite modest sofar [2]. This temporal gap can be partially circumvented by parallel algorithms.However, due to inherently sequential nature of time integration schemes, par-allelism has been exploited in the spatial domain only, while computations intime domain remain primarily sequential. One promising research avenue isto develop integration algorithms, which are parallelizable in time domain. Adifferent (or complimentary) approach is to exploit implicit integration. Im-plicit methods require solution of coupled nonlinear equations at every timestep, which is both expensive and difficult to implement. Nevertheless, theyreduce stability restrictions and enable much larger time step, typically gov-erned by accuracy considerations [3, 4]. In general, accuracy means how closeis the solution of atomic trajectories obtained by the numerical integrationto the true trajectories. We refer to this type of definition as the accuracy inlocal quantities. Molecular systems, however, are highly nonlinear and exhibitsensitive dependence on perturbations. Moreover, the initial conditions areusually assigned randomly. Thus, accurate approximation of local trajectorieson meaningful time intervals is neither obtainable nor desired. This suggeststo analyzing the accuracy with respect to the global quantities of interest,such as temperature, energy, diffusion and others.In the present manuscript we present a novel multiscale approach based on

2

the space-time variational multigrid principle for molecular systems charac-terized by diverse spatial and temporal scales. Our approach is implicit inboth space and time domains that uses typically 10 to 50 times larger timesteps than explicit methods selected based on the accuracy of global fields ofinterest. The space-time character of the method provides an improved stabil-ity without directly solving the space-time problem. The multiscale nature ofthe system is exploited in two ways: First, it substantially reduces the cost ofmultigrid iterations (see Section 2.1), and secondly, it is parallelizable in timedomain by exploiting the waveform relaxation (WR) scheme in the smoothingprocess (see Section 2.2). In Section 3 we develop the variational space-timemultigrid method suitable for MD simulations, which exploits the multiscalenature of the problem. It consists of the WR scheme aimed at capturing theresponse atomistic vibrations and the coarse scale solution in space and timeintended to resolve the slow features of the discrete medium. The coarse gridproblem is derived from the Hamilton’s principle on the subspace of coarsescale functions. In Section 4 we show that for linear case the method reducesto the sequential multigrid method [5]. Performance studies are conducted inSection 5.

2 Motivation

2.1 Multigrid as a multiscale solver

The motivation for use of multigrid ideas for multiscale problems was givenin [6, 7]. To convey the basic ideas, consider a continuum one-dimensionaltwo-scale elliptic problem for the scalar field u

ddx

(k(x)du

dx

)+ b(x) = 0

u(0) = u(L) = 0(2)

on the interval x ∈ [0, L] with oscillatory periodic piecewise constant coeffi-cients k1 and k2 and 0.5 volume fraction. In many applications of interest andin particular those described at the atomistic scale k1 À k2 yielding highlyheterogenous systems. Equation (2) is discretized using finite elements or fi-nite difference leading to a linear system Ku = f . Analyzing the eigenvaluesof K reveals the following. The eigenvalues are clustered at the two ends of thespectrum on the right half plane. More importantly, the eigenvalues clusteredclose to zero are identical to those obtained by the problem with homogenizedcoefficients [6, 7]. This character of the spectrum suggests a computationalstrategy based on the philosophy of multigrid methods. In such a multigridstrategy the smoother is designed to capture the higher frequency response ofthe fine scale model represented by a linear combination of the highest eigen-

3

modes. The auxiliary coarse model is then engineered to effectively capturethe remaining lower frequency response of the fine scale problem. For a pe-riodic heterogeneous medium, such an auxiliary coarse model coincides withthe boundary value problem with homogenized coefficients as evidenced by theidentical eigenvalues. The resulting multiscale prolongation operator is givenby

Q = Qc + Qf

where Qc and Qf are the classical (smooth) prolongation and the fine scalecorrection obtained from the discretization of the influence functions, respec-tively. The rate of convergence of the multigrid process for the two-scale prob-lem given by Eq. (2) is governed by [6, 7]

‖ei+1‖ =q

4− q‖ei‖

where ‖ei+1‖ is the norm of the error at iteration i + 1 and

q =2√

k1k2

(k1 + k2). (3)

Note that for k1 = k2 the media is homogeneous, and one recovers the clas-sical multigrid rate of 1

3[8]. However, for example if either k1

k2or k2

k1is 100,

then the convergence factor reduces to 15

and the two-scale process convergesin five iterations up to the tolerance of 10−5. In this manuscript we devisea multilevel-like strategy for efficient solution of MD equations. We view theintegration of Newton’s equations of motion as problem on a space-time grid.This observation suggests a unique space-time multigrid approach that ex-ploits multiple spatial and temporal scales of a molecular system.

2.2 The waveform relaxation scheme

One of the major barriers for efficient solution of large scale MD equations isthe sequential character of time integration methods. The waveform relaxation(WR) method is an iterative solution of space-time problems that offers par-allelization [9, 10]. Due to its implicit nature, WR provides superior stabilityand large time steps compared to explicit methods.In the WR algorithm, the space-time domain is partitioned in space intosmaller subsystems, thereafter to be referred to as the spatial blocks or sim-ply blocks. Each block is then integrated over the temporal grid. The spatialblocks can be integrated independently on different processors. The temporalgrid is divided into intervals called windows. Windows are used to accelerateconvergence and to reduce storage. At every time step an algebraic system ofequations is solved on a spatial grid. See Figure 1 for definitions. Informationtransfer between different windows takes place once the integration over the

4

Time

Spatial Slab

Spatial Block

Window

t∆

Space (1D)

Temporal grid

Atoms

Fig. 1. Slabs, windows and blocks in the waveform relaxation method

corresponding windows is completed. The main advantage of the method isthat it permits simultaneous integration of spatial blocks in each window andits ability for unstructured integration. For instance, spatial blocks involvingstiff connections will require smaller time steps than those having more com-pliant connections analogous to the multiple time step (MTS) techniques [2].The WR method has been primarily used in electrical network simulationsand parabolic initial value problems. Limited studies have been conductedfor hyperbolic [11] and second order [12] systems. In the remaining of thissection we give some technical details of the WR method focusing on non-linear systems. For highly nonlinear systems, such as those arising from MDsimulations, two versions of WR method are common. The first is a direct ex-tension of the linear WR formulations, so called Waveform Relaxation Newton(WRN) method [13]. In the MD context, the basic idea is to solve the followingnonlinear scalar equations

midν+1i = F int(dν

1, ..., dνi−1, d

ν+1i , dν

i+1, ..., dνN)

dν+1i (0) = d0i

dν+1i (0) = v0i

(4)

for every atom i in the system (without lost of generality, we assume F ext = 0).The superscripts, ν and ν+1 , denote the iteration count within a time windowt ∈ [t0, tn].Each atom (or a spatial block consisting of several atoms) in iteration ν + 1is integrated over a time window based on its previous position in time (inthe same iteration ν + 1) and the information about its neighbor positions.As opposed to classical integrators, neighboring positions are taken from the

5

iteration ν. This resembles the Jacobi smoothing for steady state problems.A schematic illustration of 1D WR employing Jacobi splitting is given inFigure 2.

Time

td

Time

νiteration

ν+1iteration

Space

� �� �� ��������� �� �� ����

� �� �� ���� � �� �� � ���� �� �� � � �� �� ���� �� ���� �� �� ��� t(d, r)

t(d, r)

x∆

t∆

� �� �� ���

� �� �� ����� �� �� ���� �� �� ����� �� ���� �� �� ���� !!" "" "" "###

$$%%& && &'' ( (( (( ())* ** ** *+++, ,, ,--. .. .. .// 0 00 011

2 22 22 23334 44 44 45556 66 66 677

Fig. 2. Jacobi waveform relaxation

An alternative approach synonymous to the Gauss-Seidel splitting is basedon updating internal forces using the information already available from theiteration ν + 1. Mathematically, the system is written as

midν+1i = F int(dν+1

1 , ..., dν+1i−1 , dν+1

i , dνi+1, ..., d

νN)

dν+1i (0) = d0i

dν+1i (0) = v0i

(5)

The Gauss-Seidel type approach leads to faster convergence rates obviouslyat the expense of more limited parallelization. The second variant is known asthe Waveform Newton (WN) [13, 14]. The idea is to approximate the internalforces in Eq. (1) as

F int = F int(dν) + D(dν)(dν+1 − dν

), (6)

where D is diagonal approximation of the Jacobian matrix obtained from thederivative of the interatomic force field. Substituting the approximation (6) forthe internal forces into (1) leads to the following system of ordinary differentialequations

Mdν+1 −D(dν)dν+1 = F int(dν)−D(dν)dν

dν+1(0) = d0

dν+1(0) = v0

(7)

The system is integrated using Newmark predictor-corrector algorithm [15].The WR iteration is terminated when the maximum residual in a time window

6

is smaller than a specified tolerance

max{‖rν+1(t)‖2} = max{‖Mdν+1 − F int(dν+1

1 , ..., dν+1i , ..., dν+1

N

)‖}. (8)

or alternatively when

max{‖dν+1(t)− dν(t)‖} ≤ ε, (9)

for some small positive constant ε. The major drawback of the WR method(both aforementioned versions) is in its slow convergence in case of a strongcoupling between spatial blocks and sizable windows [16, 17]. In the proposedvariational space-time multigrid approach (presented in Section 3) the WRtakes the role of smoother aimed at capturing the high frequency response ofatomistic vibrations. To this end we note that the combination of the WRsmoothing and the coarse grid correction is not new. The so-called multigridwaveform relaxation (MWR) has been developed in [18, 19] in the context ofparabolic partial differential equations. However, for highly nonlinear molec-ular dynamics systems we find this approach either slow to converge or notconverging at all.

3 Variational space-time multigrid method

We start by qualitative comparison between the classical spatial multigridmethod and the space-time multigrid method presented in this section. Forthe space-time problem we consider 1D atomistic chain, which correspondsto the 2D problem in space-time domain, whereas for the spatial structuredgeometric multigrid method we consider a two-dimensional spatial domainas shown in Figure 3. For the standard multigrid method in 2D the coarsegrid correction is provided by the auxiliary coarse grid (black circles and bluedashed lines in Figure 3(a)). The smoothing process (red dashed line) is thenequivalent to finding the local equilibrium positions of the remaining fine scaledegrees-of-freedom (white circles). On the other hand, in the space-time multi-grid method, the coarse grid correction corresponds to the solution on a spatialslab (blue dashed lines in Figure 3(b)), whereas smoothing corresponds to thesolution of the spatial block on a temporal grid carried out window-by-windowto reduce storage (red dashed lines). Note that in the standard multigrid, linesmoothing corresponds to solving an algebraic system of equations correspond-ing to the degrees-of-freedom positioned along the line, while in the space-timevariant, line-smoothing is a time integration process.One of the main challenges in devising an efficient multilevel approach is con-structing the coarse scale problem. For continuum systems governed by PDEs,it is possible to construct an auxiliary coarse grid based on PDEs and consti-tutive equations of the source problem. Thus the coarse grid problem can be

7

Fine

Coarse

x

y

(a) Standard geometric multigrid(1D space × 1D space)

time

space (1D)

Fine

Coarse

(b) Space-time multigrid method (1Dspace × time)

Fig. 3. Standard multigrid in 2D versus space-time multigrid

directly formulated on the auxiliary coarse model. Employing an analogousapproach to molecular equations with force fields computed from the same in-teratomic potentials for so-called auxiliary representative atoms will result inunrealistic solutions. In other word, the mathematical model for the auxiliarycoarse model (often referred to as a coarse grained model) is different than forthe original atomistic system.Developing such an efficient coarse grained model that span considerablelength and time scales has been an active research area. An adequate coarsegrained model depends on the material system under consideration. For in-stance, in a polymeric system, the most common coarse grained model is based

8

on the equation of motion of the Langevin type (also known as Brownian dy-namics)

M∂2Ri

∂2t+ C

∂Ri

∂t= F s

i

where Ri is the position of coarse grained particle i (or blob) M is its mass;C the equivalent friction between polymer chains; and F s

i represents both theGaussian random forces and the force, which mimic molecular collisions andtherefore a thermal reservoir (see for example [20, 21]). In the present man-uscript we adopt a different approach by which the coarse grained equationsare constructed directly from the fine scale using Hamilton’s principle on thesubspace of the coarse scale functions. The proposed formulation providesa superior rate of convergence, but is computationally more expensive thanevolving the coarse grained model directly since it involves computations ofthe force fields on the fine scale. Let e be the coarse scale correction aimedat updating the fine scale solution, where m ¿ N is the size of the coarsemodel. The updated fine scale solution at a certain time step is given bydν+1 ← dν +Qe where dν ∈ RN . To find the optimal correction we express theLagrangian in terms of the correction Qe

L(Qe,Qe) =1

2< M(dν + Qe), (dν + Qe) > −Φ(dν + Qe). (10)

The prolongation Q is assumed to be constant over a certain period of time.By Hamilton’s principle e(t) is the minimizer of

S[e(t)] =∫ t2

t1L(Qe,Qe)dt, (11)

written asδS

δe(t)= 0 t1 < t < t2. (12)

which is equivalent to solving the following Euler-Lagrange equations

d

dt

(∂L

∂e

)− ∂L

∂e=

δS

δe(t)= 0. (13)

Substituting the Lagrangian into (13) results in the following coarse grid prob-lem

QT MQe−QT F int(dν + Qe) = −QT Mdν

e(0) = 0

e(0) = 0

(14)

We will refer to Eq. (14) as the coarse grid correction in space and time.The system (14) can be integrated explicitly or implicitly. For the latter, ourexperience indicates (see Section 5) that it is sufficient to perform only oneor two Newton iterations to reduce the computational cost. Once the erroris calculated it is prolongated to the fine scale at each time step within a

9

window. Algorithmic details of the nonlinear space-time multigrid method aresummarized in tables corresponding to Algorithms 1, 2 and 3. Capital lettersindicate matrices and capital letters with subscript t indicate a column ina matrix corresponding to discrete time t. An illustration of the space-timemultigrid method is presented in Figure 4. Arrows depicts the informationflow.

Time

t

dt

Q rtT

coarse grid evolution

Space

Time

Timee

Qe

t

Space

� �� �� ��� � �� �� ���� �� �� ����

� �� �� ���� �� �� ���� �� �� � � �� �� ����� �� ���

� �� �� ���� �� �� ���� �� ��� � �� �� ���� �� �� ��� � �� �� ����

� �� �� ����∆x

∆t

(d, r)t

iteration ν+1

iteration ν !!" "" "##$ $$ $$ $%%

& && && &''

( (( ())

* ** ** *+++ , ,, ,, ,--

. .. .. .// 0 00 0112 22 22 233 4 44 44 455 6 66 66 6778 88 88 899 : :: :: :;;;

< << <==> >> >> >??? @ @@ @@ @AAB BB BB BCCCD DD DD DEE F FF FF FGGG

Fig. 4. Space-Time multigrid method

Algorithm 1 Nonlinear space-time multigrid method

1: [M, d0, v0, t0, tn] = setup()2: X ← [d1, .., dn] {%} initialize on space-time3: while normR ≥ tol do4: X0 = X5: [X,A] = WRNL(M, X, d0, v0, t0, tn, ν1) {%} smooth ν1 times6: X = MG(M, X,A, t0, tn) {%} coarse grid evolution7: [X,A] = WRNL(M, X, d0, v0, t0, tn, ν2) {%} smooth ν2 times8: normR ← max{‖Xt −X0

t ‖2} {%} compute residual9: end while

Remark 1: In the case of Harmonic potentials, i.e. F int(d) = −Kd the coarsegrid correction in space and time is consistent with the linear formulationsgiven in [18]

QT MQe + QT KQe = QT rs

e(0) = 0

e(0) = 0

(15)

where rs = −Kdν+1s − Mdν+1

s is the defect computed on the fine scale aftersmoothing.Remark 2: The performance of the variational space-time multigrid methodcan be improved using the windowing technique. By this approach the integra-tion domain is broken into non-overlapping subintervals [0, T1], [T1, T2], ..., [Tn−1, Tn]

10

Algorithm 2 Newmark predictor-corrector for nonlinear coarse grid evolution

1: function X = MG(M, X,A, t0, tn)2: eH ← 0 {%} initial conditions3: vH ← 04: aH ← 05: for t = t0 : ∆t : tn do6: eH = eH + ∆tvH + ∆t2

2(1− 2β)aH {%} predictor phase

7: vH = vH + ∆t(1− γ)aH

8: [aH , Q] = Solve(M,Xt, At, eH , aH) {%} solution phase9: eH = eH + ∆t2βaH {%} corrector phase

10: vH = vH + ∆tγaH

11: Xt ← Xt + QeH {%} prolongate coarse error12: end for

Algorithm 3 Nonlinear solution on a subspace

1: function [aH , Q] = Solve(M,Xt, At, eH , aH)2: for i=1,2,... do3: uh = Xt + Q ∗ (eH + ∆t2βaH)4: [J, f int

h ] = F int(uh)5: Q = Aggregation(J) {%} setup prolongation6: MH = QT MQ7: f ext

H = −QT MAt

8: Ktan = MH −∆t2βQT JQ9: f int

H = QT f inth

10: rH = f extH −MHaH + f int

H {%} nonlinear residual11: solve J∆r = r12: solve aH ← aH + ∆r13: end for

termed windows. The iteration is then performed in each window separatelyuntil convergence is reached. The final solution in a certain window is used asan initial condition for the next window. This accelerates the iteration process,reduces the amount of storage and yields better convergence rates [17]. An il-lustration of such non-overlapping windows is given in Figure 5.Remark 3: Figure 6 illustrates convergence of the space-time multigrid methodin a single window. Each line corresponds to the convergence in atom positions(measured in the relative L2 norm of the residual

( ‖ri‖2‖r0‖2

)) at a certain time

step where i indicates the iteration count. It can be seen that convergencein the initial stages of the integration is faster, or in other words solution inthe initial phase converges to a higher tolerance than towards the end of theintegration window. This suggests that once the solution in the initial phaseshas converged, it can be locked (kept fixed) and the iteration could proceedstarting with the next time step (within the same window) with the initialvalues taken from the converged solution. This may reduce the amount ofcomputational cost roughly by a factor of two.

11

Window II Window III

Space

Time

Window I

(d, r)tI

����������

��������������

��� �� �

�����

�����������������

����������

�����������������

!!"""###$$$%%%

&&&'''

((())

* ** *++,,,--

..//00011

222333444555

66677

νiteration

iteration ν−1

iteration ν+1

∆x

∆t

888999: :: :;;

<<<===

>>>??

@ @@ @AABBBCCC

DDDEE

FFFGGHHHII JJJKKK

LLLMMMNNNOO PPPQQ

RRRSSTTTUUU

VVVWW XXXYYYZZZ[[

\\\]]

^^__

``aa

bbccddeeffgghhii

jjkkllmm

nnooppqqrrssttuuvvwwxxyy

zzz{{{

||}}

~~�� ��������

���������

���� ���������

����

Fig. 5. Non-overlapping windows strategy

100

101

102

10−15

10−10

10−5

100

Spacial residual for each time

Iterations

||ri|| 2/ |

|r 0|| 2

Tolerance

time

(a) convergence in a window withoutlocking

100

101

102

10−10

10−8

10−6

10−4

10−2

100

Spacial residual for each time

Iterations

||ri|| 2/ |

|r 0|| 2

Tolerance

time

Locked

(b) convergence in a window withlocking

Fig. 6. Locking strategies to reduce work

4 Equivalence between the space-time and sequential multigridmethods

In this Section we show that when the window size coincides with the inte-gration time step and the problem is linear, the proposed space-time multi-grid method reduces to the sequential multigrid method applied at each timestep [5], for which the convergence studies have been conducted for the modelproblem [22]. We start with a standard implicit Newmark predictor-correctorintegration applied to a system with Harmonic potentials

Md + Kd = −F ext

d(0) = d0

d(0) = v0

(16)

12

The atom positions and velocities, d and v, respectively, in the predictor phaseare given as

d = d0 + ∆tv0 +∆t2

2(1− 2β)a0

v = v0 + ∆t(1− γ)a0

(17)

where the initial acceleration is computed as a0 = M−1(−F ext −Kd0)). Theresulting accelerations a is

Ka = −F ext −Kd, (18)

where

K = M + β∆t2K. (19)

The corrected values of the displacement and velocity vectors are given by

d = d + ∆t2βa

v = v + ∆tγa(20)

In the case of sequential multigrid, one pre-smoothing iteration applied toEq. (18) gives

ai+1s = ai

s + P−1s ri

s = ais + P−1

s (−F ext −Kd− Kais)

= (I − P−1s K)ai

s − P−1s (F ext + Kd)

(21)

where subscript s indicates smoothing, superscript i the iteration count, as arethe accelerations after smoothing, ri

s is the linear system residual and Ps is thesmoothing preconditioner. For simplicity Jacobi type smoother is considered,i.e.

Ps = M + β∆t2K+ ≡ D (22)

such that K+ = diag(K). The coarse grid correction is given by

QT KQa = QT ri+1s , (23)

where a is the coarse scale acceleration and ri+1s = −F ext−Kd− Kai+1

s is theresidual after smoothing. We now consider the space time variant. The JacobiWR smoothing with the splitting denoted as K− = K+ −K yields

Mdν+1 + K+dν+1 = (K+ −K)dν − F ext

dν+1(0) = d0

dν+1(0) = v0

(24)

13

Consider one time step integration using Newmark method. The predictorphase is given by

dν+1 = d = d0 + ∆tv0 +∆t2

2(1− 2β)a0

vν+1 = v = v0 + ∆t(1− γ)a0

(25)

The solution phase with the relation (20) becomes

(M + β∆t2K+)aν+1 = (K+ −K)dν − F ext −K+d

= (K+ −K)(d + β∆t2aν)− F ext −K+d

= −F ext −Kd + β∆t2K+aν − β∆t2Kaν .

(26)

Addition and subtraction of M from the right hand side leads to

(M + β∆t2K+)aν+1 = −F ext −Kd + (M + β∆t2K+ −M − β∆t2K)aν . (27)

Finally, combining Eqs. (19) and (22) yields

aν+1s = (I −D−1K)aν −D−1(F ext + Kd), (28)

which is equivalent to Eq. (21) obtained from the smoothing in space. Next,we consider the coarse grid correction in space and time

QT MQe + QT KQe = QT rs

e(0) = 0

e(0) = 0

(29)

where rs = −(M ¨dν+1s +Kdν+1

s +F ext) is the defect computed on the fine scaleafter smoothing. Once again the solution phase for the coarse scale accelera-tions ac can be simplified as

QT MQac + β∆t2QT KQac = QT (M + β∆t2K)Qac = −QT (M ¨dν+1s + Kdν+1

s + F ext)

= −QT (Maν+1s + Kdν+1

s + F ext),

(30)

and since aν+1s and dν+1

s obtained from the WR smoothing are identical to thoseobtained from the standard smoothing, it follows that the space-time variantreduces to the sequential multigrid approach. To this end, it is instructive tonote that the space-time variant provides a general framework that for thespecific case considered in this section coincides with the sequential approach.Convergence estimates for the model problem have been reported in [22].

14

5 Numerical results

In this section we conduct performance studies of the space-time multigridmethod on a model problem of an atomistic chain and a polymer melt.

5.1 Atomistic chain with Harmonic potentials

We first consider an atomistic chain with heterogeneous Harmonic potentials inperiodic arrangement with one stiff and one compliant connection; the stiffnessratio between the two connection is taken as

kstiff

ksoft= 100. We consider periodic

long interactions such that each atom interacts with 10 closest neighbors.Newmark predictor-corrector algorithm is used to integrate the equations ofmotion. The integration time step for the explicit velocity-verlet method islimited by stability considerations. The critical step is obtained by consideringhow fast a wave travels in the stiffest cell, given by

∆tcr =2√

ρ(A),

where ρ(A) is the spectral radius (highest eigenvalue) of A = M−1K [15].For the multigrid and waveform methods considered, the length of the timestep is governed by the accuracy of the coarse fields of interest, selected hereas temperature, which is the average kinetic energy of the system. Figure 7depicts the relative error in the implicit methods computed as

Er[%] =|Tempexp − Tempimp|

|Tempexp| × 100, (31)

where Tempexp is the temperature obtained by the velocity-verlet methodand Tempimp is the temperature obtained by the implicit methods for vari-ous time steps. Despite the fact that the implicit time step is 40 times largerthan that of the explicit scheme the normalized error in of the temperature, atsteady-state, is less than 0.5%. Consequently, the time step for the space-timemultigrid method is chosen as ∆timp = 40×∆texp. Table 1 reports convergenceresults for the space-time multigrid method compared to the velocity-verletand waveform relaxation method. The atomic chain consists of 1001 atomsand is integrated over 16 implicit time steps. We use one pre Jacobi smooth-ing and a coarse grid correction. The coarse grid possess homogenized materialproperties [6, 7]. Various columns in Table 1 correspond to the type of method,number of windows selected, integration time step (dt), total number of it-erations and overall CPU time. All calculations were performed on a singleprocessor machine. It can be seen that the space-time multigrid method hasthe best performance. To study the windowing strategy, we conduct a series

15

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.50

5

10

15

20

25

30

35

40

Normalized Integration Time

| (T

emp ex

p − T

emp im

p) / T

emp ex

p | ×

100

[%]

Integration error (Harmonic Potentials)

∆ timp

= 4 × ∆ texp

∆ timp

= 20 × ∆ texp

∆ timp

= 40 × ∆ texp

∆ timp

= 100 × ∆ texp

Fig. 7. Error in temperature as a function of the time step length (Harmonic po-tentials)

Table 1CPU time and iteration summary for a 1D system with Harmonic potentials

Method Windows dt iterations CPU time [sec]

MG-space-time 1 4× 10−2 11 27.39

MG-space-time 2 4× 10−2 14 22.68

MG-space-time 4 4× 10−2 24 22.14

MG-space-time 8 4× 10−2 36 20.45

MG-space-time 16 4× 10−2 64 22.07

WR 1 4× 10−2 256 226.76

WR 2 4× 10−2 347 226.18

WR 4 4× 10−2 352 153.43

WR 8 4× 10−2 386 113.70

WR 16 4× 10−2 588 101.95

velocity-verlet - 1× 10−3 - 83.15

of numerical experiments performed on atomistic chains with various num-bers of atoms. The multigrid performance in various windows is illustrated inFigure 8.

16

1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 30000

50

100

150

200

250

300

350

400

450

500

550

# of Unknowns

CPU

tim

e [s

ec]

Windows Studies (Harmonic potentials)

1 Wind

2 Wind

4 Wind

8 Wind

16 Wind

Fig. 8. Windowing strategies in multigrid (Harmonic potentials)

5.2 Polymer melts with Lennard-Jones and Harmonic potentials

As a second system, we consider a unit cell of a polymer melt as shown in Fig-ure 9. Various colors correspond to different chains with every chain consistsof 16 atoms. Lennard-Jones (LJ) potentials are used to model the interaction

0510150

1020

2

4

6

8

10

12

14

16

Y

Nondimensional polymer unit cell

X

zz

Fig. 9. 20 polymer chains consisting of 16 atoms per chain

between all pairwise atoms in the system (all atoms including atoms at differ-ent chains), and Harmonic potentials are added along the polymer axis (only

17

within a chain). Mathematically we have

Φ =N∑

i=1

N−1∑

j>1

ΦLJ(rij) +1

2

∑

bonds

kbondc (rc − r0)

2. (32)

The LJ potential describes the energy of interaction between two atoms as afunction of the distance between their centers rij. The standard pairwise LJpotential is given by

ΦLJ(rij) = 4ε

(σ

rij

)12

−(

σ

rij

)6 (33)

where rij =√

(rx,i − rx,j)2 + (ry,i − ry,j)2 + (rz,i − rz,j)2; ε is the well-depth ofthe interaction and σ is the collision diameter. The total potential energy isgiven as a sum over all pairs

Φ =1

2

N∑

i=1

N∑

j=1i 6=j

ΦLJ(rij) =N∑

i=1

N−1∑

j>1

ΦLJ(rij). (34)

In our simulations we use normalized units with ε = σ = 1 and heterogeneousHarmonic potentials to model the chains. That is, different bond stretchingconstants kbond

c are used for different chains. Specifically,

kbondc =

1 c = 1, 3, ...

10 c = 2, 4, ...(35)

and r0 = 1. This heterogeneity creates a range of time scales, and thus theexplicit velocity-verlet algorithm, due to stability requirement, is limited bythe fast vibrating components. As before, we control the accuracy in temper-ature. Figure 10 shows the normalized error in temperature for 20 polymerchains that consist of 16 atoms per chain (see Figure 9). We use the explicitmethod with ∆texp = 1×10−3 as the reference solution. The allowable error intemperature is selected to be 1.5%. This corresponds to the implicit time stepthat is 20 times larger than the explicit time step. The system is integratedover 16 implicit steps. We apply one presmoothing of Jacobi waveform Newton(WN). The iteration is terminated when ‖dν+1 − dν‖2 ≤ 10−4 for all times.The prolongation operator in the space-time multigrid method is computedbased on the aggregation approach [23]. Specifically, we form a local Jacobianmatrix for every aggregate Jagg. In our case each aggregate corresponds to onepolymer chain. We select the 32 lowest modes by solving the eigen problem

Jaggvi = λivi i = 1, .., nm (36)

18

0 0.5 1 1.5 2 2.5 3 3.5 40

2

4

6

8

10

12

14

Normalized Integration Time

| (T

emp ex

p − T

emp im

p) / T

emp ex

p | ×

100

[%]

Integration error (3D Polymer chains)

∆ timp

= 10 × ∆ texp

∆ timp

= 20 × ∆ texp

∆ timp

= 40 × ∆ texp

∆ timp

= 60 × ∆ texp

Fig. 10. Error in temperature as a function of the time step (3D polymer chains)

Table 2CPU time and iteration summary for 8 polymer chains (16 atoms per chain) withmixed Lennard Jones and Harmonic (heterogeneous) potentials

Method Winds dt iter. Nonlin. iter. Func. eval. Eigen [sec] CPU [sec]

MG-space-time 4 20× 10−3 21 60 121 13.98 428.016

MG-space-time 8 20× 10−3 25 43 87 10.01 303.484

MG-space-time 16 20× 10−3 33 33 67 5.92 225.891

WN 4 20× 10−3 46 - 151 - 491.141

WN 8 20× 10−3 70 - 125 - 415.063

WN 16 20× 10−3 106 - 107 - 357.672

velocity-verlet 1 1× 10−3 - - 321 - 1049.72

Implicit Newmark 1 20× 10−3 - 112 113 - 380.156

and construct the prolongation operator as

Q =

{v1, ..., vnm}chain1

{v1, ..., vnm}chain2

. . .

{v1, ..., vnm}chainN

.

19

The CPU time required to construct the prolongation is reported in the Ta-ble 2. For the problem considered, it is found to be negligible compared to theoverall CPU time.For the numerical experiments considered the space-time multigrid methodoutperformed the explicit and WR methods. The best performance on a sin-gle processor machine is obtained when the window size is equal to the size ofthe time step.Finally, we study the performance of the space-time multigrid method as thenumber of polymer chains increase. Figures 11(a) and 11(b) illustrates theCPU times on a single processor machine for the explicit and the space-timemultigrid method as the number of polymer chains increases (each chain con-sists of 16 atoms) in one integration time step. The explicit method requires21 force computations while the multigrid only 5. It can be seen that thespeed-up factor has been found to be around 4 independent of problem size.

5.3 Parallel implementation for Harmonic potentials

In this section we consider implementation of the explicit and space-timemultigrid methods on a single processor, two-processor, four-processor and16-processor machines for the polymer melt unit cell described in the previoussection. We use the ML package developed at Sandia for the parallel im-plementation [24]. Table 3 and Figure 12 demonstrate the time parallelizationeffect on the speed-up factor over the explicit method. The number of polymerchains in a unit cell is indicated in the first column and performance of explicitand space-time multigrid methods is compared on several processors. We usethe same material parameters as Section 5.1. We apply one pre-smoothing ofWR-Jacobi iteration with no coarsening in space, i.e., the prolongation oper-ator coincides with the identity matrix. The resulting coarse scale problem issolved approximately using a single iteration with Jacobi preconditioner. Nu-merical integration is carried out in a single window with eight implicit steps.

Table 3Parallel CPU times for polymer unit cell integrated over one window (8 implicit

steps)

# of polymer P1 P2 P4 P16

chains MG Exp MG Exp MG Exp MG Exp

340 0.46 1.78 0.20 1.12 0.10 0.91 0.03 0.47

1365 2.30 10.04 2.11 6.04 0.65 4.50 0.11 1.47

5460 9.98 44.22 7.39 32.25 3.58 20.85 0.62 8.05

The results clearly show that as the number of processors increases, the speed-

20

0 5 10 15 20 25 30 35−200

0

200

400

600

800

1000

1200 Least Square approximation (3D studies)

space (# of polymer chains)

CP

U ti

me

[sec

]

MG

Explicit

(a) CPU times after one time step

0 5 10 15 20 25 300

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Space (# of polymer chains)

η

Fexp

/Fmg

=4.2

3D studies

(b) CPU time ratio between the explicit and multigrid meth-ods

Fig. 11. CPU time studies

up factor between the MG and velocity-verlet methods increases. The mainreason for the increase in the speed-up factor is due to processors communi-cation effect. In the case of standard explicit or implicit methods processorsare communicating after every time step. However, in the MG and WR casethe communication between processors only takes place at the end of eachwindow. This results in superior parallel performance as has been suggested

21

0 2 4 6 8 10 12 14 162

4

6

8

10

12

14

Number of Processors

CPU

exp/C

PUm

g

Parallel Results

Nchains

=340

Nchains

=1365

Nchains

=5460

Fig. 12. Multigrid parallel results (polymer unit cell)

in [18].

6 CONCLUSIONS

A space-time multigrid method is developed to accelerate time integration ofmolecular dynamics equations. The method consists of the waveform relax-ation smoothing aimed at capturing the oscillatory response of the atomisticvibrations and a coarse grained model evolution intended to resolve the smoothfeatures of the atomistic media. The formulation of the coarse grained modelis based on the variational approach derived from the Hamilton’s principle.The time integration is performed in windows using the Newmark predictor-corrector method. The integration scheme is implicit and thus larger time stepsgoverned by accuracy rather than by stability of requirements may be taken.Numerical examples show significant time savings compared to standard ex-plicit integrators. A parallel version of the space-time multigrid method hasbeen developed for harmonic potentials. Significant speed-ups over the stan-dard explicit method have been observed primarily due to reduced communi-cation time between the processors. Future work will include (i) considerationof coarse grids based on the Brownian dynamics and other coarse-grainedmodels and (ii) parallel implementation for various interatomic potentials.

22

Acknowledgements

The financial supports of National Science Foundation under grants CMS-0310596, 0303902, 0408359 and Sandia National Laboratory under contractDE-ACD4-94AL85000 are gratefully acknowledged. In particular, the authorswish to thank Dr. Ray Tuminaro from Sandia National Laboratory for hisconstructive suggestions

References

[1] D.C. Rapaport. The art of molecular dynamics simulations. Cambridgeuniversity press, 1995.

[2] D. Barash, L. Yang, X. Qian, and T. Schlick. Inherent speedup limita-tions in multiple time step/particle mesh ewald algorithms. Journal ofComputational Chemistry, 24:77–88, 2003.

[3] B. J. Leimkuhler, S. Reich, and R. D. Skeel. Integration methods formolecular dynamics. IMA Volumes in Mathematics and its Applications,82:161–186, 1997.

[4] D. Janezic and B. Orel. Implicit Runge-Kutta method for moleculardynamics integration. Journal of Chemical Information and ComputerSciences, 33:252–257, 1993.

[5] J. Fish and W. Chen. Discrete-to-continuum bridging based on multigridprinciples. Computer Methods in Applied Mechanics and Engineering,193:1693–1711, 2004.

[6] J. Fish and V. Belsky. Multigrid method for periodic heterogeneous mediapart 1: Convergence studies for one-dimensional case. Computer Methodsin Applied Mechanics and Engineering, 126:1–16, 1995.

[7] J. Fish and V. Belsky. Multi-grid method for periodic heterogeneous me-dia part 2: Multiscale modeling and quality control in multidimensionalcase. Computer Methods in Applied Mechanics and Engineering, 126:17–38, 1995.

[8] W. Hackbusch. Multi-grid methods and applications. Springer, 1985.[9] E. Lelarasmee, A. E. Ruehli, and A. L. Sangiovanni-Vincentelli. The

waveform relaxation method for time-domain analysis of large scale inte-grated circuits. IEEE transactions on computer-aided design of integratedcircuits and systems, CAD-1(3):131–145, 1982.

[10] K. Burrage. Parallel and sequential methods for ordinary differential equa-tions. Oxford University Press, 1995.

[11] S. Ta”san and H. Zhang. Fourier-laplace analysis of multigrid waveformrelaxation method for hyperbolic equations. Technical Report ICASE96-53, NASA, 1996.

23

[12] B. Leimkuhler. Timestep acceleration of waveform realxation. SIAMJournal on Numerical Analysis, 35(1):31–50, 1998.

[13] R. Saleh and J. White. Accelerating relaxation algorithms for circuit sim-ulation using waveform-newton and step-size refinement. IEEE Transac-tions on computer-aided design, 9(9):951–958, 1990.

[14] Andrew Lumsdaine and Mark W. Reichelt. Waveform iterative tech-niques for device transient simulation on parallel machines. In Sixth SIAMConference on Parallel Processing for Scientific Computing, Norfolk, VA,1993.

[15] T. J. R. Hughes. The finite element method. Linear static and dynamicfinite element analysis. Prentice-Hall, 1987.

[16] U. Miekkala and O. Nevanlinna. Convergence of dynamic iteration meth-ods for initial value problems. SIAM Journal on Scientific and StatisticalComputing, 8(4):459–482, 1987.

[17] E. Giladi and H. B. Keller. Space time domain decomposition for par-abolic problems. Numerische Mathematik, 93(2):279–313, 2002.

[18] S. Vandewalle. The parallel solution of parabolic partiall differential equa-tions by multigrid waveform relaxation methods. PhD thesis, KatholikeUniversiteit Leuven, 1992.

[19] G. Horton and S. Vandewalle. A space-time multigrid method for par-abolic partial differential equations. SIAM Journal on Scientific and Sta-tistical Computing, 16(4):848–864, 1995.

[20] J. T. Padding and W. J. Briels. Coarse-grained molecular dynamics sim-ulations of polymer melts in transient and steady shear flow. Journal ofChemical Physics, 118(22):10276–10286, 2003.

[21] C. S Peskin and Schlick T. Molecular dynamics by the backward-eulermethod. Communications On Pure and Applied Mathematics, 42:1001–1031, 1989.

[22] J. Fish and W. Chen. Rve-based multilevel method for periodic hetero-geneous media with strong scale mixing. Journal of Engineering Mathe-matics, 46:87–106, 2003.

[23] J. Fish and V. Belsky. Generalized Aggregation Multilevel solver. Interna-tional Journal for Numerical Methods in Engineering, 40(23):4341–4361,1997.

[24] C. Tong and R. S. Tuminaro. ML 2.0 smoothed aggregation user’s guide.Technical Report SAND2001-8028, Sandia National Laboratories, 2000.

24