Distributional Direct Simulation Monte Carlo Methods

Christopher R. Schrock∗ and Aihua W. Wood†

Air Force Institute of Technology, Wright-Patterson AFB, OH 45433

The Direct Simulation Monte Carlo (DSMC) method has gained popularity in recentyears for treatment of flows in which the assumptions behind the continuum equationsof fluid mechanics break down. In traditional DSMC methods, simulated particles maypossess only a single velocity. This representation leads to a nonphysical representation ofthe velocity distribution function which limits convergence of the method. In a previouswork, we proved that by applying kernel density estimation techniques, strong convergencefor L

∞ solutions and pointwise convergence for bounded solutions was obtainable. Inthe current work we present numerical results for the DSMC-KDE method and presentour current developments and results relating to a Distributional DSMC method. Initialnumerical results indicate improved convergence is facilitated by such an approach overthe DSMC-KDE method alone. We propose that the development of such a method couldsubstantially improve convergence and potentially result in a substantial reduction of thevariance associated with DSMC methods.

Nomenclature

~c Molecular Velocity Vector~c′ Post Collision Molecular Velocityc Nondimensional Molecular Speedd Molecular Diameterf Molecular Velocity Distribution Function (VDF)F Molecular Speed Distribution Function

f Nanbu’s VDF Approximation

f DSMC-KDE VDF Approximation~g Relative Velocityh Kernel Bandwidthk Boltzmann ConstantK Kernel Functionm Particle Massn Particle Number DensityN Total Number of ParticlesNp Number of Simulated ParticlesNv Number of Velocity Samples per Particle~Ω Collision Orientation VectorPi Collision Probability of Particle iPij Collision Probability of Collision Pair (i, j)~r Spatial Variableσ Collision Cross Section, Standard Deviationt Time Variableτ Nondimensional Time VariableT Translational Temperature

∗PhD Candidate, Department of Mathematics and Statistics, christopher.schrock@wpafb.af.mil, Senior Member†Professor, Department of Mathematics and Statistics, aihua.wood@afit.edu

1 of 12

American Institute of Aeronautics and Astronautics

10th AIAA/ASME Joint Thermophysics and Heat Transfer Conference28 June - 1 July 2010, Chicago, Illinois

AIAA 2010-4501

This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States.

I. Introduction

The Direct Simulation Monte Carlo (DSMC) method has gained wide acceptance for the simulationof rarefied and nonequilibrium flows. Convergence of such methods was established in works publishedby Babovsky & Illner1, 2 and Wagner.3 The convergence properties established for such methods is fairlyweak, and amounts to convergence in moments of the distribution function for well-behaved test functions(bounded, continuous). It is the contention of the authors that one of the major limitations to strongerforms of convergence is the singular nature of the simulated particle velocities, and indeed we have provenmathematically that by allowing each simulated particle’s velocity to be distributed, stronger forms ofconvergence are obtainable.4

In the DSMC method, one simulates a fraction of the number of actual particles and each simulatedparticle is taken to represent W = N/Np actual particles, where N is the total number of particles and Np isthe number of simulated particles. In practice W may be on the order of 106 or more. As a simulated particlecan possess only a single velocity vector, Nanbu has shown that this is equivalent to a delta distribution overthe W particles that particle represents.5 Therefore, when considering the overall distribution function inthe gas, we obtain

f (~c) =

Np∑

i

1

Npfi =

1

Np

Np∑

i

δ (~c − ~ci) (1)

where ~ci is the particle’s velocity vector.It is well known that the steady state solution of the space homogeneous Boltzmann equation in the

absence of external forces takes on a Maxwellian form.6 From Equation (1), it is easy to see that no suchsolution can naturally arise from the Nanbu approximation simulation, and convergence is constrained tooccur only in moments. In fact, it can be shown4 that if the Boltzmann equation has a solution f , the L1

error of the Nanbu approximation to f is unity, regardless of the number of simulated particles and the sizeof the time step.

In an effort to improve convergence, we have begun development “Distributional DSMC” (DDSMC)methods. In DDSMC, the delta function representation of the distribution function is replaced by a family ofmore appropriate functions which are a better approximation of the actual distribution of molecular velocityover the millions of particles that each simulated particle represents. In this work we present preliminaryresults in the pursuit of such methods. The method improves upon Nanbu’s scheme and relies on kerneldensity estimation (KDE).

II. Kernel Density Estimation

The basic premise of the current research relies upon the replacement of the delta function approximationto the simulated particle densities. Kernel density estimation (KDE) approximates a probability densityfunction of a random variable X ∈ R

d from a set of N discrete samples as follows.7

f (x; h) =1

Nhd

N∑

i=1

K

(

x − Xi

h

)

(2)

Here, N is the number of discrete samples, d is the dimensionality of the random variable, Xi is the value ofthe ith sample, K ∈ L2 is the kernel function, satisfying

∫

K (x) dx = 1 and∫

xK (x) dx = 0, and h ∈ R+ is

the kernel bandwidth.The problem then becomes one of determining a suitable K and h with which to approximate the

distribution function. The value of h is chosen as to minimize the error between the estimator and the actualdistribution function in some sense. If h is too small, the estimator will exhibit overly oscillatory behavior.It h is too large, subtle features of the distribution function may not be captured by the estimator. Wand7

shows that the asymptotic mean square error between f and f is minimized when

h =

[

m (K)

(µ2 (K))2m (f ′′)N

]15

(3)

where,

m (g) =

∫

[g (x)]2dx

2 of 12

American Institute of Aeronautics and Astronautics

µ2 (g) =

∫

x2g (x) dx

If f is normal with variance σ2, Equation (3) becomes,

h =

[

8√

πm (K)

3 (µ2 (K))2N

]15

σ

III. DSMC-KDE

Observe that the Nanbu approximation to the distribution function may be viewed as a special case ofkernel density estimator with K = δ and h = 1. Recognizing this similarity, the authors began exploration ofa distributional DSMC method by considering a generalization of Nanbu’s method based on kernel densityestimation methods. We define the approximation to the distribution function as

f (~c; h) =1

Nph3

Np∑

i=1

K

(

~c − ~ci

h

)

(4)

where,

K (~x) = (2π)−3/2

exp

(

−‖~x‖2

2

)

(5)

h =

[

32

3√

2Np

]15

σest (6)

where σest is an estimation of the standard deviation of f . These choices are advantageous for a number ofreasons. If one chooses h such that h → 0 as Np → ∞, Equation 4 will converge to the delta representation.In the opposite limit of Np → 1, the representation becomes normal, which is the prevailing distribution inan equilibrium gas. This suggests that the accuracy of such a method will vary from an equilibrium gas allthe way to a full non-equilibrium solution depending upon the number of simulated particles.

Equation 4 may simply be interpreted as a kernel density estimator which utilizes the particle velocitiesobtained via Nanbu’s method as a sample of the overall distribution function. In this case, the stochasticsimulation remains the same as Nanbu’s method and the distribution function may be calculated fromEquation 4 post-simulation. Equation 4 may also be interpreted by assuming each particle’s velocity isdistributed according to a Maxwellian centered at ~ci. We have shown4 that this interpretation results inidentical collision selection and modeling rules for the center points of the Maxwellians in the limit as Np → ∞and that the stochastic rules governing the evolution of the simulation can be chosen to be identical toNanbu’s method. Utilizing this interpretation, we have proven4 that this method maintains the convergencedemonstrated for existing methods, achieves strong convergence for L∞, and pointwise convergence boundedsolutions, niether of which is possible with the original method. Such solutions arise frequently and are ofgreater practical interest than the general L1 case.

A brief overview of the simulation scheme is in order. As in Nanbu’s method, spatial homogeneity isassumed within each cell. Within a cell, the distribution function is given by Equation (4). Utilizing aforward Euler discretization one may approximate f at the next time step.

f (~c, t + ∆t) = f (~c, t) + ∆t∂f

∂t(~c, t) (7)

where ∂f∂t is obtained from the space homogeneous Boltzmann equation.

∂f

∂t(~c, t) =

∫

R3

∫

S+

2

[

f(

~c′(

~c, ~c1, ~Ω)

, t)

f(

~c′1

(

~c, ~c1, ~Ω)

, t)

− f (~c, t) f (~c, t)]

gσ (g, Ω)d~Ωd~c1 (8)

Substituting Equation (4) into Equations ( 8) and ( 7) and passing to the limit where Np → ∞, one obtains4

f (~c, t + ∆t) =1

Np

Np∑

i=1

[(1 − Pi) δ (~c − ~ci) + Qi] (9)

3 of 12

American Institute of Aeronautics and Astronautics

where,

Pi =∆t

Np

Np∑

j=1,j 6=i

gijσT (gij)

Qi (~c) =4∆t

Np

Np∑

j=1,j 6=i

σ (gij , χ)

gijδ

(

∥

∥~c∗∥

∥ − 1

2gij

)

σT (gij) =

∫

S+

2

σ(

gij , ~Ω)

d~Ω

~c∗ = ~c − 1

2(~ci − ~cj)

and χ is the angle between ~c∗ and ~ci − ~cj . One may see therefore, that in the appropriate limit, this resultis completely analogous to Nanbu’s original result5 but now the particle velocity distribution functions arereplaced by Gaussians, rather than delta functions. So as not to repeat prior work, we refer the reader to ourprevious work4 and Nanbu’s original work5 for full details of the implementation of the method and repeatonly the main points here.

• For each particle, calculate Pi. Generate a random number r1 in the interval (0, 1) . If Pi > r1, acceptthe particle for collision.

• Sample a collision partner j from the conditional probability distribution P ∗ik = Pik

Pi, by sampling

a second random number r2 uniformly from the interval (0, 1) and identifying the j which satisfies∑j−1

k=1 P ∗ik < r2 <

∑jk=1 P ∗

ik

• Sample the direction of ~c∗ based on Equation 9, and compute the post collision velocity of the ith

particle.

• Compute each particle’s new position according to ~r = ~r0 + ~c∆t, where ~r0 is the particle’s position atthe previous time step.

In practice, the simulation is typically evolved many times to generate an ensemble averaged solution so asto reduce statistical fluctuations.

The DSMC-KDE method is not fully distributional as envisioned by this research, but is serving as astepping stone to a more comprehensive method. This is due to the fact that the distributions are calculatedpost simulation and have no impact on collision selection or modeling. The method may be viewed as apost-processing modification to enhance convergence of Nanbu’s method.

Numerical Implementation

In order to demonstrate the DSMC-KDE method numerically, we applied the method to an exact solution ofthe space homogenous Boltzmann equation discovered by Krook and Wu.5, 8 To further simplify, we assumethe Maxwellian molecular model. The distribution function is spherically symmetric and in this case thedistribution of molecular speed is given by5

F (c, τ) = 4π (πα)− 3

2

(

α1c2 + α2c

4)

exp

(

− c2

α

)

(10)

where,

α1 =5α − 3

2α

α2 =1 − α

α2

c =c

√

2kT/m

α (τ) = 1 − 2

5e−τ

τ = πA2 (5)nt√

2b/m

4 of 12

American Institute of Aeronautics and Astronautics

The DSMC-KDE solution is shown in Figure 1. These results were obtained utilizing 320 simulated particlesand an ensemble of 100 runs. In order to obtain an understanding of accuracy and convergence of the

−4 −3 −2 −1 0 1 2 3 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Vx

f x

τ=0τ=0.1387τ=0.2789τ=0.4190τ=0.5592τ=0.6993τ=0.8394τ=0.9796τ=1.1197τ=1.2599τ=1.4000

Figure 1. DSMC-KDE Solution for x-velocity Distribution of the Krook-Wu Problem. Np = 320, Nsamp = 100, ∆τ =

1.4E − 3, Total Computational Time = 289.14 sec

method, we computed the minimum, mean, and maximum steady state L1 error for a 100 run ensembleas a function of the number of simulated particles. The results are shown in Figure 2. We see improvedconvergence over the original Nanbu DSMC method, although the benefits are not realized at low values ofNp. Finally, it is worth noting that the computational time per sample is identical to Nanbu’s method asthe actual calculations are the same. This is illustrated in Figure 3, where it is evident that the complexityvaries O

(

N2p

)

. As the KDE method may be viewed as a post simulation operation, it adds no computationalcomplexity until the actual distribution function is evaluated. If it is desired that the value of the distributionfunction is calculated at M points, a direct calculation would have a computational complexity O (MNp).It is possible to reduce the complexity to O (M + Np) by means of the Fast Gauss Transform.9

IV. Distributional DSMC (DDSMC)

A fully distributional method should possess stochastic collision selection criteria and collision modelingthat fully incorporates non-singular particle velocity distributions which are consistent even in the caseof finite Np. This requires a full re-derivation of the DSMC algorithm that should be attainable by ananalysis analogous to that which Nanbu performed to derive his original scheme. The analysis is much morecomplicated when non-singular velocity distributions are involved and as such we have developed a simplifiedscheme that improves upon the DSMC-KDE method.

We continue to utilize the KDE technique, however, we now apply it at the particle level. No longer areparticle distributions constrained to be Gaussian. We do, however, simplify the collision selection criteria byassuming that the particle distribution is “close” to a Gaussian form centered at the particle’s mean velocity.In this case, the collision selection criteria for the DSMC-KDE method can be re-used. The effect of allowinga generalized particle distribution function is taken into account in the collision modeling.

In principle, the collision modeling of two simulated particles is relatively easy to understand. One first

5 of 12

American Institute of Aeronautics and Astronautics

Figure 2. Steady State L1 Error for 100 Run Ensemble as a Function of Np

compute the combined velocity distribution function of the two particles and evolves it through ∆t utilizingsome numerical process. The exact process is somewhat flexible. One could utilize a moment method, a modelequation, or some other numerical technique traceable to the Boltzmann equation. In our previous work,4 wepresented preliminary results utilizing the BGK equation for this evolution. Unfortunately, since the BGKequation is not consistent with the Boltzmann equation, convergence is not guaranteed. In this work, weutilize the DSMC-KDE method at the particle level to evolve the combined velocity distribution function.As we have already proven that the DSMC-KDE method is consistent with the Boltzmann equation, wehave some confidence that such a method would be successful. In this case, the overall distribution functionis given by

f =1

Np

Np∑

i=1

fi

Here, fi is the ith simulated particle’s velocity distribution given by,

fi (~c) =1

Nvh3

Nv∑

j=1

K

(

~c − ~cij

h

)

where, Nv is the number of velocity samples in the kernel density estimate of each particle’s velocity distri-bution function and h is now given by

h =

[

32

3√

2NpNv

]15

σest

6 of 12

American Institute of Aeronautics and Astronautics

Figure 3. Computational Time per Sample as a Function of Np

The combined distribution function of a collision pair i, j is given by

F (~c) =1

2(fi (~c) + fj (~c)) =

1

2Nvh3

Nv∑

k=1

[

K

(

~c − ~cik

h

)

+ K

(

~c − ~cjk

h

)]

(11)

It is easily observed that Equation ( 11) is analogous to Equation ( 4) with Np = 2Nv. Therefore, thecollision selection and modeling rules previously derived may be reused to select and compute interactionsof the Gaussian centers of the combined kernel density estimator. This approach actually has a secondarybenefit, in that it allows for a particle’s distribution to evolve by self-interaction. This would be nonsensicalif the particle represented an actual single particle, but when one recalls that in practice simulated particlesrepresent vast numbers of actual particles, such interaction is physically realizable.

Numerical Implementation

To explore the DDSMC method currently proposed we applied the scheme to the same problem previouslydiscussed of Krook and Wu.8 The results of this analysis are shown in Figure 4. Notice the higher level ofdetail captured in this model than with the prior DSMC-KDE results. Figure 5 shows the mean steady stateL1 error for a 100 run ensemble as a function of Np and Nv. Here we see significantly improved results overthe original Nanbu method, as well as DSMC-KDE. Finally in Figure 6 we have the computational timeper sample as a function of Np and Nv. Note that although the computational time per sample is increasedby the additional complexity of the method, the increase in accuracy over the DSMC-KDE method faroutweighs the drawbacks. This is evident in Figures 7 and 8 where the values are plotted against the totalnumber of parameters per sample, NpNv.

7 of 12

American Institute of Aeronautics and Astronautics

−4 −3 −2 −1 0 1 2 3 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Vx

f x

τ=0τ=0.1387τ=0.2789τ=0.4190τ=0.5592τ=0.6993τ=0.8394τ=0.9796τ=1.1197τ=1.2599τ=1.4000

Figure 4. DDSMC Solution for x-velocity Distribution of the Krook-Wu Problem. Np = 20, Nv = 16, Nsamp = 100,

∆τ = 1.4E − 3, Total Computational Time = 46.27 sec

V. Conclusions and Future Work

Based upon the results of the numerical experiments undertaken to date, the DDSMC concept appears tobe quite promising. We have shown that by distributing a simulated particles velocity improved convergencecan be achieved using the DSMC-KDE method. Furthermore, we have seen that applying the DSMC-KDEapproach as a building block for a DDSMC method can result in substantial computational savings.

Although the numerical evidence indicates that a potentially significant benefit exists in applying theDDSMC method, care must be taken to formally derive and prove convergence of such a method, and ourfuture efforts will focus on the development of these results Further numerical experiments and analysiswill be undertaken to formally establish the computational complexity of such methods and to explore theirgeneralization to higher dimensional problems.

References

1Babovsky, H., “A convergence proof for Nanbu’s Boltzmann simulation scheme,” European Journal of Mechanics B:

Fluids, Vol. 8, No. 1, 1989, pp. 41–55.2Babovsky, H. and Illner, R., “A convergence proof for Nanbu’s simulation method for the full Boltzmann equation,” SIAM

Journal on Numerical Analysis, Vol. 26, No. 1, 1989, pp. 45–65.3Wagner, W., “A convergence proof for Bird’s Direct Simulation Monte Carlo method for the Boltzmann equation,” Journal

of Statistical Physics, Vol. 66, No. 3-4, 1992, pp. 1011–1044.4Schrock, C. and Wood, A., “A Modification of Nanbu’s Method and Proof of Enhanced Convergence,” Manuscript sub-

mitted for publication, 2010.5Nanbu, K., “Direct Simulation Scheme Derived from the Boltzmann Equation I. Monocomponent Gases,” Journal of the

Physical Society of Japan, Vol. 49, 1980, pp. 2042–2049.6Cercignani, C., Mathematical Methods in Kinetic Theory , Plenum Press, 1990.7Wand, M. and Jones, M., Kernel Smoothing , Chapman and Hall, 1995.8Krook, M. and Wu, T., “Exact Solutions of the Boltzmann Equation,” The Physics of Fluids, Vol. 20, No. 10, 1977,

pp. 1589–1595.9Greengard, L. and Strain, J., “The Fast Gauss Transform,” SIAM Journal of Scientific Computing , Vol. 12, 1991.

8 of 12

American Institute of Aeronautics and Astronautics

Figure 5. Steady State L1 Error for 100 Run Ensemble as a Function of Np

9 of 12

American Institute of Aeronautics and Astronautics

Figure 6. Computational Time per Sample as a Function of Np

10 of 12

American Institute of Aeronautics and Astronautics

Figure 7. L1 Error for 100 Run Ensemble as a Function of Np

11 of 12

American Institute of Aeronautics and Astronautics

Figure 8. Computational Time per Sample versus Total Number of Parameters

12 of 12

American Institute of Aeronautics and Astronautics