Upload
odessa
View
29
Download
0
Embed Size (px)
DESCRIPTION
Parallel solution of the Helmholtz equation with large wave numbers. Dan Gordon Computer Science University of Haifa. Rachel Gordon Aerospace Eng. Technion. Outline of Talk. The Kaczmarz algorithm ( KACZ ) KACZ CARP ( C omponent- A veraged R ow P rojections) Applications of CARP - PowerPoint PPT Presentation
Citation preview
July 1, 2010 Parallel solution of the Helmholtz equation 1
Parallel solution of the Parallel solution of the Helmholtz equation with Helmholtz equation with
large wave numberslarge wave numbers
Dan GordonComputer ScienceUniversity of Haifa
Rachel GordonAerospace Eng.
Technion
July 1, 2010 Parallel solution of the Helmholtz equation 2
Outline of TalkOutline of Talk The Kaczmarz algorithm (KACZ) KACZ CARP (Component-
Averaged Row Projections) Applications of CARPCARP-CG: CG acceleration of CARP Sample results with the Helmholtz
equation
July 1, 2010 Parallel solution of the Helmholtz equation 3
KACZKACZ: The Kaczmarz algorithm: The Kaczmarz algorithm
Iterative method, due to Kaczmarz (1937). Rediscovered for CT as ART
Geometric algorithm: consider the hyperplane defined by each equation
Start from an arbitrary initial point Successively project current point
onto the next hyperplane, in cyclic order
July 1, 2010 Parallel solution of the Helmholtz equation 4
KACZ: Geometric DescriptionKACZ: Geometric Description
eq. 1
eq. 2eq. 3
initialpoint
July 1, 2010 Parallel solution of the Helmholtz equation 5
KACZKACZ with Relaxation Parameter with Relaxation ParameterKACZ can be used with a relaxation
parameter =1: project exactly on the
hyperplane<1: project in front of hyperplane>1: project beyond the hyperplane Cyclic relaxation: eq. i is assigned a
relaxation parameter i
July 1, 2010 Parallel solution of the Helmholtz equation 6
Convergence Properties of Convergence Properties of KACZKACZ
KACZ with relaxation (0<<2) converges for consistent systems:– Herman, Lent & Lutz, 1978– Trummer, 1981
For inconsistent systems, KACZ converges cyclically:– Tanabe, 1971– Eggermont, Herman & Lent, 1981 (for
cyclic relaxation parameters).
July 1, 2010 Parallel solution of the Helmholtz equation 7
Algebraic formulation of Algebraic formulation of KACZKACZ Given the system Ax = b Consider the "normal equations"
system AATy = b, x=ATy Well-known fact: KACZ is SOR
applied to the normal equations The relaxation parameter of KACZ is
the usual relax. par. of SOR
July 1, 2010 Parallel solution of the Helmholtz equation 8
Block Mode & ParallelizationBlock Mode & Parallelization Block KACZ: projection onto affine
subspace defined by a block of eqns Block-sequential KACZ:
– partition eqns into blocks– each block consists of independent eqns– iterate over blocks– in each block, perform projections in
parallel
July 1, 2010 Parallel solution of the Helmholtz equation 9
CARPCARP: Component-Averaged Row Projections: Component-Averaged Row Projections
A block-parallel version of KACZ The equations are divided into blocks
(not necessarily disjoint) Initial estimate: vector x=(x1,…,xn) Suppose x1 is a variable (component
of x) that appears in 3 blocks x1 is “cloned” as y1 , z1 , t1 in the
different blocks. Perform one (or more) KACZ
iteration(s) on each block (independently, in parallel)
July 1, 2010 Parallel solution of the Helmholtz equation 10
CARPCARP – Explanation (cont) – Explanation (cont) The internal iterations in each block
produce 3 new values for the clones of x1 : y1’ , z1’ , t1’
The next iterative value of x1 is x1’ = (y1’ + z1’ + t1’)/3
The next iterate isx’ = (x1’ , ... , xn’)
Repeat iterations as needed for convergence
July 1, 2010 Parallel solution of the Helmholtz equation 11
CARPCARP as Domain Decomposition as Domain Decomposition
x x
y0 1
1
domain Adomain A domain Bdomain B
external gridexternal gridpoint of Apoint of A
clone of clone of x1
Note: domains may overlap
July 1, 2010 Parallel solution of the Helmholtz equation 12
Overview of Overview of CARPCARP
domain Adomain A domain Bdomain B
KACZKACZiterationsiterations
KACZKACZiterationsiterations
averagingaveraging
cloningcloning
KACZKACZ in superspace in superspace(with cyclic relaxation)(with cyclic relaxation)
July 1, 2010 Parallel solution of the Helmholtz equation 13
Convergence of Convergence of CARPCARP Averaging Lemma: the component-
averaging and cloning operations of CARP are equivalent to KACZ row-projections in a certain superspace (with =1)
CARP is equivalent to KACZ in the superspace, with cyclic relaxation parameters – known to converge
July 1, 2010 Parallel solution of the Helmholtz equation 14
CARPCARP Application: Solution of stiff Application: Solution of stiff linear systems from PDEslinear systems from PDEs
Elliptic PDEs w/large convection term result in stiff linear systems (large off-diagonal elements)
CARP is very robust on these systems, as compared to leading solver/preconditioner combinations
Downside: Not always efficient
July 1, 2010 Parallel solution of the Helmholtz equation 15
CARPCARP Application: Electron Tomography Application: Electron Tomography(joint work with J.-J. Fernández)(joint work with J.-J. Fernández)
3D reconstructions: Each processor is assigned a block of consecutive slices. Data is in overlapping blobs.
The blocks are processed in parallel. The values of shared variables are
transmitted between the processors which share them, averaged, and redestributed.
July 1, 2010 Parallel solution of the Helmholtz equation 16
CARP-CGCARP-CG: CG acceleration of : CG acceleration of CARPCARPCARP is KACZ in some superspace
(with cyclic relaxation parameters) Björck & Elfving (BIT 79): developed
CGMN, which is a (sequential) CG-acceleration of KACZ (double sweep, fixed relax. parameter)
We extended this result to allow cyclic relaxation parameters
Result: CARP-CG
July 1, 2010 Parallel solution of the Helmholtz equation 17
CARP-CGCARP-CG: Properties: Properties Same robustness as CARP Very significant improvement in
performance on stiff linear systems derived from elliptic PDEs
Very competitive runtime compared to leading solver/preconditioner combinations on systems derived from convection-dominated PDEs
Improved performance in ET
July 1, 2010 Parallel solution of the Helmholtz equation 18
CARP-CGCARP-CG: Properties: Properties On one processor, CARP-CG is
identical to CGMN Particularly useful on systems with
LARGE off-diagonal elements– example: convection-dominated PDEs
Discontinuous coefficients are handled without requiring domain decomposition (DD)
July 1, 2010 Parallel solution of the Helmholtz equation 19
Robustness of CARP-CGKACZ inherently normalizes the eqns After normalization, the diagonal
elements of AAT are larger than the off-diagonal ones (in each row)
This is not diagonal dominance, but it makes the normal eqns manageable
Normalization was also found to be useful for discontinuous coefficients
July 1, 2010 Parallel solution of the Helmholtz equation 20
The Helmholtz Equation Eqn: -Δu - k2u = f Wave length: = 2/k No. of grid pts per : Ng = 2/kh Shifted Laplacian approach:
– Bayliss, Goldstein & Turkel, 1983– Erlangga, Vuik & Oosterlee, 2004/06
-Δu – (ik2 u = fuses multigrid to solve the PC (PC = preconditioner)
July 1, 2010 Parallel solution of the Helmholtz equation 21
The Helmholtz Equation Bollhöfer, Grote & Schenk, 2009:
introduced algebraic multilevel PC for the Helmholtz eqn in heterogeneous media. Uses symmetric max weight matchings and an inverse-based pivoting method.
Apologies to many other contributors to this problem!
July 1, 2010 Parallel solution of the Helmholtz equation 22
ExperimentsCARP-CG was used with a fixed
relaxation parameter of 1.7 in all cases
Domain: unit square []
2nd order central difference scheme
July 1, 2010 Parallel solution of the Helmholtz equation 23
Problem 1 (with analytic sol'n)
Based on Erlangga et al '04, §6.1 Eqn: (Δ+k2)u = (k2–52)sin(x)sin(2y) bndry condition: u=0 on all sides Analytic solution: u = sin(x)sin(2y) Grid points per : Ng = 6,8,10,12 No. of processors: 1 – 32 k = 100, 300
July 1, 2010 Parallel solution of the Helmholtz equation 28
Problem 2 (homogeneous) Based on Erlangga et al '04, §6.2 Eqn: Δu + k2u = 0 Domain: unit square [0,1]x[0,1] Dirichlet bndry cond. on one side, with a
discontinuity at midpoint 1st-order absorbing bndry cond. on other
sides Grid points per : Ng = 6, 8, 10 No. of processors: 1 – 32 k = 75, 150, 300, 450, 600
July 1, 2010 Parallel solution of the Helmholtz equation 29
July 1, 2010 Parallel solution of the Helmholtz equation 30
July 1, 2010 Parallel solution of the Helmholtz equation 31
Problem 3 (heterogeneous) 3-layer heterogeneous problem Based on Erlangga et al '04, §6.3 Everything is identical to Problem 2 EXCEPT:
k=600
k=450
k=300
July 1, 2010 Parallel solution of the Helmholtz equation 34
July 1, 2010 Parallel solution of the Helmholtz equation 35
July 1, 2010 Parallel solution of the Helmholtz equation 36
July 1, 2010 Parallel solution of the Helmholtz equation 37
Comparative Timing Results
Methodtime/iter1 proc
time/iter16 proc
it-ratio1 proc
it-ratio16 proc
CARP-CG 0.0978 0.0134 1.00 1.00
Bi-CGSTAB 0.1477 0.0344 1.51 2.56
GMRES(restart=
10)0.1490 0.0212 1.52 1.58
Time/iter of Bi-CGSTAB and GMRES relative to CARP-CGit-ratio = (time/iter of algorithm) / (time/iter of CARP-CG)
Results from CARP-CG paper (PARCO, 2010)
Timing and Speedup Results
No. proc No. Iter Time (s) Speed-up Efficiency1 7025 1256 1.00 100%2 7039 822 1.53 76.4%4 7066 462 2.72 68.0%8 7115 255 4.92 61.5%15 7206 159 7.92 52.8%
Problem 2, k=600, Ng=8, grid: 763763 582,169 (complex) equations, rel-res<10-7
SummaryCARP-CG is highly scalable on the
Helmholtz eqn w/high wave numbers Applicable to discontinuous
coefficients Very simple to implement General-purpose – useful also for
other problems with large off-diagonal elements and discontinuous coefficients
July 1, 2010 Parallel solution of the Helmholtz equation 41
Other Potential ApplicationsOther Potential Applications Fourth-order schemes for the
Helmholtz equation (already have good initial results)
Maxwell equations Saddle-point problems Circuit problems Linear solvers in some eigenvalue
methods ...
July 1, 2010 Parallel solution of the Helmholtz equation 42
Relevant PublicationsRelevant Publicationshttp://cs.haifa.ac.il/~gordon/pub.htmlCARP: SIAM J Sci Comp 2005CGMN: ACM Trans Math Software 2008Microscopy: J Parallel & Distr Comp 2008Large convection + discontin coef: CMES 2009CARP-CG: Parallel Comp 2010Scaling for discont coef: J Comp & Appl Math 2010
CARP-CG SOFTWARE AVAILABLE ON REQUEST
THANK YOU!