33
149 PDE 7.2 Partial Differential Equations (PDE) 7.2 Partial Differential Equations (PDE) PDE overview Examples of PDE-s: Laplace’s equation important in many fields of science, * electromagnetism * astronomy * fluid dynamics behaviour of electric, gravitational, and fluid potentials The general theory of solutions to Laplace’s equation – potential theory In the study of heat conduction, the Laplace equation – the steady-state heat equation

7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

Embed Size (px)

Citation preview

Page 1: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

149 PDE 7.2 Partial Differential Equations (PDE)

7.2 Partial Differential Equations (PDE)

PDE overviewExamples of PDE-s:

• Laplace’s equation

– important in many fields of science,

∗ electromagnetism

∗ astronomy

∗ fluid dynamics

– behaviour of electric, gravitational, and fluid potentials

– The general theory of solutions to Laplace’s equation – potential theory

– In the study of heat conduction, the Laplace equation – the steady-stateheat equation

Page 2: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

150 PDE 7.2 Partial Differential Equations (PDE)

• Maxwell’s equations – electrical and magnetical fields’ relationships– set of four partial differential equations

– describe the properties of the electric and magnetic fields and relate themto their sources, charge density and current density

• Navier-Stokes equations – fluid dynamics (dependencies between pressure,speed of fluid particles and fluid viscosity)

• Equations of linear elasticity – vibrations in elastic materials with given prop-erties and in case of compression and stretching out

• Schrödinger equations – quantum mechanics – how the quantum state of aphysical system changes in time. It is as central to quantum mechanics as New-ton’s laws are to classical mechanics

• Einstein field equations – set of ten equations in Einstein’s theory of generalrelativity – describe the fundamental interaction of gravitation as a result ofspacetime being curved by matter and energy.

Page 3: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

151 PDE 7.3 2nd order PDEs

7.3 2nd order PDEs

We consider now only a single equation caseIn many practical cases, 2nd order PDE-s occur, for example:

• Heat equation: ut = uxx

• Wave equation: utt = uxx

• Laplace’s equation: uxx +uyy = 0.

General second order PDE has the form: (canonical form)

auxx +buxy + cuyy +dux + euy + f u+g = 0.

Assuming not all of a, b and c zero, then depending on discriminant b2−4ac:b2−4ac > 0: hyperbolic equation, typical representative – wave equation;b2−4ac = 0: parabolic equation, typical representative – heat equationb2−4ac < 0: elliptical equation, typical representative – Poisson equation

Page 4: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

152 PDE 7.3 2nd order PDEs

• In case of changing coefficient in time, equations can change their type

• In case of equation systems, each equation can be of different type

• Of course, problem can be non-linear or higher order as well

In general,

• Hyperbolic PDE-s describe time-dependent conservative physical processes likewave propagation

• Parabolic PDE-s describe time-dependent dissipative (or scattering) physicalprocesses like diffusion, which move towards some fixed-point

• Elliptic PDE-s describe systems that have reached a fixed-point and are there-fore independent of time

Page 5: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

153 PDE 7.4 Time-independent PDE-s

7.4 Time-independent PDE-s

7.4.1 Finite Difference Method (FDM)

• Discrete mesh in solving region

• Derivatives replaced with approximation by finite differences

Example. Conside Poisson equation in 2D:

−uxx−uyy = f , 0≤ x≤ 1, 0≤ y≤ 1, (19)

• boundary values as on the figure on the left:

Page 6: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

154 PDE 7.4 Time-independent PDE-s

0

y

x

0

1

0

y

x

0

1

0

• Define discrete nodes as on the figure on right

• Inner nodes, where computations are carried out are defined with

(xi,y j) = (ih, jh), i, j = 1, ...,n

– (in our case n = 2 and h = 1/(n+1) = 1/3)

Page 7: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

155 PDE 7.4 Time-independent PDE-s

Consider here that f = 0.Replacing 2nd order derivatives with standard 2nd order differences in mid-points,

we get

ui+1, j−2ui, j +ui−1, j

h2 +ui, j+1−2ui, j +ui, j−1

h2 = 0, i, j = 1, ...,n,

where ui, j is approximation of the real solution u = u(xi,y j) in point (xi,y j), and in-cludes one boundary value, if i or j is 0 or n+1. As a result we get:

4u1,1−u0,1−u2,1−u1,0−u1,2 = 0

4u2,1−u1,1−u3,1−u2,0−u2,2 = 0

4u1,2−u0,2−u2,2−u1,2−u1,3 = 0

4u2,2−u1,2−u3,2−u2,2−u2,3 = 0.

In matrix form:

Page 8: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

156 PDE 7.4 Time-independent PDE-s

Ax =

4 −1 −1 0−1 4 0 −1−1 0 4 −10 −1 −1 4

u1,1

u2,1

u1,2

u2,2

=

u0,1 +u1,0

u3,1 +u2,0

u0,2 +u1,3

u3,2 +u2,3

=

0011

= b.

This positively defined system can be solved directly with Cholesky factorisation(Gauss elimination for symmetric matrix, where factorisation A = LT L is found) oriteratively. Exact solution of the problem is:

x =

u1,1

u2,1

u1,2

u2,2

=

0.1250.1250.3750.375

.

Page 9: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

157 PDE 7.4 Time-independent PDE-s

In general case n2×n2 Laplace’s matrix has form:

A =

B −I 0 · · · 0

−I B −I . . . ...

0 −I B . . . 0... . . . . . . . . . −I0 · · · 0 −I B

, (20)

where n×n matrix B is of form:

B =

4 −1 0 · · · 0

−1 4 −1 . . . ...

0 −1 4 . . . 0... . . . . . . . . . −10 · · · 0 −1 4

.

It means that most of the elements of matrix A are zero How are such matrices called?– it is a sparse matrix

Page 10: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

158 PDE 7.4 Time-independent PDE-s

7.4.2 Finite element Method (FEM)

Consider as an example Poisson equation

−∆u(x) = f (x), ∀x ∈Ω,

u(x) = g(x), ∀x ∈ Γ,

where Laplacian ∆ is defined by

(∆u)(x) = (∂ 2u∂x2 +

∂ 2u∂y2 )(x), x =

(xy

)

In Finite element Method the region is divided into finite elements.

Page 11: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

159 PDE 7.4 Time-independent PDE-s

A region divided into Finite Elements:

Page 12: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

160 PDE 7.4 Time-independent PDE-s

Consider unit square.

y

x

The problem in Variational Formulation: Find uh ∈Vh such that

a(uh,v) = ( f ,v), ∀v ∈Vh (21)

where in case of Poisson equation

a(u,v) =∫

Ω

∇u ·∇vdx

Page 13: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

161 PDE 7.4 Time-independent PDE-s

and (u,v) =∫

Ωu(x)v(x)dx. The gradient ∇ of a scalar function f (x,y) is defined by:

∇ f =(

∂ f∂x

,∂ f∂y

)

• In FEM the equation (21) needs to be satisfied on a set of testfunctions ϕi =

ϕi(x),

– which are defined such that

ϕi =

1, x = xi

0 x = x j j 6= i

• and it is demanded that (21) is satisfied with each ϕi (i = 1, ...,N) .

• As a result, a matrix of the linear equations is obtained

Page 14: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

162 PDE 7.5 Sparse matrix storage schemes

• The matrix is identical with the matrix from (20) !

• Benefits of FEM over finite difference schemes

– more flexible in choosing discretization

– existence of thorough mathematical constructs for proof of convergenceand error estimates

7.5 Sparse matrix storage schemes

• As we saw, different discretisation schemes give systems with similar matrixstructures

• (In addition to FDM and FEM often also some other discretisation schemes areused like Finite Volume Method (but we do not consider it here))

• In each case, the result is a system of linear equations with sparse matrix.

How to store sparse matrices?How to store sparse matrices?

Page 15: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

163 PDE 7.5 Sparse matrix storage schemes

7.5.1 Triple storage format

• n×m matrix A each nonzero with 3 values: integers i and j and (in most appli-cations) real matrix element ai j. =⇒ three arrays:

indi(1:nz), indj(1:nz), vals(1:nz)of length nz, – number of matrix A nonzeroes

Advantages of the scheme:

• Easy to refer to a particular element

• Freedom to choose the order of the elements

Disadvantages :

• Nontrivial to find, for example, all nonzeroes of a particular row or column andtheir positions

Page 16: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

164 PDE 7.5 Sparse matrix storage schemes

7.5.2 Column-major storage format

For each matrix A column k a vector row_ind(j) – giving row numbers i forwhich ai j 6= 0.

• To store the whole matrix, each column nonzeros

– added into a 1-dimensonal array row_ind(1:nz)

– introduce cptr(1:M) referring to column starts of each column inrow_ind.

row_ind(1:nz), cptr(1:M), vals(1:nz)

Advantages:

• Easy to find matrix column nonzeroes together with their positions

Page 17: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

165 PDE 7.5 Sparse matrix storage schemes

Disadvantages:

• Algorithms become more difficult to read

• Difficult to find nonzeroes in a particular row

7.5.3 Row-major storage format

For each matrix A row k a vector col_ind(i) giving column numbers j forwhich ai j 6= 0.

• To store the whole matrix, each row nonzeros

– added into a 1-dimensonal array col_ind(1:nz)

– introduce rptr(1:N) referring to row starts of each row in col_ind.

col_ind(1:nz), rptr(1:N), vals(1:nz)

Page 18: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

166 PDE 7.5 Sparse matrix storage schemes

Advantages:

• Easy to find matrix row nonzeroes together with their positions

Disadvantages:

• Algorithms become more difficult to read.

• Difficult to find nonzeroes in a particular column.

7.5.4 Combined schemes

Triple format enhanced with cols(1:nz), cptr(1:M), rows(1:nz),

rptr(1:N). Here cols and rows refer to corresponding matrix A values intriple format. E.g., to access row-major type stuctures, one has to index throughrows(1:nz)

Advantages:

Page 19: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

167 PDE 7.5 Sparse matrix storage schemes

• All operations easy to perform

Disadvantages:

• More memory needed.

• Reference through indexing in all cases

Page 20: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

168 Iterative methods 8.1 Problem setup

8 Iterative methods

8.1 Problem setup

Itereative methods for solving systems of linear equations withsparse matrices

Consider system of linear equations

Ax = b, (22)

where N×N matrix A

• is sparse,

– number of elements for which Ai j 6= 0 is O(N).

• Typical example: Poisson equation discretisation on n×n mesh, (N = n×n)

– in average 5, nonzeros per A row

Page 21: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

169 Iterative methods 8.1 Problem setup

In case of direct methods, like LU-factorisation

• memory consumption (together with fill-in): O(N2) = O(n4).

• flops: 2/3 ·N3 +O(N2) = O(n6).

Banded matrix LU-decomposition

• memory consumption (together with fill-in): O(N · L) = O(n3), where L isbandwidth

• flops: 2/3 ·N ·L2 +O(N ·L) = O(n4).

Page 22: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

170 Iterative methods 8.2 Jacobi Method

8.2 Jacobi Method• Iterative method for solving (22)

• With given initial approximation x(0), approximate solution x(k), k = 1,2,3, ...of (22) real solution x are calculated as follows:

– i-th component of x(k+1), x(k+1)i is obtained by taking from (22) only the

i-th row:Ai,1x1 + · · ·+Ai,ixi + · · ·+Ai,NxN = bi

– solving this with respect to xi, an iterative scheme is obtained:

x(k+1)i =

1Ai,i

(bi−∑

j 6=iAi, jx

(k)j

)(23)

Page 23: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

171 Iterative methods 8.2 Jacobi Method

The calculations are in essence parallel with respect to i – no dependence onother componens x(k+1)

j , j 6= i. Iteration stop criteria can be taken, for example:∥∥∥x(k+1)−x(k)∥∥∥< ε or k+1≥ kmax, (24)

– ε – given error tolerance

– kmax – maximal number of iterations

• memory consumption (no fill-in):

– NA6=0 – number of nonzeroes of matrix A

• Number of iterations to reduce∥∥∥x(k)−x

∥∥∥2< ε

∥∥∥x(0)−x∥∥∥

2:

#IT≥ 2lnε−1

π2 (n+1)2 = O(n2)

Page 24: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

172 Iterative methods 8.2 Jacobi Method

• flops/iteration ≈ 10 ·N = O(n2), =⇒

#IT · flopsiteration

=Cn4 +O(n3) = O(n4).

coefficent C in front of n4 is:

C ≈ 2lnε−1

π2 ·10≈ 2 · lnε−1

• Is this good or bad?This is not very good at all... We need some better methods, because

– For LU-decomposition (banded matrices) we had C = 2/3

Page 25: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

173 Iterative methods 8.3 Conjugate Gradient Method (CG)

8.3 Conjugate Gradient Method (CG) C a l c u l a t e r(0) = b−Ax(0) wi th g i v e n s t a r t i n g v e c t o r x(0)

f o r i = 1 , 2 , . . .s o l v e Mz(i−1) = r(i−1) # we assume here t h a t M = I nowρi−1 = r(i−1)T z(i−1)

i f i ==1p(1) = z(0)

e l s eβi−1 = ρi−1/ρi−2

p(i) = z(i−1)+βi−1p(i−1)

e n d i fq(i) = Ap(i) ; αi = ρi−1/p(i)T q(i)

x(i) = x(i−1)+αip(i) ; r(i) = r(i−1)−αiq(i)

check c o n v e r g e n c e ; c o n t in u e i f neededend

Page 26: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

174 Iterative methods 8.3 Conjugate Gradient Method (CG)

• memory consumption (no fill-in):

NA6=0 +O(N) = O(n2),

where NA6=0 – # nonzeroes ofA

• Number of iterations to achieve∥∥∥x(k)−x

∥∥∥2< ε

∥∥∥x(0)−x∥∥∥

2:

#IT≈ lnε−1

2

√κ(A) = O(n)

• Flops/iteration ≈ 24 ·N = O(n2) , =⇒

#IT · flopsiteration

=Cn3 +O(n2) = O(n3),

Page 27: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

175 Iterative methods 8.3 Conjugate Gradient Method (CG)

where C ≈ 12lnε−1 ·√

κ2(A).

=⇒ C depends on condition number of A! This paves the way for preonditioningtechnique

Page 28: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

176 Iterative methods 8.4 Preconditioning

8.4 PreconditioningIdea:Replace Ax = b with system M−1Ax = M−1b.Apply CG to

Bx = c, (25)

where B = M−1A and c = M−1b.But how to choose M?Preconditioner M = MT to be chosen such that

(i) Problem Mz = r being easy to solve

(ii) Matrix B being better conditioned than A, meaning that κ2(B)< κ2(A)

Page 29: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

177 Iterative methods 8.4 Preconditioning

Then#IT(25) = O(

√κ2(B))< O(

√κ2(A)) = #IT(22)

butflops

iteration(25) =

flopsiteration

(22)+ (i) >flops

iteration(22)

• =⇒We need to make a compromise!

• (In extreme cases M = I or M = A)

• Preconditioned Conjugate Gradients (PCG) Method

– obtained if to take in previous algorithm M 6= I

Page 30: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

178 Iterative methods 8.5 Preconditioner examples

8.5 Preconditioner examplesDiagonal Scaling (or Jacobi method)

M = diag(A)

(i) flopsIteration = N

(ii) κ2(B) = κ2(A)

⇒ Is this good?no large improvement to be expeted

Page 31: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

179 Iterative methods 8.5 Preconditioner examples

Incomplete LU-factorisation

M = LU ,

• L and U – approximations to actual factors L and U in LU-decompoition

– nonzeroes in Li j and Ui j only where Ai j 6= 0 (i.e. fill-in is ignored in LU-factorisation algorithm)

(i) flopsIiteration = O(N)

(ii) κ2(B)< κ2(A)

How good is this preconditioner?Some improvement at least expected!

κ2(B) = O(n2)

Page 32: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

180 Iterative methods 8.5 Preconditioner examples

Gauss-Seidel method do k=1,2,...

do i=1,...,n

x(k+1)i =

1Ai,i

(bi−

i−1

∑j=1

Ai jx(k+1)j −

n

∑j=i+1

Ai, jx(k)j

)(26)

enddo

enddoNote that in real implementation, the method is done like:

do k=1,2,...

do i=1,...,n

xi =1

Ai,i

(bi−∑

j 6=iAi jx j

)(27)

enddo

enddoDo you see a problem with this preconditioner (with PCG method)?But the preconditioner is not symmetric, which makes CG not to converge!

Page 33: 7.2 Partial Differential Equations (PDE) PDE overview · 149PDE7.2 Partial Differential Equations ... 7.2 Partial Differential Equations (PDE) PDE overview ... General second order

181 Iterative methods 8.5 Preconditioner examples

Symmetric Gauss-Seidel methodTo get the symmetric preconditioner, another step is added:

do k=1,2,...

do i=1,...,n

xi =1

Ai,i

(bi−∑ j 6=i Ai jx j

)enddo

enddo

do k=1,2,...

do i=n,...,1

xi =1

Ai,i

(bi−∑ j 6=i Ai jx j

)enddo

enddo