Lecture 3 - Mathematicsmath.mit.edu/~stoopn/18.086/Lecture3-4.pdf · Lecture 3 18.086. Info ... Figure 6.3: Stability regions IGI 5 1 for popular methods (stabi1ity.m on the cse site)

Lecture 318.086

Info• PSet 1 will be posted this afternoon/evening (due in 2 weeks)

• Until Feb. 22: Decide for a computational project (50% grade) and submit proposal to me by email (content: see next slide)

• March 28: Submit short (max 1page) written mid-term project report

• Until Friday, May 6: Submit written final project report

• May 5, 10, 12: Presentation (in class): 12 minutes + 3 minutes questions/discussion

• Content of project: Free choice, as long as related to course contents/computational aspect(can also go beyond!). No recycling of existing/old projects!Implementation/testing/comparison of methods is ok

Term project• When submitting your project proposal (Feb. 22), use this template:

• Project title

• Project background: Does it relate to your work in another field (e.g. your thesis)? If yes, briefly outline the questions and goals of your work in the other field.

• Questions and Goals: Briefly describe the questions you wish to investigate in your project. What are your expectations?

• Plan: Which language do you plan to program in? Do you intend to use special software? Does your project work relate to the work of other people at MIT?

Plan for today• Runge-Kutta time integration

• Basic time integration algorithm

• 1-way wave equation

FD schemes for ODEs• Equation: u’ = f(u) with u(0) given

• First order accuracy (p=1): Euler methods

• Second order (p=2): global error O(Δt2), local error O(Δt3):

Trapezoidal implicit

Adams-Bashford explicit

Backward differences explicit

Runge-Kutta 2 explicit

Un+1 � Un

�t=

1

2[f(Un+1, tn+1) + f(Un, tn)]

Un+1 � Un

�t=

3

2f(Un, tn)�

1

2f(Un�1, tn�1)

Un+1 � Un

�t=

1

2[f(Un, tn) + f(Un +�tf(Un, tn), tn+1)]

Lecture: Stability & Accuracy

6.2 Finite Difference Methods 467

3un+1 - 4 u n + un-1 Backward differences/BDF2 2At = f (Un+l, tn+l) . (19)

What about stability? The trapezoidal method (15) is stable even for stiff equations, when a is very negative: l - $a At (left side) will be larger than 1 + l a At (right side). (19) is even more stable and accurate. The Adams method (17) will be stable if At is small enough, but there is always a limit on At for explicit systems.

Here is a quick way to find the stability limit -a& 5 C in (17) when a is real. The limit occurs when the growth factor is exactly G = -1. Set Un+l = - 1 and Un = 1 and Un-l = - 1 i n (17 ) . Solve for a when f (u , t ) = au:

-2 3 Stability limit in (17) - - - 1 - a + - a gives a n t = -1. So C = 1 . (20)

a t 2 2

We now have three second-order methods (15)-(17)-(19), all definitely useful. The reader might suggest including both Un-l and fn-1 to increase the accuracy to third order. Sadly, this method is violently unstable (Problem 5). We may extend (19) by older values of U in backward differences, or extend (17) by older f (U). But including both U and f (U) for extreme accuracy produces instability for all At.

Multistep Methods: Explicit and Implicit

By using p earlier values of U , the accuracy can be increased to order p. Backward Euler has p = 1, and BDF2 in (19) has p = 2. Each VU is U(t) - U(t - At):

1 1 Backward differences (V + Z ~ 2 + . . + -VP) Un+1 = At f (Un+1, tn+l) . (21) P

MATLAB's stiff code odel5s varies from p = 1 to p = 5 depending on the local error. The alternative is to use older values of f (U, t) instead of U. Explicit comes first:

Adams- Bashforth Un+1 - Un = At(b1 fn + . . . + bp fn-,+I) (22) The table shows the numbers b up to p = 4, starting with Euler for p = 1.

The fourth-order method is often a good choice, although astronomers go above p = 8.

order of accuracy p = l p = 2 p = 3 p = 4

b l b2 b3 b4

1 312 -112

23/12 -16112 5/12 55/24 -59124 37/24 -9124

limit on -aAt for stability

2 1

611 1 3/10

constant c in error DE

5/ 12 112

318 2511720

FD schemes for ODEs• Runge-Kutta is actually a family of methods!

• The most famous is Runge-Kutta-4 (RK4), which has p=4:

RK4 explicit

(ode45 in matlab) with

k1 =1

2f(Un, tn)

k2 =1

2f(Un +�tk1, tn+1/2)

k3 =1

2f(Un +�tk2, tn+1/2)

k4 =1

2f(Un + 2�tk3, tn+1)

Un+1 � Un

�t=

1

3[k1 + 2k2 + 2k3 + k4]

Stability diagrams• So far, we obtained stability conditions such as (RK2):

• It makes sense (see lecture) to consider complex z, and draw the stability region in the complex plane

|1 + z +1

2z2| 1


Adams-Bashforth

5ii Adams-Moulton

Backward Differences Runae-Kutta

Figure 6.3: Stability regions IGI 5 1 for popular methods (stabi1ity.m on the cse site). Explicit methods are stable for a At inside the curves; implicit methods outside.

Runge-Kutta Methods

If evaluations of f (u, t ) are not too expensive, Runge-Kutta methods are highly competitive (and self-starting). Runge-Kutta of orders 4 and 5 is the basis for ode45. These are compound one-step methods, using Euler's Un + At fn inside f :

Simplified Runge- Kutta un+l-un 1 At = -[fn + f (Un + Atfn , tn+~)] . 2 (25)

You see the compounding of f . For u ' = au the growth factor G captures (At)2:

Comparing with the exact growth ea At, this confirms second-order accuracy. Stability hits a limit at aAt = -2 where G = 1. Now let a At = r be complex:

Stability limit for RK2 1 [GI = Il+x+-x21 = 1 for z = a + i b 2

The stability limit is a closed curve in the complex plane through x = a At = -2. Figure 6.3 shows all the numbers x (eigenvalues in the matrix case) at which IGI 5 1.


Adams-Bashforth

5ii Adams-Moulton

Backward Differences Runae-Kutta

Figure 6.3: Stability regions IGI 5 1 for popular methods (stabi1ity.m on the cse site). Explicit methods are stable for a At inside the curves; implicit methods outside.

Runge-Kutta Methods

If evaluations of f (u, t ) are not too expensive, Runge-Kutta methods are highly competitive (and self-starting). Runge-Kutta of orders 4 and 5 is the basis for ode45. These are compound one-step methods, using Euler's Un + At fn inside f :

Simplified Runge- Kutta un+l-un 1 At = -[fn + f (Un + Atfn , tn+~)] . 2 (25)

You see the compounding of f . For u ' = au the growth factor G captures (At)2:

Comparing with the exact growth ea At, this confirms second-order accuracy. Stability hits a limit at aAt = -2 where G = 1. Now let a At = r be complex:

Stability limit for RK2 1 [GI = Il+x+-x21 = 1 for z = a + i b 2

The stability limit is a closed curve in the complex plane through x = a At = -2. Figure 6.3 shows all the numbers x (eigenvalues in the matrix case) at which IGI 5 1.

(z = a Δt for u’=au)

Basic ODE time integration• Given: RHS f(u,t), initial conditions u(0)

• Solve u’(t) = f(u,t):

Nsteps=1000; %Perform 1000 integration steps dt = 0.1;u = zeros(Nsteps+1); % Preallocate space for solution u % Set initial conditions: u(1)=u0; t = 0;

% Forward Euler: for i=1:1000

u(i+1) = u(i) + dt*f(u(i),t); t=t+dt;

end% Plot solution u etc.

If you use integrator that requires U from past: Start doing a single Forward Euler step to get the first 2 values of U, then use your integrator of choice!

1-way wave equation

• So far we only considered ODE’s: u’ = f(u,t) and we only considered methods for the time integration

• Now: PDEs! Simplest example: 1-way wave equation:

• We need to use “good” time integration and good approximation of spatial derivatives.

ut

= cux

1-way wave equation• Properties of :u

t

= cux

• Conservative equation (modes neither grow nor shrink)

lecture

• c is the speed by which the wave travels

• wave does not change its form

• Solution to initial condition : u(x, 0) = e

ikx

u(x, t) = e

ik(x+ct)

lecture

Spatial finite difference (FD) schemes• Common FD schemes for 1D ux:

u(x)

x

Forward diff, O(Δx):

Backw. diff, O(Δx):i

Central diff, O(Δx2):

Lecture: accuracyBut this is only spatial accuracy! How does it affect overall accuracy?

FD for the 1-way wave equation

Upwind/forward

differences

Lax-Wendroff

Lax-Friedrich

• Now: How to do time and space discretization in optimal way?

U(x, t+�t)� U(x, t)

�t

= c

U(x+�x, t)� U(x, t)

�x

U(x, t+�t)� U(x, t)

�t

= c

U(x+�x, t)� U(x��x, t)

2�x

+�t

2c

2U(x+�x, t)� 2U(x, t) + U(x��x, t)

�x

2

U(x, t+�t)� 12 (U(x+�x, t) + U(x��x, t))

�t

= c

U(x+�x, t)� U(x��x, t)

2�x

• Multiplying by △t gives various terms on RHS with factor r = c

�t

�x

CFL condition for stability: r≤1 (lecture)

until here

say next time why r<1 is stability criterium

Courant-Friedrichs-Lewy condition

• The CFL condition is necessary but not sufficient for stability! r = c

�t

�x

1

• Sufficient criteria for stability: Neumann stability analysis

• Idea: Remember, wave equation has solution in terms of exponentials: => We found |G|=1 analytically, but how does this look for discrete approximations? => Will tell us stability for each mode k separately

u(x, t) = G(k, t)eikx

Neumann stab. analysis for upwing and Lax-Friedrichs: Lecture

0 r 1• We find: Upwind:Lax-Friedrichs: �1 r 1

Lecture 418.086

Last time

• 1-way wave equation (also called advection equation)

• CFL criterium for stability

• von Neumann stability analysis

• solution in terms of single modes:

r = c

�t

�x

1

ut

= cux

u(x, t) = G(k, t)eikx

• |G|=1 analytically• |G|<1 for stability of numerical scheme

FD for 1-way wave eq: summary

Upwind differences

Lax-Wendroff

Lax-Friedrich

U(x, t+�t)� U(x, t)

�t

= c

U(x+�x, t)� U(x, t)

�x

U(x, t+�t)� U(x, t)

�t

= c

U(x+�x, t)� U(x��x, t)

2�x

+�t

2c

2U(x+�x, t)� 2U(x, t) + U(x��x, t)

�x

2

U(x, t+�t)� 12 (U(x+�x, t) + U(x��x, t))

�t

= c

U(x+�x, t)� U(x��x, t)

2�x

O(Δt)

O(Δt2)

O(Δt) !

0 r 1

�1 r 1

�1 r 1

r = c

�t

�x

Lecture

Notation• We have seen different schemes for 1-way wave equation.

The schemes are often called stencils and depicted like this:

480 Chapter 6 Initial Value Problems

Lax-Friedrichs G matches the next term in the exact growth factor only if r2 = 1:

GLF = cos kAx + ir sin kAx = 1 + irkAx - AX)^ + . - - (22) 1 2 2 Gexact = e = 1 + irk Ax + 2i r (k AX)^ + - - -

In the exceptional cases r = 1 and r = -1, G agrees with Gexact. Staying exactly on the characteristic line, UJ,,+1 matches the true u ( j A x , t + At) . For r2 < 1, Lax- Friedrichs has an important advantage and disadvantage:

U,,,+1 is a positive combination of old values. But accuracy is only first-order.

4. Lax-Wendroff is stable for -1 < r 5 1. The LW difference equation combines U,,, and Uj-l,n and Uj+1,, to compute U,,n+l:

Lax- Wendroff 1 zk Ax 1 -zkAx G = (1 - r2) + -(r2 + r)e + -(r2 - r)e . 2 2 (23)

This is G = 1 - r2 + r2 cos kAx + ir sin kAx. At the dangerous frequency kAx = T ,

the growth factor is 1 - 2r2. That stays above -1 if r2 5 1. Problem 5 shows that IGI < 1 for every kAx. Lax-Wendroff is stable when-

ever the CFL condition r 2 < - 1 is satisfied. The wave can go either way (or both ways) since c can be negative. LW is the most accurate of these five methods.

upwind wrong way centered Lax-Friedrichs Lax-Wendroff stable unstable unstable stable stable i f r L 1 all At all At if Irl 5 1 if Irl 5 1

5. The centered methods of maximum accuracy are stable for -1 < r < 1. Lax-Wendroff uses three values at time level n for accuracy two. For every even p = 2q, there are p+ 1 coefficients a_,, . . . , a, so that the difference equation U,,n+l = C amU,+,,, has accuracy p. Matching G = C ameimkAx with the exact factor e - - eikrAx gives p + 1 equations C amm& rj for the a's.

Using all those values, the CFL condition allows the possibility of stability out to -q 5 c A t lAx 5 q. The actual requirement is -1 5 r 5 1.

Lax-Wendroff

Lax-Friedrich

U(x, t+�t)� U(x, t)

�t

= c

U(x+�x, t)� U(x��x, t)

2�x

+�t

2c

2U(x+�x, t)� 2U(x, t) + U(x��x, t)

�x

2

U(x, t+�t)� 12 (U(x+�x, t) + U(x��x, t))

�t

= c

U(x+�x, t)� U(x��x, t)

2�x

Real-world accuracyUpwind:

R. J. LeVeque — AMath 585–6 Notes 183

0 1 2 3 4 5 6−0.5

0

0.5

1Upwind solution at time 4

0 1 2 3 4 5 6−0.5

0

0.5

1Lax−Wendroff solution at time 4

Figure 13.4: Numerical solution using upwind (diffusive) and Lax-Wendroff (dispersive) methods.

13.6.2 Lax-Wendroff

If the same procedure is followed for the Lax-Wendroff method, we find that all O(k) terms drop out ofthe modified equation, as is expected since this method is second order accurate on the advection equa-tion. The modified equation obtained by retaining the O(k2) term and then replacing time derivativesby spatial derivatives is

vt + avx +16ah2

!1 −

"ak

h

#2$

vxxx = 0. (13.37)

The Lax-Wendroff method produces a third order accurate solution to this equation. This equationhas a very different character from (13.35). The vxxx term leads to dispersive behavior rather thandiffusion. This is clearly seen in Figure 13.4, where the Un

j computed with Lax-Wendroff are comparedto the true solution of the advection equation. The magnitude of the error is smaller than with theupwind method for a given set of k and h, since it is a higher order method, but the dispersive termleads to an oscillating solution and also a shift in the location of the main peak, a phase error.

The group velocity for wave number ξ under Lax-Wendroff is

cg = a − 12ah2

!1 −

"ak

h

#2$ξ2

which is less than a for all wave numbers. (The concept of group velocity is explained in Section 13.7.)As a result the numerical result can be expected to develop a train of oscillations behind the peak, withthe high wave numbers lagging farthest behind the correct location.

If we retain one more term in the modified equation for Lax-Wendroff, we would find that the U nj

are fourth order accurate solutions to an equation of the form

vt + avx +16ah2

!1 −

"ak

h

#2$

vxxx = −ϵvxxxx, (13.38)

Lax-Wendroff:


0 1 2 3 4 5 6−0.5

0

0.5


0 1 2 3 4 5 6−0.5

0

0.5



13.6.2 Lax-Wendroff


vt + avx +16ah2

!1 −

"ak

h

#2$

vxxx = 0. (13.37)




cg = a − 12ah2

!1 −

"ak

h

#2$ξ2




vt + avx +16ah2

!1 −

"ak

h

#2$


Real-world accuracy478 Chapter 6 Initial Value Problems

1 2 -

1 .

08. Upwind 06.

04.

02.

0

Figure 6.6: Three approximations to a sharp signal show smearing and oscillation.

For an ideal difference equation, we only want dissipation very close to the shock. This can avoid oscillation without losing accuracy. A lot of thought has gone into high resolution met hods (Section 6.6), to capture shock waves cleanly.

Stability of the Four Finite Difference Methods

Accuracy requires G to stay close to the true eicbAt. Stability requires G to stay inside the unit circle. If IGI > 1 at frequency k, the solution Gnezkx will blow up.

We now check IGI < 1, in the four methods. CFL was only a necessary condition ! 1. Forward differences in space and time: AU/At = cAU/Ax (upwind). Equation (11) was G = 1 - r + reikAx. If the Courant number is 0 < r 5 1, then 1 - r and r will be positive. The triangle inequality gives IGI < 1:

Stability for 0 5 r < 1 zk Ax - IGl 5 11-rI+Ire I = 1 - r + r = l . (17)

This sufficient condition 0 5 c At/ Ax < 1 agrees with the CFL necessary condition U ( x , nAt) depends on the initial values between x and x + nAx. That domain of dependence must include the point x + c n At. (Otherwise, changing the initial value at the point x + c n A t would change the true solution u but not the approximation U.) Then 0 < c nAt < nAx means that 0 5 r 5 1.

Figure 6.7 shows G in the stable case r = and the unstable case r = % (when At is too large). As k varies, and eAAX goes around a unit circle, the complex number G = 1 - r + reikAx goes in a circle of radius r. The center is 1 - r. Always G = 1 at zero frequency (constant solution, no growth).

2. Forward difference in time, centered difference in space. This combination is never stable ! The shorthand U,,, will stand for U(jAx, nAt):

step-like initial condition:

Can we understand this behavior?

Modified equations

• see lecture notes

Real-world accuracyUpwind:


0 1 2 3 4 5 6−0.5

0

0.5


0 1 2 3 4 5 6−0.5

0

0.5



13.6.2 Lax-Wendroff


vt + avx +16ah2

!1 −

"ak

h

#2$

vxxx = 0. (13.37)




cg = a − 12ah2

!1 −

"ak

h

#2$ξ2




vt + avx +16ah2

!1 −

"ak

h

#2$


Smeared out due to diffusive term in modified equations!

Real-world accuracyLax-Wendroff:


0 1 2 3 4 5 6−0.5

0

0.5


0 1 2 3 4 5 6−0.5

0

0.5



13.6.2 Lax-Wendroff


vt + avx +16ah2

!1 −

"ak

h

#2$

vxxx = 0. (13.37)




cg = a − 12ah2

!1 −

"ak

h

#2$ξ2




vt + avx +16ah2

!1 −

"ak

h

#2$


Dispersion due to term vxxx in modified equations

High freq. modes travel slower (=> on the left of the envelope)

Phase vs. group velocity• Remember from physics:


−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 0

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 0.4

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 0.8

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 1.2

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 1.6

Figure 13.6: The oscillatory wave packet satisfies the dispersive equation ut + aux + buxxx = 0. Alsoshown is a black dot, translating at the phase velocity cp(ξ0) and a Gaussian that is translating at thegroup velocity cg(ξ0).

phase velocity

group velocity

Dispersion in LW schemeR. J. LeVeque — AMath 585–6 Notes 187

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 0

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 0.4

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 0.8

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 1.2

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 1.6



−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 0

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 0.4

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 0.8

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 1.2

−3 −2 −1 0 1 2 3−1

−0.5

0

0.5

1

time = 1.6


Documents

Lecture 3 - Mathematicsmath.mit.edu/~stoopn/18.086/Lecture3-4.pdf · Lecture 3 18.086. Info ... Figure 6.3: Stability regions IGI 5 1 for popular methods (stabi1ity.m on the cse site)