MTMW12:Introduction to Numerical Modelling

Dr Hilary Weller

September - December 2021

• The first few weeks will be about calculations in Python.• Then numerical analysis and implementing and testing numericalmethods for atmosphere and ocean models.• Please ask questions, comment and give feedback on “Questions anddiscussion” on Blackboard.• There are students online• This is being recorded• Fill in the notes with gaps• Bring a device

Week Lecture content Practicals Hand out/in1/5 Integration, Integration, 28 Sept

roots and Python roots and Python No deadline2/6 More Python More 5 Oct

including arrays programming No deadline3/7 Taylor series Differentiation 12 Oct

and interpolation Deadline 19 OctPeer review 26 Oct (5%)

4/8 Navier Stokes Assignment 4 19 Oct (20%)and diffusion Diffusion Deadline 2 Nov

5/9 Fourier series Assignment 4and numerical analysis Diffusion

6/10 Week 6 (1-5 Nov)7/11 Linear advection Assignment 5 9 Nov

and numerical analysis Advection8/12 Further Assignment 5

advection Advection9/13 Drop in session No practical Code deadline

23 Nov (5%)10/14 Drop in session No practical Report deadline29 Nov 10 Dec (30%)11/15 Test 40%2

Further Resources (Not essential reading)Python• “A Hands-On Introduction to Using Python in the Atmospheric and Oceanic

Sciences” by Johnny Wei-Bing Lin. http://www.johnny-lin.com/pyintro.• “Think Python. How to Think Like a Computer Scientist” by Allen B. Downey .http://www.greenteapress.com/thinkpython/thinkpython.html

• “Numerical Methods in Engineering with Python 3” by Jaan Kiusalaashttps://doi.org/10.1017/CBO9781139523899

• http://www.python.org• http://docs.python.org/tutorial/• http://matplotlib.sourceforge.net/users/index.html• http://learnpythonthehardway.org/• http://www.ibiblio.org/g2swap/byteofpython/read/• https://www.linkedin.com/learning/• “Making Use of Python” by Rashi Gupta• “Python Essential Reference” by David M. Beazley (Addison Wesley)• “Learning Python” Mark Lutz (O’Reilly Media)• Free courses in Python and other languages https://www.codecademy.com/

3

Numerical Methods• Weller, H. Introduction to Numerical Methods for Weather and Climate Models.

Draft of a chapter for a book.http://www.met.reading.ac.uk/~sws02hs/teaching/PDEsNumerics/WellerPrimer.pdf

• Wikipedia http://en.wikipedia.org• Durran, D. R. Numerical methods for fluid dynamics (Springer).• LeVeque, R. Numerical Methods for Conservation Laws (Springer)• Ortega, J.M. and Poole, W.G. An Introduction to Numerical Methods for Differential

Equations. 1981 (Pitman Publishing Inc)• Ferziger, J. H. and Peric, M. Computational Methods for Fluid Dynamics (Springer).• Numerical Recipes: The Art of Scientific Computing, Third Edition (2007),

(Cambridge University Press). http://www.nr.com/fortranOther People’s lecture notes for similar courses• https://people.maths.ox.ac.uk/trefethen/pdetext.htmlDiscussion siteIf you have questions about the module or assignments outside lesson times, you canask them on the discussion site for this module on Blackboard. You are encouraged tolook there for answers first and answer questions if you can.

4

Contents

1 Numerical Methods in Python 141.1 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.1.1 1-point Gaussian quadrature . . . . . . . . . . . . . . . . . . . . 151.1.2 1-point Gaussian quadrature in Python3 . . . . . . . . . . . . . 16

1.2 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171.2.1 Doc-strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.3 Roots of Non-linear Equations . . . . . . . . . . . . . . . . . . . . . . . 201.3.1 The Bisection Method . . . . . . . . . . . . . . . . . . . . . . . 211.3.2 The Bisection Method in Python . . . . . . . . . . . . . . . . . 22

1.4 Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231.4.1 Working with Real Numbers . . . . . . . . . . . . . . . . . . . . 241.4.2 Representation of Real Numbers on Computers . . . . . . . . . . 26

1.4.2.1 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . 271.4.3 Summing small and large numbers . . . . . . . . . . . . . . . . 281.4.4 Summing lots of numbers . . . . . . . . . . . . . . . . . . . . . 28

1.4.4.1 Binary Conversion . . . . . . . . . . . . . . . . . . . . 285

1.5 Indentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291.5.1 Tabs or spaces for indentation - very important to read this . . . 29

1.6 Manually Wrap Long Lines . . . . . . . . . . . . . . . . . . . . . . . . 291.7 Global and Local Variables . . . . . . . . . . . . . . . . . . . . . . . . . 291.8 Order of Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301.9 Help with Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311.10 Python Functions and Packages . . . . . . . . . . . . . . . . . . . . . . 311.11 Python Arrays and Lists . . . . . . . . . . . . . . . . . . . . . . . . . . 33

1.11.1 Copying lists and arrays . . . . . . . . . . . . . . . . . . . . . . 341.11.2 Range Commands . . . . . . . . . . . . . . . . . . . . . . . . . 35

1.12 Python Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361.13 Plotting Graphs using the Python library matplotlib . . . . . . . . . . . 37

1.13.1 A first example . . . . . . . . . . . . . . . . . . . . . . . . . . . 371.13.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2 Better Software Practices 392.1 Some Good Programming Practices . . . . . . . . . . . . . . . . . . . . 392.2 Backup and Version Control . . . . . . . . . . . . . . . . . . . . . . . . 42

6

2.2.1 Instructions for using Gitlab . . . . . . . . . . . . . . . . . . . . 432.3 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.3.1 Making Assertions . . . . . . . . . . . . . . . . . . . . . . . . . 452.3.2 Raising Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.3.2.1 Value and Type Errors . . . . . . . . . . . . . . . . . . 472.4 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3 Taylor Series and Finite Differences 493.1 The Taylor Series: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.1.1 Quiz (select all that apply) https://forms.office.com/r/

DBBCSp1rcy . . . . . . . . . . . . . . . . . . . . . 543.2 Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . 563.3 Taylor Series to find Finite Difference Gradients . . . . . . . . . . . . . . 57

3.3.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573.3.2 Exercises (answers at the end) . . . . . . . . . . . . . . . . . . . 59

7

3.4 Order of Accuracy of Numerical Solutions . . . . . . . . . . . . . . . . . 603.5 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.5.1 Taylor Series to Find Finite Difference Formulae for Interpolation 633.5.2 Exercises (answers on Blackboard) . . . . . . . . . . . . . . . . 64

4 The Navier Stokes (NS) Equations 654.1 The Potential Temperature Equation . . . . . . . . . . . . . . . . . . . 664.2 Advection of Pollution . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

4.2.1 Pure Linear Advection . . . . . . . . . . . . . . . . . . . . . . . 674.2.2 Advection/Diffusion with Sources and Sinks . . . . . . . . . . . 68

4.3 The Momentum Equation . . . . . . . . . . . . . . . . . . . . . . . . . 694.3.1 Coriolis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694.3.2 The Pressure Gradient Force . . . . . . . . . . . . . . . . . . . . 704.3.3 Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764.3.4 The Complete Navier Stokes Equations . . . . . . . . . . . . . . 77

4.4 Boundary Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.4.1 Fixed Value (Dirichlet) . . . . . . . . . . . . . . . . . . . . . . . 784.4.2 Fixed Gradient (Neumann) . . . . . . . . . . . . . . . . . . . . 78

8

4.4.3 Periodic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.5 Linearity of Partial Differential Equations (PDEs) . . . . . . . . . . . . . 79

5 Solution of the Diffusion Equation 805.1 Numerical Solution using Finite Differences . . . . . . . . . . . . . . . . 815.2 Implicit and Explicit Schemes . . . . . . . . . . . . . . . . . . . . . . . 845.3 Analytic Solution of the Diffusion Equation (no need to memorise) . . . 85

5.3.1 Solution for step-function initial conditions (optional) . . . . . . 865.3.2 Superposition of Solutions . . . . . . . . . . . . . . . . . . . . . 86

6 Fourier Analysis 886.1 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896.2 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 926.3 Discrete Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . 936.4 Differentiation and Interpolation . . . . . . . . . . . . . . . . . . . . . . 956.5 Spectral Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 966.6 Wave Power and Frequency . . . . . . . . . . . . . . . . . . . . . . . . 966.7 Analysing Power Spectra . . . . . . . . . . . . . . . . . . . . . . . . . . 97

9

7 Numerical Analysis 1007.1 What is Numerical Analysis and why is it done . . . . . . . . . . . . . . 1017.2 Some Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1027.3 Convergence, errors and order of accuracy . . . . . . . . . . . . . . . . 1047.4 Lax-Equivalence Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 1057.5 Von-Neumann Stability Analysis . . . . . . . . . . . . . . . . . . . . . . 1067.6 Von-Neumann Stability Analysis of FTCS . . . . . . . . . . . . . . . . . 107

8 Linear Advection 1098.1 Eulerian Finite Difference Advection Schemes . . . . . . . . . . . . . . . 111

8.1.1 Forward in Time, Backward in Space (FTBS) . . . . . . . . . . . 1118.1.2 Accuracy of FTBS . . . . . . . . . . . . . . . . . . . . . . . . . 112

8.2 Centred in Time, Centred in Space (CTCS) and FTCS . . . . . . . . . . 1138.2.1 Accuracy of CTCS . . . . . . . . . . . . . . . . . . . . . . . . . 114

8.3 Some Explicit Eulerian Advection Schemes . . . . . . . . . . . . . . . . 115

9 Numerical Analysis of Advection Schemes 1169.1 Domain of Dependence . . . . . . . . . . . . . . . . . . . . . . . . . . 118

10

9.1.1 Domain of Dependence of a PDE . . . . . . . . . . . . . . . . . 1199.1.2 Domain of Dependence of a Numerical Method . . . . . . . . . . 1209.1.3 Courant-Friedrichs-Lewy (CFL) criterion: . . . . . . . . . . . . . 1219.1.4 The Domain of Dependence of the CTCS Scheme . . . . . . . . 122

9.2 Von-Neumann Stability Analysis of Advection Schemes . . . . . . . . . . 1239.2.1 Von-Neumann Stability Analysis of FTBS . . . . . . . . . . . . . 1239.2.2 What should A be for real linear advection? . . . . . . . . . . . . 1259.2.3 Von Neumann Stability Analysis of CTCS . . . . . . . . . . . . . 126

9.3 Conservation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1289.3.1 Conservation of higher moments . . . . . . . . . . . . . . . . . 129

9.4 Exercises: Analysis of Advection Schemes (answers on blackboard) . . . 130

10 More Advection Schemes 13110.1 Artificial Diffusion to Remove Spurious Oscillations . . . . . . . . . . . . 13210.2 Semi-Lagrangian Advection . . . . . . . . . . . . . . . . . . . . . . . . 13310.3 Implicit Time Stepping for Advection . . . . . . . . . . . . . . . . . . . 13510.4 The Finite Volume Method . . . . . . . . . . . . . . . . . . . . . . . . 136

10.4.1 The Finite Volume Method in one dimension, for constant u . . . 13711

10.5 Lax-Wendroff and Warming and Beam . . . . . . . . . . . . . . . . . . 13810.6 Total Variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14110.7 Total Variation Diminishing (TVD) schemes . . . . . . . . . . . . . . . 142

10.7.1 Limiter functions . . . . . . . . . . . . . . . . . . . . . . . . . . 143

11 Waves, Dispersion and Dispersion Errors 14411.1 Background on Waves . . . . . . . . . . . . . . . . . . . . . . . . . . . 14511.2 Dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14711.3 Some Animations and Videos which demonstrate Dispersion . . . . . . . 15211.4 Phase speed of linear advection . . . . . . . . . . . . . . . . . . . . . . 15311.5 Phase speed and dispersion errors of CTCS . . . . . . . . . . . . . . . . 15411.6 What does k∆x mean? . . . . . . . . . . . . . . . . . . . . . . . . . . . 15611.7 What dispersion relations mean for numerical methods . . . . . . . . . . 157

12

Numerical Modelling of the Atmosphere and OceanWeather, ocean and climate forecasts predict the winds, temperature and pressurefrom numerical solution of the Navier-Stokes equations:https://www.windy.comAdditional equations predict the transport of moisture and pollutants in theatmosphere from the winds:Pullution Plumes on YoutubeAll of these simulations involve numerical integration. We will start here ...Multiple choice answers here https://forms.office.com/r/DBBCSp1rcy

13

14

Chapter 1: Numerical Methods in Python1.1 Numerical IntegrationIntegration cannot always be done exactly because:• Not all functions can be integrated analytically• An analytic function is not known, just the values at some points.There are a number of methods to integrate a function numerically. Eg:• The trapezium rule• Simpson’s rule• n-point Gaussian quadrature

1.1.1 1-point Gaussian quadratureTo integrate function f (x) approximatelybetween x = a and x = b. Divide theinterval into N parts of length∆x = (b−a)/N. Then

∫ b

af (x)dx≈

a) (b−a) f(a+b

2

)

b) (b−a) 12 ( f (a)+ f (b))

c) ∑Ni=1 ∆x f

(a+ i∆x

)

d) ∆x∑N−1i=0 f

(a+(i+ 1

2)∆x)X

a bx

0.0

0.2

0.4

0.6

0.8

1.0

f

1-point quadrature has 1evaluation of f in each interval.

15

1.1.2 1-point Gaussian quadrature in Python3# Calculate the approximate integral of sin (x) between a and b using# 1−point Gaussiaun quadrature using N intervals

import numpy as np # Import numerical l ib rary

# Set integration l imits and number of interva lsa = 0b = np. piN = 20# from these calculate the interval lengthdx = (b − a)/N# I n i t i a l i s e the integralS = 0.0

# Sum contribution from each interval (end of loop at end of indentation)for i in range(0 ,N): # note : ":" before indented region

S += np. sin (a + ( i +0.5)∗dx) # note : i takes values 0 then 1

S ∗= dx # same as S = S∗dx

exact =−np. cos(b) + np. cos(a) # the exact values for comparisonprint ( ’Gaussian quadrature integral with ’ , N, ’ interva ls i s ’ , S, \

’\nExact integral = ’ , exact , ’ , error = ’ , S − exact)

Exercise: Work through the above code in order to calculate S using N=2. (Give theanswer in terms of π and square roots.)a) π√

2/2≈ 2.22 X, b) π√

2/4≈ 1.11, c) 0.0861 , d) 0.1937You should now be able to complete assignment 1 question 1a,b

16

1.2 FunctionsYou can use built in functions:x = 3.5y = 3.5 + xx = max(x , y)

What is x? a) 7X b) 3.5 c) 10.5 d) 0You can use functions from libraries like numpy:import numpy as npx = np. exp(0)

What is x? a) 0 b) 1X c) 2.718 d) -1You can define your own functions. Note the colon and the indentation.def closerToZero(a ,b) :

"""Return the argument that i s closer to zero"""i f abs(a) < abs(b) :

return aelse :

return b

z = closerToZero(1.1,−1.1)

What is z? a) -1.1 X, b) 1.1, c) 0

17

Functions can change their arguments but only inside the function:def maxAbs(a ,b) :

"""Make both arguments non−negative and return the bigger one"""a = abs(a)b = abs(b)return max(a ,b)

x = 1y = −2z = maxAbs(x , y)print (x , y , z)

What gets printed? a) 1 2 2, b) 1 2 -2, c) 1 -2 -2, d) 1 -2 2 XFunctions can return zero, one or many thingsdef sayHello () :

"""Say hel lo to the ca l l e r """print ("Hello")

def sayHelloTo(name):"""Say hel lo to the name"""print ("Hello " , name)

def swap(a ,b) :"""Swap the order of a and b in the return"""return b,a

[x , y ] = swap(1 ,2)

What are x and y? a) 2 and 1X, b) 1 and 2

18

Functions can be used to avoid global variables:def main() :

"""Put a scr ipt inside a main function"""[x , y ] = swap(closerToZero(1 ,2) , maxAbs(3 ,4))print ( ’x = ’ , x , ’ y = ’ , y)

main()

What gets printed? a) x = 4, y = 1Xb) x = 1, y = 4

c) x = 3, y = 1d) x = 3, y = 4

1.2.1 Doc-stringsDoc-strings are the text inside quotes after the def line of a function. They explain1. What the function does2. What the arguments are3. What is returnedFunctions should ALWAYS have doc-strings!If you typehelp(functionName)

python will print out the doc-string.

19

1.3 Roots of Non-linear Equations• The Navier Stokes equations are non-linear and so difficult to solve numerically.• Many much simpler non-linear equations cannot be solved analytically. Eg:

x5 + x+1 = 0sinx+ x+1 = 0

• There are a number of methods to solve f (x) = 0 approximately, eg:– Fixed point iteration– Newton-Raphson

– Bracketing methods ∗ Bisection∗ Regula falsi∗ Secant

20

1.3.1 The Bisection MethodTo find an approximate solution, c, to f (x) = 0 with error < ε (so that | f (c)|< ε).The bisection method starts from two points either side of the root, a and b. Themid-point, c, of a and b is calculated and then the mid point becomes either a or b sothat a and b are still either side of the root. This continues until the mid-point, c, isclose enough to the root.1. Find a and b either side of a root

(so that f (a) f (b)< 0).

0.0 0.2 0.4 0.6 0.8 1.0x

1.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

2. Calculate the mid-point, c = (a+b)/23. Find f at the mid-point, ie f (c).4. The solution is sufficiently close to c if | f (c)|< ε .5. Otherwise, if f (a) and f (c) are opposite signs,

set c as the new value for b, but if f (b) and f (c)are opposite signs then set c as the new a.

6. Go to 2• Bisection is simple and robust but slow.• ∴ often used to get a first guess, before using a more rapidly converging method

21

1.3.2 The Bisection Method in Python

This code defines function,solveByBisect, which finds a root offunction, f, between a and b by bisection.The function also has arguments withdefault parameters (nmax and e).• The main code is within a function,

“solveByBisect”. There are no globalvariables. (Global variables are definedoutside functions and visible from withinall functions. This can lead to bugs.)• The second else occurs if you do not

break out of the for loopExercise What will these produce?1. solveByBisect(sin,-1,1)

a) 0.0 X, b) 3.14159,c) raise Exception("No root found")

2. solveByBisect(exp,-0.1, 0.5)a) 1.0, b) 0.0,c) raise Exception("No root found")X

def solveByBisect( f , a , b, nmax=100, e=1e−6):"""Find a root of function f by bisectionbetween a and b to tolerance e . Maximumnmax iterat ions . Returns the root ."""

for i t in range(nmax) :c = 0.5∗(a+b)i f f (a)∗ f (c) < 0:

b = celse :

a = ci f abs( f (c)) < abs(e ) :

breakelse :

ra ise Exception("No root found")return c

def poly5(x ) :"""Returns a quintic polynomial of x"""return x∗∗5 + x + 1

root = solveByBisect(poly5 , −1, 1)print ("Root of poly5 i s " , root , ". Check " ,

"poly5( root) = " , poly5( root ))

22

1.4 Data TypesNumbers in Python 3 take one of the following types:int An integer whose size is (limited by the computer memory

(sys.maxsize)float A real number in the range ±1.7976931348623157×10308

(sys.float_info.max)Smallest float 2.2250738585072014×10−308 (sys.float_info.min)Smallest float distinguishable from 1: 2.220446049250313×10−16

(sys.float_info.epsilon)complex A complex number consisting of two floating point numbers

with complex arithmetic (eg complex(1,2) or 1+2j)Dividing an integer by an integer gives the integer part of the number in Python 2.7but gives a float in Python 3.A float in python is “double precision”: it takes up 64 bits of memory (on a 64 bitcomputer).A numpy.float32 is “single precision”.A numpy.float16 is “half precision”.

23

1.4.1 Working with Real NumbersWhat will this code produce?def pressure ( force , radius ) :

"""Calculate the pressure over a c i rcu lar area of given radiusand force . Returns the pressure"""

from numpy import pi# Pressure = force/areai f radius != 0:

area = pi∗radius∗∗2pressure = force/areaprint ( ’ pressure = ’ , pressure )

else :print ( ’ error , cannot calculate pressure over \

a zero radius c i rc le ’ )

pressure ( force=10, radius=1e−164)

a) error, cannot calculate pressure over a zero radius circleb) pressure = infc) pressure = 3.18309886e+328d) ZeroDivisionError: float division by zero XWhy? Radius is not exactly zero but area is truncated to zero and so an error isgenerated on division by zero.How can we avoid errors like this?

24

Avoid division by zero and avoid comparing real numbers for equalitydef pressure ( force , radius ) :

"""Calculate the pressure over a c i rcu lar area of given radiusand force . Returns the pressure"""

from numpy import pi# Import system package which defines a small f loat ing point number

import sys

eps = sys . float_info . epsilon # see help( sys . float_info )area = pi∗radius∗∗2

i f abs(area) > eps :pressure = force/areaprint ( ’ pressure = ’ , pressure )

else :print ( ’ radius = ’ , radius , ’ gives area = ’ , area , \

’ which i s too small to calculate the pressure ’ )

pressure ( force=10, radius=1e−164)

What will this code produce?a) radius = 1e-164 gives area = 0.0 which is too small to calculate the pressure Xb) pressure = infc) pressure = 3.18309886e+328d) ZeroDivisionError: float division by zero

25

1.4.2 Representation of Real Numbers on Computers• Integers can be stored by a computer exactly if they are not too large.• Real numbers are stored with limited precision. They are truncated to the number of

significant figures that the machine can record.• Real numbers are stored in a binary form of scientific notation. For example

−1.01×211 or 1.101×2−101

– The (positive part of the) first number is called the mantissa– The (positive part of the) second number is called the exponent:

±(mantissa)×2±exponent

– On a 32-bit computer the 32 bits could be assigned as follows:± mantissa ± exponent

1 bit 23 bits 1 bit 7 bits– the mantissa is a binary number between 1 and less than 2.– the exponent is a binary integer• If you try to create a real number with more significant figures than will fit into the

mantissa or exponent, something will be lost. This is called rounding error. Thiscan lead to some problems with real numbers:

26

1.4.2.1 ExercisesCalculators store real numbers similarly to computers. On your calculator, calculatethese and explain the answers. Is there a rounding error, what is it caused by?Possible problems: 1. not enough space in mantissa, 2. not enough space in exponent.

1. 1×1010 +1×10−10 = a) 1×1010 X, b) 10000000000.0000000001,c) 1.00000000000000000001×1010, d) 0, e) InfProblem: Truncation errors when adding large and small reals. (Not enough bitsavailable for the mantissa). These problems accumulate and can cause big problems!

2. 1×10−10 +1×10−10 = a) 2×10−10 X, b) 0, c) 1.0000000001×10−10, d) 0, e) InfProblem: No problem. Two small numbers can be added accurately

3. (1×1099)2 = a) 1×10198, b) 2×1099, c) 1×1099, d) Error XProblem: The exponent won’t fit in the number of bits available.

4. (1×10−99)2 = a) 0X, b) 1×10−198, c) 2×10−99, d) ErrorProblem: The exponent won’t fit in the number of bits available.

5.√

1+5×10−10 = a) 1.0000000002, b) 1 X, c) 0, d) ErrorProblem: On my calculator, there is enough space in the mantissa to store1+5×10−10 exactly but the number is even closer to 1 after the square root sothere is no longer enough space in the mantissa.

27

1.4.3 Summing small and large numbersThe sum of a large and a small floating point number leads to rounding errors. If thisis done many times, errors can accumulate and become significant.1.4.4 Summing lots of numbersIf you have to sum lots of numbers, eventually you will get to a stage where you areadding a relatively small number to a much larger number. This will lead to roundingerrors.1.4.4.1 Binary ConversionDecimals cannot always be converted to binary numbers precisely. For example, thedecimal number, 0.01 cannot be expressed exactly in binary see table below).Therefore the binary representation in the computer will be truncated which will notnecessarily convert back to the same value.

The decimal number 0.01 in binary, truncated and then converted backDecimal Binary0.01 → 0.00000010100011110101110000101 ...

↓ rounding0.009999275... ← 0.00000010100011110101

28

1.5 IndentationLoops and conditionals in Python are controlled by indentation and colons.1.5.1 Tabs or spaces for indentation - very important to read thisIf you mix tabs and spaces in Python, it will behave in unexpected ways. The pythoncode that I am giving you has spaces not tabs. If you are using Spyder go to Tools ->Preferences-> Editor-> Advanced setting and make sure that “4 spaces” is selected for“Indentation characters”Show spaces too see what is going wrong. In Spyder go to Tools -> Preferences->Editor-> Display and select “Show blank spaces”.1.6 Manually Wrap Long LinesWrap long lines – unless inside brackets, use \ at the end of the line so that the nextline is a continuation. Set up your text editor to avoid lines longer than 80 characters.For example:gradient = (value2 − value1) \

/(x2 − x1)

1.7 Global and Local VariablesA global variable is a variable that can be accessed from all functions, even if it is notan argument to that function. This can be very dangerous and lead to bugs in yourcode. A local variable is a variable that is only defined within a specific function. Avoidglobal variables by putting all code the defines variables within functions.

29

1.8 Order of OperatorsThe order of operator in Python is similar to in mathematics.The operators closer to the top of the list are done first.Python Operator Description** Exponentiation+x, -x Positive, negative*, /, % Multiplication, Division and Remainder+, - Addition and subtraction<<, >> Shifts& Bitwise AND^ Bitwise XOR| Bitwise OR<, <=, >, >=, <>, !=, == Comparisonsis, is not Identity testsin, not in Membership testsnot Boolean NOTand Boolean ANDor Boolean OR

Use brackets if you need the order of operators to be different:In Mathematics ax2 +bx+ c In Python a*x**2 + b*x + c

(a+b)2

c+d (a + b)**2/(c + d)Do not use more brackets than are needed!

30

1.9 Help with Python• Type help() to enter the interactive help and quit to quit online help

1.10 Python Functions and PackagesPython 3 built in functions:abs() delattr() hasattr() max() round()all() dict() hash() memoryview() set()any() dir() help() min() setattr()ascii() divmod() hex() next() slice()bin() enumerate() id() object() sorted()bool() eval() input() oct() staticmethod()breakpoint() exec() int() open() str()bytearray() exit() isinstance() ord() sum()bytes() filter() issubclass() pow() super()callable() float() iter() print() tuple()chr() format() len() property() type()classmethod() frozenset() list() range() vars()compile() getattr() locals() repr() zip()complex() globals() map() reversed() __import__()

Type help(functionName) to see what any of them do

31

In order to get access to more functions you will need to import packages. Somepackages you might use:numpy Mathematical functions and multidimensional arrays supportmatplotlib 2D plotting librarymath Some more mathematical functions that do not operate on arrays

There are 4 ways that you can use the functions in a package:1. from numpy import *

Now you can use all the functions in numpy2. from numpy import sin

Now you can use the function sin from numpy3. import numpy

Now you can use all the functions from numpy but you must refer to them as:numpy.sin(3) etc

4. import numpy as npNow you can use all the functions from numpy but you must refer to them as:np.sin(3)etc

5. In order to find out what functions are defined in a library (say numpy), type:import numpydir(numpy)

32

1.11 Python Arrays and ListsA list of variables can be declared usingmyList = [8 , 3 , 5 , 1]

The elements are numbered from zero and can be accessed using square brackets sothatExample Returns Example Returns Example ReturnsmyList[0] 8 myList[1] 3 myList[3] 1myList[-1] 1 myList[-2] 5myList[4] IndexError: list index out of range

We will mostly be using numpy arrays which can be declared in many ways. Forexample, for an array of five zeros:import numpy as nparraySize = 5myArray = np. zeros (arraySize )

These can be set or accessed using a loopfor i in range(arraySize ) :

myArray[ i ] = 2∗ i + 3print (myArray)

What does this code print out? https://forms.office.com/r/DBBCSp1rcy

a) [3. 5. 7. 9. 11.] X, b) [5. 7. 9. 11. 13.], c) [3. 5. 7. 9. 11. 13.], d) [0 0 0 0 0]

33

1.11.1 Copying lists and arraysAssignment statements in Python do not copy objects, they create bindings between atarget and an object (http://docs.python.org/2/library/copy.html).So if I do:a = [ ’Jack ’ , ’ J i l l ’ , ’Tim’ , ’Dave’ ]b = aa[0] = ’Fred ’print (b)

I get:[ ’_________’ , ’ J i l l ’ , ’Tim’ , ’Dave’ ]

[ ’Fred ’ , ’ J i l l ’ , ’Tim’ , ’Dave’ ]

because “a” and “b” are just two names for the same list. However if I want twoseparate arrays and I want to copy the contents of one array to the other:import numpy as npa = np. array ( [ ’ Jack ’ , ’ J i l l ’ , ’Tim’ , ’Dave’ ] )b = a.copy()a [0] = ’Fred ’print (b)

I get:[ ’________’ , ’ J i l l ’ , ’Tim’ , ’Dave’ ]

[ ’ Jack ’ , ’ J i l l ’ , ’Tim’ , ’Dave’ ]

34

1.11.2 Range Commands• range([start,] stop[, step])

creates integers in turn start to just before stop every stepExamples Returnsrange(4) [0, 1, 2, 3]range(1,4) [1, 2, 3]range(2,7,2) [2, 4, 6]range(7,2,-2) [7, 5, 3]

range doesn’t create a list, it returns the numbers one by one as needed. Thereforeless memory is used if the values are used in turn for a loop.• numpy.arange([start,] stop[, step,])

Creates an array of integer or non-integer valuesExamples Returnsarange(4) array([0, 1, 2, 3])arange(3.,1.,-0.5) array([3., 2.5, 2., 1.5])

35

1.12 Python DictionariesDictionaries can be used to store groups of variables so that they can be passedtogether around functions. For example, a dictionary of physical properties of air:airPhysProp = ’R’ : 287.058, # Gas constant (J/kg/K)

’cp ’ : 1003.8, # Specif ic heat capacity (J/kg/K)’mu’ : 1.2e−5 # Dynamic viscos ity (kg/m/s)

These can be accessed as, for example:airPhysProp [ ’R’ ]

The dictionary can be used by functions representing physical laws that rely on physicalproperties, for example the ideal gas law:def pressureIdealGas(rho , T, physProps ) :

"""Returns the pressure of and ideal gas of density rho , temperature Tand physical properties given in dictionary physProps which includesentry physProps ["R"] for the gas constant ."""

return rho∗physProps ["R"]∗T

This code uses the above dictionary and function. What is the output?Ts = np. array ([273. , 283. , 293. , 303.])ps = np. zeros_like (Ts)for i in range( len (Ts)) :

ps [ i ] = pressureIdealGas (1.2 , Ts[ i ] , airPhysProp)print (ps)

a) [101876.8842 105608.6382 109340.3922 113072.1462],b) [94040.2008 97484.8968 100929.5928 104374.2888] X

36

1.13 Plotting Graphs using the Python library matplotlib1.13.1 A first exampleFirst import the plotting library (and numpy for array, sin and cos):import numpy as np # get access to arraysimport matplotlib . pyplot as plt # the plotting functions

Next create the x and y data and plotx = np. linspace (0 ,1 ,51) # create x − datay = np. sin (x) # compute y − dataplt . plot (x , y) # create plotplt . plot (x , 3∗y) # to create another plotplt .show() # needed i f you are not using Sypder

In Spyder:• You can see the variables in the variable explorer• You may want to see the graph in a separate screen and add more lines to the same

graphTools −> Preferences −> IPython console −> Graphics −> Backend −> Automatic

Then restart the kernel• Show the history• Copy the history to a file, edit, save to a sensible place and re-run

37

More control over graphs and save outputimport numpy as npimport matplotlib . pyplot as pltx = np. linspace (−np. pi , np. pi , 100) # 100 points from −pi to pi inc lus ivey1 = np. sin (x)y2 = np. sin(3∗x)plt . ion () # interact ive mode onplt . c l f () # clear any exist ing f igureplt . plot (x , y1 , label=’sin (x) ’) # plot the f i r s t function with a legendplt . plot (x , y2 , label=’sin (3x) ’) # and the second (over−plot )plt . legend() # shows the legendplt . grid () # show a gridplt . xlabel ( ’x ’ ) # insert this x labelplt . axis ( [ x [0 ] , x[−1] , −1, 1]) # sets the axis rangesplt . savefig ( ’sineWave . pdf ’ )

See copious online support and examples for matplotlib1.13.2 Exercises1. How do you change the color and line-style of graphs2. How to plot symbols instead of lines3. How to change the placing of ticks and grid intervals4. How to plot a log-log plot5. How do you put the legend in a different place?

38

Chapter 2: Better Software PracticesThere are numerous very bad programming practices in science. We will cover a fewways in which things could be done better.2.1 Some Good Programming Practices1. Plan your code before (or after) writing it. Work out what variables you will need

and arrays sizes. Work out how to structure your code using functions.2. Comments:(a) All files should have over-arching comments describing the file contents.(b) All functions should have doc-strings to describe and document them. They are in

double quotes and appear immediately after the function definition line.(c) Doc-strings should describe all arguments and output of a function.(d) Blocks of code and loops should have comments describing what they do.(e) Do not make obvious comments (eg loop from 0 to n−1 every 1).(f) Make sure that comments are up to date with the code.(g) Consider using print statements instead of comments to monitor progress.3. Testing and error catching (section 2.3)4. Backup and version control (section 2.2)

39

5. Write efficient code:(a) Do not declare (a lot) more arrays than you need.(b) Where possible, avoid conditionals inside loops. For example, do not test a

conditional every time around a loop if you know exactly where it will be different.Good u[0] = . . .

u[N] = . . .for i in xrange(1 ,N):

u[ i ] = . . .

Bad for i in xrange(0 ,N+1):i f i == 0:

u[ i ] = . . .e lse i f i == N:

u[ i ] = . . .e lse :

u[ i ] = . . .The second version is more complex code, longer and more expensive to run.

(c) When looping over an array, do not recalculate the whole array every time around:import numpy as npx = np. linspace (0 ,1 ,11)for i in range( len (x )) :

y = x∗∗2

6. Coding to Avoid Errors:(a) Avoid code duplication by using functions and by getting the structure right.(b) Avoid very deep nested loops. Use functions instead.(c) Use variables rather than having input parameters as numbers and pass variables as

arguments to functions rather than re-defining them in each function.(d) Use data structures to avoid long argument lists to functions.

40

(e) If there are any dependencies between variables, calculate these in the code.(f) Avoid global variables. (In Python this means declare all variables inside functions.)(g) The output should be reproducible. Do not rely on editing the code to get all the

output needed for your report.7. Code readability:(a) Put space between functions, space between blocks, space either side of = and

space where needed to make the code more readable. For example:y = a*x**2 + b*x + c not y=x ** 2+b*x+c

(b) Do not use more brackets than you need.(c) Wrap long lines – use \ at the end of the line so that the next line is a

continuation. Set up your text editor to avoid lines longer than 80 characters.(d) Make the code look like the maths.(e) Use good variable names (eg use i to loop over an array x in direction x.)8. Re-write. It is difficult to get the design right first time. Once you have got it all

working, it is time to re-structure, improve readability, efficiency and extensibility.9. If you are new to using arrays and looping over arrays, make sure that you can

initialise and manipulate arrays by hand rather than using inbuilt python handling ofarrays. Once you are confident, you should use numpy array manipulations instead ofexplicit loops as this will be faster and take fewer lines of code.

41

2.2 Backup and Version ControlIdeally everyone who writes code should use a version control system like git and aremote code repository such as github, bitbucket or gitlab. In this module we will beusing a gitlab remote repository without a full version control system. Regardless ofthe version control system or repository you use, you should do the following:• Back up all of your code regularly (or commit to a repository). Exclude output of

code and graphics from backups. You should be able to reproduce all output andgraphics just by re-running the code.• Keep code and output in separate folders.• Check that you can reproduce all of your code output and plots by re-running your

code without making any edits.• Include a README file with your code describing how to run the code in order to

reproduce all output.

42

2.2.1 Instructions for using Gitlab• Log into https://gitlab.act.reading.ac.uk and click “Create a Project”

and call it MTMW12_assig2• Make the repository private and select “Initialize repository with a README”• Click “Create Project"• Add text to the README file by clicking on it and choosing Edit. To save the

changes, first write a commit message, then press “Commit changes”.• To upload files, go to the MTMW12_assig2 page, click “Repository”, then click +

then “Upload File”. Select a file, write a commit message then “Upload file”.• To replace files in the repository with files on your computer, click on a file ingitlab and then select “Replace”.• To download files, click on the file then clicking on the download button .• To download the whole repository, go to the project page and click .• Whenever you make changes you will be asked to write a commit message and then

commit. All previous versions of the project are stored so you can find an old versionby looking at the commit messages.• In order to automatically keep the version on the remote repository up to date with

the version on your computer you would need git. We are not using git in thismodule so you will need to ensure that versions are up to date yourself.

43

2.3 TestingAll code should be tested. Some examples of how functions can be tested:• If a function returns a variable, test that it returns exactly the right variable for

particular arguments. Give an example• If you know that a function will fail for a given argument check that the function

throws an exception when these arguments are given. Example If it divides by zeroNot enough testing is done in scientific programming which hampers the progress ofscience and weakens scientific arguments. Please help to fix this.There are some excellent modules to help with testing in Python that are not coveredhere, for example, unittest and nosetest. Tools like coverage can show youhow much of your code you have tested. There is good integration with someintegrated development environments (IDEs), such as PyCharm, which helps withmost aspects of writing, testing, debugging, and running code.

44

2.3.1 Making AssertionsWhat outputs do these commands give? Try them out.

assert 2 + 2 == 4assert 2 + 2 == 5assert max(1 , 2) == 2assert max(1 , 2) == 1assert 1 < 2assert 2 < 1assert abs(np. sin (np. pi )) < 1e−6assert abs(np. sin (np. pi )) < 1e−16

The testing described next will use the code solveByBisect.py available athttps://gitlab.act.reading.ac.uk/sws02hs/mtmw12_assig2/blob/master/solveByBisect.py

45

We will test the bisection root finding function:def solveByBisect( f , a , b, nmax=100, e=1e−6):

"""Find a root of function f by bisection between a and b to tolerance e .Maximum nmax iterat ions . Returns the root ."""

# Iterate unt i l the solution i s within the error or too many iterat ionsfor i t in range(nmax) :

c = 0.5∗(a+b)i f f (a)∗ f (c) < 0:

b = celse :

a = ci f abs( f (c)) < abs(e ) :

break # Break out of the for loopelse : # I f you haven ’ t broken out of the for loop by the end ,

ra ise Exception("No root found by solveByBisect")

return c

We can make assertions about what solveByBisect should do for particular sets ofarguments and the error message to give if the wrong answer is foundfrom numpy import sinx = solveByBisect( sin ,−1,1)assert abs(x) < 1e−6# ORassert abs( sin (solveByBisect( sin ,−1,2))) < 1e−6

What will the output be?No output

46

2.3.2 Raising ExceptionsSee also https://docs.python.org/2/tutorial/errors.htmlThe standard types of exception that can be raised in Python are at https://www.tutorialspoint.com/python/standard_exceptions.htm

2.3.2.1 Value and Type ErrorsValue and type errors are exceptions to check that the correct types or ranges of valuesare passed to functions. For example, the solveByBisect function could include:

i f nmax <= 0:ra ise ValueError ( ’Argument nmax to solveByBisect should be >0’)

i f e <= 0:ra ise ValueError ( ’Argument e to solveByBisect should be >0’)

i f not( is instance (a , f loat )) or not( is instance (b, f loat )) :ra ise TypeError( ’Arguments a and b to solveByBisect should be a floats ’ )

i f not( cal lable ( f )) :ra ise Exception( ’A cal lable function must be sent to solveByBisect ’ )

Use TypeError sparingly; it is not Pythonic and can make your code inflexible. Itmay be more useful, for example, to convert integer values of a and b to floating pointvalues.a = float (a)b = float (b)

These will raise exceptions of a and b cannot be converted to floats.

47

2.4 DebuggingGood testing will reduce the need for debugging. You should be able to debug by handwithout a dedicated tool, using print statements:• Re-produce the problem with the simplest code and with arrays set as small as

possible so that code runs quickly and you could re-produce the calculations by hand.• You need to know where things go wrong. Start by putting in print statements at a

point in the code shortly before everything went wrong. Print out variables that youthink might be related to the problem and check that their values are as expected.• Print out relevant variables further up where you think that everything was working.• Using more print statements, find where in between things go wrong.• When you have found and corrected the bug, delete the de-bugging code rather than

commenting it out. You do not want your code littered with commented out code.You could save the debug statements in a branch of your repository.• Think how the bug happened and how it could be avoided with better testing.Further InformationIDEs (Interactive Development Environments) have interactive debugging tools whichallow you to set breakpoints in the code and inspect variable values.

48

Chapter 3: Taylor Series and Finite DifferencesThis chapter introduces Taylor series and how they are used in numerical analysis tofind numerical approximations and estimate their accuracy.• Many functions can be expressed as Taylor series• Taylor series are infinite polynomials

For example an exponential:

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.02

1

0

1

2

3

4

5

6ex

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.02

1

0

1

2

3

4

5

6ex

1

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.02

1

0

1

2

3

4

5

6ex

1

1 +x

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.02

1

0

1

2

3

4

5

6ex

1

1 +x

1 +x+ x2

2!

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.02

1

0

1

2

3

4

5

6ex

1

1 +x

1 +x+ x2

2!

1 +x+ x2

2!+ x3

3!

2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.02

1

0

1

2

3

4

5

6ex

1

1 +x

1 +x+ x2

2!

1 +x+ x2

2!+ x3

3!

1 +x+ x2

2!+ x3

3!+ x4

4!

49

For example a sine wave:

6 4 2 0 2 4 61.5

1.0

0.5

0.0

0.5

1.0

1.5

sinx

6 4 2 0 2 4 61.5

1.0

0.5

0.0

0.5

1.0

1.5

sinx

x

6 4 2 0 2 4 61.5

1.0

0.5

0.0

0.5

1.0

1.5

sinx

x

x− x3

3!

6 4 2 0 2 4 61.5

1.0

0.5

0.0

0.5

1.0

1.5

sinx

x

x− x3

3!

x− x3

3!+ x5

5!

6 4 2 0 2 4 61.5

1.0

0.5

0.0

0.5

1.0

1.5

sinx

x

x− x3

3!

x− x3

3!+ x5

5!

x− x3

3!+ x5

5!− x7

7!

The more terms you include, the more accurate it should get.

50

Functions with discontinuities cannot be expressed as Taylor series:

6 4 2 0 2 4 61.5

1.0

0.5

0.0

0.5

1.0

1.5square wave

A polynomial cannot jump between 1 and -1

51

Functions with discontinuities in some of their derivatives cannot beexpressed as Taylor series:

Eg. cosine bell, f (x) =

1/2(1+ cosx) |x|< π0 otherwise

6 4 2 0 2 4 60.5

0.0

0.5

1.0

1.5

cosine bell1−1

2x2

2!

1−12x2

2!+1

2x4

4!

1−12x2

2!+1

2x4

4!−1

2x6

6!

A polynomial cannot be uniform in one place and non-uniform in another

52

3.1 The Taylor Series:A function, f , can be represented as a Taylor series about position a if:• it is continuous near a and• all of its derivatives are continuous near a

Using the notation ∆x = x−a:

f (x) = f (a)+∆x f ′(a)+∆x2

2!f ′′(a)+

∆x3

3!f ′′′(a)+ · · ·+ ∆x j

j!f ( j)(a)+ · · ·

where f ′(x) = d fdx (x), f ′′(x) = d2 f

dx2 (x), ...• If infinitely many terms are used then this approximation is exact near a.• If terms of order n and above are discarded then the error is approximately

proportional to ∆xn. Then the approximation is nth order accurate• A third order accurate approximation for f (x) has error proportional to ∆x3:

f (x) = f (a)+∆x f ′(a)+∆x2

2!f ′′(a)+O

(∆x3) .

We say that the error is of order ∆x3 or O(∆x3).

53

3.1.1 Quiz (select all that apply) https://forms.office.com/r/DBBCSp1rcy

1. What is the infinite Taylor series for function f at position x+∆x about x

(a) f (x+∆x) = f (x)+∆x f ′(x)+ ∆x2

2! f ′′(x)+ ∆x3

3! f ′′′(x)+ · · ·+ ∆x j

j! f ( j)(x)+ · · · X(b) f (x) = f (x+∆x)+∆x f ′(x+∆x)+ ∆x2

2! f ′′(x+∆x)+ · · ·+ ∆x j

j! f ( j)(x)+ · · ·(c) f (x+∆x) = ∑∞

j=0∆x j

j! f ( j)(x) X(d) f (x) = ∑∞

j=0∆x j

j! f ( j)(x+∆x)2. Write the third order approximation for f (x−∆x) in terms of f (x), f ′(x) and f ′′(x).

Write the error term using the O(∆xn) notation.(a) f (x−∆x) = f (x)−∆x f ′(x)−O

(∆x3)

(b) f (x−∆x) = f (x)−∆x f ′(x)+ ∆x2

2! f ′′(x)+O(∆x3) X

(c) f (x−∆x) = f (x)−∆x f ′(x)− ∆x2

2! f ′′(x)−O(∆x3)

(d) f (x−∆x) = f (x)−∆x f ′(x)− ∆x2

2! f ′′(x)+O(∆xn)

54

1. Write the third order approximation for f (x+∆x) in terms of f (x−∆x), f ′(x−∆x)and f ′′(x−∆x).

(a) f (x+∆x) = f (x)+∆x f ′(x)+ ∆x2

2! f ′′(x)+O(∆x3)

(b) f (x−∆x) = f (x)−∆x f ′(x)+ ∆x2

2! f ′′(x)+O(∆x3)

(c) f (x+∆x) = f (x−∆x)+2∆x f ′(x−∆x)+2∆x2 f ′′(x−∆x)+O(∆x3) X

(d) f (x+∆x) = f (x−∆x)+2∆x f ′(x−∆x)+4∆x2 f ′′(x−∆x)+O(∆x3)

(e) f (x) = f (x−∆x)+∆x f ′(x−∆x)+ ∆x2

2! f ′′(x−∆x)+O(∆x3)

55

3.2 Numerical DifferentiationConsider points, x0,x1, · · ·x j, · · ·xn which are a distance ∆x apart (x j = j∆x). Assumethat we know the value of the function f (x) at these points, as shown in figure 3.1.

x0 x1 x2 x j x j+1x∆ x j−1 x

f f jj−1

f j+1

f

Figure 3.1: Values of a function f at points x0, x1, · · · ,x j, · · · .Some possible estimate of f ′j = f ′(x j) are:

forward difference backward difference centred differencef ′j ≈

f j+1− f j∆x f ′j ≈

f j− f j−1∆x f ′j ≈

f j+1− f j−12∆x

Taylor series can be used to estimate derivatives and find their order of accuracy.

56

3.3 Taylor Series to find Finite Difference GradientsIn order to use a Taylor series (below) to find an approximation for f ′

f (x+∆x) = f (x)+∆x f ′(x)+ ∆x2

2! f ′′(x)+ ∆x3

3! f ′′′(x)+ · · ·+ ∆x j

j! f ( j)(x)+ · · ·1. write down the knowns2. consider where we want to find f ′

3. consider what order of accuracy we want4. write down Taylor series for some of the knowns5. eliminate the additional unknowns to find f ′

3.3.1 Example1. Assume that we know f j = f (x j)

f j−1 = f (x j−1) = f (x j−∆x)f j+1 = f (x j+1) = f (x j +∆x)

2. and we want to find f ′j.3. For 3 knowns we wonder if we can get second order accuracy4. We don’t know f ′j−1 or f ′j+1 so no Taylor series about x j−1 or x j+1. Let’s try Taylor

series for f j+1 and f j−1 about x j

f j+1 = f j +∆x f ′j +∆x2

2! f ′′j +∆x3

3! f ′′′j +O(∆x4)

f j−1 = f j−∆x f ′j +∆x2

2! f ′′j − ∆x3

3! f ′′′j +O(∆x4)

57

5. Eliminate f ′′j by taking the difference of the two equations

f j+1− f j−1 = 2∆x f ′j +∆x3

3f ′′′j +O(∆x4)

Rearrange to get f ′j

f ′j =f j+1− f j−1

2∆x− ∆x2

3!f ′′′j +O(∆x3)

We cannot eliminate f ′′′j so this is part of the error:

f ′j =f j+1− f j−1

2∆x+O(∆x2)

The error, ε , is proportional to ∆x2 (ε ∝ ∆x2) so this approximation is second orderaccurate.

This is a worked example on my YouTube page:https://youtu.be/xPTnH0PAAXs?list=PLEG35I51CH7UttDft9tOB0ej_kkAJF368called TaylorSeries2

58

3.3.2 Exercises (answers at the end)1. Use the Taylor series to find an approximation for f ′j in terms of f j and f j−1. What

order accuracy is it?

2. Derive an uncentred, second order difference formula for f ′j that uses f j, f j+1 andf j+2. (And show that it is second order accurate)

3. Find an uncentred approximation for f ′′j using f j, f j+1 and f j+2. What orderaccurate is it?

4. Derive a second order approximation for f ′b from fa, fb and fc at x locationsa < b < c when the grid spacing is not regular. (And show that it is second orderaccurate)

59

3.4 Order of Accuracy of Numerical SolutionsIn order to demonstrate the order of accuracy (or order of convergence) of a numericalmethod, we can calculate the error of solving a problem that has an analytic solution.Assume that the error, ε = A∆xn where ∆x is the resolution, n is the order of accuracyand A is unknown. Given two resolutions, ∆x1 and ∆x2, and two resulting errors, ε1and ε2, eliminate A to find n:

ε1 = A∆xn1 and ε2 = A∆xn

2

=⇒ ε1

ε2=

(∆x1

∆x2

)n

Take logarithms (any base) of both sides and remember log identities:

logε1

ε2= n log

(∆x1

∆x2

)

=⇒ logε1− logε2 = n(log∆x1− log∆x2)

=⇒ n =logε1− logε2

log∆x1− log∆x2(3.1)

Thus the order of convergence can be estimate using errors at two resolutions.

60

The order can also be estimated graphically. From eqn (3.1) we can see that n is thegradient of a line plotted on log-log paper. For example a numerical method calculatesthe gradient of sinx and gives these results:

∆x numerical gradient of sinx at x = 0 Error, ε (Difference from cos(0))0.4 0.97355 -0.026450.2 0.99335 -0.006660.1 0.99833 -0.00167

Plot |ε| as a function of ∆x on log paper andestimate the gradient.

10-2 10-1 100

∆x

10-3

10-2

10-1

|erro

r|Next use eqn (3.1) to find two estimates of n. Do the values agree with the gradient ofthe curve? They should all be close to 2.

61

3.5 Interpolation

An Example:• Function f is known at points x1 and

x2 (values f1 and f2)• We want to estimate the value of f at

point xi in between x1 and x2 x1

x2

1f

if

2f

xi

β∆x (1−β)∆ x

∆x

x

• Exercise: Use linear interpolation (ie assume that fi lies on a straight line betweenf1 and f2): to find f at xiHint: First write down expressions for ∆x, β and the gradient, f ′ between x1 and x2.Then find an expression for f at xi along the straight line between x1 and x2.

∆x = x2− x1 β = xi−x1x2−x1

f ′ = f2− f1x2−x1

=⇒ fi = (1−β ) f1 +β f2

62

3.5.1 Taylor Series to Find Finite Difference Formulae for InterpolationAn Example:Assume that we know f j = f (x j) and f j+1 = f (x j+1) and we want to find theinterpolated value, f j+1/2

, mid-way between x j and x j+1.• Start by writing down Taylor series for f j and f j+1 about f j+1/2

f j+1 = f j+1/2+ ∆x

2 f ′j+1/2+ 1

2!

(∆x2

)2f ′′j+1/2

+ 13!

(∆x2

)3f ′′′j+1/2

+O(∆x)4

f j = f j+1/2− ∆x

2 f ′j+1/2+ 1

2!

(∆x2

)2f ′′j+1/2

− 13!

(∆x2

)3f ′′′j+1/2

+O(∆x)4

• Eliminate the largest unknown, f ′j+1/2by adding the two equations

f j + f j+1 = 2 f j+1/2+ ∆x2

4 f ′′j+1/2+O(∆x)4

• Rearrange to find f j+1/2and express the error based on the largest unknown

f j+1/2= ( f j + f j+1)/2+O(∆x)2

• So this is a second-order accurate approximation

63

3.5.2 Exercises (answers on Blackboard)1. Derive a centred, second order difference interpolation formula for f j that uses f j−1

and f j+1. (And show that it is second order accurate)2. Derive a centred fourth order difference formula for f ′j+1/2

that uses f j−1, f j, f j+1

and f j+2. (And show that it is fourth order accurate)3. Find the highest possible order approximation for f ′j that uses f j−2, f j−1, f j and

f j+1 and find its order of accuracy.4. Show that the first order forward difference formula for f ′j is exact for linear

functions, f (x) = ax+b.5. Show that the centred second order difference formula for f ′j is exact for quadratic

functions f (x) = ax2 +bx+ c.

64

Chapter 4: The Navier Stokes (NS) EquationsWeather and climate prediction models solve equations of motion to predict the wind,air density and temperature for a compressible, rotating atmosphere:The Lagrangian derivative: DΨ

Dt = ∂Ψ∂ t +u ·∇Ψ

Momentum (NS) DuDt = -2ΩΩΩ×u− ∇p

ρ +g+µu(∇2u+ 1

3∇(∇ ·u))

Continuity DρDt +ρ∇ ·u = 0

Potential temperature DθDt = Q+µθ ∇2θ

An equation of state, eg perfect gas law, p = ρRT

u Wind vector g Gravity vector (downwards)t Time θ Potential temperature, T (p0/p)κ

ΩΩΩ Rotation rate of planet κ heat capacity ratio ≈ 1.4ρ Density of air Q Source of heatp Atmospheric pressure µu, µθ Diffusion coefficients (constant in eqns above)

These equation do not need to be memorised. They are displayed to show why we learnabout numerical methods for solving the diffusion and advection equations.• Do you need to revise vector calculus in order to understand ∇ ·u, ∇p and u ·∇Ψ?

65

4.1 The Potential Temperature EquationDθDt = ∂θ

∂ t + u ·∇θ = Q + µθ ∇2θLagrangian Rate of change Advection Heat Diffusionderivative at fixed point of θ source of θ

This is a numerical solution ofthe potential temperatureequation with fixed winds(arrows) and a small heatsource.θ will be created anddestroyed by the heat source,Q , it will be moved around bythe wind field, u, and θ willbe diffused by a diffusioncoefficient, µθ

66

4.2 Advection of Pollution4.2.1 Pure Linear AdvectionAdvection of concentration φ without diffusion or sources or sinks:

DφDt

=∂φ∂ t

+u ·∇φ = 0 (4.1)

Changes of φ are produced by the component of the wind in the same direction asgradients of φ . In order to understand why the u ·∇φ term leads to changes in φ ,consider a region of polluted atmosphere where the pollutant has the concentrationcontours shown below:

φ = 0.1

φ = 0.2

φ = 0.3

φ = 0.4

u

Exercise: Draw on the figure thedirections of the gradients of φ andthus mark with a +, − or 0 locationswhere u ·∇φ is positive, negative andzero. Thus deduce where φ isincreasing, decreasing or staying thesame based on equation 4.1. Henceoverlay contours of φ at a later time.

67

4.2.2 Advection/Diffusion with Sources and SinksDΨDt = ∂Ψ

∂ t + u ·∇Ψ = S + µΨ∇2ΨLagrangian Rate of change Advection Sources Diffusionderivative at fixed point of Ψ and sinks of Ψ

68

4.3 The Momentum EquationDuDt = -2ΩΩΩ×u - ∇p

ρ + g + µu(∇2u+ 1

3∇(∇ ·u))

Lagrangian Coriolis Pressure Gravitational Diffusionderivative gradient acceleration

4.3.1 CoriolisInertial Oscillations governed by part of the momentum equation:

∂u∂ t

= -2ΩΩΩ×u

• A drifting buoy set inmotion by strong westerlywinds in the Baltic Sea inJuly 1969.• Once the wind subsides, the

upper ocean follows inertiacircles

69

4.3.2 The Pressure Gradient Force

If the pressure gradient force is the onlylarge term in the momentum equation,then together with the continuityequation and perfect gas law, we getequations for acoustic waves:

∂u∂ t

+1ρ0

∇p = 0

∂ p∂ t

+ρ0c2∇ ·u = 0

where ρ0 is a reference density and c isthe speed of sound.

70

Pressure Gradients lead to very fast acceleration - Acoustic Waves

71

Geostrophic Balance: Pressure Gradients versus Coriolis

72

Geostrophic turbulence: pressure gradients, Coriolis and the (non-linear)advection of velocity by velocity

73

Gravitational Acceleration: Explosive Comulonimbus

74

Revision of Second Derivatives https://forms.office.com/r/DBBCSp1rcyGiven this y, which is d2y/dx2 a

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0x

6

4

2

0

2

4

6

b X c

0.0 0.2 0.4 0.6 0.8 1.0x

100

80

60

40

20

0

20

40

0.0 0.2 0.4 0.6 0.8 1.0x

40

20

0

20

40

60

80

100

75

4.3.3 DiffusionDiffusion of φ with coefficient µφ :

∂φ∂ t = µφ ∇2φ

And in 1d:∂φ∂ t = µφ

∂ 2φ∂x2

The second derivative of φ is high attroughs and low in peaks of φ . Thereforediffusion tends to remove peaks andtroughs and make a profile more smooth:

Questions:• Which equations have diffusion?• What causes diffusion?• Is diffusion a large term of the

equations of atmospheric motion?

Diffusion of two gases (Youtube)

Diffusion of a noisy profile(zero gradient boundary conditions)

0.0 0.2 0.4 0.6 0.8 1.0x

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

φ

76

4.3.4 The Complete Navier Stokes EquationsWith moisture, phase changes, radiation, ... HadGEM simulaiton from Pier Luigi Vidale

77

4.4 Boundary ConditionsFor a partial differential equation (PDE) to have a solution, extra information is neededat the boundaries. These are boundary conditions and there are some different types:4.4.1 Fixed Value (Dirichlet)For the diffusion equation, the dependent variable (for example temperature) can befixed at the boundary. A fixed temperature at a boundary might be maintained by athermostat. The diffusion equation can also represent wind in the boundary layer. Theground leads to a fixed wind speed of zero at the boundary.For the advection equation a fixed value is needed at inlet boundaries to definewhat is entering the domain. No boundary condition is needed at outlets or walls.4.4.2 Fixed Gradient (Neumann)A gradient implies a flux into or out of the domain proportional to the gradient. If theboundary condition is zero gradient then there will be no flux at the boundary and sothis boundary condition will not alter the conservation inside the domain. A zerogradient boundary condition for temperature implies an insulating boundary.4.4.3 PeriodicThis boundary condition can be used to represent an unrolled circular domain – exactlywhat leaves one boundary enters the opposite boundary. Two separate boundaries arealways given identical properties.

78

4.5 Linearity of Partial Differential Equations (PDEs)• A partial differential equation is linear if it is linear in the unknown functions and

their derivatives. (So f 2 or f ∂ f/∂x is a non-linear term for unknown function f .)• Which of these PDEs are linear and which non-linear

∂ f∂ t +u∂ f

∂x = 0 linear ∂ f∂ t + x3 ∂ f

∂x = 0 linear

∇2 f = ∂ 2 f∂x2 +

∂ 2 f∂y2 = 0 linear ∂ f

∂ t + f ∂ f∂x = 0 non-linear

∂ f∂ t −K ∂ 2 f

∂x2 = 0 linear

• If h(x) and and g(x) are both solutions of a linear PDE, then, for constants a and b,ah(x)+bg(x) is also a solution– super-position of solutions• Which are the non-linear terms of the Navier-Stokes equations?

79

80

Chapter 5: Solution of the Diffusion EquationThe one-dimensional diffusion equation is

∂φ∂ t = K ∂ 2φ

∂x2 (5.1)

5.1 Numerical Solution using Finite Differences• An Eulerial discretisation – separate discretisations for time and space• Split up space and time into discrete intervals separated by numbered points:

x j = j ∆x, tn = n ∆t with φ (n)j = φ(x j, tn)

for j = 0,1,2..., n = 0,1,2, ... assuming ∆x and ∆t are constant• The time derivative, ∂φ/∂ t, can be represented by forward, backward or centred

finite differences (see section 3.2):

Order of accuracy

Forward time FT ∂φ∂ t

(n)

j ≈φ (n+1)

j −φ (n)j

∆t 1st order

Backward time BT ∂φ∂ t

(n+1)

j ≈ φ (n+1)j −φ (n)

j∆t 1st order

Centred time CT ∂φ∂ t

(n)

j ≈φ (n+1)

j −φ (n−1)j

2∆t 2nd order

81

• The spatial derivative, ∂ 2φ/∂x2 = φ ′′j , can be represented using φ j−1, φ j and φ j+1.Either:– Use Taylor series to find φ ′′j in terms of φ j−1, φ j and φ j+1 or– Estimate the gradients at x j±1/2

and then estimating the gradient of the gradients.φ

x j+1x jx j−1

φ j

φ j+1

φ j−1

x

x j+1/2x j−1/2

These both give:

Order

Centred space CS φ ′′j ≈φ j+1−2φ j+φ j−1

∆x2 2nd order

82

This leads to three potential schemes for the diffusion equation. Derive these schemesby substituting the various finite difference schemes into equation 5.1. Substitute inthe non-dimensional diffusion coefficient, d = K∆t/∆x2. Take care to use the correcttime-level:

Scheme Order of accuracy intime space

FTCS φ (n+1)j = φ (n)

j +d(φ (n)

j+1−2φ (n)j +φ (n)

j−1)

1 2

BTCS φ (n+1)j = φ (n)

j +d(φ (n+1)

j+1 −2φ (n+1)j +φ (n+1)

j−1)

1 2

CTCS φ (n+1)j = φ (n−1)

j +2d(φ (n)

j+1−2φ (n)j +φ (n)

j−1)

2 2

83

5.2 Implicit and Explicit SchemesExplicit Values at the next time-step are determined from values at the

current (and previous) time-steps. Straightforward.Implicit Values at the next time-step are determined from values at the next

time step!• This leads to a set of simultaneous equations• How do we solve a set of linear simultaneous equations?

- create a matrix equation and then solve using, eg Gaussianelimination• How do we solve a set of non-linear simultaneous equations?

- multi-dimensional generalisation of Newton-Raphson method -hard!• Why would we want to use an implicit method?...

Advanced Optional Exercise: Assume that all of the j values, φ (n)j (for a particular

n) can be represented as a vector: φφφ (n) =(

φ (n)0 ,φ (n)

1 , · · · ,φ (n)j , · · · ,φ (n)

nx−1

)T. Find a

matrix M such that BTCS is defined as Mφφφ (n+1) = φφφ (n) given x : 0→ 1 and periodicboundary conditions so that φ0 = φnx .

84

5.3 Analytic Solution of the Diffusion Equation (no need tomemorise)

The linear partial differential equation:∂φ∂ t = K ∂ 2φ

∂x2 (5.2)has solutions of the form:

φ = Aexp(−Kk2t)cos(kx+B)+C (5.3)where A, B, C and k are arbitrary constants.Exercise: Try it out: substitute equation (5.3) into equation (5.2).If the initial conditions can be represented as a sum of cosines with differentwavenumbers (k):

φ(x,0) =n

∑k=0

Ak cos(kx+Bk) (5.4)

then, since the diffusion equation is linear, the analytic solution at any time can bewritten down using the super-position of solutions:

φ(x, t) =n

∑k=0

Ak exp(−Kk2t)cos(kx+Bk).

The higher the wavenumber, the faster it is diffused.Question: Can any initial conditions be expressed as in equation (5.4)? See nextchapter on Fourier analysis

85

5.3.1 Solution for step-function initial conditions (optional)

For initial conditions consisting of a step function inan infinite domain

φ(x,0) =

1 x≥ 00 x < 0

the analytic solution is φ(x, t) = 12erf

(x√4Kt

)where erf is the error function, available

in the python math and scipy libraries.5.3.2 Superposition of SolutionsSince the diffusion equation is linear, sums of solutions are also solutions. If f (x, t) andg(x, t) are solutions of ∂φ/∂ t = K∂ 2φ/∂x2 then

φ(x, t) = a f (x+b, t)+ cg(x+d, t)

is also a solution.Exercise: Using the superposition of solutions and the error function solution, find thesolution to the diffusion equation with the initial condition:

φ(x,0) =

1 α ≤ x≤ β0 otherwise

86

Signposting• Will the diffusion schemes described in this chapter give accurate solutions to the

diffusion equation given sufficient resolution?• Will we be able to take large time-steps to reach a solution quickly?• What is the point of implicit time stepping?In order to answer these questions, we will need von-Neumann stability analysis.von-Neumann stability analysis relies on Fourier analysis. An introduction/revision ofFourier analysis will be presented for 3 reasons:1. It is needed for stability analysis2. Some weather forecasting models use spectral decompositions (eg ECMWF) which

are a spherical version of Fourier decompositions3. Fourier analysis is used to analyse climate data

87

88

Chapter 6: Fourier Analysis6.1 Fourier SeriesAny periodic, integrable function, f (x) (defined on [−π,π]), can be expressed as aFourier series; an infinite sum of sines and cosines:

f (x) =a0

2+

∞

∑k=1

ak coskx+∞

∑k=1

bk sinkx (6.1)

• The ak and bk are the Fourier coefficients.• The sines and cosines are the Fourier modes.• k is the wavenumber - number of complete waves that fit in the interval [−π,π]

sinkx for different values of k

−π −π/2 0 π/2 π

x

1.0

0.5

0.0

0.5

1.0

k=1

k=2

k=4

• The wavelength is λ = 2π/k• When more Fourier modes are included,

their sum will get closer to the function.

89

Sum of First 4 Fourier Modes of a Periodic Function

−π −π/2 0 π/2 π

x

1.0

0.5

0.0

0.5

1.0Fourier ModesFourier ModesFourier ModesFourier Modes

−π −π/2 0 π/2 π

x

4

2

0

2

4Original functionSum of first 4 Fourier modes

90

The first four Fourier modes of a square wave• The additional oscillations are “spectral ringing”.• Each mode can be seen as motion around a circle.• The motion around each circle has a speed and a

radius. These represent the wavenumber and theFourier coefficients. Which is which?

a. Speed is wavenumber, radius is coefficient.X b. Speed is coefficient, radius is wavenumber

91

Equivalently, equation (6.1) can be expressed as an infinite sum of exponentials usingthe relation eiθ = cosθ + isinθ where i =

√−1:

f (x) =a0

2+

∞

∑k=1

ak coskx+∞

∑k=1

bk sinkx =∞

∑k=−∞

Akeikx. (6.2)

with Ak =1/2 (ak− ibk) , A−k =

1/2 (ak + ibk)

Exercise Check that formulae for Ak are correct for a value of k.

6.2 Fourier Transform• The Fourier Transform transforms a function f which is defined over space (or time)

into the frequency domain, so that it is defined in terms of Fourier coefficients.• The Fourier transform calculates the Fourier coefficients as:

ak =1π

∫ π

−πf (x)cos(kx)dx , bk =

1π

∫ π

−πf (x)sin(kx)dx

92

6.3 Discrete Fourier TransformA discrete Fourier Transform converts a list of 2N +1 equally spaced samples, fn, in[−π,π) of a complex valued, periodic function, f , to the list of the first 2N +1complex valued Fourier coefficients:

Ak =1

2N +1

N

∑n=−N

fn e−i 2πnk2N+1 . (6.3)

for k =−N, ...N.The truncated Fourier series:

f (x)≈N

∑k=−N

Akeikx (6.4)

is an approximation to the function f which fits the sampled points, fn, exactly.On a computer this is done with a Fast Fourier Transform (or fft). The inverseFourier transform (sometimes called ifft) transforms the Fourier coefficients back tothe f values (transforming from spectral back to real space):

f−N , f−N+1, ..., f0, f1, f2, · · · fNfft−−→ A−N ,A−N+1, ...,A0,A1, · · ·AN

A−N ,A−N+1, ...,A0,A1, · · ·ANifft−−−→ f−N , f−N+1, ..., f0, f1, f2, · · · fN

93

ExerciseCheck the formulae in equations (6.3) and (6.4) by defining your own function, f , at2N +1 equally space points in (−π,π) . Use these function values to calculate theFourier coefficients from equation (6.3) then plug them back into equation (6.4) tocheck that you retrieve the same function values at the original points. To make this assimple as possible, try using N = 1 and f = 2(1− cosx) so that f−1 is defined atx =−2π/3, f0 = 0 is defined at x = 0 and f1 is defined at x = 2π/3.

94

6.4 Differentiation and InterpolationIf we know the Fourier coefficients, Ak, of a function f then we can calculate thegradient of f at any point, x: If

f (x) =∞

∑k=0

ak coskx+∞

∑k=0

bk sinkx =∞

∑k=−∞

Akeikx (6.5)

then ∂ f∂x = f ′(x) = (select all that are correct)

a.∞

∑k=0−kak sinkx+

∞

∑k=0

kbk coskx X b.∞

∑k=−∞

i k Akeikx X (6.6)

c.∞

∑k=0

ak sinkx+∞

∑k=0

bk coskx d.∞

∑k=−∞

Akeikx (6.7)

and the second derivative, f ′′(x) =

a.∞

∑k=0−k2ak coskx−

∞

∑k=0

k2bk sinkx X b.∞

∑k=−∞

−k2 Akeikx X (6.8)

c.∞

∑k=0

k2ak coskx+∞

∑k=0

k2bk sinkx d.∞

∑k=−∞

k2 Akeikx (6.9)

These have spectral accuracy; the order of accuracy is as high as the number of points.Similarly equation 6.1 or 6.2 can be used directly to interpolate f onto an undefinedpoint, x. Again, the order of accuracy is spectral.

95

6.5 Spectral Models• ECMWF use a spectral model.• The prognostic variables are transformed between physical and spectral space usingffts and iffts.• Gradients are calculated very accurately in spectral space6.6 Wave Power and Frequency• If a function, f , has Fourier coefficients, ak and bk, then wavenumber k has power

a2k +b2

k .• A plot of wave frequency versus power is referred to as the power spectrum. Before

we learn how power spectra are used, we will have some revision questions...

Quiz on Fourier Series https://forms.office.com/r/nNL0Kh5t4P

96

6.7 Analysing Power SpectraDaily rainfall at a station in the Middle East

0

10

20

30

40

50

60

0 5 10 15 20

rain

fall

(m

m)

years

daily rainfallFourier filtered using 40 wave numbers

Observations about theTruncated Fourierfiltered rainfall:• very smooth (only low

wavenumbers included)• includes negative

values – “spectralringing”

Power Spectrum of Middle East Rainfall

10-6

10-5

10-4

10-3

10-2

10-1

100

10-1

100

101

102

po

wer

Number per year

number per year = wavenumber×365/total number of days

Observations• dominant frequency at one

year (annual cycle)• power at high frequencies (ie

daily variability)

97

Time Series of the Nino 3 sea surface temperature (SST)The SST in the Nino 3 region of the equatorial Pacific is a diagnostic of El Nino

23

24

25

26

27

28

29

30

SS

T (

deg

C)

1990 1995 2000 2005 2010

raw2 years and slowerannual cycle How were the dashed lines

generated?• Annual cycle is the Fourier

mode at 1 year• The “two years and slower”

filtered data is the sum of allthe Fourier modes of thesefrequencies.

Power Spectrum of Nino 3 SST

1e−06

1e−05

0.0001

0.001

0.01

0.1

1

po

wer

0.01 0.1 1 10

Frequency (per year)

Observations• Dominant frequency at 1 year (annual

cycle)• Less power at high frequencies (SST

varies slowly)• Power at 1-10 years (El Nino every 3-7

years)

98

Time Series of the Quasi-Biennial Oscillation (QBO)The QBO is an oscillation of the equatorial zonal wind between easterlies andwesterlies in the tropical stratosphere which has a mean period of 28 to 29 months:

-30

-20

-10

0

10

20

1950 1960 1970 1980 1990 2000 2010

equ

ato

rial

win

d (

m/s

)

Power Spectrum of QBO

101

102

103

104

105

106

107

108

10-2

10-1

100

101

po

wer

frequency (per year)

Observations• Dominant frequency at

close to 2 years• Less power at high

frequencies• Less power at long

time-scales

99

100

Chapter 7: Numerical Analysis7.1 What is Numerical Analysis and why is it done• Numerical analysis involves mathematically analysing numerical methods in order to

predict how they will behave when they are used.• Numerical analysis is important because– Model development by trial and error is very time consuming– We cannot test our models for every possible situation. We need evidence that

they will work for all situations– We gain insight into how numerical methods work and so how to design better ones• This chapter gives some definitions of terms used in numerical analysis and describes

von-Neumann stability analysis

101

7.2 Some Definitions1. Convergence: A finite difference scheme is convergent if the solutions of the

scheme converge to the solutions of the PDE as ∆x and ∆t tend to zero.2. Consistency: A finite difference scheme is consistent with a PDE if the errors in

approximating all of the terms tend to zero as ∆x and/or ∆t tend to zero. (Terms ofthe finite difference scheme are typically analysed using Taylor series.)

3. Order of accuracy: Error ∝ ∆xn (error is O(∆xn)) means scheme is nth orderaccurate. Errors of an nth order scheme converge to zero with order n.

4. Stability: Errors do not tend to infinity for any number of time steps.(a) Conditionally stable - if stable only for a sufficiently small time-step(b) Unconditionally stable - if stable for any time-step(c) Unconditionally unstable - if unstable for any time-step5. Conservation: If, eg mass, energy, potential vorticity are conserved by the PDEs,

are they conserved by the numerical scheme?6. Boundedness: If the initial conditions are bounded between values a and b then a

bounded solution will remain bounded between a and b for all time.7. Monotonicity: Monotonic schemes do not generate oscillations. No new local

maxima or minima are created or amplified.Quiz: https://forms.office.com/r/61QvZu01Lz

102

These are solutions of the diffusion equation solved using FTCS. Based on yourknowledge of the FTCS diffusion scheme and based on the numerical solutions below,which of properties 2,3,4,6,7 apply to these numerical solutions and why?

Consistent Yes (from Taylor series analysis) Order of Accuracy 1stMonotone No, see time-step 40 Bounded No, see time-step 70Stable No (see time-step 150) How do we measure convergence ...

103

7.3 Convergence, errors and order of accuracyIf an analytic solution, f (x, t), to a PDE is known, then errors of a numerical solution,φ (x, t), can be calculated as functions of space and time. Errors over space can besummarised by integrating over space to calculate error norms:

`1 (φ) =∑ j ∆x|φ j− f

(x j)|

∑ j ∆x| f(x j)|

`2 (φ) =

√∑ j ∆x

(φ j− f

(x j))2

√∑ j ∆x f

(x j)2

`∞ (φ) =max |φ j− f

(x j)|

max | f(x j)| .

If a numerical scheme is described as nth order accurate, then these error metricsshould converge to zero at a rate proportional to ∆xn:

`i ∝ ∆xn.

Using these error measures we can find the convergence of implementations ofnumerical methods. But how can we predict if a numerical method will be convergent...

104

7.4 Lax-Equivalence TheoremThe Lax equivalence theorem is the fundamental theorem in the analysis of finitedifference methods for the numerical solution of partial differential equations. It statesthat for a consistent finite difference method for a well-posed linear initial valueproblem, the method is convergent if and only if it is stable:

consistency+ stability⇐⇒ convergenceSo for a finite difference numerical method we need to demonstrate:• finite difference approximations for each of the terms of a PDE is at least first-order

accurate (using, eg Taylor series)• the method is stableHow do we prove stability? ...

105

7.5 Von-Neumann Stability Analysis• Assume that the solution at time level n+1 can be represented as an amplification

factor, A, multiplied by the solution at time-level n:φ (n+1) = Aφ (n)

• The amplification factor will tell us the following about the numerical method:|A|< 1 ∀ k, ∆x stable and damping|A|= 1 ∀ k, ∆x neutrally stable|A|> 1 for any k, ∆x unstable (amplifying)A≥ 0 ∀ k, ∆x non-oscillatory

• How are we going to find A? ...• φ can be expressed as a sum of Fourier modes with wavenumbers k:

φ =∞

∑k=−∞

Akeikx (7.1)

• Consider the stability of a solution for individual wavenumbers• Assume a uniform grid, x = j∆x

• Substitute φ (n)j = Aneik j∆x into the equation for the linear numerical method.

• Rearrange to give an equation for the amplification factor, A(k,∆x,∆t)• For a linear method on a uniform grid, A will not be a function of x or t, because we

are only considering one wavenumber106

7.6 Von-Neumann Stability Analysis of FTCSFTCS for the diffusion equation is:

φ (n+1)j = φ (n)

j +d(φ (n)

j+1−2φ (n)j +φ (n)

j−1)

(7.2)

where d = K∆t/∆x2. Substitute in φ (n)j = Aneik j∆x :

An+1eik j∆x = Aneik j∆x +d(Aneik( j+1)∆x−2Aneik j∆x +Aneik( j−1)∆x) (7.3)Cancel powers of Aneik j∆x and rearrange to find A in terms of d and k∆x:

A = 1+d(eik∆x−2+ e−ik∆x) (7.4)We need to find the magnitude of A so we need to write it down in real and imaginaryform. So substitute eik∆x = cosk∆x+ isink∆x and e−ik∆x = cosk∆x . . .− isink∆x:

A = 1−2d(1− cosk∆x) (7.5)We require the scheme to be stable for all wavenumbers, k. So find for what values ofd the scheme is stable for all values of k∆x.

−1≤ A≤ 1⇐⇒ −1≤ 1−2d(1− cosk∆x)≤ 1⇐⇒ 1≥−1+2d(1− cosk∆x)≥−1 add one to all 3 sides⇐⇒ 2≥ 2d(1− cosk∆x)≥ 0 divide through by 2d (which is always positive)

⇐⇒ 0≤ (1− cosk∆x)≤ 1d

107

Sketch 1− cosk∆x to find itsmaximum and minimum values.From this we can see that . . .0≤ (1− cosk∆x)≤ 2 . . . for any value of k∆x.For stability we need (1− cosk∆x)≤ 1

d for every value of k∆x so we need:

2≤ 1d⇐⇒ d ≤ 1

2⇐⇒ K∆t/∆x2 ≤ 1

2⇐⇒ ∆t ≤ ∆x2

2KIf in addition we do not want the solution to oscillate in sign from one time-step (ienon-oscillatory) to the next then we require:

0≤ A≤ 1⇐⇒ 0≤ 1−2d(1− cosk∆x)≤ 1⇐⇒ 0≥−1+2d(1− cosk∆x)≥−1⇐⇒ 1/2≥ d(1− cosk∆x)≥ 0⇐⇒ 1/(2(1− cosk∆x))≥ d ≥ 0 since 0≤ 1− cosk∆x≤ 2

⇐ d ≤ 14⇐⇒ ∆t ≤ ∆x2

4KSo for large K or for high resolution (small ∆x) a very small time-step must be takenfor FTCS to be stable.Exercises: Find the time-step restrictions for BTCS and CTCS diffusion schemes.

108

Chapter 8: Linear AdvectionAdvection is the process by which everything is moved around by the wind or by oceancurrents. Temperature and concentrations of atmospheric constituents are movedaround by linear advection. Non-linear advection is the movement of the wind by thewind. A reminder of the linear advection equation for component φ , moved around bythe wind, u(x, t), with sources and sinks represented by S(x, t):

∂φ∂ t

+u ·∇φ = S (8.1)

Here again is a solution ofthe linear advectionequation with sources ofpollution from citieswhich is moved around byknown winds. You cansee the video athttps://youtu.be/x9bqzX7pBlA

109

The linear advection equation in one spatial dimension, x, with constant wind, u, andwith no sources or sinks is: ∂φ

∂ t +u∂φ∂x = 0

If u > 0, ∂φ/∂x > 0 =⇒ φ increasing or decreasing? decreasingIf u < 0, ∂φ/∂x < 0 =⇒ φ increasing or decreasing? decreasingAdvection with u > 0 and periodic boundaries:

Analytic solution: For initial condition φ0(x), the solution at t is φ(x, t) = φ0(x−ut).Exercise: You can check that this is the analytic solution. (Hint: define X = x−ut,calculate ∂X

∂x and ∂X∂ t and use the chain rule, ∂φ

∂x = ∂φ∂X

∂X∂x and , ∂φ

∂ t = ∂φ∂X

∂X∂ t )

110

8.1 Eulerian Finite Difference Advection Schemes8.1.1 Forward in Time, Backward in Space (FTBS)To solve ∂φ

∂ t +u∂φ∂x = 0 numerically:

• Divide space into nx equal intervals of size∆x so that x j = j∆x for j = 0,1, ...,nx

φ j+1

φ jφ φ j−1

x1 x2∆x

x j x j+1 xx j−1x0

• Divide time into time steps ∆t(so that tn = n∆t, n = 0,1,2, ....)• Define φ (n)

j = φ(x j, tn).

• Approximate ∂φ/∂ t at x j, tn by a forward difference: ∂φ (n)j

∂ t =φ (n+1)

j −φ (n)j

∆t

• Approximate ∂φ/∂x at x j, tn by a backward difference: ∂φ (n)j

∂x =φ (n)

j −φ (n)j−1

∆x• Substitute these into the linear advection equation at x j, tn to give FTBS:

φ (n+1)j −φ (n)

j∆t +u

φ (n)j −φ (n)

j−1∆x = 0 (8.2)

• This can be re-arranged to get φ (n+1)j on the LHS and all other terms on the RHS so

that we can calculate φ at the new time step at all locations based on values atprevious time steps. Also substitute in the Courant number, c = u∆t/∆x:

φ (n+1)j = φ (n)

j − c(φ (n)j −φ (n)

j−1)

(8.3)111

8.1.2 Accuracy of FTBSAdvection of an initial profile once around the domain (periodic boundaries, c = 0.2,100 space intervals, 500 time steps)

• Why is it so inaccurate? First-order accurate in space and time• Why is it so diffusive? Find the leading error term of the Taylor series of the

approximation for ∂φ/∂x = φ ′ in FTBS.

φ j−1 ≈ φ j−∆xφ ′j +∆x2

2!φ ′′j .

⇒ φ ′j ≈φ j−φ j−1

∆x+

∆x2

φ ′′j .

The error is proportional to φ ′′ = ∂ 2φ/∂x2 which is the spatial term from the ...diffusion ... equation. Adding this error to the advection equation diffuses thesolution.

112

8.2 Centred in Time, Centred in Space (CTCS) and FTCS

To solve ∂φ∂ t

+u∂φ∂x

= 0 numerically:• Approximate ∂φ/∂ t at x j, tn by a centred

difference, using φ (n−1)j and φ (n+1)

j :∂φ (n)

j∂ t =

φ (n+1)j −φ (n−1)

j2∆t

φ j+1

φ jφ φ j−1

x1 x2∆x

x j x j+1 xx j−1x0

• Approximate ∂φ/∂x at x j, tn by a centred

difference: ∂φ (n)j

∂x =φ (n)

j+1−φ nj−1

2∆x• Substitute these into the linear advection equation at x j, tn and re-arrange to get

φ (n+1)j on the LHS and all other terms on the RHS. Also substitute in the Courant

number, c = u∆t/∆x:

φ (n+1)j = φ (n−1)

j − c(φ (n)

j+1−φ (n)j−1)

. (8.4)• This is a three-time-level formula (it involves values of φ at times tn−1, tn and tn+1.

To start the simulation, values of φ are needed at times t0 and t1. However, onlyφ(x, t0) is available. So another scheme (such as FTCS) must be used to obtainφ (1) = φ(x, t1):

FTCS: φ (n+1)j = φ (n)

j −c2(φ (n)

j+1−φ (n)j−1)

(8.5)

113

8.2.1 Accuracy of CTCSAdvection of an initial profile once around the domain (periodic boundaries, c = 0.2,100 space intervals, 500 time steps)

• What is the order of accuracy of CTCS? Second-order in space and time• Are FTBS and CTCS explicit or implicit? Second-order in space and time• Why all the oscillations? ... We will come back to this• Take the quiz on this chapter

114

8.3 Some Explicit Eulerian Advection SchemesSchemes to solve the 1d advection equation, ∂φ

∂ t +u∂φ∂x = 0 assuming x = j ∆x, t = n ∆t

and constant ∆x and ∆t and using Courant number c = u∆t∆x .

Order Stabilitytime space constraint

FTBS: φ (n+1)j = φ (n)

j − c(

φ (n)j −φ (n)

j−1

)1 1 0≤ c≤ 1

FTCS: φ (n+1)j = φ (n)

j − c2

(φ (n)

j+1−φ (n)j−1

)1 2 unstable ∀c

CTCS: φ (n+1)j = φ (n−1)

j − c(

φ (n)j+1−φ (n)

j−1

)2 2 −1≤ c≤ 1

FTFS: φ (n+1)j = φ (n)

j − c(

φ (n)j+1−φ (n)

j

)1 1 −1≤ c≤ 0

CTBS: φ (n+1)j = φ (n−1)

j −2c(

φ (n)j −φ (n)

j−1

)2 1 unstable ∀c

CTFS φ (n+1)j = φ (n−1)

j −2c(

φ (n)j+1−φ (n)

j

)2 1 unstable ∀c

115

116

Chapter 9: Numerical Analysis of Advection SchemesFor analysing diffusion schemes in chapter 7, we used:• Taylor series• von-Neumann stability analysis• Lax equivalence theorem.For analysing advection schemes we will also consider:• the domain of dependence• conservation• diffusion errors (section 8.1.2)• dispersion errors (chapter 11)

117

9.1 Domain of DependenceThe temperature at any point depends on the temperature upwind at a previous time.The concentration of a pollutant depends on the concentration upwind.The domain of dependence of the solution of a PDE at position x and at time t is theset of points at a previous time that influence the solution at position x and at time t.If the wind is blowing from the west at 10 miles/hour, the following points are in thedomain of dependence of Reading at 10am:• 10 miles to the west at 9am• 15 miles to the west at ... 8:30am• 20 miles to the west at 8am

118

9.1.1 Domain of Dependence of a PDEThe domain of dependence of the solution of a PDE at position x and at time t is theset of points at a previous time that influence the solution at position x and at time t.Let us consider the 1D linear advection equation for dependent variable φ andadvecting velocity u:

φt +uφx = 0Remember the analytic solution:

φ(x, t) = φ(x−ut,0)

Draw the domain of dependence ofx = 12m, t = 8s assuming u = 1.5m/s:

4 6 8 10 1220

0

2

4

6

8

x

t

For exact linear advection, thedomain of dependence of point(x, t) is the straight line of slope1/u through (x, t)

119

9.1.2 Domain of Dependence of a Numerical MethodThe domain of dependence of point j, time step n consists of all the previous pointsthat influence the solution at point j, time step n. For example for FTBS:

φ (n+1)j = φ (n)

j − c(φ (n)j −φ (n)

j−1)

φ (n+1)j depends on φ (n)

j and φ (n)j−1 and these in turn depend on φ (n−1)

j , φ (n−1)j−1 and

φ (n−1)j−2 etc. The domain of dependence of φ (n+1)

j can be drawn on a grid of points:t

n−1

n−2

j+1jj−1j−2j−3x

n+1

n

Also draw lines for the analytic solution for c =−1,0,1,2. (Remember u = c∆x/∆t andthe lines have slope 1/u)

120

9.1.3 Courant-Friedrichs-Lewy (CFL) criterion:• The domain of dependence of the numerical solution should include (surround) the

domain of dependence of the original PDE.• The CFL criterion is necessary but not sufficient for stability of the numerical method.What does this tell us about the stability of FTBS?The numerical domain of dependence contains the physical domain of dependencewhen 0≤ c≤ 1 so FTBS is unstable for c > 1 and c < 0.

121

9.1.4 The Domain of Dependence of the CTCS Scheme

CTCS: φ (n+1)j = φ (n−1)

j − c(φ (n)j+1−φ (n)

j−1)

Draw the domain of dependence of φ (n+1)j for CTCS and for the analytic solution for

c =±1.t

n−1

n−2

j+1jj−1j−2j−3x

n+1

n

For what values of c will CTCS be unstable? c <−1 and c > 1Except at the initial time, the solution is found on two sets of points that are notcoupled. The solution can oscillate between two unrelated solutions. This is amanifestation of the computational mode of CTCS.The domain of dependence and the CFL criterion can tell us when some schemes areunstable. How about proving stability? ...

122

9.2 Von-Neumann Stability Analysis of Advection Schemes9.2.1 Von-Neumann Stability Analysis of FTBSFTBS for linear advection is:

φ (n+1)j = φ (n)

j − c(φ (n)

j −φ (n)j−1)

(9.1)

Substitute in φ (n)j = Aneik j∆x :

An+1eik j∆x = Aneik j∆x− cAn(eik j∆x− eik( j−1)∆x) (9.2)Cancel powers of Aneik j∆x and rearrange to find A in terms of c and k∆x:

A = 1− c(1− e−ik∆x) . (9.3)We need to find the magnitude of A so write down real and imaginary parts usinge−ik∆x = cosk∆x− isink∆x:

A = 1− c(1− cosk∆x)− icsink∆x . (9.4)and calculate |A|2 = AA∗ (A multiplied by its complex conjugate):

|A|2 = 1−2c(1− cosk∆x)+ c2(1−2cosk∆x+ cos2 k∆x)+ c2 sin2 k∆x

=⇒ |A|2 = 1−2c(1− c)(1− cosk∆x)

We need to find the values of c for which |A| ≤ 1:

123

|A| ≤ 1 ⇐⇒ |A|2 ≤ 1⇐⇒ 1−2c(1− c)(1− cosk∆x)≤ 1⇐⇒ c(1− c)(1− cosk∆x)≥ 0

We know that 1− cosk∆x≥ 0 for all k∆x so FTBS is stable whenc(1− c)≥ 0 ⇐⇒ 0≤ c≤ 1.We have proved that FTBS is unstable if u < 0 or if u∆t

∆x > 1.We will now define the Upwind scheme FTBS when u≥ 0

FTFS when u < 0which is first order accurate in space and time, conditionally stable and damping.ExerciseWrite down the FTFS advection scheme and find its stability limits.

124

9.2.2 What should A be for real linear advection?For initial conditions consisting of a single Fourier mode: φ(x,0) = Akeikx, the solutionat time ∆t is:

φ(x,∆t) = Akeik(x−u∆t) .

This can be represented as a factor times the initial conditions:φ(x,∆t) = e−iku∆t Akeikx .

Therefore the amplification factor is A = e−iku∆t which has |A|= 1 and so, under theinfluence of linear advection waves should not amplify or decay.

125

9.2.3 Von Neumann Stability Analysis of CTCS

The amplification factor, A, is found by substituting φ (n)j = Aneik j∆x into CTCS:

φ (n+1)j = φ (n−1)

j − c(φ (n)

j+1−φ (n)j−1). (9.5)

This gives: An+1eik j∆x−An−1eik j∆x + c(Aneik( j+1)∆x−Aneik( j−1)∆x) = 0

=⇒ A2 +(2icsink∆x)A−1 = 0

=⇒ A =−icsink∆x±√

1− c2 sin2 k∆x

Case (i) |c| ≤ 1 =⇒ c2 sin2 k∆x≤ 1

=⇒ |A|2 = 1− c2 sin2 k∆x+ c2 sin2 k∆x = 1

=⇒ The solution is stable and not dampingCase (ii) |c|> 1 =⇒ c2 sin2 k∆x > 1 for some k∆x

=⇒ |A|2 = (csink∆x±√

c2 sin2 k∆x−1)2

=⇒ At least one of the roots has |A|> 1. The solution is unstable126

So CTCS is conditionally stable: it is stable for |c| ≤ 1Regardless of c, there are always two possible values of A. This means that there willalways be two possible solutions; a realistic solution (the physical mode) and theun-realistic, oscilating solution; the spurious computational mode.

127

9.3 ConservationFor a property, φ , advected with velocity, u: (φt +uφx = 0) we can show that the totalmass of φ is conserved. Define the total mass to be

M =∫ 1

0 φ dx

and assume that φ has periodic boundary conditions so that φ(0, t) = φ(1, t) ∀t. Then:dMdt = d

dt

(∫ 10 φdx

)=∫ 1

0dφdt dx=−∫ 1

0 udφdx dx =−u

∫ 10 dφ =−u [φ ]10 = 0

since φ(0, t) = φ(1, t). Therefore M is conserved.Is M conserved by a numerical scheme? Consider FTBS advection:

φ (n+1)j = φ (n)

j − c(φ (n)

j −φ (n)j−1).

From this we can calculate M(n+1) as a function of M(n):M(n+1) =

nx

∑j=1

∆xφ (n+1)j = ∆x

nx

∑j=1

(φ (n)

j − c(φ (n)

j −φ (n)j−1))

= M(n)− c∆x( nx

∑j=1

φ (n)j −

nx

∑j=1

φ (n)j−1

)

= M(n)− c∆x( nx

∑j=1

φ (n)j −

nx−1

∑j=0

φ (n)j

)= M(n)− c∆x

(φ (n)

nx −φ (n)0

)= M(n)

due to the periodic boundaries, φ (n)nx = φ (n)

0 . ∴ M(n+1) = M(n): mass is conserved.128

9.3.1 Conservation of higher momentsWe can show that the variance of φ is also conserved under linear advection. Thevariance is:

V =∫ 1

0φ 2dx−M2.

Then we can calculate the rate of change of variance under linear advection, assumingthat Mt = 0:

dVdt

=∫ 1

0

dφ 2

dtdx =

∫ 1

02φ

dφdt

dx =−∫ 1

02φu

dφdx

dx =−2u∫ 1

0φdφ =−u

[φ 2]1

0 = 0.

Is V conserved by a numerical scheme? Consider FTBS advection:φ (n+1)

j = φ (n)j − c

(φ (n)

j −φ (n)j−1).

From this we can calculate V (n+1) as a function of V (n) (ignoring M for brevity):

V (n+1) =nx

∑j=1

∆x(

φ (n+1)j

)2= ∆x

nx

∑j=1

(φ (n)

j − c(φ (n)

j −φ (n)j−1))2

= ∆xnx

∑j=1

(φ (n)

j

)2−2c∆x

nx

∑j=1

φ (n)j φ (n)

j +2c∆xnx

∑j=1

φ (n)j φ (n)

j−1 + c2∆xnx

∑j=1

(φ (n)

j −φ (n)j−1

)2

=V (n)−2c(1− c)

(V (n)−∆x

nx

∑j=1

φ (n)j φ (n)

j−1

)

The term multiplying 2c(1− c) is always greater than zero and so the variance alwaysdecreases when using FTBS.

129

Revision: Remind yourself of the definitions of terms used in numerical analysis fromsection 7.2.Exercise: Look back at the solutions of FTBS and CTCS presented in chapter 8.Which of properties 1-7 might these solutions have?Take this quiz which examines solution of the advection equation using FTCS.9.4 Exercises: Analysis of Advection Schemes (answers on

blackboard)1. Derive expressions for the amplification factors for the following advection schemes:(a) FTCS(b) BTCS(c) CTBS2. Which of the above schemes suffer from a spurious computational mode?3. Sketch of the domain of dependence for these schemes4. For FTCS and BTCS(a) Find for what Courant numbers the schemes are stable.(b) Are the schemes damping, amplifying or neutral?5. Determine if mass is conserved by CTCS

130

Chapter 10: More Advection SchemesThe advection schemes defined so far have some problems:• The first-order scheme is diffusive• The second-order scheme generates grid-scale oscillations• They both have time-step restrictionsThis chapter defines advection schemes that overcome some of these problems. Noneed to memorise. There are two fundamental limits to be considered:The CFL criterion: The domain of dependence of the numerical solution mustcontain the domain of dependence of the original PDE.The implication is that most explicit schemes have time step restrictions.Godunov’s theorem:Linear numerical schemes for solving the advection equation that do not generatingnew extrema (monotone scheme), can be at most first-order accurate.A linear numerical method is one where the next time step values are a linearcombination of values at the previous time step(s). (Logical for solving a linear PDE).All the schemes so far have been linear (including implicit schemes). HoweverGodunov’s theorem says that if they are second-order accurate or higher, they willgenerate oscillations.

131

10.1 Artificial Diffusion to Remove Spurious OscillationsNumerical schemes are designed not to produce spurious oscillations. However, once aforecasting model is put together, often spurious oscillations are generated. These canbe removed by adding artificial diffusion to the equations. For example, the linearadvection equation with diffusion:

∂φ∂ t +u •∇φ −K∇2φ = 0 (10.1)

or in one dimension: ∂φ∂ t +u∂φ

∂x −K ∂ 2φ∂x2 = 0 (10.2)

• This is dampens spurious oscillations and real features• It is only (but frequently) used as a last resort.• More scale-selective filtering can be achieved using ∇4 rather than ∇2:

∂φ∂ t +u •∇φ +K∇4φ = 0 (10.3)

Stablity limits of the advection-diffusion schemeThe advection diffusion scheme:

φ (n+1)j −φ (n−1)

j + c(φ (n)j+1−φ (n)

j−1)−2d(φ (n−1)j+1 −2φ (n−1)

j +φ (n−1)j−1 ) = 0 (10.4)

where c = u∆t/∆x and the diffusion number, d = K∆t/∆x2 has the stability constraintc2 +4d ≤ 1. The algebra is complicated and does not need to be reproduced.Questions: How much diffusion is needed to control oscillations? How much does thisreduce accuracy?

132

10.2 Semi-Lagrangian AdvectionCFL criterion: The domain of dependence of the numerical solution must contain thedomain of dependence of the original PDE.• So to allow long time-steps, construct the numerical domain of dependence to

contain the physical domain of dependence.• Recall the linear advection equation:

∂φ∂ t +u∂φ

∂x = 0• and its analytic solution: φ(x, t) = φ(x−ut,0)

x j

x k+1x k

tn+1

x jd x

u

tn

• Semi-Lagrangian advection is defined from this:φ(x j, tn+1) = φ(x j−u∆t, tn) = φ(x jd, tn)x jd = x j−u∆t is the departure point of point x j.• So interpolate φ from known points onto x jd.• First find grid index k such that xk ≤ x jd ≤ xk+1: (Use floor(x) meaning the integer≤ x)k = floor(x jd/∆x) =floor((x j−u∆t)/∆x) =floor( j− c)• Interpolate from xk−1, xk, xk+1, ... onto x jd (for example using cubic-Lagrange

interpolation)Exercise: Find β = (x jd− xk)/∆x as a function of only j, k and c given x j = j∆x andc = u∆t/∆x. β = j− k− c

133

Advantage: Explicit scheme without Courant number restrictionProblem: The advected quantity is not conservedOrder of Accuracy: Ignoring errors in calculating the departure point,semi-Lagrangian should achieve an order of accuracy of O[(∆x)p+1/∆t] where p is theorder of the interpolation (Durran, 2010).Cubic Lagrange interpolation (no need to memorise)If f is known at n points then a polynomial of degree n−1 can be fit to estimate f . Ifn = 4 this is cubic Lagrange interpolation. If f is known at points fk−1, fk, fk+1 andfk+2 a distance ∆x apart then f at position x = xk +β∆x is estimated as:

f (x) =−16

β (1−β )(2−β ) fk−1 +12(1+β )(1−β )(2−β ) fk

+12(1+β )β (2−β ) fk+1−

16(1+β )β (1−β ) fk+2

134

10.3 Implicit Time Stepping for AdvectionThis is only very rarely used in atmospheric modelling because the solutions loseaccuracy fast for c > 1 (unlike semi-Lagrangian). Implicit versions of the finitedifference schemes defined in chapter 8 can be derived:

Order Stabilitytime space constraint

BTBS φ (n+1)j = φ (n)

j − c(

φ (n+1)j −φ (n+1)

j−1

)1 1 c≥ 0

BTCS φ (n+1)j = φ (n)

j − c2

(φ (n+1)

j+1 −φ (n+1)j−1

)1 2 stable ∀c

CNBS φ (n+1)j = φ (n)

j − c(

φ (n)j +φ (n+1)

j2 − φ (n)

j−1+φ (n+1)j−1

2

)2 1 c≥ 0

CNCS φ (n+1)j = φ (n)

j − c2

(φ (n)

j+1+φ (n+1)j+1

2 − φ (n)j−1+φ (n+1)

j−12

)2 2 stable ∀c

CN

means “Crank-Nicolson” which is the same as trapezoidal implicit.

135

10.4 The Finite Volume MethodVolume integrals of predicted variables (eg Φ = ρφ where ρ is density) are calculatedon small volumes (cells with volume V ) by calculating fluxes over cell walls/edges:

1 3

2

4

VΦf3

f1

f2

f4

∂V Φ∂ t

=4

∑i=1

fi

Where fi is the flux of Φ through edge i.The finite volume method can also be derived bydiscretising the flux form of the advection equation

∂Φ∂ t

+∇ · (u Φ) = 0where ρ is density.

The flux (or conservative) form can be discretised using Gauss’s divergence theorem:

∇ ·uΦ≈ 1V

∫

V∇ ·uΦ dV =

1V

∫

SΦu ·dS =

1V ∑

ifi

where volume V has surface S, dS is the outward pointing normal to surface S andfi = Φu ·dS is the flux over edge i.Conservation: the total mass of Φ is conserved because exactly what leaves one cellenters another.There are numerous finite volume methods, defined by how to estimate fi fromsurrounding Φ and u values.

136

10.4.1 The Finite Volume Method in one dimension, for constant u

For constant u: f j+1/2= uφ j+1/2

.φ j−1/2 φ j+1/2

φ j−2 φ j−1 φ j+1 φ j+2φ j

So to solve: ∂φ∂ t =−u∂φ

∂x we use∂φ j

∂ t=−

uφ j+1/2−uφ j−1/2

∆xIt remains to estimate φ j±1/2

fromsurrounding values of φ j.Exercise Demonstrate the following:

• If we choose centred in time for ∂φ j∂ t and

φ j+1/2= 1

2

(φ j +φ j+1

)we recover CTCS

• If we choose forward in time for ∂φ j∂ t and

φ j+1/2= φ j we recover FTBS

Solution on notes on blackboard.Using the finite volume method, we can solve equations on arbitrary meshes.

137

10.5 Lax-Wendroff and Warming and BeamLax-Wendroff and Warming and Beam are examples of finite volume schemes. Thegeneric finite volume method in one dimension is

φ (n+1)j −φ (n)

j

∆t=−

uφ j+1/2−uφ j−1/2

∆x

Lax-Wendroff: φ j+1/2= 1/2(1+ c)φ (n)

j + 1/2(1− c)φ (n)j+1

Warming and Beam: φ j+1/2= 1/2(3− c)φ (n)

j − 1/2(1− c)φ (n)j−1

Notes:• They are both second-order accurate in space and time.• They are NOT monotonic or bounded• Lax-Wendroff is stable for |c| ≤ 1, Warming and Beam is stable for c ∈ [0,2]

138

Lax-Wendroff Derivation (optional)

φ j+1/2is estimated to be the average value that

passes through position x j+1/2between time-steps

n and n+1. Therefore we need to calculate theaverage value of φ between positions x j+1/2

and adistance u∆t upstream of x j+1/2

.

φ j

x j x j+1/2x j+1

u∆t

φ j+1

In order to calculate the average, we assume that φ varies linearly between x j andx j+1. Then we can calculate φ at x = x j+1/2− 1/2u∆t .

This could be written φ (n+1/2)j+1/2

= φ (n) (x j+1/2− 12u∆t

)

Using the Courant number, c = u∆t/∆x, this gives:φ(

x j+1/2− 1/2u∆t

)= φ

(1/2(x j + x j+1− c∆x

))

So we need to linearly interpolate φ (n)j and φ (n)

j+1 onto position 1/2(x j + x j+1− c∆x).Define interpolation parameter β =

(1/2(x j + x j+1− c∆x)− x j)/∆x = 1/2(1− c). Then

⇒ φ(1/2(x j + x j+1− c∆x

))= (1−β )φ (n)

j +βφ (n)j+1 =

1/2(1+ c)φ (n)j + 1/2(1− c)φ (n)

j+1

139

Lax-Wendroff Warming and Beam

φ j+1/2= 1/2(1+ c)φ (n)

j + 1/2(1− c)φ (n)j+1 φ j+1/2

= 1/2(3− c)φ (n)j − 1/2(1− c)φ (n)

j−1

Advection of a top-hat profile to the right using c = 0.2, 100 points

Smooth ahead of the discontinuity Smooth behind the discontinuityGodunov’s theorem: Linear numerical schemes having the property of not generatingnew extrema (monotone scheme), can be at most first-order accurate.Before defining non-linear, monotone schemes, let us define a measure ofmonotonicity.

140

10.6 Total VariationTotal variation, TV, is used to measure the number and strength of oscillations:

TV =nx−1

∑j=0|φ j+1−φ j|

Calculate the total variation of these functions and submit answers athttps://forms.office.com/r/XXenK9KRh6:

0 2 4 6 8 100

1

2

3

4

5

6

7

φ

(a)

0 2 4 6 8 100

1

2

3

4

5

6

7

φ

(b)

0 2 4 6 8 100

1

2

3

4

5

6

7

φ

(c)

0 2 4 6 8 100

1

2

3

4

5

6

7

φ

(d)

0 2 4 6 8 100

1

2

3

4

5

6

7

φ

(e)

0 2 4 6 8 100

1

2

3

4

5

6

7

φ

(f)

• A total variation diminishing (TVD) scheme has TV (n+1) ≤ TV (n).Question: Why is total variation used rather than boundedness to measure thegeneration of spurious oscillations?

141

10.7 Total Variation Diminishing (TVD) schemes• A total variation diminishing (TVD) scheme has TV (n+1) ≤ TV (n)

• First-order upwind is the only linear TVD scheme.• We start with the linear advection equation discretised using the finite volume

method:φ (n+1)

j −φ (n)j

∆t=−

uφ j+1/2−uφ j−1/2

∆x. (10.5)

• φ j+1/2, is a weighted average of a high order flux (φH) and a low order flux (φL):

φ j+1/2= Ψ j+1/2

φH +(1−Ψ j+1/2) φL

where Ψ is a limiter function.• Ψ needs to be close to one where the solution is smooth and close to zero where the

solution changes rapidly

– Lax-Wendroff can be the high-order flux: φH = 1/2(1+ c)φ j +1/2(1− c)φ j+1

– First-order upwind must be the low-order flux: φL =

φ j if u≥ 0φ j+1 otherwise

• So what should Ψ be?

142

10.7.1 Limiter functionsThe limiter function, Ψ, is based on the ratio of the upwind gradient to the localgradient which for u > 0 is:

r j+1/2=

φ j−φ j−1

φ j+1−φ jFor a second-order accurate, TVD scheme, Ψ must be in the unshaded region of theSweby diagram:For second-order accuracy

Ψ(r) ∈[r,1] if r ≤ 1[1,r] if r ≥ 1

For total variation diminishing

Ψ(r) ≤ min(2, 2r) 1 0 1 2 3 4 5r

0

1

2

3

4Lax-WendroffWarming and BeamTVD limits

For example, the Van Leer Limiter function is Ψ(r) = (r+ |r|)/(1+ |r|)There are many other limiter functions that give TVD schemes. See, eghttps://en.wikipedia.org/wiki/Flux_limiter

Consider: How should r be defined if φ j+1−φ j = 0?

143

144

Chapter 11: Waves, Dispersion and Dispersion Errors11.1 Background on WavesA travelling wave can be described by the equation

y = asin(kx−ωt) (11.1)

y(x, t) is the height of the wave atposition x, time t

a is the amplitude of the wavek is the wavenumber (number of

whole waves between 0 and 2π)ω is the angular wave frequency

(the number of oscillations intime 2π at a fixed point)

u = ∂x∂ t is the wave speed (the rate

of change of x along the curve fora fixed y)

0 /2 3 /2 21.00

0.75

0.50

0.25

0.00

0.25

0.50

0.75

1.00 a = 1k = 1

= 1t = 0.0

a sin(kx t)

145

How do variations in k and ω influence the wave and its propagation according toy = asin(kx−ωt)

Quiz: https://forms.office.com/r/DBBCSp1rcyWhat are the relationships between wave length, λ , the wavefrequency, ω , wavenumber, k and wave speed, u (select all thatapply)?(a) k = 2πλ (b) λ = 2π/k X (c) λ = 2πk(d) u = ω/k X (f) u = k/ω (e) u = ωk

146

11.2 DispersionDispersion occurs when a combination of waves of different wavenumbers propagate atdifferent speeds.

• u = ω1k1

= ω2k2

so wave of both wavelengths propagate at the same speed• No dispersion occurs – the function does not change shape

147

• u1 =ω1k1

= 10ω2k2

so the short wavelength waves are much slower than the long waves• Dispersion occurs – the function changes shape

148

• u1 =ω1k1

= 13

ω2k2

so the short wavelength waves travel more quickly• Dispersion occurs – the function changes shape

149

For a function to propagate without changing shape, all of the Fourier modesmust propagate at the same speed:This shows the propagation of a Gaussian and propagation of the first 9 Fourier modes.

150

Propagation of a Gaussian with high wavenumber waves propagating moreslowly

The original Gaussian function changes shape

151

11.3 Some Animations and Videos which demonstrateDispersion

• Some description and animations from Dr. Dan Russell, Grad. Prog. Acoustics, PennState:http://www.acs.psu.edu/drussell/Demos/Dispersion/dispersion.html

• A video of the wake of a motor-boat from JNHeyman (41 seconds):https://www.youtube.com/watch?v=lWi_KpBy8kU

152

11.4 Phase speed of linear advectionWe can predict if advection schemes have dispersion errors. We know:• the one-dimensional linear advection equation:

φt +uφx = 0

• its analytic solution:φ(x, t) = φ(x−ut,0)

• under the action of the linear advection equation, waves propagate with speed u.Question: Is linear advection dispersive? No (since u does not depend on k)• If we have an initial condition consisting of a single Fourier mode, eikx, then the

solution at time ∆t is eik(x−u∆t)

• So for real linear advection with phase speed u, the amplification factor ise−iku∆t

• Write the amplification factor of a numerical method as A = re−iθ for wavenumber k.Alternatively, for consistency with real linear advection, we can write theamplification factor as A = re−ikun∆t where un is the phase speed of the numericalwave. Equating the two equations for A gives un = θ/(k∆t)• If un depends on k then there will be numerical dispersion

153

11.5 Phase speed and dispersion errors of CTCSThe amplification factor of CTCS is

A =−icsink∆x±√

1− c2 sin2 k∆xwhere c = u∆t/∆x. Therefore the phase-speed is:

un =1

k∆ttan−1

(csink∆x

±√

1− c2 sin2 k∆x

).

This can be simplified by substituting in u∆t = c∆x and sinα = csink∆x:un

u=± α

ck∆x.

There are two possiblephase-speeds for each mode.These can be plotted against k∆xto find out how waves propagatewhen advected by CTCS. A plot ofun/u against k∆x (or ω = kuagainst k) is called a dispersionrelation. This is the dispersionrelation for CTCS for c = 0.4:

0 π/4 π/2 3π/4 π

k∆x

1.0

0.5

0.0

0.5

1.0

un/u

154

The dispersion relation for CTCS for c = 0.4 plotted as ω/u against k for∆x = 1

0 π/4 π/2 3π/4 π

k∆x

1.5

1.0

0.5

0.0

0.5

1.0

1.5

ωn/u

numericaltrue

What do these tell us aboutCTCS?• There are two possible solutions:– the physical mode (up/u≥ 0)– the computational mode

(up/u < 0)• The computational mode

propagates in the wrong direction• All waves propagate too slowly• For the physical mode, when

waves are well resolved (small k orsmall ∆x), they propagate atnearly the correct speed (this iswhy it is called the physical mode)• Grid-scale waves (k∆x = π) do not

propagate

155

11.6 What does k∆x mean?• Grid fine relative to waves

– Waves well resolved• Numerical methods accurate

• Grid scale waves havek∆x = π• Numerical methods lose

accuracy

156

11.7 What dispersion relations mean for numerical methodsThese are the results of solving the linear advection equation using two finite volumemethods:

Lax-Wendroff Warming and BeamAdvection of a top-hat profile to the right using c = 0.2, 100 points

157

These are the dispersion relations for the two schemes. Which dispersion relation isrelated to which scheme and why?

0 π/4 π/2 3π/4 πk∆x

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

ω/u

Scheme aScheme btrue (i) a is Lax-Wendroff

b is Warming and Beam(ii) a is Warming and Beam

b is Lax-Wendroff

• Scheme b corresponds to Lax-Wendroff because poorly resolved waves propagate tooslowly so oscillations behind the discontinuities where high wavenumber modesassociated with the discontinuity lag behind the longer wavelength modes.• Scheme a corresponds to Warming and Beam because the poorly resolved waves

propagate too quickly, leading to oscillations ahead of the discontinuity.

158