Embed Size (px)
Citation preview
Modeling and Simulation with ODE (MSE)
Massimo Fornasier
Fakultat fur MathematikTechnische Universitat Munchen
[email protected]://www-m15.ma.tum.de/
Johann Radon Institute (RICAM)Osterreichische Akademie der Wissenschaften
[email protected]://hdsparse.ricam.oeaw.ac.at/
Department of MathematicsTechnische Universitat Munchen
Lecture 11
Why are you here?In this course we shall provides the rudiments (the basics!) of thenumerical solution of differential equations, i.e, equations involvingthe derivatives of a (possibly) multivariate functionu : Ω ⊂ Rd → R:
F (x1, . . . , xd , u(x),∂
∂x1u(x), . . . ,
∂
∂xdu(x),
∂2
∂x1∂x1,
∂2
∂x1∂x2, . . . ,
∂2
∂x1∂xd, . . . ) = 0.
Often one of the variables indicates the time, e.g., x1 = t, and theequation governs a phenomena which evolves in time:
∂u
∂t+ F (x , u,Du,D2u, . . . ) = 0.
Differential equations were for thefirst time formulated in the 17th
century by I. Newton (1671) and byG.W. Leibniz (1676).
Just old boring stuff?
Yes, of course!?But also NO, because the history did not stop ...
used by Euler, Maxwell, Boltzmann, Navier, Stokes, Einstein,Prandtl, Schrodinger, Pauli, Dirac,Turing, Black&Scholes ...........
I provide an important modeling tool for the physical sciences,theoretical chemistry, biology, socio-economic sciences,engineering sciences ....
I accessible to numerical simulations
For instance ...Fourier in Theorie analytique de la chaleur (1822) formulated and solvedanalytically the equation governing the heat conduction in homogenous media:
∂u
∂t(t, x)− α
(∂2u
∂x21
(t, x) +∂2u
∂x22
(t, x) +∂2u
∂x23
(t, x)
)= 0.
A similar equation is studied by Fornasier and March in 2007 to recolor ancientItalian frescoes from fragments:
See Restoration of color images by vector valued BV functions and variationalcalculus (M. Fornasier and R. March), SIAM J. Appl. Math., Vol. 68 No. 2,2007, pp. 437-460.
http://www.ricam.oeaw.ac.at/people/page/fornasier/SIAM_Fornasier_
March_067187.pdf
Illustrations of differential equations and some of their(modern) applications
Reference: P. A. Markowich, Applied Partial Differential Equations - A Visual
Approach, Springer, 2006
Slides: http:
//homepage.univie.ac.at/peter.markowich/galleries/vortrag.pdf
I Gas dynamics Boltzmann equationI Fluid/gas dynamics: Navier-Stokes/Euler EquationsI Kinetic modeling of granular flowsI Chemotaxis and formation of biological patternsI Semiconductor modelingI Free boundary problems and interfacesI Reaction-diffusion equationsI Monge-Kantorovich optimal transportationI Wave equationsI Digital image processingI Socio-Economic modeling
From partial differential equations to ordinary differentialequations
As we shall see in the course of this lecture, several partialdifferential equations (PDE) can be reduced, after discretization ofthe space variable, to a system of ordinary differential equations(ODE):
∂u
∂t+ F (t, x , u,Du,D2u, . . . ) = 0⇒ u′h(t) + Fh(u(t)) = 0,
where h is a space discretization parameter.
The first part of this course is dedicated to the use of ODE formodeling physical and engineering problems, while the second partof the course is dedicated to their numerical solution by some ofthe most relevant numerical methods.
Program of the course in a nutshell
Modeling by ODE:
I First-order ODEs
I Second-order linear ODEs
I Higher-oder linear ODEs
I Systems of ODEs
Numerical methods for ODE:
I forward Euler method
I theta method
I Adams methods and more general multi-step methods
I Runge-Kutta methods
I stability and stiffness
I Backward differentiation formulae
References for this course
Besides the slides provided after the lecture online ...Books:
K11 E. Kreyszig, Advanced Engineering Mathematics, 10th Edition, JohnWiley & Sons, Inc., 2011
I09 A. Iserles, A First Course in the Numerical Analysis of DifferentialEquations (2nd ed.), Cambridge University Press, 2009.
QSG10 A. Quarteroni, F. Saleri, P. Gervasio, Scientific Computing with Matlaband Octave (3rd ed.), Springer, 2010.
Lecture notes:
F13 M. Fornasier, Numerik der gewohnlichen Differentialgleichungen,http://www-m3.ma.tum.de/foswiki/pub/M3/NumerikDG13/WebHome/
Numerik_2013-08-04.pdf (password required)
All you find at the webpage of the course:
http://www-m15.ma.tum.de/Allgemeines/ModelingSimulation
References do not mean ...
How does a book work? ...
References do not mean ...
that you do not come to the lecture and think to be smart ...
’cause I will NOT let you escape ...
At the end there will be the exam ... and I’ll be waiting for youthere (REMEMBER IT!) ...you better show up! (just a friendly - Italian - invitation not toskip the lectures :-) !)
Just the classical email from the “smart” student ...
Sehr geehrter Herr Prof. Fornasier,mein Name ist [SmartStudentName]. Gestern habe ich die Prufung in”Numerisches Programmieren II (CSE)” geschrieben. Ich habe extrem viel furdiese Prufung gelernt, da ich mich vor allem in PDEs spezialisieren mochte undim nachsten Semester auch “Numerik der PDEs” bei Prof.[ReallySmartProfName] horen werde. Mich zieht es innerhalb von CSE auchimmer mehr in Richtung Numerik und daher investiere ich viel Kraft in diesesThema. Ich habe mich mit wirklich allen Satzen, Beweisen und auch denThemen nach FEM (hyp./par. PDEs, die Sie nicht mehr behandelt haben) undauch daruber hinaus mit dem Buch von Iserles, “PDEs” von Lawrence Evansund weiterer Literatur beschaftigt und bin der Meinung, dass ich fur einenCSE-Studenten viel mehr Wissen als erforderlich besitze und vieleZusammenhange sehr gut verstanden habe.An dieser Stelle mochte ich meinen Unmut uber die gestrige Prufungausdrucken. Ich hatte bereits in NP1 eine 1.0 und dies war auch gestern meinZiel. Vermutlich habe ich nun gar nicht bestanden oder wenn, dann vielleichthochstens mit einer 4.0.[...]
Vielen herzlichen Dank.
Eh, eh, eh! He has NOT attended the lecture!
Let’s start ... why ODE?
As just mentioned, ODE comes often after the space discretizationof PDE.
But ODE have their own dignity and relevance, since, actually,PDE can be derived as the “limit” of sytems of ODE governing theevolution of particles driven by Newton laws:
F = m · a.
Hence, ODE ⇒ PDE ⇒ ODE.
Particle systemsBesides in physics, large particle systems arise in many modernapplications:
Image halftoning via variational
dithering.
Dynamical data analysis: R. palustris
protein-protein interaction network.
Large Facebook “friendship” network
Computational chemistry: molecule
simulation.
Social dynamics
We consider large particle systems ofform:
xi = vi ,
vi =∑N
j=1 H(xj − xi , vj − vi ),
Several“social forces” encoded in theinteraction kernel H:
I Repulsion-attraction
I Alignment
I ...
Possible noise/uncertainty by addingstochastic terms.
Patterns related to different balance of
social forces.
Understanding how superposition of re-iterated binary “socialforces” yields global self-organization.
An example inspired by nature
Mills in nature and in our simulations.
J. A. Carrillo, M. Fornasier, G. Toscani, and F. Vecil, Particle, kinetic,
hydrodynamic models of swarming, within the book “Mathematical modeling
of collective behavior in socio-economic and life-sciences”, Birkhauser (Eds.
Lorenzo Pareschi, Giovanni Naldi, and Giuseppe Toscani), 2010.
http://www.ricam.oeaw.ac.at/people/page/fornasier/bookfinal.pdf
The genearal ODEOur goal is to use equations of the type
y ′ = f (t, y), t ≥ t0, y(t0) = y0, (1)
for modeling and then for simulation. (In the previous modeling,yi = (xi , vi ) and f (t, y)i = (vi ,
∑Nj=1 H(xj − xi , vj − vi )).)
I f is a map of [t0,∞)× Rd to Rd and the initial conditiony0 ∈ Rd is a given vector
I f is “nice”, obeying, in a given vector norm, the Lipschitzcondition
‖f (t, x)− f (t, y)‖ ≤ λ‖x − y‖, ∀x , y ∈ Rd , t ≥ t0 (2)
Here λ > 0 is a real constant that is independent of thechoice of x and y . In this case we say that f is a Lipschitzcontinuous function.
I Subject to (2), it is possible to prove that the ODE system(1) possesses a unique solution (see, for instance [Theorem3.5, F13] pag. 52).
Just an idea of the proof of existence and uniqueness(Picard 1890, Lindelof 1894)
Actually (1) can be rewritten by integration
y(t) = y0 +
∫ t
t0
f (s, y(s))ds = g(y)(t). (3)
Hence, a solution y(t) is a fixed point trajectory of the equationy = g(y). How can one solve fixed point equations? Well, if gwere a contraction, i.e., g is a Lipschitz continuous function withLipschitz constant 0 < Λ < 1, then the iteration
yn+1 = g(yn), n ≥ 0 (4)
converges always to the unique fixed point!
Graphical interpretation and mathematical explanation
Indeed, assume that such fixed point y exists then
‖y−yn+1‖∗ = ‖g(y)−g(yn)‖∗ ≤ Λ‖y−yn‖∗ ≤ Λn‖y−y 0‖∗ → 0, n→∞,
because Λ < 1. The tricky part of the proof is to show that thereexists a fixed point and that there exists always a norm ‖ · ‖∗ whichmakes g a contraction as soon as f is Lipschitz continuous.
Did you noticed that ...... what we wrote in (4) is actually an ALGORITHM?!
OMG! Really, an ALGORITHM?!
... is it a bad thing?!Actually, no, that’s what you are here for! Or, perhaps yes ... Ohwell, nevertheless, an ALGORITHM!
For the brave student: if you find out a way to discretize (4) and a way of
properly “scaling” the equation so that the iteration always converges on a
finite set of time point t0 < t1 < · < tn, then let me know! Actually this course
is (IMPLICITLY) all about this!
From Picard-Lindelof back to Euler: “greed is good!”Instead of solving globally the fixed point equation (3) by amultitude of iterations (4), we may consider the simpler idea ofsolving it locally, step by step, by iterating the approximation
y(t) = y0+
∫ t
t0
f (s, y(s))ds ≈ y(t0)+(t−t0)f (t0, y(t0)), for t ≈ t0.
(5)Given a sequence t0, t1 = t0 + h, t2 = t0 + 2h, ... , where h > 0 isa time step, we denote by yn a numerical estimate of the exactsolution y(tn), n = 0, 1, . . . . Motivated by (31), we choose
y1 = y0 + hf (t0, y0).
If h is small, it should not be that wrong! But then, why not tocontinue, assuming that we did not that bad before, at t2, t3 andso on. In general, we obtain the recursive scheme
yn+1 = yn + hf (tn, yn), (6)
the celebrated Euler method.
Graphical interpretationConsider the Euler method applied to the logistic equation
y ′ = y(1− y), y(0) =1
10,
with step h = 1:
It’s clear that at each step we produce an error, but our goal is not to avoid
any (numerical error)! (Eventually nobody is perfect!) Our final goal is to have
a practical method that approximates the analytic solution with increasing
accuracy (i.e., decreasing error) the more computational effort we do.
Yes, but what hell is this logistic equation? Is it justmathematical nonsense?!
The logistic equation is a model
OMG! Really, a MODEL?!
of population growth first published by Pierre Verhulst (1845,1847). Its solution is the function
y(t) =1
1 + e−t, t ∈ R.
Try it out and check that y ′(t) = y(t)(1− y(t)) (Exercise)!
Yes, but what hell is this logistic equation? Is it justmathematical nonsense?!
The logistic function finds applications in a range of fields, including artificialneural networks, biology, especially ecology, biomathematics, chemistry,demography, economics, geoscience, mathematical psychology, probability,sociology, political science, and statistics.In population dynamics one describes the growth of the population P(t) by
dP(t)
dt= rP(t)
(1− P(t)
K
).
The term rP(t) on the right-hand side tells that for r > 0 the larger is thepopulation the stronger the growth will be (he, he, he) ... however the negativeterm − r
KP(t)2 instead indicates the drop of the population due to - interacting
- competition. This antagonistic effect is called the bottleneck, and is modeledby the value of the parameter K > 0. By setting y = P(t)
Kone obtains
y ′(t) = ry(t)(1− y(t)),
which for r = 1 is just our equation.
How to implement Euler’s method in MATLAB
Try out Euler’s method on the logistic equation:
f = @(t,x) x.*(1-x); [t,yy]=ode45(f,[0:5],1/10); y(1)=1/10;h=1; for n=1:5, y(n+1)=y(n)+h*y(n)*(1-y(n)); end,plot([0:5],y,’r*-’,[0:5],yy,’b’) h=1/2; for n=1:10,y(n+1)=y(n)+h*y(n)*(1-y(n)), end, hold on,plot([0:0.5:5],y,’g-*’)
Convergence of a numerical method
Assume that h > 0 is variable and h→ 0. On each grid t0,t1 = t0 + h, t2 = t0 + 2h, ... we associate a different numericalsequence yn = yn,h, n = 0, 1, . . . , bt∗/hc (not necessarily producedby the Euler method!).
A method is said to be convergent if, for every ODE (1) with aLipschitz function f and every t∗ > 0 it is true that
limh→0
maxn=0,1,...,bt∗/hc
‖yn,h − y(tn)‖ = 0,
where bαc ∈ Z is the integer part of α ∈ R.
Convergence means that, for every Lipschitz function,the numerical solution tends to the true solution as thegrid becomes increasingly fine.
Convergence of the Euler method
TheoremThe Euler method (32) is convergent
Proof. (For this proof we assume that the Taylor expansion of f has uniformlybounded coefficients, i.e., f is analytic, implying that y is analytic as well.) Letus consider the error en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0.Taylor expansion of y(t)
y(tn+1) = y(tn) + hy ′(tn) +O(h2) = y(tn) + hf (tn, y(tn)) +O(h2).
Subtracting this equation to (32), we obtain
en+1,h = en,h + h[f (tn, yn)− f (tn, y(tn))] +O(h2).
Triangle inequality and (2) imply
‖en+1,h‖ ≤ ‖en,h‖+ h‖f (tn, yn)− f (tn, y(tn))‖+ ch2
≤ (1 + hλ)‖en,h‖+ ch2.
By induction over this estimate we get
‖en,h‖ ≤c
λh[(1 + hλ)n − 1], n = 0, 1, . . .
Convergence of the Euler method continues ...
We notice now that 1 + ξ ≤ eξ for all ξ > 0, hence (1 + hλ) ≤ ehλ and(1 + hλ)n ≤ ehnλ. But n = 0, 1, . . . , bt∗/hc and n ≤ t∗/h, implying
(1 + hλ)n ≤ et∗λ
and‖en,h‖ ≤
[ c
λ(et∗λ − 1)
]· h. (7)
Since[
cλ
(et∗λ − 1)]
is independent of n and h, it follows
limh→0
max0≤nh≤t∗
‖en,h‖ = 0.
The error bound in (33) tells us that actually Euler’s methodconverges with order q = 1 since it decays as O(hq). However,letus stress that the constant
[cλ(et
∗λ − 1)]
is by far over-pessimisticand should not be used for numerical puroposes (it’s justtheoretical!).
Beyond logistic equation?
Other applications of ODEs (extracted from the book [K11])
First oder ODEs
We shall consider first-order ODEs. Such equations contain onlythe first derivative y ′ and may contain y and any given functionsof x . Hence we can write them as
F (x , y , y ′) = 0,
or often in the explicit form
y ′ = f (x , y),
as we already got used to read it. Example: x−3y ′ − 4y 2 = 0 isimplicit and y ′ = 4y 2x3 is its explicit form.
Solution of a ODE
A solution y = y(x) of a ODE is a differentiable function definedon an real interval [a, b], with −∞ ≤ a < b ≤ +∞ which formallyfulfills the equation
y ′(x) = f (x , y(x)),
at every x ∈ (a, b). Example: y(x) = c/x is a solution of theequation
xy ′ = −y .
Indeed, differentiate y(x) = c/x to get y ′(x) = −c/x2. Multiplythe latter by x and obtain xy ′(x) = −c/x = −y(x)!
Solution by direct calculus
One trivial version of ODE is the one where there is no dependenceon y on the right-hand side, i.e.,
y ′ = f (x , y) = f (x).
But then, by integration we get
y(x) =
∫ x
x0
f (ξ)dξ + y(x0),
where y(x0) is an unknown value! Hence
y =
∫f (ξ)dξ + c ,
is a family of possible solutions. Example: y ′(x) = cos(x) hassolutions y(x) = sin(x) + c .
Family of solutions
Linear first order equation
The functiony(x) = ecx , (8)
has derivativey ′(x) = cecx = cy(x).
Hence the exponential function (8) is actually a solution of theequation
y ′ = cy ,
where the right-had side f (x , y) = cy is a linear function in y .
Some terminology
We see that each ODE in these examples has a solution thatcontains an arbitrary constant c. Such a solution containing anarbitrary constant c is called a general solution of the ODE.
Geometrically, the general solution of an ODE is a family ofinfinitely many solution curves, one for each value of the constantc . If we choose a specific c we obtain what is called a particularsolution of the ODE. A particular solution does not contain anyarbitrary constants.
Initial value problems
In most cases the unique solution of a given problem, hence aparticular solution, is obtained from a general solution by an initialcondition y(x0) = y0, with given values x0 and y0, that is used todetermine a value of the arbitrary constant c .Example: Solve the initial value problem
y ′ = 3y , y(0) = 5.7.
The general solution is given by
y(x) = ce3x .
We need to detemine c . The we plug y(0) = 5.7 = ce30 = c.Hence, c = 5.7 and
y(x) = 5.7e3x .
Radioactivity. Exponential DecayPhysical Information. Experiments show that at each instant aradioactive substance decomposes and is thus decaying in timeproportional to the amount of substance present: if y(t) is theamount of radioactive substance at the time t, we have
y ′ = −ky ,
where the constant k is positive, so that, because of the minus, wedo get decay. The value of k is known from experiments forvarious radioactive substances (e.g., k = 1.4× 10−11 sec−1,approximately, for radium 226
88 Ra).
Exponential decay of a radioactive substance
Geometrical interpretation
The equationy ′ = f (x , y),
has a simple geometric interpretation. From calculus you know thatthe derivative y ′(x) of y(x) is the slope of y(x). Hence a solutioncurve that passes through a point (x0, y0) must have, at that point,the slope y ′(x0) equal to the value of f at that point; that is,
y ′(x0) = f (x0, y0).
Geometrical interpretationThis gives a direction field (or slope field) into which you can thenfit (approximate) solution curves. This may reveal typicalproperties of the whole family of solutions. The figure below showsa direction field for the ODE
y ′ = y + x .
Direction field of y ′ = y + x , with three approximate solution curves passing
through (0, 1), (0, 0), (0,−1), respectively
Separable ODEs. ModelingMany practically useful ODEs can be reduced to the form
g(y)y ′ = f (x) (9)
by purely algebraic manipulations. Then we can integrate on bothsides with respect to x , obtaining∫
g(y)y ′dx =
∫f (x)dx + c . (10)
On the left we can switch to y as the variable of integration. Bycalculus, y ′dx = dy , so that∫
g(y)dy =
∫f (x)dx + c . (11)
If f and g are continuous functions, the integrals exist, and byevaluating them we obtain a general solution. This method ofsolving ODEs is called the method of separating variables, and (9)is called a separable equation.
Example 1
The ODE y ′ = 1 + y 2 because it can be written
1
1 + y 2dy = dx ,
By integration we obtain arctan y = x + c or y = tan(x + c). It isvery important to introduce the constant of integrationimmediately when the integration is performed! If we wrotearctan y = x , then y = tan x , and then introduced c , we wouldhave obtained y = tan x + c , which is not a solution (whenc 6= 0). Verify this.
Example 2
The ODE y ′ = (x + 1)e−xy 2 is separable; we obtainy−2dy = (x + 1)e−xdx . By integration
−y−1 = −(x + 2)e−x + c, y(x) =1
(x + 2)e−x − c.
Example 3. Initial value problem (IVP): the bell-shapedfunction
Solvey ′ = −2xy , y(0) = 1.8.
Solution: by separation of variables
1
ydy = −2xdx ,
and by integrationln(y) = −x2 + c ,
ory = ce−x
2,
where c = e c . This is the general solution. From the initialcondition we have 1.8 = y(0) = c . Hence the IVP has solutiony(x) = 1.8e−x
2.
Otzi: my grangrangranfather!
Otzi: the iceman well-preserved natural mummy
of a Chalcolithic (Copper Age) man from about XXX BC
In September 1991 the famous Iceman (Oetzi), a mummy from theNeolithic period of the Stone Age found in the ice of the OetztalAlps (hence the name “Oetzi”) in Southern Tyrolia near theAustrian-Italian border (like me!), caused a scientific sensation.When did Oetzi approximately live and die if the ratio of carbon 14
6C to carbon 12
6 C in this mummy is 52.5% of that of a livingorganism?
Physical facts
In the atmosphere and in living organisms, the ratio of radioactivecarbon 14
6 C (made radioactive by cosmic rays) to ordinary carbon126 C is constant. When an organism dies, its absorption of 14
6 C bybreathing and eating terminates. Hence one can estimate the ageof a fossil by comparing the radioactive carbon ratio in the fossilwith that in the atmosphere. To do this, one needs to know thehalf-life of 14
6 C, which is 5715 years (CRC Handbook of Chemistryand Physics, 83rd ed., Boca Raton: CRC Press, 2002, page 1152,line 9).
Mathematical modelingRadioactive decay is governed by the ODE y ′ = ky . By separationand integration (where t is the time and y0 is the initial ratio of146 C to 12
6 C)
dy
y= kdt, ln(|y |) = kt + c , y = y0ekt , (y0 = ec).
Next we use the half-life H = 5715 to determine k. When t = H,half of the original substance is still present. Thus
y0ekH = 0.5y0, ekH = 0.5, k =ln 0.5
H= −0.693
5715= −0.0001213.
Finally, we use the ratio 52.5% for determining the time t whenOetzi died (actually, was killed),
ekt = e−0.0001213t = 0.525, t =ln(0.525)
−0.0001213= 5312.
Answer: About 5300 years ago!!!
Heating an office
Suppose that in winter the daytime temperature in a certain officebuilding is maintained at 20C . The heating is shut off at 22. andturned on again at 6. On a certain day the temperature inside thebuilding at 2 was found to be 18C . The outside temperature was4C at 22 and had dropped to −6C by 6. What was thetemperature inside the building when the heat was turned on at 6?
Heating an office
Physical information. Experiments show that the time rate ofchange of the temperature T of a body B (which conducts heatwell, for example, as a copper ball does) is proportional to thedifference between T and the temperature of the surroundingmedium (Newton’s law of cooling).
Heating an office
Newton’s law:dT
dt= k(T − TA).
Unfortunately we do NOT know TA, external temperature, which is a functionof time. Then we decide to approximate it to the mean TA ≈ 4−6
2= −1. Hence
dT
T + 1= kdt, ln |T + 1| = kt + C , T (t) = −1 + cekt .
We choose 22 as time t = 0. Then the given initial condition T (0) = 20. Bysubstitution
20 = T (0) = −1 + cek0 = −1 + c c = 21.
Now we determine k. T (4) = 18 at 2 where t = 4 is 2 in the morning.
18 = T (4) = −1 + 21e4k ,19
21= e4k , k =
1
4ln(0.90) = −0.025.
Hence
T (t) = −1 + 21e−0.025t , T (8) = −1 + 21e−0.025×8 ≈ 16.3C .
Leaking tank
It concerns the outflow of water from a cylindrical tank with a holeat the bottom. You are asked to find the height of the water in thetank at any time if the tank has diameter 2 m, the hole hasdiameter 1 cm, and the initial height of the water when the hole isopened is 2.25 m. When will the tank be empty?
Leaking tank
Physical information. Under the influence of gravity the outflowingwater has velocity
v(t) = 0.600√
2gh(t) (Torricelli’s law),
where h(t) is the height of the water above the hole at time t, andg = 980cm/sec2 is the acceleration of gravity at the surface of theearth.
Leaking tank
To get an equation, we relate the decrease in water level h(t) to the outflow.The volume ∆V of the outflow during a short time ∆t is
∆V = Av∆t, A is the area of the hole
and it has to be equal to the change ∆V ∗ of volume in the tank, given by
∆V ∗ = −B∆h,
where B is the cross-sectional area of the tank and ∆h is the decrease of theheight h(t) of the water. Hence, by Torricelli’s law
−B∆h = Av∆t, or∆h
∆t= −A
Bv = −A
B0.600
√2gh(t).
By letting ∆t → 0 we obtain the wonderful ODE
dh
dt= −A
B0.600
√2gh = −26.56
A
B
√h.
Leaking tank
This is our model, a first-order ODE, which is actually separable.A/B is constant. Separation and integration gives
dh√h
= −26.56A
Bdt, 2
√h(t) = c − 26.56
A
Bt.
Dividing by 2 and squaring gives h(t) = (c − 13.28At/B)2.Inserting 13.28A/B = 0.000332 yields the general solution
h(t) = (c − 0.000332t)2.
The initial height h(0) = 225cm and c2 = 225 or c = 15, thus theparticular solution is
h(t) = (15− 0.000332t)2.
Hence h(t) = 0 for t = 45181 sec or t ≈ 12.6 hours.
Extended method: reduction to separable form
Certain nonseparable ODEs can be made separable bytransformations that introduce for y a new unknown function. Wediscuss this technique for a class of ODEs of practical importance,namely, for equations
y ′ = f(y
x
),
where f is a differentiable function. The form of the ODE suggeststhe introduction of a new variable function u = y/x . By productrule y ′ = u′x + u. Substitution into the equation leads to
u′x + u = f (u), ordu
f (u)− u=
dx
x,
if f (u)− u 6= 0.
ExampleSolve
2xyy ′ = y 2 − x2.
Solution. To get the usual explicit form we divide by 2xy and obtain
y ′ =y 2 − x2
2xy=
y
2x− x
2y.
By using the subsitution y = ux we obtain
u′x + u =u
2− 1
2um
2udu
1 + u2= −dx
x.
By integration we get
ln(1 + u2) = − ln |x |+ c = ln1
|x | + c.
By applying the exponentials on both sides we get
1 + u2 = c/x or 1 + (y/x)2 = c/x .
Multiplying by x2 both sides yeilds
x2 + y 2 = cx , or(
x − c
2
)2
+ y 2 =c2
4.
Graphical interpretation
This general solution represents a family of circles passing throughthe origin with centers on the x-axis.
Second-Order Linear ODE
A second-order ODE is called linear if it can be written
y ′′ + p(x)y ′ + q(x)y = r(x),
and nonlinear if it cannot be written in this form. The distinctivefeature of this equation is that it is linear in y and its derivatives,whereas the functions p, q, and r on the right may be any givenfunctions of x . If the equation begins with, say, f (x)y ′′, thendivide by f (x) to have the standard form with y ′′ as the first term.
(Non)homogenous equations
If r(x) ≡ 0 (that is, r(x) = 0 for all x considered; read “r(x) isidentically zero”), then the equation reduces to
y ′′ + p(x)y ′ + q(x)y = 0
and is called homogeneous. If r(x) 6= 0, then the equation iscalled nonhomogeneous. An example of a nonhomogeneouslinear ODE is
y ′′ + 25y = e−x cos x ,
and a homogeneous linear ODE is
xy ′′ + y ′ + xy = 0 ⇒ y ′′ + y ′/x + y = 0
Finally, an example of a nonlinear ODE is
y ′′y + y ′2
= 0.
The functions p and q are called the coefficients of the ODEs.
Solutions
Solutions are defined similarly as for first-order ODEs. A functiony(x) is called a solution of a (linear or nonlinear) second-orderODE on some open interval I if y is defined and twicedifferentiable throughout that interval and is such that the ODEbecomes an identity if we replace the unknown y, the derivative y ′,and the second derivative y ′′.
Homogeneous Linear ODEs: Superposition PrincipleLinear ODEs have a rich solution structure. For the homogeneousequation the backbone of this structure is the superpositionprinciple or linearity principle, which says that we can obtainfurther solutions from given ones by adding them or by multiplyingthem with any constants. Example: The functions y = cos x andy = sin x are both solutions of the homogeneous linear ODE
y ′′ + y = 0
for all x. Verify this by differentiation and substitution:(cos x)′′ = − cos x and similarly (sin x)′′ = − sin x! However, dueto the linearity of the equations any other function of the type
y(x) = c1 cos x + c2 sin x
is again a solution of the equation!!! Indeed
(c1 cos x+c2 sin x)′′ = c1(cos x)′′+c2(sin x)′′ = −(c1 cos x+c2 sin x).
Homogeneous Linear ODEs: Superposition Principle
In this example we have obtained from y1(= cos x) and y2(= sin x)a function of the form
y = c1y1 + c2y2
(c1, c2 arbitrary constants). This is called a linear combination ofy1 and y2. In terms of this concept we can now formulate theresult suggested by our example, often called the superpositionprinciple or linearity principle.
Fundamental Theorem for the Homogeneous Linear ODE
TheoremFor a homogeneous linear ODE, any linear combination of twosolutions on an open interval I is again a solution on I . Inparticular, for such an equation, sums and constant multiples ofsolutions are again solutions.
Proof.Let y1 and y2 be solutions of
y ′′ + p(x)y ′ + q(x)y = 0
on I . Then by substituting y = c1y1 + c2y2 and its derivatives into the equation,and using the familiar rule (y = c1y1 + c2y2)′ = c1y ′1 + c2y ′2, etc., we get
y ′′ + p(x)y ′ + q(x)y
= (c1y1 + c2y2)′′ + p(x)(c1y1 + c2y2)′ + q(x)(c1y1 + c2y2)
= c1y ′′1 + c2y ′′2 + p(x)(c1y ′1 + c2y ′2) + q(x)(c1y1 + c2y2)
= c1y ′′1 + p(x)c1y ′1 + q(x)c1y1︸ ︷︷ ︸:=0
+ c2y ′′2 + p(x)c2y ′2 + q(x)c2y2︸ ︷︷ ︸:=0
= 0.
Caution!
Don’t forget that this highly important theorem holds forhomogeneous linear ODEs only but does not hold fornonhomogeneous linear or nonlinear ODEs, as the followingexample illustrates. Verify by substitution that the functionsy = 1 + cos x and y = 1 + sinx are solutions of thenonhomogeneous linear ODE
y ′′ + y = 1,
but their sum is not a solution.
Also nonlinearity breaks the superposition principle!
Verify by substitution that the functions y = x2 and y = 1 aresolutions of the nonlinear ODE
y ′′y − xy ′ = 0,
but their sum is not a solution!!
Initial value problems
For a second-order homogeneous linear ODE, an initial valueproblem consists of the equation and two initial conditions
y(x0) = K0, y ′(x0) = K1.
These conditions prescribe given values of the solution and its firstderivative (the slope of its curve) at the same given x0 in the openinterval considered. The conditions are used to determine the twoarbitrary constants c1 and c2 in a general solution
y = c1y1 + c2y2
of the ODE; here, y1 and y2 are suitable solutions of the ODE.This results in a unique solution, passing through the point(x0,K0) with K1 as the tangent direction (the slope) at that point.That solution is again called a particular solution of the ODE.
Examples
Solve the initial value problem
y ′′ + y = 0, y(0) = 3.0, y ′(0) = −0.5.
Solution. General solution. The functions cos x and sin x are solutions of theODE and
y = c1 cos x + c2 sin x .
be a general solution. Particular solution. We need the derivativey ′ = −c1 sin x + c2 cos x . From this and the initial values we obtain, sincecos 0 = 1 and sin 0 = 0,
y(0) = c1 = 3.0,
andy ′(0) = c2 = −0.5.
This gives as the solution of our initial value problem the particular solution
y = 3.0 cos x − 0.5 sin x .
Linear independence!
Our choice of y1 and y2 was “independent” enough to satisfy bothinitial conditions. Now let us take instead two proportionalsolutions y1 = cos x and y2 = k cos x , so that
y = c1 cos x + c2(k cos x) = C cos x
where C = c1 + c2k . We are no longer able to satisfy two initialconditions with only one arbitrary constant C . Consequently, indefining the concept of a general solution, we must excludeproportionality and we need linear independence of two solutions!
Linear space structure
DefinitionA general solution of an ODE
y ′′ + p(x)y ′ + q(x)y = 0
on an open interval I is a solution y = c1y1 + c2y2 in which y1 andy2 are solutions of the equation on I that are not proportional, andc1 and c2 are arbitrary constants. These y1, y2 are called a basis(or a fundamental system) of solutions of the equation on I . Aparticular solution on I is obtained if we assign specific values to c1
and c2.
Linear space structure
Actually, we can reformulate our definition of a basis by using aconcept of general importance. Namely, two functions y1 and y2
are called linearly independent on an interval I where they aredefined if
k1y1(x) + k2y2(x) = 0
everywhere on I implies k1 = 0 and k2 = 0. And y1 and y2 arecalled linearly dependent on I if k1y1(x) + k2y2(x) = 0 also holdsfor some constants k1, k2 not both zero.
Definition (Basis reformulated)
A basis of solutions on an open interval I is a pair of linearlyindependent solutions on I .
Example
Verify by substitution that y1 = ex and y2 = e−x are solutions ofthe ODE y ′′ + y = 0. Then solve the initial value problem
y ′′ − y ′ = 0, y(0) = 6, y ′(0) = −2.
Solution. As (e±x)′ = ±ex it is easily shown that (e±x)′′ − e±x = 0. They are
not proportional, ex/e−x = e2x 6= const. Hence ex , e−x form a basis for all x .
The rest is just as before by finding c1 and c2 of y = c1ex + c2e−x by the given
initial conditions. The final answer is y = 2ex + 4e−x . This is the particular
solution satisfying the two initial conditions.
Reduction of the order
It happens quite often that one solution can be found byinspection or in some other way. Then a second linearlyindependent solution can be obtained by solving a first-order ODE.This is called the method of reduction of order.
ExampleFind a basis of solutions of the ODE
(x2 − x)y ′′ − xy ′ + y = 0.
Solution. Inspection shows that y1 = x is a solution because y ′1 = 1 andy ′′1 = 0, so that the first term vanishes identically and the second and thirdterms cancel. The idea of the method is to substitute
y = uy1 = ux , y ′ = u′x + u, y ′′ = u′′x + 2u.
into the ODE. This gives
(x2 − x)(u′′x + 2u′)− x(u′x + u) + ux = 0.
ux and xu cancel and we are left with an ODE, which we divide by x , order,and simplify to obtain
(x2 − x)u′′ + (x − 2)u′ = 0.
This ODE is of first order in v = u′, namely, (x2 − x)v ′ + (x − 2)v = 0!Separation of variables and integration now gives
dv
v= − x − 2
x2 − xdx = (
1
x − 1− 2
x)dx , ln |v | = ln |x − 1| − 2 ln |x | = ln
|x − 1|x2
.
Example continued ...
We need no constant of integration because we want to obtain a particularsolution; similarly in the next integration. Taking exponents and integratingagain, we obtain
v =x − 1
x2=
1
x− 1
x2, u =
∫vdx = ln |x |+ 1
x,
hence y2 = ux = x ln |x |+ 1. Since y1 = x and y2 = ux = x ln x + 1 are linearly
independent (their quotient is not constant), we have obtained a basis of
solutions, valid for all positive x .
The general methodIn this example we applied reduction of order to a homogeneous linear ODE
y ′′ + p(x)y ′ + q(x)y = 0.
We assume a solution y1 on an open interval I to be known and want to find abasis. For this we need a second linearly independent solution y2 on I . To gety2, we substitute
y = y2 = uy1, y ′ = u′y1 + u′y1, y ′′ = u′′y1 + 2u′y ′1 + uy ′′,
in the equation, obtaining
u′′y1 + u′(2y ′1 + py1) + u (y ′′1 + py ′1 + qy1)︸ ︷︷ ︸:=0
= 0.
Hence, for v = u′ we obtain
v ′ + v2y ′1 + py1
y1= 0,
which can be solved by separation of variables!
dv
v= −2y ′1 + py1
y1dx , ln |v | = −2 ln |y1| −
∫pdx or v =
1
y 21
e∫pdx .
Hence, the desired second solution y2 = uy1 = y1
∫vdx . As the quotient
y2/y1 = u =∫
vdx cannot be constant because v > 0, y1 and y2 are a basis for
the space of solutions!
Homogeneous Linear ODEs with Constant Coefficients
We shall now consider second-order homogeneous linear ODEswhose coefficients a and b are constant,
y ′′ + ay ′ + by = 0.
These equations have important applications in mechanical andelectrical vibrations. To solve it, we recall that the solution of thefirst-order linear ODE with a constant coefficient k
y ′ + ky = 0,
isy(x) = ce−kx .
Solution strategy
This gives us the idea to try with solutions of the type y(x) = eλx ,for which
y ′(x) = λeλx , y ′′(x) = λ2eλx ,
hence(λ2 + aλ+ b)eλx = 0, ∀x
and λ is a solution of the characteristic equationλ2 + aλ+ b = 0. From algebra we know that the roots are
λ± =1
2(−a±
√a2 − 4b),
leading to two solutions
y1(x) = eλ+x , y2(x) = eλ−x
CasesWe have three cases
I a2 − 4b > 0: two distict real roots;I a2 − 4b = 0: one double root;I a2 − 4b < 0: two conjugate complex roots;
In the first case, we have that
y(x) = c1y1(x) + c2y2(x),
is the general solution of the equation, since y1 and y2 are certainly notproportional. In the second case, λ = −a/2 and there is only one solution giveny1(x) = e−a/2x . In order to obtain a second solution needed for a basis, we usethe order reduction technique. So we consider y2 = uy1. Plugging this into theequation we obtain, as we already computed
u′′y1 + u′(2y ′1 + ay1) = 0.
Now notice that actually 2y ′1 + ay1 = 0! In fact y1(x) = e−a/2x , hencey ′1(x) = − a
2y1(x). We are left with
u′′y1 = 0, or u′′ = 0, or u(x) = c1x + c2.
Hence, y2(x) = xy1 and a general solution of the equation is
y(x) = (c1 + xc2)e−a/2x .
Short introduction to complex numbers
Complex numbers are introduced to give meaning to situationswhere the square root of a negative number has to be considered.This happens, for instance, when one considers the discriminant ofa second order polynomial equation ax2 + bx + c = 0
∆ = b2 − 4ac,
which might happen to be negative (for instance fora = b = c = 1). To solve this issue one artifically introduces a newnumber, called the imaginary identity, indicated by the letter i andcorresponding to the square root of −1:
i :=√−1 or i2 := −1
Hence√−10 =
√10√−1 =
√10i . In general, a complex number
is a number that can be expressed in the form a + bi , where a andb are real numbers and i is the imaginary unit. In this expression, ais the real part and b is the imaginary part of the complex number.
How do we operate on complex numbers?Complex numbers are actually vectors expressed in the two dimensional realspace spanned by the basis
1, i,hence they can be identified with the Euclidean plane.
Identification of the complex number a + bi with a vector on the plane
Hence, one operates on complex numbers as they were vectors according to thefollowing addition rule
(a1 + b1i) + (a2 + b2i) = (a1 + a2) + (b1 + b2)i .
Multiplication follows by applying the distributive property and rememberingthat i2 = −1
(a1 + b1i)(a2 + b2i) = a1a2 + (a1b2 + a2b1)i − b1b2.
Exponentials of complex numbers: Euler formula
We shall show that
ea+ib = ea(cos b + i sin b). (12)
If b = 0 (hence the number is real) then cos 0 = 1 and sin 0 = 0and the identity holds. If a = 0 then we have by Maclaurin series
ea+ib = e ib = 1 + (ib) +(ib)2
2!+
(ib)3
3!+
(ib)4
4!+ . . .
= 1− b2
2!+
b4
4!− · · ·+ i
(b − (b)3
3!+
(b)5
5!+ . . .
)= cos b + i sin b.
The identity (12) is called the Euler formula.
Complex representations of cos and sin
By using the Euler formula we find new ways of writing cos andsin. We note that e−it = cos(−t) + i sin(−t) = cos t − i sin t, sothat by addition and subtraction of this and Euler’s formula,
cos t =1
2(e it + e−it), sin t =
1
2i(e it − e−it).
Cases continued ...In the third case the roots are complex
λ± = −a
2± iω,
where ω2 = b − 14a2. Then the solutions
y1 = e−a/2x cos(ωx), y2 = e−a/2x sin(ωx)
turn out to be linearly independent! Hence in this case the general solution isgiven by
y(x) = e−a/2x(c1 cos(ωx) + c2 sin(ωx)).
But how these solutions are derived? From Euler formulae iξ = (cos(ξ) + i sin(ξ)) we obtain
cos ξ =1
2(e iξ + e−iξ), sin(ξ) =
1
2(e iξ − e−iξ).
Henceeλ+x = e(− a
2+iω)x = e(− a
2x)(cos(ωx) + i sin(ωx)),
eλ−x = e(− a2−iω)x = e(− a
2x)(cos(ωx)− i sin(ωx)).
Adding these functions and multiplying by 12
gives us y1, while substracting the
second from the first and multiplying by 12i
gives us y2.
Example
Solve the initial value problem
y ′′ + 0.4y ′ + 9.04y = 0, y(0) = 0, y ′(0) = 3.
Solution. General solution. The characteristic equation is λ2 + 0.4λ+ 9.04 = 0.It has the roots −0.2± 3i . Hence ω = 3, and a general solution is
y(x) = e−0.2x(A cos 3x + B sin 3x).
Particular solution. The first initial condition gives y(0) = A = 0. Theremaining expression is y = Be−0.2x sin 3x . We need the derivative (chain rule!)
y ′ = B(−0.2e−0.2x sin 3x + 3e−0.2x cos 3x).
From this and the second initial condition we obtain y ′(0) = 3B = 3. HenceB = 1. Our solution is
y = e−0.2x sin 3x .
Typical damped oscillation
The Figure shows y and the curves of e−0.2x and −e−0.2x
(dashed), between which the curve of y oscillates. Such “dampedvibrations” (with x = tbeing time) have important mechanical andelectrical applications.
Summary
ModelingLinear ODEs with constant coefficients have importantapplications in mechanics and in electrical circuits.
Now we model and solve a basic mechanical system consisting of amass on an elastic spring (a so-called “massspring system”), whichmoves up and down.
Mass-spring systems
We take an ordinary coil spring that resists extension as well ascompression. We suspend it vertically from a fixed support andattach a body at its lower end, for instance, an iron ball. We lety = 0 denote the position of the ball when the system is at rest(Fig. b). Furthermore, we choose the downward direction aspositive, thus regarding downward forces as positive and upwardforces as negative. We now let the ball move, as follows. We pullit down by an amount y > 0 (Fig. c). This causes a spring force
F = −ky Hooke’s law,
proportional to the stretch. The minus sign indicates that F1
points upward, against the displacement.
Motiong driven by the force
The motion of our massspring system is determined by Newtonssecond law
my ′′ = F or my ′′ + ky = 0.
This is a homogeneous linear ODE with constant coefficients, withgeneral solution
y(t) = A cos(ω0t) + B sin(ω0t), ω0 =
√k
m.
This motion is called a harmonic oscillation. Its frequency isf = ω0/(2π) Hertz (= cycles>sec). The frequency f is called thenatural frequency of the system.
Solutions
Damping
To our model m′′ = −ky we now add a damping force
F1 = −cy ′,
obtaining my ′′ = −ky − cy ′; thus the ODE of the damped mass-spring system is
my ′′ + cy ′ + ky = 0.
We assume this damping force to be proportional to the velocityy ′. This is generally a good approximation of friction effects forsmall velocities.
Solution
The ODE (5) is homogeneous linear and has constant coefficients.Hence we can solve it by the methods we just saw. Thecharacteristic equation is
λ2 +c
mλ+
k
mλ = 0.
By the usual formula for the roots of a quadratic equation weobtain
λ± = −α± β := − c
2m± 1
2m
√c2 − 4mk
Damping ...
It is now interesting that depending on the amount of dampingpresent whether a lot of damping, a medium amount of dampingor little damping three types of motions occur, respectively:
OverdampingIf the damping constant c is so large that c2 > 4mk , then thecorresponding general solution ofis
y(t) = c1e−(α−β)t + c2e−(α+β)t
Both exponents are negative because β2 = α2 − k/m < α2 andα, β > 0.
Critical dampingCritical damping is the border case between nonoscillatory motions(Case I) and oscillations (Case III). It occurs if the characteristicequation has a double root, that is, if c2 = 4mk , so that β = 0.Then the corresponding general solution of is
y(t) = (c1 + c2t)e−αt .
UnderdumpingCase III. Underdamping This is the most interesting case. It occursif the damping constant c is so small c2 < 4mk . Then β in is nolonger real but pure imaginary, say,
β = iω.
Hence the corresponding general solution is
y(t) = e−αt(Acosωt + Bsinωt) = Ce−αt cos(ωt − δ),
where C 2 = A2 + B2 and tan δ = B/A. This represents dampedoscillations.
Existence and uniquenessWe shall discuss the general theory of homogeneous linear ODEs
y ′′ + p(x)y ′ + q(x)y = 0, (13)
with continuous, but otherwise arbitrary, variable coefficients p andq. We are concerned with the existence and form of a generalsolution as well as its uniqueness of initial value problemsconsisting of such an ODE and two initial conditions
y(x0) = K0, y ′(x0) = K1, (14)
with given x0,K0,K1. The two main results will state that such aninitial value problem always has a solution which is unique, andthat a general solution
y = c1y1 + c2y2
includes all solutions.Notice that no such theory was needed for constant-coefficientequations because everything resulted explicitly from ourcalculations!!!
Existence and uniqueness
TheoremIf p(x) and q(x) are continuous functions on some open intervalI ⊂ R and x0 is in I , then the initial value problem consisting of(13) and (14) has a unique solution y(x) on the interval I.
No existence proof (it can be done by using the Picard-Lindelofiteration strategy). We address the uniqueness proof.
Uniqueness proofProof. Define y(x) = y1(x)− y2(x) and clearly y(x0) = 0 = y ′(x0). Set
z(x) = y(x)2 + y ′(x)2.
We have z ′ = 2yy ′ + 2y ′y ′′ and by y ′′ = −p(x)y ′ − q(x)y we obtain
z ′ = 2yy ′ − 2py ′2 − 2qyy ′. (15)
As (y ± y ′)2 = y 2 ± 2yy ′ + y ′2 ≥ 0 we have z = y 2 + y ′
2 ≥ 2|yy ′|. From (15)we obtain
z ′ ≤ z + 2|p|y ′2 + |q|z ≤ z + 2|p|z + |q|z = (1 + 2|p|+ |q|)︸ ︷︷ ︸:=h
z = hz
A similar estimate is obtained for −z ′ to arrive to z ′ ≤ hz , z ′ ≥ −hz . DenoteF1 = e−
∫h(x)dx and F2 = e
∫h(x)dx and
(F1z)′ = (F1z ′ + F ′1z) = F1z ′ − F1hz = F1(z ′ − hz) ≤ 0, (F2z)′ ≥ 0.
Hence F1z is nonincreasing and F2z is nondecreasing and for x ≥ x0
(F1z)(x) ≤ (F1z)(x0) = 0, (F2z)(x) ≥ (F2z)(x0) = 0.
As F1 > 0 and F2 > 0 we obtain
z ≤ 0, z ≥ 0, or z = y 2 + y ′2 ≡ 0, x ≥ x0,
hence y1(x) = y2(x) for x ≥ x0. Similar argument for x < x0.
Linear Dependence and Independence of SolutionsTwo functions y1 and y2 are called linearly independent on aninterval I where they are defined if
k1y1(x) + k2y2(x) = 0
everywhere on I implies k1 = 0 and k2 = 0. And y1 and y2 arecalled linearly dependent on I if k1y1(x) + k2y2(x) = 0 also holdsfor some constants k1, k2 not both zero.
TheoremLet the ODE have continuous coefficients p(x) and q(x) on anopen interval I . Then two solutions y1 and y2 of (13) on I arelinearly dependent on I if and only if their “Wronskian”
W (y1, y2) = y1y ′2 − y2y ′1
is 0 at some x0 in I . Furthermore, if W = 0 at an x = x0 in I , thenW = 0 on the entire I ; hence, if there is an x1 in I at which W isnot 0, then y1, y2 are linearly independent on I .
Proof of the theorem(a) Let y1 and y2 be linearly dependent on I . Then y1 = ky2 and
W (y1, y2) = y1y ′2 − y2y ′1 = ky2y ′2 − y2ky ′2 = 0.
(b) Assume now W (y1, y2) = 0 at some x = x0 and we show that y1 and y2 arelinearly dependent on I . Consider the linear system
k1y1(x0) + k2y2(x0) = 0
k1y ′1(x0) + k2y ′2(x0) = 0
in the unknown k1, k2. Multiply the first equation by y ′2 and the second by −y2
and add the resulting equations, to obtain
0 = k1W (y1(x0), y2(x0)).
Multiply the first equation by −y ′1 and the second by y1 and add the resultingequations, to obtain
0 = k2W (y1(x0), y2(x0)).
If W were not 0 at x0, we could divide by W and conclude that k1 = k2 = 0. If
W (y1(x0), y2(x0)) = 0 then there exist k1 and k2, not both zero, solving the
system. Setting y = k1y1 + k2y2, we have that y is again a solution of (13) and
y(x0) = y ′(x0) = 0. Since the solution y∗ ≡ 0 is also a solution and there is
uniqueness, we have y∗ = y ≡ 0, i.e., y1 and y2 are mutually proportional.
Proof of the theorem ...
We show the last statement of the theorem. (c) Assume that W (x0) = 0, thenby (b) y1 and y2 are linearly dependent and by (a) W ≡ 0. Hence in the caseof linear dependence it cannot happen that W (x1) 6= 0 at an x1 in I . If it doeshappen, it thus implies linear independence as claimed.
Wronskian determinant
Students familiar with second-order determinants may have noticedthat
W (y1, y2) =
∣∣∣∣ y1 y2
y ′1 y ′2
∣∣∣∣ = y1y ′2 − y2y ′1.
This determinant is called the Wronskian of two solutions y1 andy2 of (13). Note that its four entries occupy the same positions asin the linear system of the proof!
Example of application
1 .The functions y1 = cos(ωx) and y2 = sin(ωx) are solutions ofy ′′ + ω2y = 0. Their Wronskian is
W (cos(ωx), sin(ωx)) =
∣∣∣∣ cos(ωx) sin(ωx)−ω sin(ωx) ω cos(ωx)
∣∣∣∣ = ω.
The solutions are linearly independent if and only if ω 6= 0.
2. A general solution of y ′′ − 2y ′ + y = 0 on any interval isy = (c1 + c2x)ex . (Verify!). The corresponding Wronskian is not0, which shows linear independence of ex and xex on any interval.Namely,
W (ex , xex) = e2x 6= 0.
Existence of general solutions
TheoremIf p(x) and q(x) are continuous on an open interval I , then (13)has a general solution on I .
Proof. By the the existence and uniqueness theorem for initial value problems
(13) has a solution y1 satisfying y1(x0) = 1 and y ′1(x0) = 0, and another one y2
such that y2(x0) = 0 and y ′2(x0) = 1. Consequently W (y1, y2)(x0) = 1 6= 0,
hence y1 and y2 are linearly independent and y = c1y1 + c2y2 is the constructed
general solution.
Exhausting all solutions!
TheoremIf the ODE (13) has continuous coefficients p(x) and q(x) on someopen interval I , then every solution y = Y (x) on I is of the form
Y (x) = C1y1(x) + C2y2(x)
where y1, y2 is any basis of solutions of (13) on I and C1, C2 aresuitable constants.Proof. Consider now the linear system
k1y1(x0) + k2y2(x0) = Y (x0)
k1y ′1(x0) + k2y ′2(x0) = Y ′(x0)
with unknowns k1 and k2. This system has unique solutions since its
(Wronskian) determinant W (y1, y2)(x0) 6= 0 does not vanish, being y1 and y2
linearly indepedent. Let us now define y∗(x) = k1y1(x) + k2y2(x) the
corresponding solution, such that y∗(x0) = Y (x0) and y∗′(x0) = Y ′(x0). Being
y∗ also a solution with the same initial values, and the solutions unique (!) we
have Y ≡ y∗.
Nonhomogeneous second order linear equationsWe consider the theory of the solution of
y ′′ + p(x)y ′ + q(x)y = r(x), (16)
for r 6= 0. We shall see that a “general solution” of it is the sum ofa general solution of the corresponding homogeneous ODE,
y ′′ + p(x)y ′ + q(x)y = 0, (17)
and a “particular solution” of (16).
DefinitionA general solution of the nonhomogeneous ODE (16) on an openinterval I is a solution of the form
y(x) = yh(x) + yp(x);
here, yh = c1y1 + c2y2 is a general solution of the homogeneousODE (17) on I and yp is any solution of (16) on I containing noarbitrary constants. A particular solution of (16) on I is a solutionobtained by assigning specific values to c1 and c2 in yh.
Nonhomogeneous and homogeneous solutions
Theorem
(a) The sum of a particular solution yp of (16) and a solution yhof (17) is again a particular solution of (16);
(b) The difference of two particular solutions of (16) is a solutionof (17).
Proof. (a) Let L[y ] := y ′′ + p(x)y ′ + q(x)y the differentialoperator governing the equations. This operator is linear in y .Hence, L[yp + yh] = L[yp] + L[yh] = r + 0 = r . Hence, yp + yh is aparticular solution of (16). (b) Let y 1
p and y 2p two particular
solutions of (16) and yh = y 1p − y 2
p . ThenL[yh] = L[y 1
p − y 2p ] = L[y 1
p ]− L[y 2p ] = r − r = 0. Hence yh is a
solution of (17).
Exhausting all the solutions
TheoremIf the coefficients p(x), q(x), and the function r(x) in (16) arecontinuous on some open interval I , then every solution of (16) onI is obtained by assigning suitable values to the arbitrary constantsc1 and c2 in a general solution of (16) on I .
Proof. Let y∗ be a solution of (16). Certainly, by previous resultswe know that there exist general solutions yh = c1y1 + c2y2 of (17)and there exists a particular solution yp of (16) (by a constructionwhich we will show in the following). Hence by the previoustheorem we have that Y = y∗ − yp is again a solution of thehomogeneous equation (17). As
Y (x0) = y∗(x0)− yp(x0), Y ′(x0) = y∗′(x0)− y ′p(x0),
Y is particular solution (17) obtained by assigning suitable valuesto c1, c2 in yh. Hence, y∗ = Y + yp.
Method of Undetermined Coefficients
To solve the nonhomogeneous ODE or an initial value problem for(16), we have to solve the homogeneous ODE (17) and find anysolution yp of (16) and sum them up. But, how can we find asolution yp of (16)? One method is the so-called method ofundetermined coefficients. More precisely, the method ofundetermined coefficients is suitable for linear ODEs with constantcoefficients a and b
y ′′ + ay ′ + by = r(x) (18)
when r(x) is an exponential function, a power of x , a cosine orsine, or sums or products of such functions. These functions havederivatives similar to r(x) itself. This gives the idea.
Method of Undetermined Coefficients
Choice Rules for the Method of Undetermined Coefficients
(a) Basic Rule. If r(x) in (18) is one of the functions in the first column inthe Table, choose yp in the same line and determine its undeterminedcoefficients by substituting yp and its derivatives into (18).
(b) Modification Rule. If a term in your choice for yp happens to be asolution of the homogeneous ODE corresponding to (18), multiply thisterm by x (or by x2 if this solution corresponds to a double root of thecharacteristic equation of the homogeneous ODE).
(c) Sum Rule. If r(x) is a sum of functions in the first column of the Table,choose for yp the sum of the functions in the corresponding lines of thesecond column.
Comments on the rules
The Basic Rule applies when r(x) is a single term. TheModification Rule helps in the indicated case, and to recognizesuch a case, we have to solve the homogeneous ODE first. TheSum Rule follows by noting that the sum of two solutions of (18)with r = r1 and r = r2 (and the same left side!) is a solution of(18) with r = r1 + r2. (Verify!) The method is self-correcting. Afalse choice for yp or one with too few terms will lead to acontradiction. A choice with too many terms will give a correctresult, with superfluous coefficients coming out zero.
Example: Application of the Basic Rule (a)Solve the initial value problem
y ′′ + y = 0.001x2, y(0) = 0, y ′(0) = 1.5
Solution. Step 1. General solution of the homogeneous ODE. The ODEy ′′ + y = 0 has the general solution yh = A cos x + B sin x . Step 2. Solution ypof the nonhomogeneous ODE. We first try yp = Kx2. Then y ′′p = 2K . Bysubstitution, 2K + Kx2 = 0.001x2. For this to hold for all x , the coefficient ofeach power of x (x2 and x0) must be the same on both sides; thus K = 0.001and 2K = 0, a contradiction. The second line in the Table suggests the choiceyp = K2x2 + K1x + K0. Then
y ′′p + yp = 2K2 + K2x2 + K1x + K0 = 0.001x2.
Equating the coefficients of x2, x1, x0 on both sides, we have K2 = 0.001,K1 = 0, 2K2 + K0 = 0. Hence K0 = −2K2 = −0.002. This givesyp = 0.001x2 − 0.002, and
y = yh + yp = A cos x + B sin x + 0.001x2 − 0.002.
Step 3. The solution of the initial value problem goes as usual to identify A
and B.
Example: Application of the Modification Rule (b)Solve the initial value problem
y ′′ + 3y ′ + 2.25y = −10e−1.5x , y(0) = 1, y ′(0) = 0.
Solution. Step 1. General solution of the homogeneous ODE. Thecharacteristic equation of the homogeneous ODE isλ2 + 3λ+ 2.25 = (λ+ 1.5)2 = 0. Hence the homogeneous ODE has thegeneral solution
yh = (c1 + c2x)e−1.5x . Verify it!.
Step 2. Solution yp of the nonhomogeneous ODE. The function e−1.5x on theright would normally require the choice Ce−1.5x . But we see from yh that thisfunction is a solution of the homogeneous ODE, which corresponds to a doubleroot of the characteristic equation. Hence, according to the Modification Rulewe have to multiply our choice function by x2. That is, we choose
yp = Cx2e−1.5x .
By substitution in the equation, this gives C = −5. Hence the given ODE hasthe general solution
y = yh + yp = (c1 + c2x)e−1.5x − 5x2e−1.5x .
Step 3. The solution of the initial value problem is done as usual to determine
c1 and c2 from the initial conditions.
Example: Application of the Sum Rule (c)Solve the initial value problem
y ′′+ 2y ′+ 0.75y = 2 cos x−0.25 sin x + 0.09x , y(0) = 2.78, y ′(0) = −0.43.
Solution. Step 1. General solution of the homogeneous ODE. Thecharacteristic equation of the homogeneous
λ2 + 2λ+ 0.75 = (λ+ 1/2)(λ+ 3/2) = 0
which gives the general solution yh = c1e−x/2 + c2e−3x/2. Step 2. Particularsolution of the nonhomogeneous ODE. We write yp = yp1 + yp2 and, followingthe Table, (c) and (b),
yp1 = K cos x + M sin x
andyp2 = K1x + K0.
By substitution in the equation we obtain K = 0, M = 1, and K1 = 0.12,K0 = −0.32. Hence a general solution of the ODE is
y = c1e−x/2 + c2e−3x/2 + sin x + 0.12x − 0.32.
Step 3. The solution of the initial value problem goes as usual to identify c1
and c2.
Forced oscillators and electrical circuits
Applications of these methods for the solution of the equationsgoverning forced oscillators and electrical circuits will be presentednext week in the exercises.
Solution by Variation of Parameters
The method of variation of parameters and is credited toLagrange. Lagrange’s method gives a particular solution yp of
y ′′ + p(x)y ′ + q(x)y = r(x), (19)
for r 6= 0, of the form
yp(x) = −y1
∫y2r
Wdx + y2
∫y1r
Wdx (20)
where y1, y2 form a basis of solutions of the correspondinghomogeneous ODE
y ′′ + p(x)y ′ + q(x)y = 0, (21)
and W is the Wronskian of y1, y2.
Example
Solve the nonhomogeneous ODE
y ′′ + y = sec x =1
cos x.
Solution A basis of solutions of the homogeneous ODE on any interval isy1 = cos x , y2 = sin x . This gives the Wronskian
W (y1, y2) = cos x cos x − sin x(− sin x) = 1.
From (??), choosing zero constants of integration, we get the particularsolution of the given ODE
yp = − cos x
∫sin x sec xdx + sin x inf cos x sec xdx = cos x ln | cos x |+ x sin x .
We obtain the answer
y = yh + yp = (c1 + ln | cos x |) cos x + (c2 + x) sin x .
How can one get the Lagrange’s formula?
The idea is to start from a general solution
yh(x) = c1y1(x) + c2y2(x)
of the homogeneous ODE on an open interval I and to replace theconstants (“the parameters”) c1 and c2 by functions u(x) andv(x); We shall determine u and v so that the resulting function
yp(x) = u(x)y1(x) + v(x)y2(x)
is a particular solution of the nonhomogeneous ODE. Now yp mustsatisfy (19), but we need a second condition as well to determinethe two unknowns u, v . Indeed, our calculation will show that wecan determine u and v such that yp satisfies(19) and u and vsatisfy as a second condition the equation
u′y1 + v ′y2 = 0.
ComputationBy considering the latter equation one derives
y ′p = u′y1 + v ′y2 + uy ′1 + vy ′2 = uy ′1 + vy ′2.
An additional differentiation gives
y ′′p = u′y ′1 + uy ′′1 + v ′y ′2 + vy ′′2 .
By substitution into the equation we get
u(y ′′1 + py ′1 + qy1) + v(y ′′2 + py ′2 + qy2) + u′y ′1 + v ′y ′2 = r .
Since y1, y2 are solutions of the homog. equ. we get
u′y ′1 + v ′y ′2 = r ,
which, together withu′y1 + v ′y2 = 0,
is a linear system in the unknowns u′, v ′. This sytems is solvable forW (y1, y2) 6= 0 and has the solutions
u′ = −y2r
W, v ′ =
y1r
W, or u = −
∫y2r
Wdx , v =
∫y1r
Wdx .
Higher order linear ODEsThe concepts and methods of solving linear ODEs of order n = 2extend nicely to linear ODEs of higher order n, that is, n = 3, 4,etc. This shows that the theory so far explained for second-orderlinear ODEs is attractive, since it can be extended in astraightforward way to arbitrary n. We do so now and we considernonhomogenous equations of the type
y (n) + pn−1(x)y (n−1) + · · ·+ p1(x)y ′ + p0(x)y = r(x). (22)
If r(x) is identically zero, r(x) ≡ 0 (zero for all x considered,usually in some open interval I ), then (22) becomes thehomogenous equation
y (n) + pn−1(x)y (n−1) + · · ·+ p1(x)y ′ + p0(x)y = 0. (23)
A solution of an nth-order ODE on some open interval I is afunction y(x) that is defined and n times differentiable on I and issuch that the ODE becomes an identity if we replace the unknownfunction and its derivatives by y and its corresponding derivatives.
Superposition principle
The basic superposition or linearity principle extends to nth orderhomogeneous linear ODEs as follows.
TheoremFor a homogeneous linear ODE (23), sums and constant multiplesof solutions on some open interval I are again solutions on I . (Thisdoes not hold for a nonhomogeneous or nonlinear ODE!)
The proof is a simple generalization of that for second order and Ileave it to you.
Basis for solutions
Our further discussion parallels and extends that for second-orderODEs. So we next define a general solution of (23), which willrequire an extension of linear independence from 2 to n functions.
DefinitionA general solution of (23) on an open interval I is a solution of(23) on I of the form
y(x) = c1y1(x) + . . . cnyn(x)
where y1, . . . , yn is a basis (or fundamental system) of solutions of(23) on I ; that is, these solutions are linearly independent on I , asdefined below. A particular solution of (23) on I is obtained if weassign specific values to the n constants c1, . . . , cn.
Linear Independence and Dependence
Consider n functions y1(x), . . . , yn(x) defined on some interval I .These functions are called linearly independent on I if the equation
k1y1(x) + · · ·+ knyn(x) = 0 (24)
on I implies that all k1, . . . , kn are zero. These functions are calledlinearly dependent on I if this equation also holds on I for somek1, . . . , kn not all zero.
Linear Independence and Dependence
If and only if y1, . . . , yn are linearly dependent on I , we can express(at least) one of these functions on I as a “linear combination” ofthe other n − 1 functions , that is, as a sum of those functions,each multiplied by a constant (zero or not). This motivates theterm “linearly dependent.” For instance, if (24) holds with k1 6= 0,we can divide by k1 and express y1 as the linear combination
y1 = − 1
k1(k2y2 + · · ·+ knyn).
Note that when n = 2 (and just for this case!), these conceptsreduce to to mutual proportionality!
ExampleSolve the fourth-order ODE
y iv − 5y ′′ + 4y = 0,
equation, where y iv = d4ydx4 .
Solution. As done of second order, we substitute y = eλx .Omitting the common factor eλx , we obtain the characteristic
λ4 − 5λ2 + 4 = 0.
This is a quadratic equation in µ = λ2, namely,
µ2 − 5µ+ 4 = 0.
The roots are µ = 1 and 4. Hence λ = ±1,±2. This gives foursolutions. A general solution on any interval is
y = c1e2x + c2ex + c3e−x + c4e−2x ,
provided those four solutions are linearly independent. This is truebut will be shown later.
Initial Value Problem
An initial value problem for the ODE (23) consists of (23) and ninitial conditions
y(x0) = K0, y′(x0) = K1, . . . , y
(n−1)(x0) = Kn−1,
with given x0 in the open interval I considered, and givenK0, . . . ,Kn−1.
Existence and Uniqueness
TheoremIf the coefficients p0(x), . . . , pn−1(x) of (23) are continuous onsome open interval I and x0 is in I , then the initial value problem(23), then an initial value problem associated to this equation hasa unique solution y(x) on I .
The existence requires again the use of a functional fixed pointargument, based on the Picard-Lindelof iteration and it is omitted.Uniqueness can be proved by a slight generalization of theuniqueness proof we made for the second order equations.
ExampleSolve the following initial value problem on any open interval I onthe positive x-axis containing x = 1
x3y ′′′ − 3x2y ′′ + 6xy ′ − 6y = 0, y(1) = 2, y ′(1) = 1, y ′′(1) = −4.
Solution. Step 1. General solution. We try y = xm (why?!?). Bydifferentiation and substitution,
m(m − 1)(m − 2)xm − 3m(m − 1)xm + 6mxm − 6xm = 0.
Dropping xm and ordering gives m3 − 6m2 + 11m− 6 = 0. If we guess the rootm = 1, then we can divide by m − 1 and find the other roots 2 and 3, thusobtaining the solutions x , x2, x3, which are linearly independent !!! Hence ageneral solution is y = c1x + c2x2 + c3x3, valid on any interval I , even when itincludes x = 0 where the coefficients of the ODE divided by x3 are notcontinuous. Step 2. Particular solution. The derivatives arey ′ = c1 + 2c2x + 3c3x2 and y ′′ = 2c2 + 6c3x . From this, and y and the initialconditions, we get by setting x = 1
(a) y(1) = c1 + c2 + c3 = 2
(b) y ′(1) = c1 + 2c2 + 3c3 = 1
(c) y ′′(1) = 2c2 + 6c3 = −4.
This can be easily solved giving as solution y = 2x + x2 − x3.
Linear Independence of Solutions. Wronskian again!
Linear independence of solutions is crucial for obtaining generalsolutions. Although it can often be seen by inspection, it would begood to have a criterion for it. This extended criterion uses theWronskian W of n solutions y1, . . . , yn defined as the nth-orderdeterminant
W (y1, . . . , yn) =
∣∣∣∣∣∣∣∣y1 y2 . . . yny ′1 y ′2 . . . y ′n. . . . . . . . . . . .
y(n−1)1 y
(n−1)2 . . . y
(n−1)n
∣∣∣∣∣∣∣∣ .Note that W depends on x since y1, . . . , yn do.
Wronskian criterionTheoremLet the ODE (23) have continuous coefficients p0(x), . . . , pn−1(x) on an openinterval I . Then n solutions y1, . . . , yn of (23) on I are linearly dependent on Iif and only if their Wronskian is zero for some x = x0 in I . Furthermore, if W iszero for x = x0, then W is identically zero on I . Hence if there is an x1 in I atwhich W is not zero, then y1, . . . , yn are linearly independent on I , so that theyform a basis of solutions of (23) on I .
Proof. (a) Let y1, . . . , yn be linearly dependent solutions. Then, there areconstants k1, . . . , kn not all zero, such that for all x ∈ I ,k1y1(x) + · · ·+ knyn(x) = 0. By n − 1 differentiations of the latter equation weobtain for all x ∈ I
k1y1(x) + · · ·+ knyn(x) = 0
k1y ′1(x) + · · ·+ kny ′n(x) = 0
. . . . . .
k1y(n−1)1 (x) + · · ·+ kny (n−1)
n (x) = 0.
This is a homogeneous linear system of algebraic equations with a nontrivial
solution k1, . . . , kn. Hence its coefficient determinant must be zero for every x
on I, by Cramer’s theorem. But that determinant is the Wronskian W . Hence
W is zero for every x on I .
Wronskian criterion ...
(b) Conversely, if W is zero at an x0 in I , then the system with x = x0 has a
solution k1, . . . , kn, not all zero, by the same theorem. With these constants we
define the solution y∗ = k1y1 + · · ·+ knyn of (23). By the system this solution
satisfies the initial conditions. But another solution satisfying the same
conditions is y = 0. Hence y∗ = y by the uniqueness. Together,
y∗ = k1y1 + · · ·+ knyn = 0 on I . This means linear dependence of y1, . . . , yn.
(c) If W is zero at an x0 in I , we have linear dependence by (b) and then
W ≡ 0 by (a). Hence if W is not zero at an x1 in I, the solutions y1, . . . , yn
must be linearly independent on I .
Existence of general solutions
TheoremIf the coefficients p0(x), . . . , pn−1(x) of (23) are continuous onsome open interval I , then (23) has a general solution on I .
Proof. We choose any fixed x0 in I. The ODE has n solutionsy1, . . . , yn, where yj satisfies initial conditions with Kj−1 = 1 andall other K ’s equal to zero. Their r Wronskian at x0 equals 1,because in this case the Wronskian matrix is actually the identitymatrix of order n! Hence for any n those solutions y1, . . . , yn arelinearly independent on I . They form a basis on I , andy = c1y1 + · · ·+ cnyn is a general solution of (23) on I.
General Solution Includes All SolutionsTheoremIf the ODE (23) has continuous coefficients p0(x), . . . , pn−1(x) on some openinterval I , then every solution y = Y (x) of (23) on I is of the form
Y (x) = C1y1(x) + · · ·+ Cnyn(x)
where y1, . . . , yn is a basis of solutions and C1, . . . ,Cn are suitable constants.
Proof. Let Y be a given solution and y = c1y1 + · · ·+ cnyn a general solution.We choose any fixed x0 in I and show that we can find constants c1, . . . , cn forwhich y and its first n − 1 derivatives agree with Y and its correspondingderivatives at x0. That is, we should have at x = x0
c1y1(x0) + · · ·+ cnyn(x0) = Y (x0)
c1y ′1(x0) + · · ·+ cny ′n(x0) = Y ′(x0)
. . . . . .
c1y(n−1)1 (x0) + · · ·+ cny (n−1)
n (x0) = Y (n−1)(x0).
Due to linear independence of y1, . . . , yn, the Wronskian of this system is non
vanishing , and there is a unique solution to this system C1, . . .Cn giving a
particular solution y∗ = C1y1 + · · ·+ Cnyn. But then y∗ and Y are both
solutions of the same equation with the same initial conditions, hence
Y = y∗ = C1y1 + · · ·+ Cnyn by uniqueness.
Homogeneous Linear ODEs with Constant Coefficients
We generalize the results from n = 2 to arbitrary n. We want tosolve an nth-order homogeneous linear ODE with constantcoefficients, written as
y (n) + an−1y (n−1) + · · ·+ a1y ′ + a0y = 0. (25)
We substitute y = eλx to obtain the characteristic equation
λn + an−1λn−1 + · · ·+ a1λ+ a0 = 0.
Clearly if λ is a root, then y = eλx is a solution of (25).
Distinct real roots
If all the n roots λ1, . . . , λn are real and different, then the n solutions eλj x ,j = 1, . . . , n constitute a basis for all x . The corresponding general solution is
y = c1eλ1x + . . . cneλnx .
Indeed, the solutions are linearly independent, because the Wronskian turns outto be an exponential times the determinant of a Vandermonde matrix(exercise)!
W (eλ1x , . . . , eλnx) = e(λ1+···+λn)x
∣∣∣∣∣∣∣∣1 1 . . . 1λ1 λ2 . . . λn
. . . . . . . . . . . .λn−1
1 λn−12 . . . λn−1
n
∣∣∣∣∣∣∣∣:= e(λ1+···+λn)x |V |
The Vandermonde matrix V has nontrivial determinant |V | 6= 0 if and only ifλ1, . . . , λn are real and different, because
|V | = (−1)(n(n−1)/2)∏j<k
(λj − λk).
(Try to prove this statement by induction!)
Simple complex roots
If complex roots occur, they must occur in conjugate pairs sincethe coefficients of the polynomial are real. Thus, if λ± = γ ± iνare the conjugate solutions, then two corresponding linearlyindependent solutions are
y1 = eγx cos(νx), y2 = eγx sin(νx).
Multiple real roots
If a real double root occurs, say, λ1 = λ2, then y1 = y2, and wetake y1 and xy1 as corresponding linearly independent solutions.This is as we did by means of the order reduction method forsecond order equations. More generally, if λ is a real root of orderm, then m corresponding linearly independent solutions are
eλx , xeλx , . . . , xm−1eλx .
Multiple Complex Roots
In this case, real solutions are obtained as for complex simple rootsabove. Consequently, if λ+ = γ + iν is a complex double root, sois the conjugate λ− = γ − iν. Corresponding linearly independentsolutions are
eγx cos(νx), eγx sin(νx), xeγx cos(νx), xeγx sin(νx).
For complex triple roots (which hardly ever occur in applications),one would obtain two more solutions x2eγx cos(νx), x2eγx sin(νx),and so on.
Nonhomogeneous Linear ODEs
We now turn from homogeneous to nonhomogeneous linear ODEsof nth order. We write them in standard form
y (n) + pn−1(x)y (n−1) + · · ·+ p1(x)y ′ + p0(x)y = r(x). (26)
and r 6= 0. As for second-order ODEs, a general solution of (26)on an open interval I of the x-axis is of the form
y(x) = yh(x) + yp(x),
where y = c1y1 + · · ·+ cnyn is a general solution of thecorresponding homogeneous equation
y (n) + pn−1(x)y (n−1) + · · ·+ p1(x)y ′ + p0(x)y = 0, (27)
and yp is any solution of (26). If (26) has continuous coefficientsand right-hand-side then general solutions of the type above existand include all solutions.
Initial value problems
An initial value problem for (26) consists of (26) and n initialconditions
y(x0) = K0, y′(x0) = K1, . . . , y
(n−1)(x0) = Kn−1,
with given x0 in the open interval I considered, and givenK0, . . . ,Kn−1. The conditions are used to determine the unknowncoefficients c1, . . . , cn in the expression of the general solution yh.
Method of Undetermined CoefficientsWe need to determine a particular solution yp for (26). For aconstant-coefficient equation
y (n) + an−1y (n−1) + · · ·+ a1y ′ + a0y = r(x),
and special r(x) as described for second-order equations (such assums and products of polynomials, trigonometric functions,exponentials), such yp can be determined by the method ofundetermined coefficients, according to the following rules:
(A) Basic rule: it’s the same as for second order equations;
(B) Modification rule: . If a term in your choice for yp(x) is asolution of the homogeneous equation, then multiply this termby xk , where k is the smallest positive integer such that thisterm times xk is not anymore a solution of (27);
(C) Sum rule: it’s the same as for second order equations.
The practical application of the method is the same as for secondorder equation and we do not repeat it anymore.
Method of Variation of ParametersThe method of variation of parameters also extends to arbitraryorder n. It gives a particular solution yp for the nonhomogeneousequation (26) (in standard form with y (n) as the first term!) by theformula
on an open interval I on which the coefficients of (26) and r(x) arecontinuous. In the formula the functions y1, . . . , yn form a basis ofthe homogeneous ODE (r ≡ 0), with Wronskian W , and Wj
(j = 1, . . . , n) is obtained from W by replacing the j th column ofW by the column [0 0 . . . 0 1]T :
Example
Solve the nonhomogeneous Euler-Cauchy equation
x3y ′′′ − 3x2y ′′ + 6xy ′ − 6y = x4 ln x (x > 0).
Solution. Step 1. General solution of the homogeneous ODE. Substitution ofy = xm and the derivatives into the homogeneous ODE and deletion of thefactor xm gives
m(m − 1)(m − 2)− 3m(m − 1) + 6m − 6 = 0.
The roots are 1, 2, 3 and give as a basis
y1 = x , y2 = x2, y3 = x3.
Hence the corresponding general solution of the homogeneous ODE is
yh = c1x + c2x2 + c3x3.
Step 2. Determinants. These are:
Example ...
Example ...
Step 3. Integration. We also need the right side r(x) of our ODE in standardform, obtained by division of the given equation by the coefficient x3 of y ′′′ ;thus, r(x) = x ln x . We have the simple quotients W1/W = x/2,W2/W = −1, W3/W = 1/(2x). Hence the formula now becomes (exercise!)
yp = x
∫x/2x ln x dx − x2
∫x ln x dx + x3
∫1/(2x)x ln x dx
= 1/6x4(ln x − 11/6).
Towards systems of ODEs
An nth-order ODE
y (n) = F (t, y , y ′, . . . , y (n−1))
can be converted to a system of n first-order ODEs by setting
y1 = y , y2 = y ′, y3 = y ′′, . . . , yn = y (n−1).
This system is of the form
y ′1 = y2
y ′2 = y3
. . . . . . . . .
y ′n−1 = yn
y ′n = F (t, y1, y2, . . . , yn)
Systems of ODEs
The first-order system just showed is a special case of the moregeneral system
y ′1 = f1(t, y1, y2, . . . , yn)
y ′2 = f2(t, y1, y2, . . . , yn)
. . . . . . . . .
y ′n−1 = fn−1(t, y1, y2, . . . , yn)
y ′n = fn(t, y1, y2, . . . , yn) (28)
We can write such a system a a vector equation by introductingthe column vectors y = [y1 . . . yn]T and f = [f1 . . . fn]T . This gives
y′ = f(t, y).
Solution concept and initial value problems
A solution of the system (28) on some interval is a set of ndifferentiable functions y1, . . . , yn on the same interval that satisfythe equations of the system. An initial value problem for thesystem consists of its equations (28) and n given initial conditions
y1(t0) = K1, y2(t0) = K2, . . . , yn(t0) = Kn,
in vector form, y(t0) = K, where t0 is a specified value of t in theinterval considered and the components of K = [K1 . . .Kn]T aregiven numbers.
Existence and uniqueness
TheoremLet f1, . . . , fn as below be continuous functions having continuouspartial derivatives ∂f1/∂y1, . . . ∂f1/∂yn, . . . , ∂fn/∂yn in somedomain R of t, y1, y2, . . . , yn-space containing the point(t0,K1, . . . ,Kn) (actually it is sufficient that the functions fj areLipschitz continuous as defined at the very beginning of thiscourse!!!). Then the system (28) has a solution on some intervalt0 − α < t < t0 + α. α > 0 satisfying the initial values, and thissolution is unique.
Again, the proof of this very general theorem generalizes all theexistence and uniqueness results we saw before and it can be basedon a Picard-Lindelof iteration.
Linear systems of ODEsExtending the notion of a linear ODE, we call (28) a linear systemif it is linear in y1, . . . , yn; that is, if it can be written
y ′1 = a11(t)y1 + · · ·+ a1n(t)yn + g1(t)
y ′2 = a21(t)y1 + · · ·+ a2n(t)yn + g2(t)
. . . . . . . . .
y ′n = an1(t)y1 + · · ·+ ann(t)yn + gn(t) (29)
As a vector equation, this becomes
y′ = Ay + g,
where
Linear systems of ODEs ...
This system is called homogeneous if g = 0, nonhomogeneousotherwise.As soon as the ajk and gj are continuous we can apply the previousresult and obtain the existence and uniqueness of solutions as soonas initial values are fixed. Additionally, it holds also forhomogenous linear system the superposition principle, i.e., if y1
and y2 are solutions, then any linear combination of themy = c1y1 + c2y2 is again a solution of
y′ = Ay.
Bases for solutions
By a basis or a fundamental system of solutions of thehomogeneous system y′ = Ay on some interval J we mean alinearly independent set of n solutions y1, . . . , yn on that interval.We can write all the solutions as column of a matrix
Y = [y1, . . . yn],
whose determinant is called again the Wronskian of y1, . . . , yn,which we denote by
W (y1, . . . , yn) =
∣∣∣∣∣∣∣∣y 1
1 . . . yn1
y 12 . . . yn
2
. . . . . . . . .y 1n . . . yn
n
∣∣∣∣∣∣∣∣
Constant-Coefficient Systems
Let us assume thaty′ = Ay.
has constant coefficients aij , i.e., the do not depend on t. Now asingle ODE y ′ = ky has the solution of the type y = Cekt . So letus try
y(t) = xeλt .
By substitution into the equation we obtain
y′ = λxeλt = Ay = Axeλt .
Dividing by eλt one obtains
Ax = λx,
hence λ is an eigenvalue of A and x a corresponding eigenvector.
A fundamental system of solutions
We assume that A has a linearly independent set of n eigenvectors.This holds in most applications, in particular if A is symmetric(akj = ajk) or skew-symmetric (akj = −ajk) or has n differenteigenvalues. Let those eigenvectors be x1, . . . , xn and let themcorrespond to eigenvalues λ1, . . . , λn (which may be all different,or someor even allmay be equal). Then the corresponding solutionsare
y1(t) = x1eλ1t , . . . , yn(t) = xneλnt .
Their Wronskian is given by
W (y1, . . . , yn) =
∣∣∣∣∣∣∣∣y 1
1 . . . yn1
y 12 . . . yn
2
. . . . . . . . .y 1n . . . yn
n
∣∣∣∣∣∣∣∣ = e(λ1+···+λn)t
∣∣∣∣∣∣∣∣x1
1 . . . xn1
x12 . . . xn
2
. . . . . . . . .x1n . . . xn
n
∣∣∣∣∣∣∣∣ .The latter is not vanishing as soon as the vectors x1, . . . , xn arelinearly independent!
Summarizing ...
TheoremIf the constant matrix A in the system
y′ = Ay.
has a linearly independent set of n eigenvectors, then thecorresponding solutions
y1(t) = x1eλ1t , . . . , yn(t) = xneλnt .
form a basis of solutions, and the corresponding general solution is
y = c1x1eλ1t + · · ·+ cnxneλnt .
How to search for solutions otherwise?If a matrix has not a set of n linearly independent eigenvectors,how can we solve the equation
y′ = Ay?
The idea is to introduce the so-called exponential matrix
eB = I + B +B2
2+
B3
3!+ · · ·+ Bk
k!+ . . . .
By considering the function t → etA and differentiating it bydifferentiating each term of the Taylor-like expansion ofetA = I + tA + t2A2
2 + . . . one obtains
(etA)′ = AetA.
Hencey(t) = etAx,
does solve y′ = Ay, for all x. This observation is useful in certaincircumstances, but, in general, it is hard to compute etA directly.
How to search for solutions otherwise...?Here is the procedure to follow, to solve a system which may be “defective”(not having a basis of eigenvectors!):
I Find the eigenvalues of the matrix A;
I For each eigenvalue λ (appearing as a root of the characteristic
polynomial with multiplicity m), find a basis v1, . . . , vk for the
corresponding eigenspace.
I If k = m, then the solutions to the system y′ = Ay comingfrom this eigenspace are v1eλt , . . . , vmeλt .
I If the eigenspace is defective (in other words, if k < m), thenfor each eigenvector w in the basis v1, . . . , vk look for a chainof generalized eigenvectors satisfying
(A− λI )w2 = w
(A− λI )w3 = w2
. . . . . . . . .
(A− λI )w` = w`−1.
Then the solutions to the system for this eigenspace are
weλt ,(tw + w2
)eλt ,
(t2
2w + tw2 + w3
)eλt , . . . ,
How to search for solutions otherwise...?
. . .
(t`−1
(`− 1)!w +
t`−2
(`− 2)!w2 · · ·+ tw`−1 + w `
)eλt .
I If y1, . . . , yn are the n solution functions obtained as in the previous step,then the general solution to the system is given by
y = c1y1 + . . . cnyn.
ExampleFind the general solution to the system
y ′1 = 5y1 − 9y2,
y ′2 = 4y1 − 7y2.
Solution. The matrix associated to the system is[5 −94 −7
].
We have that
det(A− λI ) = (5− λ)(−7− λ)− (4)(−9) = (λ+ 1)2.
This there is a double eigenvalue λ = −1. To compute the eigenvectors forλ = −1 we want to find w = [a b]T such that
(A− λI )w = 0 =
[6 −94 −6
]·[
ab
]=
[00
]or 2a− 3b = 0.
So the eigenvectors are of the form [a 2/3a]T and the eigenspace is only
1-dimensional with a basis given by, e.g., w = v = [3 2]T . We have only one
eigenvector w, so we need to compute a chain of generalized eigenvectors to
find the remaining solution to the system.
Example
We want to find w2 such that[6 −94 −6
]·[
ab
]=
[32
].
This gives our choice of w2 = [2 1]T . Now we have the chain of proper legnth(namely 2), so we can write down the two solutions for this eigenspace: they are[
32
]e−t and
[32
]te−t +
[21
]e−t .
Hence a general solution is given by
y = c1
[32
]e−t + c2
([32
]te−t +
[21
]e−t
).
Nonhomogeneous Linear Systems of ODEs
We discuss methods for solving nonhomogeneous linear systems ofODEs
y′ = Ay + g (30)
where the vector g(t) is not identically zero. We assume g(t) andthe entries of the n × n matrix A(t) to be continuous on someinterval J of the t-axis. From a general solution yh(t) of thehomogeneous system y′ = Ay on J and a particular solution yp(t)on J [i.e., a solution containing no arbitrary constants], we get asolution ,
y = yh + yp,
which is called a general solution of (30) on J because it includesevery solution of (30) on J.This follows from similar arguments as we used for scalar equations.
Method of Undetermined Coefficients
Just as for a single ODE, this method is suitable if the entries of Aare constants and the components of g are constants, positiveinteger powers of t, exponential functions, or cosines and sines. Insuch a case a particular solution yp is assumed in a form similar tog; this is similar to our previous uses of this method, except for theModification Rule. It suffices to show this by an example.
Example
Find a general solution of
y′ = Ay + g =
[−3 11 −3
]y +
[−62
]e−2t .
Solution. A general equation of the homogeneous system is
yh = c1
[11
]e−2t + c2
[1−1
]e−4t .
Since λ = −2 is an eigenvalue of A, the function e−2t on the right side alsoappears in yh, and we must apply the Modification Rule by setting
yp = ute−2t + ve−2t ,
rather than just yp = ute−2t . Note that the first of these two terms is theanalog of the modification rule applied in the past, but it would not besufficient here!!! By substitution
yp ′ = ue−2t − 2ute−2t − 2ve−2t = Aute−2t + Ave−2t + g.
Example ...
Equating the te−2t -terms on both sides, we have −2u = Au. Hence u is aneigenvector of A corresponding to λ = −2; thus u = a[1 1]T with any a 6= 0.Equating the other terms gives
u− 2v = Av +
[−62
].
Collecting terms and reshuffling gives
v1 − v2 = −a− 6
−v1 + v2 = −a + 2.
By addition 0 = −2a− 4 or a = −2, and then v2 = v1 + 4, say, v1 = k andv2 = k + 4, and we can choose for instance k = 0. This gives the final answer:
y = yh + yp = c1
[11
]e−2t + c2
[1−1
]e−4t − 2
[11
]te−2t +
[04
]e−2t .
Method of Variation of Parameters
Let us consider a particular solution to
y′ = Ay + g
given byy := Yv,
where Y is the matrix
Y = [y1, . . . yn],
whose columns constitute a system of fundamental solutions of thehomogenous equation
y′ = Ay,
and v(t) is a nonconstant vector to be determined.
Method of Variation of Parameters ...
By differentiation, we obtain
y′ = Y′v + Yv′ = AYv + Yv′,
for the property Y′ = AY! But, since y := Yv is a solution of thenonhomogenous equation we get
AYv + Yv′ = Y′ = AY + g = AYv + g,
or
Yv′ = g, or v′ = (Y)−1g or v =
∫(Y(t))−1g(t)dt.
Notice that Y is invertible because its determinant coincides withthe Wronskian which is nonzero, as y1, . . . , yn are a basis!
Critical PointsLet us consider homogeneous linear systems with constantcoefficients in dimension two.
We already seen that the solutions are related to the characteristic(eigenvector/eigenvalue) equation
Ax = λx,
for which y = [y1 y2]T := xeλt is a solution. We define a criticalpoint a point at which
dy2
dy1=
y ′2dt
y ′1dt=
a21y1 + a22y2
a11y1 + a12y2,
becomes 0/0 undetermined. Notice that this is a place whereAy = 0 (hence a position from where in principle there is nodynamics occurring).
Critical Points ...
There are several types of critical points. Consider the equationdetermining them
det(A− λI ) = λ2 − (a11 + a22)λ+ det(A) = 0.
This quadratic equation λ2 − pλ+ q = 0 with coefficients p, q anddiscriminant ∆ = p2 − 4q for p = a11 + a22 and q = det(A) hastwo solutions
λ± =1
2(p ±
√∆).
From (λ− λ+)(λ− λ−) = λ2 − pλ+ q its easy to derive
p = λ+ + λ−, q = λ+λ−.
Critical Points ...
Improper nodeAn improper node is a critical point P0 at which all the trajectories,except for two of them, have the same limiting direction of thetangent. The two exceptional trajectories also have a limitingdirection of the tangent at P0 which, however, is different.
Proper nodeA proper node is a critical point P0 at which every trajectory has adefinite limiting direction and for any given direction d at P0 thereis a trajectory having d as its limiting direction.
Saddle point
A saddle point is a critical point P0 at which there are twoincoming trajectories, two outgoing trajectories, and all the othertrajectories in a neighborhood of P0 bypass P0.
Centre pointA center is a critical point that is enclosed by infinitely manyclosed trajectories.
Spiral point
A spiral point is a critical point P0 about which the trajectoriesspiral, approaching P0 as t →∞ (or tracing these spirals in theopposite sense, away from P0).
Stability
Critical points may also be classified in terms of their stability.Stability concepts are basic in engineering and other applications.They are suggested by physics, where stability means, roughlyspeaking, that a small change (a small disturbance) of a physicalsystem at some instant changes the behavior of the system onlyslightly at all future times t. For critical points, the followingconcepts are appropriate.
Stability ...
DefinitionA critical point P0 is called stable if, roughly, all trajectories thatat some instant are close to P0 remain close to P0 at all futuretimes; precisely: if for every disk Dε of radius ε > 0 with center P0
there is a disk Dδ of radius δ > 0 with center P0 such that everytrajectory that has a point P1 (corresponding to t = t1, say) in Dδ
has all its points corresponding to t > t1 in Dε.P0 is called unstable if P0 is not stable.P0 is called stable and attractive (or asymptotically stable) if P0
is stable and every trajectory that has a point in Dδ approaches P0
as t →∞.
Stability ...
Stability Criteria for Critical Points
Nonlinear systems
The study of nonlinear ODEs or, more in general, nonlinearsystems of ODEs
y ′1 = f1(t, y1, y2, . . . , yn)
y ′2 = f2(t, y1, y2, . . . , yn)
. . . . . . . . .
y ′n−1 = fn−1(t, y1, y2, . . . , yn)
y ′n = fn(t, y1, y2, . . . , yn)
is much harder and does not usually come to explicit solutions!There are four methods to approach nonlinear systems.
Nonlinear systems ...
1. Compute a first integral for the equation, i.e., a function E (y)which is constant on the trajectories. Hence, its level setsy : E (y) = c are trajectory points for solutions. Thistechnique usually is applicable on conservative physicalsystems (where a well-defined energy E is known andconserved along the dynamics!).
Nonlinear systems ...
Consider for instance the equation of the pendulum
θ′′ +g
Lsin θ = 0,
where g is the gravity, L is the length of the pendulum, and theta is the angleof oscillation. We can transform this problem into
y ′1 = f1(y1, y2) = y2
y ′2 = f2(y1, y2) = −g
Lsin y1.
It’s not difficult to show that E(y1, y2) =y2
22− g
Lcos y1 is an integral, i.e., a
constant function along trajectories. Hence, we can get a picture of the
trajectories by drawing the level sets of E .
Nonlinear systems ...
Nonlinear systems ...
2. Linearization of the system around a critical point; This means that wemake a linear approximation of the system around a critical point P0 (i.e,f(t,P0) = 0)
y = f(t, y) = f(t,P0)︸ ︷︷ ︸:=0
+∇y f(t,P0)(y − P0) +O(‖y − P0|2).
By defining A = ∇y f(t,P0) and g(t) = −∇y f(t,P0)P0 +O(‖y − P0|2),then one studies the linear system
y′ = Ay + g,
in particular its stability etc.
3. Qualitative analysis of solutions: Assume, for instance, that n = 1 and
y ′ = f (t, y).
The locations (t, y) ∈ R2 where f (t, y) > 0 are locations of growingsolutions, i.e., y(t)′ > 0, and f (t, y) > 0 are locations of decayingsolutions y(t)′ < 0. These simple observations can easily allow to skecthtrajectories on the (t, y)-plane, keeping into account that trajectories cannever intersect! More can be done ...
Nonlinear systems ...
4. When life becomes nasty ... then we absolutely need to use
a numerical solver!
From Picard-Lindelof back to Euler: “greed is good!”Instead of solving globally the fixed point equation (3) by amultitude of iterations (4), we may consider the simpler idea ofsolving it locally, step by step, by iterating the approximation
y(t) = y0+
∫ t
t0
f (s, y(s))ds ≈ y(t0)+(t−t0)f (t0, y(t0)), for t ≈ t0.
(31)Given a sequence t0, t1 = t0 + h, t2 = t0 + 2h, ... , where h > 0 isa time step, we denote by yn a numerical estimate of the exactsolution y(tn), n = 0, 1, . . . . Motivated by (31), we choose
y1 = y0 + hf (t0, y0).
If h is small, it should not be that wrong! But then, why not tocontinue, assuming that we did not that bad before, at t2, t3 andso on.In general, we obtain the recursive scheme
yn+1 = yn + hf (tn, yn), (32)
the celebrated Euler method.
Graphical interpretationConsider the Euler method applied to the logistic equation
y ′ = y(1− y), y(0) =1
10,
with step h = 1:
It’s clear that at each step we produce an error, but our goal is not to avoid
any (numerical error)! (Eventually nobody is perfect!) Our final goal is to have
a practical method that approximates the analytic solution with increasing
accuracy (i.e., decreasing error) the more computational effort we do.
How to implement Euler’s method in MATLAB
Try out Euler’s method on the logistic equation:
f = @(t,x) x.*(1-x); [t,yy]=ode45(f,[0:5],1/10); y(1)=1/10;h=1; for n=1:5, y(n+1)=y(n)+h*y(n)*(1-y(n)); end,plot([0:5],y,’r*-’,[0:5],yy,’b’) h=1/2; for n=1:10,y(n+1)=y(n)+h*y(n)*(1-y(n)), end, hold on,plot([0:0.5:5],y,’g-*’)
Convergence of a numerical method
Assume that h > 0 is variable and h→ 0. On each grid t0,t1 = t0 + h, t2 = t0 + 2h, ... we associate a different numericalsequence yn = yn,h, n = 0, 1, . . . , bt∗/hc (not necessarily producedby the Euler method!).
A method is said to be convergent if, for every ODE (1) with aLipschitz function f and every t∗ > 0 it is true that
limh→0
maxn=0,1,...,bt∗/hc
‖yn,h − y(tn)‖ = 0,
where bαc ∈ Z is the integer part of α ∈ R.
Convergence means that, for every Lipschitz function,the numerical solution tends to the true solution as thegrid becomes increasingly fine.
Convergence of the Euler method
TheoremThe Euler method (32) is convergent
Proof. (For this proof we assume that the Taylor expansion of f has uniformlybounded coefficients, i.e., f is analytic, implying that y is analytic as well.) Letus consider the error en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0.Taylor expansion of y(t)
y(tn+1) = y(tn) + hy ′(tn) +O(h2) = y(tn) + hf (tn, y(tn)) +O(h2).
Subtracting this equation to (32), we obtain
en+1,h = en,h + h[f (tn, yn)− f (tn, y(tn))] +O(h2).
Triangle inequality and (2) imply
‖en+1,h‖ ≤ ‖en,h‖+ h‖f (tn, yn)− f (tn, y(tn))‖+ ch2
≤ (1 + hλ)‖en,h‖+ ch2.
By induction over this estimate we get
‖en,h‖ ≤c
λh[(1 + hλ)n − 1], n = 0, 1, . . .
Convergence of the Euler method continues ...
We notice now that 1 + ξ ≤ eξ for all ξ > 0, hence (1 + hλ) ≤ ehλ and(1 + hλ)n ≤ ehnλ. But n = 0, 1, . . . , bt∗/hc and n ≤ t∗/h, implying
(1 + hλ)n ≤ et∗λ
and‖en,h‖ ≤
[ c
λ(et∗λ − 1)
]· h. (33)
Since[
cλ
(et∗λ − 1)]
is independent of n and h, it follows
limh→0
max0≤nh≤t∗
‖en,h‖ = 0.
The error bound in (33) tells us that actually Euler’s methodconverges with order q = 1 since it decays as O(hq). However,letus stress that the constant
[cλ(et
∗λ − 1)]
is by far over-pessimisticand should not be used for numerical puroposes (it’s justtheoretical!).
Example
Let’s consider
y ′ = −100y , y(0) = 1, t∗ = 1.
The error estimate (33) would give us the following bound on theerror
|en,h| ≤ 2.69× 1045h,
while the exact and numerical solutions are respecivelyy(t) = e−100t and yn = (1− 100h)n, giving
|yn − yn,h| = |(1− 100h)n − e−100nh|,
which is by far smaller than 2.69× 1045h!
Consistency error and orderEuler’s method can be rewritten
yn+1 − yn − hf (tn, yn) = 0.
How well this equation represents the original solution? Let’s plug the originalsolution in this equation
y(tn+1)− y(tn)− hf (tn, y(tn)) = y(tn+1)− y(tn)− hy ′(tn) = O(h2), (34)
by Taylor’s expansion error. This residual error O(h2) is called the consistencyerror. In general, given an arbitrary time-stepping method
yn+1 = Yn(f , h, y0, y1, . . . , yn),
for the ODE (1), we say that it is of consistency order p if
y(tn+1)− Yn(f , h, y(t0), y(t1), . . . , y(tn)) = O(hp+1).
for every analytic f and n = 0, 1, . . . . Hence, by (34) Euler’s method hasconsistency order 1!
Never confuse the order of convergence with the consistency order! They may
well be different! A method can have a positive consistency order but not be
convergent! (see later)
Trapezoidal rule alias Crank-Nicolson method
Euler’s method approximates the derivative by a constant in[tn, tn+1], namely by its value at tn. Clearly, the “leftist”approximation is not very good (it’s NOT a political comment!Left can be good :-) )and it makes more sense to make the constant approximation ofthe derivative equal to the average of its values at the endpoints:
y(t) = y0+
∫ t
t0
f (s, y(s))ds ≈ y(t0)+1
2(t−t0)[f (t0, y(t0))+f (t, y(t))],
(35)for t ≈ t0. This brings us to consider the trapezoidal rule or theCrank-Nicolson method
yn+1 = yn +h
2[f (tn, yn) + f (tn+1, yn+1)]. (36)
Consistency order of the Crank-Nicolson method
To obtain the order of (36), we substitute the exact solution,
y(tn+1)− y(tn)− h
2[f (tn, y(tn)) + f (tn+1, y(tn+1))]
= [y(tn) + hy ′(tn) +h2
2y ′′(tn) +O(h3)]
− (y(tn) +h
2y ′(tn) + [y ′(tn) + hy ′′(tn) +O(h2)]) = O(h3).
Therefore the trapezoidal rule is of order 2. This is very good, butwe should be careful (as we shall see later) already to concludethat the method is convergent. Actually we need to prove it.
Convergence of the Crank-Nicolson method
TheoremThe Crank-Nicolson method (36) is convergent.
Proof. (For this proof we assume again that f is analytic.) Let us consider theerror en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0. Substracting
y(tn+1) = y(tn) +h
2[f (tn, y(tn)) + f (tn+1, y(tn+1))] +O(h3).
(see for instance [Section 2.2.2, F13] pag. 21 for this asymptotic error) to (36),we obtain
‖en+1,h‖ ≤ ‖en,h‖+h
2[‖f (tn, yn)− f (tn, y(tn))‖+ ‖f (tn+1, yn+1)− f (tn+, y(tn+1))‖]
+ ch3
≤ ‖en,h‖+hλ
2[‖en,h‖+ ‖en+1,h‖] + ch3.
By assumption that h > 0 gets so small that hλ < 2, we get
‖en+1,h‖ ≤
(1 + 1
2hλ
1− 12hλ
)‖en,h‖+
(c
1− 12hλ
)h3.
Convergence of the Crank-Nicolson method continues ...
By induction over this estimate we get
‖en,h‖ ≤c
λh2
[(1 + 1
2hλ
1− 12hλ
)n
− 1
], n = 0, 1, . . .
Since 0 < hλ < 2, it is true that(1 + 1
2hλ
1− 12hλ
)= 1 +
(hλ
1− 12hλ
)≤∞∑`=0
1
`!
(hλ
1− 12hλ
)`= exp
(hλ
1− 12hλ
).
But n = 0, 1, . . . , bt∗/hc and n ≤ t∗/h, implying
‖en,h‖ ≤
[c
λexp
(t∗λ
1− 12hλ
)]· h2. (37)
It followslimh→0
max0≤nh≤t∗
‖en,h‖ = 0.
The error bound in (37) tells us that actually the methodconverges with order q = 2 since it decays as O(hq).
But not so easy ...Notice that the Crank-Nicolson method
yn+1 = yn +h
2[f (tn, yn) + f (tn+1, yn+1)]︸ ︷︷ ︸
:=Fn(yn+1)
.
does not allow for the explicit/direct computation of yn+1 from ynas it occurs for (32). Indeed it is a so-called implicit method and itrequires the solution of the nonlinear equation (36) in the knownyn+1. How can we do it? Well, for instance, we observe thatactaully yn+1 = Fn(yn+1) is a fixed point of an equation and thatFn is a contraction for h sufficiently small! Therefore we cancompute
yn+1 = limm→∞
ymn+1,
where the sequence (ymn+1)m is generated (similarly to the
Picard-Lindelof iteration!!) by
ym+1n+1 = Fn(ym
n+1), m ≥ 0.
Newton ... again
The solution of the equation
0 = G (yn+1) = yn+1 − Fn(yn+1)
can also be solved by Newton’ method:
ym+1n+1 = ym
n+1 −∇G (ymn+1)−1 · G (ym
n+1), m = 0, 1, . . .
where ∇G (ymn+1) is the Jacobian matrix of G computed at ym
n+1.Again we can compute
yn+1 = limm→∞
ymn+1,
How to implement the Crank-Nicolson method inMATLAB?
Try this out:
f = @(t,x) -x + 2*exp(-t)*cos(2*t); t(1)=0; x(1)=0;y(1)=0; h=1/2; for n=1:(10/h), y(n+1)=fsolve(@(x)x-y(n)-h/2*(f(t(n),y(n))+f(t(n+1),x)),y(n)); end,semilogy(t,abs(y-xx),’k’,t, h2*ones(size(t)),’r:’)
Example
Euler’s method and the trapezoidal rule, as applied to y ′ = −y + 2e−tcos2t,
y(0) = 0. The logarithm of the error, ln(|yn − y(tn)|), is displayed for h = 1/2
(solid line), h = 1/10 (broken line) and h = 1/50 (broken-and-dotted line).
Logarithm of the error
If ‖e‖ ≈ chp, then ln e ≈ ln c + p ln h. Denoting by e(1) and e(2)
the errors corresponding to step sizes h(1) and h(2) respectively, itfollows that
ln e(2) ≈ ln e(1) + p ln(h(2)/h(1)).
The ratio of consecutive step sizes being five, we expect the errorto decay by (at least) a constant multiple of ln 5 ≈ 1.6094 and2 ln 5 ≈ 3.2189 for Euler and the trapezoidal rule respectively.
Midpoint rule
To derive the the average
y ′(t) ≈ 1
2[f (tn, yn) + f (tn+1, yn+1)]
is a sensible choice. Similar reasoning leads, however, to analternative approximation,
y ′(t) ≈ f (tn +1
2h,
1
2(yn + yn+1)), t ∈ [tn, tn+1]
and to the implicit midpoint rule
yn+1 = yn + hf (tn +1
2h,
1
2(yn + yn+1))
Exercise: prove that midpoint rule is of second order and that itconverges! The implicit rule is a special case of the Runge-Kuttamethod (more to come).
The theta-method
Both Euler’s method and the trapezoidal rule fit the generalpattern
yn+1 = yn + h[θf (tn, yn) + (1− θ)f (tn+1, yn+1)], (38)
for θ = 1 and θ = 1/2 respectively.
Consistency order
Substituting the exact solution y(t) into the formula
y(tn+1)− y(tn)− h[θf (tn, y(tn)) + (1− θ)f (tn+1, y(tn+1))]
= y(tn+1)− y(tn)− h[θy ′(tn) + (1− θ)y ′(tn+1)]
= [y(tn) + hy ′(tn) +1
2h2y ′′(tn) +
1
6h3y ′′′(tn)]
− y(tn)− hθy ′(tn) + (1− θ)[y ′(tn) + hy ′′(tn) +1
2h2y ′′′(tn)]
+O(h4)
= (θ − 1
2)h2y ′′(tn) + (
1
2θ − 1
3)h3y ′′′(tn) +O(h4).
Therefore the method is of order 2 for θ = 1/2 (the trapezoidalrule) and otherwise of order one.
Error estimates and convergence
Subtracting (as usual!) the last expression from
yn+1 − yn + h[θf (tn, yn) + (1− θ)f (tn+1, yn+1)] = 0,
one obtains
en+1 = en + θh[f (tn, y(tn) + en)− f (tn, y(tn))]
+ (1− θ)h[f (tn+1, y(tn+1) + en+1)f (tn+1, y(tn+1))]− 1
12 h3y ′′′(tn) +O(h4), θ = 12 ,
+(θ − 12 )h2y ′′(tn) +O(h3), θ 6= 1
2 .
By using the Lipschitz continuity of f and proceeding as alreadydone so far, one can show that the θ-methods are alwaysconvergent, for all θ ∈ [0, 1]. Exercise!
An important byproduct: implicit Euler’s method
The choice θ = 0 is of great practical relevance. The first-orderimplicit method
yn+1 = yn + hf (tn+1, yn+1), n = 0, 1, . . . (39)
is called the backward Euler’s method (or implicit Euler’s method)and is a favourite algorithm for the solution of stiff ODEs (more tocome).
Adams method: introduction
Let us suppose again that yn is the numerical solution attn = t0 + nh. Assume that
ym = y(tm) + O(hs+1), m = 0, 1, . . . , n + s − 1, (40)
where s ≥ 1 is a given integer. Let us specify now how can weadvance the solution from tn−s+1 to tn+s . First write
y(tn+s) = y(tn+s−1)+
∫ tn+s
tn+s−1
y ′(τ)dτ = y(tn+s−1)+
∫ tn+s
tn+s−1
f (τ, y(τ))dτ .
We shall compute the integral by extrapolation, constructing theinterpolation polynomial of f on the nodes f (t`, y(t`)) for` = n, n + 1, n + s − 1 and then compute the integral of thispolynomial between tn+s−1 and tn+s .
Interpolation polynomial
Explicitly,
p(t) =s−1∑m=0
pm(t)f (tn+m, yn+m),
where
pm(t) =s−1∏
`=0,` 6=m
t − tn+`
tn+m − tn+`=
(−1)s−1−m
m!(s − 1−m)!
s−1∏`=0,` 6=m
(t − tn
h− `)
=(−1)s−1−m
m!(s − 1−m)!
s−1∏`=0,`6=m
(t − t0 − hn
h− `)
are the Lagrange interpolation polynomials (see [Section 1.1, F13]pag. 8).
Construction of the schemeExercise: show that (40) implies
p(tm) = f (tm, ym) = y ′(tm)+O(hs+1), m = n, n+1, . . . , n+s−1,
(use the Lipschitz continuity of f ... |y ′(tm)− f (tm, ym)| =|f (t, y(tm))− f (tm, ym)| ≤ Lf |y(tm)− ym| = O(hs+1) . Hence, pinterpolates y ′ up to an error of order at least O(hs): indeed, bythe theory of [Theorem 1.2, F13] we can deduce that
p(t) = y ′(t) +O(hs) = f (t, y(t)) +O(hs), ∀t ∈ [n + s−1, n + s].
Hence if we substitute p(t) to f (t, y(t)) in∫ tn+s
tn+s−1f (τ, y(τ))dτ
and yn+s−1 to y(tn+s−1), we produce and error of orderO(hs+1) and we obtain the scheme
yn+s = yn+s−1 + hn+s−1∑m=n
bmf (tn+m, yn+m), (41)
which is, by construction, of consistency order p = s.
On the coefficients of the s-step Adams-Bashforth method
Notice that, by a change of variables in the integrals,
bm = h−1
∫ tn+s
tn+s−1
pm(τ)dτ =(−1)s−1−m
m!(s − 1−m)!
∫ 1
0
s−1∏`=0,`6=m
[s−(`+1)−ξ]dξ
do not depend on n or h and can be used for subsequent iterations.
Examples
For s = 1 this scheme coincides with the Euler’s method, whereass = 2 gives explicitly
yn+2 = yn+1 + h[3
2f (tn+1, yn+1)− 1
2f (tn, yn)],
and s = 3 gives
yn+3 = yn+2 + h[23
12f (tn+2, yn+2)− 4
3f (tn+1, yn+1) +
5
12f (tn, yn)],
ComparisonLogarithm of the errors of Euler, 2-step, and 3-step Adams-Basgforth methodsfor the equation
y ′ = y 2, y(0) = 1.
When h is halved, Euler’s error decreases linearly, the error of 2-step AB decays
quadratically and 3-step AB displays cubic decay. Eulers method, the 2-step
AB, and the 3-step AB correspond to the solid, broken and broken-and-dotted
lines respectively.
Consistency order of multistep methods
Write a general s-step method in the form
s∑m=0
amyn+m = hs∑
m=0
bmf (tn+m, yn+m), n = 0, 1, . . . (42)
where am, bm, m = 0, 1, . . . , s, are given constants, independent ofh, n and the underlying equation. It is conventional to normalizeas = 1. When bs = 0 the method is explicit (as in theAdams-Bashford methods), otherwise again implicit. Adapting thedefition of consistency order to these methods we say that (42) hasorder p if
ψ(t, y) =s∑
m=0
amy(t+mh)−hs∑
m=0
bmf (t+mh, y(t+mh)) = O(hp+1),
for h→ 0, for all sufficiently smooth y and there is one functionwhich do not allow to improve this order.
Characterization of the order
The method (42) can be characterized in terms of the polynomials
ρ(w) :=s∑
m=0
amwm, σ(w) :=s∑
m=0
bmwm.
TheoremThe multistep method is of order p ≥ 1 if and only if there existsc 6= 0 such that
ρ(w)− σ(w) ln w = c(w − 1)p+1 +O(|w − 1|p+2), w → 1.
ProofRecall
ψ(t, y) =s∑
m=0
amy(t + mh)− hs∑
m=0
bm f (t + mh, y(t + mh))︸ ︷︷ ︸:=y′(t+mh)
,
We assume that y is analytic and that its radius of convergence exceeds sh.Taylor + exchanging sums:
ψ(t, y) =s∑
m=0
am
∞∑k=0
y (k)(t)
k!mkhk − h
s∑m=0
bm
∞∑k=0
y (k+1)(t)
k!mkhk
=
(s∑
m=0
am
)y(t) +
∞∑k=1
1
k!
(s∑
m=0
mkam − ks∑
m=0
mk−1bm
)hky (k)(t).
Thus for having order p it is necessary and sufficient that
s∑m=0
am = 0,s∑
m=0
mkam = ks∑
m=0
mk−1bm, k = 1, . . . , p,
s∑m=0
mp+1am 6= (p + 1)s∑
m=0
mpbm.
Proof continues ..
Let w = ez ; then w → 1 corresponds to z → 0. Taylor + exchanging sumsagain gives
ρ(z)− zσ(z) =s∑
m=0
amemz − zm∑s=0
bmemz
=s∑
m=0
am
∞∑k=0
1
k!mkzk −
s∑m=0
bm
∞∑k=0
1
k!mkzk+1
=∞∑k=0
1
k!
(s∑
m=0
mkam
)zk −
∞∑k=1
1
(k − 1)!
(s∑
m=0
mk−1bm
)zk .
Hence,ρ(z)− zσ(z) = czp+1 +O(zp+2)
for some c 6= 0 iff the conditions written above hold. The result follows by
restoring w = ez (remember that z = ln w = (w − 1) +O(|w − 1|2) for
w → 1).
An application to the 2-step Adams-Bashforth
Recall
yn+2 = yn+1 + h[3
2f (tn+1, yn+1)− 1
2f (tn, yn)].
An explicit computation shows that for ξ = w − 1
ρ(w)−σ(w) ln(w) = (ξ+ξ2)−(1+3
2ξ)(ξ−1
2ξ2+
1
3ξ+. . . ) =
5
12ξ3+O(ξ4),
thus order 2 is validated. Exercise: try the same computation forthe 3-step Adams-Bashforth method.
Yeah! Happiness, joy!
We have a method for constructing consistent methods of arbitraryorder!
Bad news ... this sometimes does not help for convergence!
Consider the two-step implicit scheme
yn+2−3yn+1+2yn = h[13
12f (tn+2, yn+2)−5
3f (tn+1, yn+1)− 3
12f (tn, yn)].
Exercise: show that it has consistency order 2.Now consider the ODE y ′ = 0 with y(0) = 1, and unique solutiony ≡ 1. A single step reads
yn+2 − 3yn+1 + 2yn = 0,
whose general solution is given by yn = c1 + c22n, n = 0, 1, ....Suppose that c2 6= 0 and we need y0, y1 to initiate the iteration.The assumption c2 6= 0 is equivalent to y0 6= y1. Hence, in thiscase the method does NOT converge at all: in fact let t > 0 andh→ 0 and n→∞ so that hn→ t. Obviously n→∞ and|yn| → ∞, which is far from y(t) ≡ 1, ooops :-)
One smart student may say ...
... Ok but you have assumed a “stupid” choice of y0 6= y1. Wereyou choosing them equal, you would have ended up on the rightsolution, don’t you? Well, not really ... the motivation is thatalthough y0 = y1 in the first iteration, then after a while, due toroundoff errors yn 6= yn+1 and you are again dead in the water!
Touch to believe ...
Breakdown in the numerical solution of y ′ = −y , y(0) = 1, by anonconvergent numerical scheme, showing how the situationworsens with decreasing step size.
Dahlquist equivalence theoremWe say that a polynomial obeys the root condition if all its zerosreside in the closed complex unit disc and all its zeros of unitmodulus are simple.
Theorem (The Dahlquist equivalence theorem)
Suppose that the error in the starting values y1, y2, . . . ys−1 tendsto zero as h→ 0. The multistep method (42) is convergent if andonly if it is of order p ≥ 1 and the polynomial ρ obeys the rootcondition.
Touch to believe ... check that roots of
ρ(w) = w 2 − 3w + 2 = (w − 1)(w − 2)
are indeed w = 1, 2 and this explains the non-convergence of thecorresponding method. However, all Adam-Bashforth methods aresafe, because for them
ρ(w) = w s−1(w − 1).
Exercise!
Dahlquist first barrier
The multistep method (42) has 2s + 1 parameters and thanks tothe consistence order theorem we can tune them in order tooptimize the order (i.e., we can choose them to make it thelargest). The outcome, an (implicit) s-step method of order 2s, isunfortunately not convergent for s ≥ 3 (we have already seen thecase s = 3!). In general, it is possible to prove that the maximalorder of a convergent s-step method is at most 2b(s + 2)/2c forimplicit schemes and just s for explicit ones; this is known as theDahlquist first barrier.
Good construction
Choose an arbitrary s-degree polynomial ρ that obeys the rootcondition (how to do that?) and such that ρ(1) = 0 (how to dothat? Why do we need it?). Dividing the order condition by ln(w)
we obtain σ(w) = ρ(w)ln(w) +O(|w − 1|p). Suppose first p = s + 1
and expand the fraction in Taylor series around w = 1 and let σ bethe polynomial coinciding with the Taylor expansion up to orderO(|w − 1|s+1). The outcome is a convergent (!!!), s-step methodof order s + 1. Similarly , to obtain an explicit method of order s,we let σ be an (s − 1)th degree polynomial (to force bs+1 = 0)that matches the series to order O(|w − 1|s).
An example and Adams-Moulton methods
Try, for example, s = 2 and ρ(w) = w 2 − w . The solution isσ(w) = − 1
12 + 23 w + 5
12 w 2 for the implicit case, whereas for theexplicit case, where σ is linear we have p = 2, and so recover,unsurprisingly, the Adams-Bashforth scheme.The choice ρ(w) = w s−1(w − 1) is associated with Adamsmethods. We have already seen the explicit Adams-Bashforthschemes; their implicit counterparts are called Adams-Moultonmethods.
Explicit Runge-Kutta methods
Consider
y(tn+1) = y(tn) +
∫ tn+1
tn
f (s, y(s))ds
= y(tn) + h
∫ 1
0f (tn + hs, y(tn + hs))ds,
and replace the second integral by a (Gaussian) quadrature. Theoutcome might have been the method
yn+1 = yn + hν∑
j=1
bj f (tn + cjh, y(tn + cjh)), n = 0, 1, . . . ,
except that we do not know the value of y at the nodes tn + c1h,tn + c2h, . . . , tn + cνh. We must resort to an (additional)approximation!
Construction
We denote our approximation of y(tn + cjh) by ξj , j = 1, 2, . . . , ν.To start with, we let c1 = 0, since then the approximation isalready provided by the former step of the numerical method,ξ1 = yn. The idea is to construct the ξj ’s updating yn by a linearcombination of f (tn + c`h, ξ`) for ` < j . Specifically, we let
ξ1 = yn
ξ2 = yn + ha21f (tn, ξ1)
ξ3 = yn + ha31f (tn, ξ1) + ha32f (tn + c2h, ξ2)
. . . = . . .
ξν = yn + hν−1∑`=1
aν`f (tn + c`h, ξ`)
yn+1 = yn + hν∑`=1
bj f (tn, ξ`)
Butcher tableau
The matrix A = (aij)i ,j=1,...,ν , with zero entries where cofficientsare not used in the definition of the scheme, is called the RKmatrix, while the vectors b = (b1, . . . bν)T and c = (c1, . . . , cν)T
are the RK-weights and RK nodes respectively. We say that themethod has ν stages.A Runge-Kutta scheme is represented oftenby means of the so-called Butcher tableau
c | A−−− + −−−
| bT(43)
How to construct an explicit Runge-Kutta method?
Let us start assuming that
j−1∑i=1
aji = cj , j = 2, 3, . . . , ν.
This is necessary and sufficient condition for a consistent methodto be invariant under autonomization (see [Lemma 3.19, F13]).Thesimplest device to derive RK methods (only valid for p ≥ 3!)consists of verifying the order for the scalar autonomous equation
y ′ = f (y), t ≥ t0, y(0) = y0.
How to construct an explicit Runge-Kutta method?
Let us start observing that Let show how to do it for ν = 3 only.We write below f ∼ f (y(tn)) and y ∼ y(tn) unless differentlyspecified.
ξ1 = y
⇒ f (ξ1) = f
ξ2 = y + hc2f
⇒ f (ξ2) = f (y + hc2f ) = f + hc2fy f +1
2h2c2
2 fyy f 2 +O(h3)
ξ3 = y + h(c3 − a32)f (ξ1) + ha32f (ξ2)
= y + hc3f + h2a32c2fy f +O(h3)
⇒ f (ξ3) = f (y + hc3f + h2a32c2fy f ) +O(h3)
= f + hc3fy f + h2(1
2c2
3 fyy f 2 + a32c2f 2y f ) +O(h3).
How to construct an explicit Runge-Kutta method?
Therefore
yn+1 = yn + hb1f + hb2(f + hc2fy f +1
2h2c2
2 fyy f 2)
+hb3[f + hc3fy f + h2(1
2c2
3 fyy f 2 + a32c2f 2y f )] +O(h4)
= yn + h(b1 + b2 + b3)f + h2(c2b2 + c3b3)fy f
+h3[1
2(b2c2
2 + b3c33 )fyy f 2 + b3a32c2f 2
y f ] +O(h4).
How to construct an explicit Runge-Kutta method?Since for a high-order consistent method we need
y ′(tn) ≈ f , y ′′(tn) ≈ ffy , y ′′′(tn) ≈ fyy f 2 + f 2y f ,
we have
y(tn+1) ≈ y + hf +1
2h2ffy +
1
6h3(fyy f 2 + f 2
y f ) +O(h4),
and comparing the terms of the exapansion up to the order threewith
yn+1 = yn + h(b1 + b2 + b3)f + h2(c2b2 + c3b3)fy f
+h3[1
2(b2c2
2 + b3c33 )fyy f 2 + b3a32c2f 2
y f ] +O(h4)
we obtain
b1+b2+b3 = 1, b2c2+b3c3 =1
2, b2c2
2 +b3c23 =
1
3, b3a32c2 =
1
6.
(44)
Classical RK methods
Third order three-stage explicit RK method:
0 |12 | 1
21 | −1 2
−−− + −−− −−− −−−| 1
623
16
(45)
and the so-called Nystrom scheme
0 |23 | 2
323 | 0 2
3−−− + −−− −−− −−−
| 14
38
38
(46)
Classical RK methods
The fourth order method is not beyond the capability of the Taylorexpansion technique presented, but it is significantly harder to besolved. The best-known fourth-order four-stage RK method is
0 |12 | 1
212 | 0 1
21 | 0 0 1
−−− + −−− −−− −−− −−−−| 1
613
13
16
(47)
To obtain order 5 we need six stages, and matters becomeconsiderably worse for higher orders.
Implicit RK methods
The idea behind implicit RungeKutta (IRK) methods is to allowthe vector functions ξ1, ξ2, . . . , ξν to depend upon each other in amore general manner. Thus, let us consider the scheme
ξj = yn + hν∑
i=1
aji f (tn + cih, ξi ), j = 1, 2, . . . , ν,
yn+1 = yn + hν∑
i=1
bj f (tn + cjh, ξj).
Here A = (aji )j ,i=1,...,ν is an arbitrary matrix (while before it wasstrictly lower triangular). We impose the convention
ν∑i=1
aji = cj , j = 1, 2, 3, , ν.
Disadvantage and advantages
The progressing of an implicit RK scheme one needs to solve ateach time a system of ν equations in the unknown ξj ’s to be ableto implement a step. This is clearly higher complexity, but IRKpossess important advantages; in particular they may exhibitsuperior stability properties (we shall see that later).
Unique solvability for IRK methods
Let us consider ki ’s so defined
ξj = y + hν∑
i=1
aji f (t + cih, ξi )︸ ︷︷ ︸:=ki
, j = 1, 2, . . . , ν.
Clearly we can define a set of equations for them
ki = f (t + cih, y + hν∑
j=1
aijkj), i = 1, 2, . . . , ν.
TheoremIf f is Lipschitz-continuous then for h > 0 small enoughk = (k1, . . . , kν), and hence ξ = (ξ1, . . . , ξν), are uniquely definedby the above system of nonlinear equations.
Unique solvability for IRK methodsSketch of a proof. Let us assume that f is globaly Lipschitzcontinuous with constant Λ. Set k = (k1, . . . , kν)T ∈ Rν×d and
Fi (k) = f (t + cih, y + hν∑
j=1
aijkj), i = 1, . . . , ν
F (k) = (F1(k), . . . ,Fν(k))T .
Let us consider the norm ‖k‖ = max1≤i≤ν ‖ki‖2. Thenk = (k1, . . . , kν)T of the Runge-Kutta step satisfies the fixed pointequation
k = F (k).
Now it holds
‖F (k)− F (k)‖ ≤ Λh max1≤i≤ν
ν∑j=1
|aij | ‖kj − k j‖2
≤ Λh‖A‖∞‖k − k‖.
For h < 1Λ‖A‖∞ the function F is a contraction with a unique fixed
point.
Larger time stepsOne of the advantages of IRK methods is their stability for largetime steps when dealing with stiff equations. The previous methodof computation is not so efficient, as it requires a barrier to h. Wecan therefore consider the following Newton iteration: Let usdefine zi = ξi − y , i = 1, . . . , ν
G (z) = z − h
∑ν
j=1 a1j f (t + cjh, y + zj)...∑ν
j=1 aνj f (t + cjh, y + zj)
and
z0 = 0
∇G (z`)∆z` = − G (z`), z`+1 = z` + ∆z`.
The drawback of the iteration is as usual that the Jacobi-Matrix∇G has to be evaluated and inverted at each step. A simplesolution is to substitute ∇G (z`) by ∇G (z0), giving the so-calledQuasi-Newton method.
Example of a two stages implicit Runge-Kutta method
Let us consider the method
ξ1 = yn +1
4h[f (tn, ξ1)− f (tn +
2
3h, ξ2)]
ξ2 = yn +1
12h[3f (tn, ξ1) + 5f (tn +
2
3h, ξ2)]
yn+1 = yn +1
4h[f (tn, ξ1) + 3f (tn +
2
3h, ξ2)].
In short tableau form
0 | 14 −1
423 | 1
45
12−−− + −−− −−−
| 14
34
Try to prove that this method is of consistency order 3.
How can be construct Runge-Kutta methods? Usecollocation
Define the (unique!) polynomial u ∈ Pνu(tn) = yn
u(tn + cih) = f (tn + cih, u(tn + cih)), i = 1, . . . , ν,(48)
for 0 ≤ c1 < · · · < cν ≤ 1, the so-called relative nodes. Then onedefine the evolution
yn+1 = u(tn + h).
The evolution defined by (48) is called a collocation method
Indeed collocation methods are Runge-Kutta methods!
Let p1, . . . , pν be the Lagrangian polynomial associated to the(interpolation) nodes c1, . . . , cν (see [Section 1.1, ,F13]), so that
pj(ci ) = δij , i , j = 1, . . . , ν.
TheoremThe collocation method on the nodes c = (c1, . . . , cν) is animplicit Runge-Kutta method (b, c ,A) where
aij =
∫ ci
0pj(θ)dθ, i , j = 1, . . . , ν, (49)
and
bi =
∫ 1
0pi (θ)dθ, i = 1, . . . , ν. (50)
Recall how an implicit RK is formulated ...
Define
ki = f (tn + cih, yn + hν∑
j=1
aijkj), i = 1, 2, . . . , ν.
Then
yn+1 = yn + hν∑
i=1
biki .
ProofSet
kj = u(tn + cjh), j = 1, . . . , ν.
The Lagrange interpolation basis on c1, . . . , cν gives
u(tn + θh) =ν∑
j=1
kjpj(θ).
Integration over θ yields
u(tn + cih) = yn + h
∫ ci
0u(tn + θh)dθ
= yn + h
∫ ci
0
ν∑j=1
kjpj(θ)dθ
= yn + hν∑
j=1
[∫ ci
0pj(θ)dθ
]kj
= yn + hν∑
j=1
aijkj .
Proof continues ...
If we sustitute this expression in formula (48)
u(tn + cih)︸ ︷︷ ︸=ki
= f (tn + cih, u(tn + cih)︸ ︷︷ ︸=yn+h
∑νj=1 aijkj
),
we obtain
ki = f (tn + cih, yn + hν∑
j=1
aijkj) (51)
and analogously it follows
yn+1 = u(tn + h) = yn + h
∫ 1
0u(tn + θh)dθ
= yn + hν∑
i=1
biki .
(52)
We obtain so an implicit Runge-Kutta method.
Collocation as numerical quadrature ...
We define
y(tn + h) =
∫ tn+1
tn
f (t)dt, (53)
so thaty(t) = f (t), y(tn) = 0. (54)
The application of the collocation method will give us
y(tn+1) ≈ yn+1 = hν∑
i=1
biki
= hν∑
i=1
bi f (tn + cih),
(55)
over the nodes c1, . . . , cν , i.e., a Newton-Cotes integration formula!
