A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less

A little on Newton, Raphson and Halley,a little more on Simpson.

Why we should or should not care

Trond Steihaug1

1Department of Informatics, University of Bergen.Currently visiting scholar Oxford University

Tuesday July 2, 2019

22nd EuroAD Workshop, Imperial College 1-2 July 2019

Trond Steihaug Newton, Raphson, Halley & Simpson 1 / 21

Content

Computational science existed long before the dawn of computers.

Four towering figures: Newton, Raphson, Halley and Simpson

The methods of Newton and Raphson

The classical test example of Newton

Halley’s method

Early “rediscoveries” of Halley’s method

Simpson

Other contributors (Ward, Maclaurin, Jones, . . . )

Kepler’s equation

History of numerical analysis is a part of history of mathematics


This is not about derivatives . . .

Newton evaluating fluxions Griewank evaluating derivatives


Nonlinear equations

Newton’s method first in print in Wallis Algebra 1685.

Raphson’s method in 1690, no reference to Wallis or Newton.

Halley’s (two methods) in 1694 with reference to Raphson

Newton

Let g0(s) = f (x0 + s)For k = 0, 1, 2, . . . ,m − 1

Compute gk(s) ≈ gk(s)Solve for sk in gk(s) = 0Let gk+1(s) = gk(sk + s)

x = x0 +∑m−1

k=0 sk

gk(s) = gk(sk) + g ′(sk)s

Raphson

For k = 0, 1, 2, . . . ,m − 1Compute gk(s) ≈ f (xk + s)Solve for sk in gk(s) = 0Let xk+1 = xk + sk

x = xm

Raphson: gk(s) = f (xk) + f ′(xk)s


Newton’s method before Wallis 1685

De analysiThe method appears in the manuscript De analysi peræquationes numero terminorum infinitas around 1669.De analysi was printed in 1711 in an edition edited by WilliamJones.The commentary section of the translation to English of thiswork that Stewart in 1745 observes that the methods ofNewton and Raphson are the same.

De methodisAn updated version of the method appears in the manuscriptDe methodis fluxionum et serierum infinitarum written in 1671De methodis is translated and commented by Colson in 1736.Published in print in 1685. it is reasonable to assume thatWallis based his printed presentation in 1685 on a transcript ofthe manuscript from 1671.

Leipniz’ Excerpta from De analysi 1676. (Unknowndistribution)


The test example of Newton 1669/71: f (x) = x3 − 2x − 5

Proponatur æquatio y3 − 2y − 5 = 0 resolvenda, Et sit 2numerus utcunque inventus qui minus quam decima sui partediffert a radice quæsita.Let the equation y3 − 2y − 5 = 0 be proposed for solution and letthe number 2 be found, one way or other, which differs from therequired root by less than its tenth part. Newton’s handwriting in the

manuscript De methodis MS Add. 3960.14, Cambridge University Library,

Cambridge, UK. Transcribed and translated by Whiteside

f (x) = x3 − 2x − 5 and x0 = 2


Newton’s method f (x) = x3 − 2x − 5 and x0 = 2

Using the notation on previous slide we have:

g0(s) = f (2 + s) = s3 + 6s2 + 10s − 1 and s0 = 110 .

g1(s) = g0( 110 + s) =

= s3 +63

10s2 +

1123

100s +

61

1000and s1 = − 61

11230

.g2(s) = g1(− 61

11230 + s) =

= s3+35 283

5 615s2+

351 906 913

31 528 225s+

32 878 756

177 030 983 375and s2 = −

32 878 756

1 975 957 316 495

.

x = 2 + 110 −

6111230 −

32 878 7561 975 957 316 495 = 4 138 744 325 037

1 975 957 316 495 .

The sequence is 2, 2.1, 2.094568121104185, 2.094551481698199.


Newton’s cubic equation

To simplify the computation Newton and Wallis aretruncating (in second iteration) 61/11230 to 0.0054. Thismakes computing s2 by hand easier.

The sequence of corrections or updates is then 0.1, -0.0054and -0.00004852 with final iterate x = 2.09455148

Final error 1.5 10−9 (3 iterations) (compared to the root).

-1 -0.5 0 0.5 1 1.5 2 2.5

x

-8

-6

-4

-2

0

2

4

6Newton Example

x3 -2x-5

-0.15 -0.1 -0.05 0 0.05 0.1

s

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5Newton shifted polynomials

g0(s)

g1(s)

g2(s)


Raphson’s method

Raphson’s method (x0 = 2):

s0 = − f (2)f ′(2) = −1

10 and x1 = 2110 .

s1(s) = − f ( 2110

)

f ′( 2110

)= −

6110001123100

and x2 = 2110 −

6111230 = 11761

5615 .

s2 = − f ( 117615615

)

f ′( 117615615

)= −

32 878 756177 030 983 375

351 906 91331 528 225

= − 32 878 7561 975 957 316 495 .

x ≈ x3 = x2 + s2 = 4 138 744 325 0371 975 957 316 495 .

Raphson is generating the sequence of iterates 2.1, 2.0946,2.094551483, and 2.0945514815427104141 with an error of3.7 10−13 in the final iterate (4 iterations)


Raphson and Halley

gk(s) = f (xk) + f ′(xk)s +1

2f ′′(xk)s2

Solving the quadratic equation gk(s) = 0 will give what is calledHalley’s irrational method or using the approximation√

a2 − b ≈ a− ab

2a2 − 12b

where a and b are real numbers giving Halley’s rational method.

Method Update sk

Raphson sk = − f (xk )f ′(xk )

Halley Rational sk = − f ′(xk )f (xk )

f ′(xk )2− 12f ′′(xk )f (xk )

Halley Irrational sk = − f ′(xk )−√

f ′(xk )2−2f (xk )f ′′(xk )

f ′′(xk )


Halley’s rational method on f (x) = x3 − 2x − 5, x0 = 2

xk+1 = xk + sk , sk = − f ′(xk)f (xk)

f ′(xk)2 − 12 f′′(xk)f (xk)

Halley’s rational formula and two iterations

s0 = − f ′(2)f (2)

f ′(2)2− 12f ′′(2)f (2)

= 553 and x1 = 111

53 ,

f (x1) = 25·11533 , f

′(x1) = 5·6269532 and f ′′(x2) = 2·32·37

53 ,

s1 = 1103344052079190773 , x2 = 1 090 082 546 191

52 079 190 773

The sequence 2, 2.094339622641509, and 2.094551481540164with an error 2.2 10−12 in the final iterate.

Halley well aware of triple number of figures in the iterates.


Early rediscovery of Halley rational method

Halley’s approximation

gk(s) = f (xk) + f ′(xk)s +1

2f ′′(xk)s2

Replace s2 ≈ − f (xk )f ′(xk )s.

The approximate function is then

gk(s) = f (xk) +

(f ′(xk)− 1

2

f (xk)

f ′(xk)f ′′(xk)

)s.

This derivation was used by Wolff in 1713, Holliday 1745 andMclaurin in 1748.This derivation is usually credited Frame from 1944.This derivation is also suitable for deriving Halley’s method tosystems of nonlinear equations. Let F : Rn 7→ Rn and

s(1)k = −F ′(xk)−1F (xk). Then(

F ′(xk) +1

2F ′′(xk)s

(1)k

)sk = F (xk), xk+1 = xk + sk .

This can be rewritten as

xk+1 = xk−

{I +

1

2L(xk)

[I − 1

2L(xk)

]−1}

(F ′(xk))−1F (xk), k = 0, 1, . . . ,

where

L(x) = (F ′(x))−1F ′′(x)(F ′(x))−1F (x), x ∈ Rn.

which is the ’standard’ way to express the Halley method.


Simpson’s Contribution 1740

Raphson’s test cases:(9 out of 33) (to the left) and Simpson’s cases

Raphson x0 Iter

x2 − 2 1 4

x3 − 37945 33 3

x4 − 2741583974 229 2

x5 − 2327834559873 298 1

x2 + 587x − 987459 746 2

x2 − 5x − 31 8 3

−x2 + 8x − 14 2 4

x3 + 24x − 587914 83 2

−x3 + 300x − 1000 3 4

Simpson x0 Iter

−x3 + 300x − 1000 3.5 2√

1− x +√

1− 2x2 +√

1− 3x3 − 2 .5 2(y +

√y2 − x2 − 10

x +√

y2 + x − 12

) (56

)2 49

(x − x

(x+y)2

)− 25(1− x2

(1+y)2 )

81(1− x2

(1+y)2 )− 49( xy− xy

(1+x)2 )

(.8.6

)2

(xx + yy − 1000xy + yx − 100

) (4.52.5

)2

Simpson uses fluxion (derivative) notation.


Other contributions

Wallis 1685, 1693 (Short discussion of Raphson’s method)

Halley 1694 contains three examples

Raphson 1690, 1697 (2nd edition). The Newton testcase istaken out!

Ward 1695

Sault 1694 (6 of Raphson’s examples)

Parson and Wastell 1704

Jones 1706 (Halley rational and irrational)

Wolff 1713 (Raphson and Halley) (Translated to English in1739)

Ronayne 17717 (Raphson)

Colson 1736 (Methods of Newton and Raphson nearly agree)

Simpson 1740 (Kepler’s equation) and 1745

Stewart 1745

Diderot 1751


Newton’s and Raphson’s methods are the same

In 1745 Stewart made a translation and extensivecommentaries of Newton’s text De analysi transcribed andedited by Jones in 1711.

Stewart is the first to point out that the methods of Newtonand Raphson are the same:

And as to Mr. Raphson’s Method of Approximation in the ex-tracting the Roots of Equations, published in his Analysis Equa-tionum universalis, it is, in effect, the very same with our Au-thor’s Method here laid down;

He also derives Halley rational and irrational formulas.

In the expanded French translation in 1751 by Diderot of thedictionary by Chambers from 1728, Diderot uses Raphson’smethod but adds

Cette methode pour approcher les racines des equationsnumeriques, est due a M. Newton.


The approach of Colin Maclaurin (1698–1746)

In Maclaurin treatise of algebra from 1748.

Let g0(s) = f (x0 + s)For k = 0, 1, 2, . . . ,m − 1

Compute gk(s) ≈ gk(s)Solve for sk in gk(s) = 0Let gk+1(s) = gk(sk + s)

x = x0 +∑m−1

k=0 sk

gk(s) = gk(sk) + g ′(sk)s + 12g′′(sk)s2

x3 − 15x2 + 63x − 50 = 0 and x3 − 2x − 5 = 0 with linearapproximation (The Newton way)

Substitution 1 + y for x in x3 − 15x2 + 63x − 50 = 0 gives−1 + 36y − 12y2 + y3 = 0. Maclaurin takes quadraticapproximation and and in effect use Halley’s rational method.

Better would be replace s2 ≈ − f (xk )f ′(xk )s and s3 ≈

(f (xk )f ′(xk )

)2s


Kepler’s Equation M = E − e sin(E )

M = 2πt

T,

a = (semi)major axis ,

b = (semi)minor axis,

e =

√1− b2

a2

Derived by Newton in Philosophiae Naturalis PrincipiaMathematica 1687, 1713, 1726 (3rd ed).

Newton derived a solution technique that was shown by JohnCouch Adams in 1882 to be Newton-Raphson.

Newton gave no numerical example.


Simpson and Kepler’s equation

Simpson is using the time after passing of the aphelion(farthest point in the elliptic orbit)

Simpson derived the equivalent equation

E + e sin(E ) = D

and a solution technique in 1740 (Newton–Raphson)

Example I: Equation E + 2.86479 sin(E ) = 72.21

Demonstrated (using degrees) with 2 iterations.

Simpson: Sequence: 70◦, 69◦ 31′ 16100

′′(69.536), and

69◦ 31′ 34′′ (69.5261)The sequence is (rounded) 70, 69.5261, 69.5262


A (more) modern approach

Joseph Fenn was an Irish mathematics master. His New andComplete System of Algebra or Specious Arithmetic (1767) can beconsidered one of the first “modern” algebra texts. The term“specious arithmetic” is defined as “the calculus of quantities asdesignated by letters of the alphabet.”

Name f (x) x0 Iterations

Chap LXI√1− x +

√1− 2x2 +

√1− 3x3 − 2 .5 2

Chap LXIII

(y +

√y 2 − x2 − 10

x +√

y 2 + x − 12

) (56

)2

Chap LXIV

49(x − x

(x+y)2

)− 25(1− x2

(1+y)2 )

81(1− x2

(1+y)2 )− 49( xy− xy

(1+x)2 )

(.8.6

)2

All examples from Simpson using Leibniz notation.


Concluding Remarks

“Newton = Raphson” was known around 1750

Traub, 1964: The Halley method must share with the secantmethod the distinction of being the most frequentlyrediscovered method in the literature.

Halley’s rational method rediscovered also in the 1700s.

Simpson first to consider “non-polynomial” equations.

Fourier of 1818 and 1831 the Newton-Raphson is attributedto Newton. The immense popularity of Fourier’s writing led tothe universal adoption of the name ”Newton’s method” forthe Newton-Raphson process

Joseph Lagrange in 1798 derives Raphson’s method and writesthat the Newton’s method and Raphson’s method are the samebut presented differently and Raphson’s method is plus simple quecelle de Newton.

Thank you for listeningTrond Steihaug Newton, Raphson, Halley & Simpson 20 / 21

Worth Reading

Tjalling J. Ypma, Historical development of theNewton-Raphson method. SIAM Review Vol.37(4), pp. 531 -551, 1995.

Nick Kollerstrom. Thomas Simpson and ’Newton’s method ofapproximation’: an enduring myth. BJHS, Vol. 25, pp.347-54, 1992.

David J. Thomas. JOSEPH RAPHSON, F.R.S.. Notes Rec.R. Soc. Lond. Vol.4(4), pp. 151-167. 1990

Trond Steihaug. Computational science in the eighteenthcentury. Test cases for the methods of Newton, Raphson, andHalley: 1685 to 1745, Numerical Algorithms. 2019.https://doi.org/10.1007/s11075-019-00724-8,


Documents

A little on Newton, Raphson and Halley, a little more on ... EuroAd Workshop - Trond Steiha… · the number 2 be found, one way or other, which di ers from the required root by less