Embed Size (px)
Citation preview
CIS
541
–N
umer
ical
Met
hods
Rog
er C
raw
fisO
hio
Stat
e U
nive
rsity
CIS
541
–N
umer
ical
Met
hods
Roo
t Fin
ding
Aug
ust 1
2, 2
005
OSU
/CIS
541
3
Root
Fin
ding
Top
ics
•B
i-sec
tion
Met
hod
•N
ewto
n’s m
etho
d•
Use
s of r
oot f
indi
ng fo
r sqr
t() a
nd re
cipr
ocal
sqrt(
)•
Seca
nt M
etho
d•
Gen
eral
ized
New
ton’
s met
hod
for s
yste
ms o
f non
-lin
ear e
quat
ions
–Th
e Ja
cobi
anm
atrix
•Fi
xed-
poin
t for
mul
as, B
asin
s of A
ttrac
tion
and
Frac
tals
.
Aug
ust 1
2, 2
005
OSU
/CIS
541
4
Mot
ivat
ion
•M
any
prob
lem
s can
be
re-w
ritte
n in
to a
fo
rm su
ch a
s:–
f(x,
y,z,
…) =
0–
f(x,
y,z,
…) =
g(s
,q,…
)
Aug
ust 1
2, 2
005
OSU
/CIS
541
5
Mot
ivat
ion
•A
root
, r, o
f fun
ctio
nf o
ccur
s whe
n f(r
)= 0
.•
For e
xam
ple:
–f(x
) = x
2–
2x –
3ha
s tw
oro
ots a
t r=
-1 a
nd r
= 3.
•f(-
1) =
1 +
2 –
3 =
0•
f(3)=
9 –
6 –
3 =
0
–W
e ca
n al
so lo
ok a
t fin
its f
acto
red
form
.f(x
)= x
2–
2x–
3 =
(x +
1)(
x –
3)
Aug
ust 1
2, 2
005
OSU
/CIS
541
6
Fact
ored
For
m o
f Fun
ctio
ns
•Th
e fa
ctor
ed fo
rm is
not
lim
ited
to
poly
nom
ials
.•
Con
side
r:f(x
)= x
sin
x –
sin
x.A
root
exi
sts a
t x=
1.f(x
) = (x
–1)
sin
x
•O
r, f(x
) = si
n πx
=> x
(x–
1) (x
–2)
…
Aug
ust 1
2, 2
005
OSU
/CIS
541
7
Exam
ples
•Fi
ndx,
such
that
–xp
= c,⇒
xp–
c =
0•
Cal
cula
te th
e sq
rt(2)
–x2
–2
= 0
•B
allis
tics
–D
eter
min
e th
e ho
rizon
tal d
ista
nce
at w
hich
the
proj
ectil
e w
ill in
ters
ect t
he te
rrai
n fu
nctio
n.
()(
)2
20
22
xx
x−
==
−+
Aug
ust 1
2, 2
005
OSU
/CIS
541
8
Root
Fin
ding
Alg
orith
ms
•C
lose
d or
Bra
cket
ed te
chni
ques
–B
i-sec
tion
–R
egul
a-Fa
lsi
•O
pen
tech
niqu
es–
New
ton
fixed
-poi
nt it
erat
ion
–Se
cant
met
hod
•M
ultid
imen
sion
al n
on-li
near
pro
blem
s–
The
Jaco
bian
mat
rix•
Fixe
d-po
int i
tera
tions
–C
onve
rgen
ce a
nd F
ract
al B
asin
s of A
ttrac
tion
Aug
ust 1
2, 2
005
OSU
/CIS
541
9
Bise
ctio
n M
etho
d
•B
ased
on
the
fact
that
the
func
tion
will
ch
ange
sign
s as i
t pas
ses t
hru
the
root
.•
f(a)*
f(b) <
0
•O
nce
we
have
a ro
ot b
rack
eted
, we
sim
ply
eval
uate
the
mid
-poi
nt a
nd h
alve
the
inte
rval
.
Aug
ust 1
2, 2
005
OSU
/CIS
541
10
Bise
ctio
n M
etho
d
•c=
(a+b
)/2
ab
c
f(a)>
0
f(b)<
0
f(c)>
0
Aug
ust 1
2, 2
005
OSU
/CIS
541
11
Bise
ctio
n M
etho
d
•G
uara
ntee
d to
con
verg
e to
root
if o
ne e
xist
s w
ithin
the
brac
ket.
cb
a
a =
cf(a
)>0
f(b)<
0f(c
)<0
Aug
ust 1
2, 2
005
OSU
/CIS
541
12
Bise
ctio
n M
etho
d
•Sl
owly
con
verg
es to
the
root
bc
a
b =
cf(b
)<0
Aug
ust 1
2, 2
005
OSU
/CIS
541
13
Bise
ctio
n M
etho
d
•Si
mpl
e al
gorit
hm:
Given: a and b, such that f(a)*f(b)<0
Given: error tolerance, err
c=(a+b)/2.0; // Find the midpoint
While( |f(c)| > err ) {
if( f(a)*f(c) < 0 ) // root in the left half
b = c;
else // root in the right half
a = c;
c=(a+b)/2.0; // Find the new midpoint
} return c;
Aug
ust 1
2, 2
005
OSU
/CIS
541
14
Rela
tive
Erro
r
•W
e ca
n de
velo
p an
upp
er b
ound
on
the
rela
tive
erro
r qui
te e
asily
.
,b
ax
ca
xa
x−
−≥
≤
ab
xc
Aug
ust 1
2, 2
005
OSU
/CIS
541
15
Abso
lute
Err
or
•W
hat d
oes t
his m
ean
in b
inar
y m
ode?
–er
r 0≤
|b-a
|–
err i+
1≤
err i/
2 =
|b-a
|/2i+
1
•W
e ga
in a
n ex
tra b
it ea
ch it
erat
ion!
!!•
To re
ach
a de
sire
d ab
solu
teer
ror t
oler
ance
:–
err i+
1 ≤
err to
l⇒
2
2
log
tol
n
tol
ba
err b
an
err
−≤
⎛−
⎞≥
⎜⎟
⎝⎠
Aug
ust 1
2, 2
005
OSU
/CIS
541
16
Abso
lute
Err
or
•Th
e bi
sect
ion
met
hod
conv
erge
s lin
early
or
first
-ord
er to
the
root
.•
If w
e ne
ed a
n ac
cura
cy o
f 0.0
001
and
our
initi
al in
terv
al (b
-a)=
1, th
en:
2-n<
0.00
01 ⁼⇒
14 it
erat
ions
•N
ot b
ad, w
hy d
o I n
eed
anyt
hing
els
e?
Aug
ust 1
2, 2
005
OSU
/CIS
541
17
A N
ote
on F
unct
ions
•Fu
nctio
ns c
an b
e si
mpl
e, b
ut I
may
nee
d to
ev
alua
te it
man
y m
any
times
.•
Or,
a fu
nctio
n ca
n be
ext
rem
ely
com
plic
ated
. Con
side
r:•
Inte
rest
ed in
the
conf
igur
atio
n of
air
vent
s (po
sitio
n,
orie
ntat
ion,
dire
ctio
n of
flow
) tha
t mak
es th
e te
mpe
ratu
re in
the
room
at a
par
ticul
ar p
ositi
on
(teac
her’
s des
k) e
qual
to 7
2°.
•Is
this
a fu
nctio
n?
Aug
ust 1
2, 2
005
OSU
/CIS
541
18
A N
ote
on F
unct
ions
•Th
is fu
nctio
n m
ay re
quire
a c
ompl
ex th
ree-
dim
ensi
onal
hea
t-tra
nsfe
r cou
pled
with
a
fluid
-flo
w si
mul
atio
n to
eva
luat
eth
e fu
nctio
n. ⇒
hour
s of c
ompu
tatio
nal t
ime
on
a su
perc
ompu
ter!
!!•
May
not
nec
essa
rily
even
be
com
puta
tiona
l.•
Tech
niqu
es e
xist
ed b
efor
e th
e B
abyl
onia
ns.
Aug
ust 1
2, 2
005
OSU
/CIS
541
19
Root
Fin
ding
Alg
orith
ms
•C
lose
d or
Bra
cket
ed te
chni
ques
–B
i-sec
tion
–R
egul
a-Fa
lsi
•O
pen
tech
niqu
es–
New
ton
fixed
-poi
nt it
erat
ion
–Se
cant
met
hod
•M
ultid
imen
sion
al n
on-li
near
pro
blem
s–
The
Jaco
bian
mat
rix•
Fixe
d-po
int i
tera
tions
–C
onve
rgen
ce a
nd F
ract
al B
asin
s of A
ttrac
tion
Aug
ust 1
2, 2
005
OSU
/CIS
541
20
Regu
laFa
lsi
•In
the
book
und
er c
ompu
ter p
robl
em 1
6 of
se
ctio
n 3.
3.•
Ass
ume
the
func
tion
is li
near
with
in th
e br
acke
t.•
Find
the
inte
rsec
tion
of th
e lin
e w
ith th
e x-
axis
.
Aug
ust 1
2, 2
005
OSU
/CIS
541
21
Regu
laFa
lsi
ab
c
() (
)
()
()
()
()
()
()
()
()
0(
)
0(
)(
)(
)(
)(
)(
)
fa
fb
yx
fb
xb
ab
fa
fb
yc
fb
cb
ab
ab
fb
cb
fa
fb
fb
ab
cb
fa
fb
−=
+−
−−
==
+−
−−
=+
−− −
=−
−
f(c)<
0
Aug
ust 1
2, 2
005
OSU
/CIS
541
22
Regu
laFa
lsi
•La
rge
bene
fit w
hen
the
root
is m
uch
clos
er
to o
ne si
de.
–D
o I h
ave
to w
orry
abo
ut d
ivis
ion
by z
ero?
ac
b
Aug
ust 1
2, 2
005
OSU
/CIS
541
23
Regu
laFa
lsi
•M
ore
gene
rally
, we
can
stat
e th
is m
etho
d as
: c=wa
+ (1
-w)b
–Fo
r som
e w
eigh
t, w
, 0≤w
≤1.
–If
|f(a
)| >>
|f(b
)|, th
en w
< 0
.5•
Clo
ser t
o b.
Aug
ust 1
2, 2
005
OSU
/CIS
541
24
Brac
ketin
g M
etho
ds
•B
rack
etin
g m
etho
ds a
re ro
bust
•C
onve
rgen
ce t
ypic
ally
slow
er th
an o
pen
met
hods
•U
se to
find
app
roxi
mat
e lo
catio
n of
root
s•
“Pol
ish”
with
ope
n m
etho
ds•
Rel
ies o
n id
entif
ying
two
poin
ts a
,bin
itial
ly su
ch
that
: •f(a
) f(b
) < 0
•G
uara
ntee
d to
con
verg
e
Aug
ust 1
2, 2
005
OSU
/CIS
541
25
Root
Fin
ding
Alg
orith
ms
•C
lose
d or
Bra
cket
ed te
chni
ques
–B
i-sec
tion
–R
egul
a-Fa
lsi
•O
pen
tech
niqu
es–
New
ton
fixed
-poi
nt it
erat
ion
–Se
cant
met
hod
•M
ultid
imen
sion
al n
on-li
near
pro
blem
s–
The
Jaco
bian
mat
rix•
Fixe
d-po
int i
tera
tions
–C
onve
rgen
ce a
nd F
ract
al B
asin
s of A
ttrac
tion
Aug
ust 1
2, 2
005
OSU
/CIS
541
26
New
ton’
s Met
hod
•O
pen
solu
tion,
that
requ
ires o
nly
one
curr
ent g
uess
.•
Roo
t doe
s not
nee
d to
be
brac
kete
d.•
Con
side
r som
e po
int x
0.–
If w
e ap
prox
imat
e f(x
)as a
line
abo
ut x
0, th
en
we
can
agai
n so
lve
for t
he ro
ot o
f the
line
.
00
0(
)(
)()
()
lx
fx
xx
fx
′=
−+
Aug
ust 1
2, 2
005
OSU
/CIS
541
27
New
ton’
s Met
hod
•So
lvin
g, le
ads t
o th
e fo
llow
ing
itera
tion:
01
00
1()
0(
)(
)(
)(
)i
ii
i
lx
fx
xx
fx fx
xx
fx
+
=
=−
′
=−
′
Aug
ust 1
2, 2
005
OSU
/CIS
541
28
New
ton’
s Met
hod
•Th
is c
an a
lso
be se
en fr
om T
aylo
r’s S
erie
s.•
Ass
ume
we
have
a g
uess
, x0,
clos
e to
the
actu
al ro
ot. E
xpan
d f(x
)abo
ut th
is p
oint
.
•If
dx
is sm
all,
then
dxn
quic
kly
goes
to z
ero.
2
()
()
()
()
02!
i
ii
ii
xx
xx
fx
xf
xxf
xf
x
=+∆
∆′
′′+∆
=+∆
++
≡L
1(
)(
)i
ii
i
fx
xx
xf
x+
∆≈
−=−
′
Aug
ust 1
2, 2
005
OSU
/CIS
541
29
New
ton’
s Met
hod
•G
raph
ical
ly, f
ollo
w th
e ta
ngen
t vec
tor d
own
to th
e x-
axis
inte
rsec
tion.
x ix i
+1
Aug
ust 1
2, 2
005
OSU
/CIS
541
30
New
ton’
s Met
hod
•Pr
oble
ms
dive
rges
x 0
1
2
3
4
Aug
ust 1
2, 2
005
OSU
/CIS
541
31
New
ton’
s Met
hod
•N
eed
the
initi
al g
uess
to b
e cl
ose,
or,
the
func
tion
to b
ehav
e ne
arly
line
ar w
ithin
the
rang
e.
Aug
ust 1
2, 2
005
OSU
/CIS
541
32
Find
ing
a sq
uare
-roo
t
•Ev
er w
onde
r why
they
cal
l thi
s a sq
uare
-ro
ot?
•C
onsi
der t
he ro
otso
f the
equ
atio
n:•
f(x)=
x2 -
a
•Th
is o
f cou
rse
wor
ks fo
r any
pow
er:
0,p
pa
xa
pR
⇒−
=∈
Aug
ust 1
2, 2
005
OSU
/CIS
541
33
Find
ing
a sq
uare
-roo
t
•Ex
ampl
e: √
2 =
1.41
4213
5623
7309
5048
8016
8872
4209
7
•Le
t x0
be o
ne a
nd a
pply
New
ton’
s met
hod.
2
1
0 1 2
()
2
21
22
21 1
23
11.
5000
0000
002
12
13
417
1.41
6666
6667
22
312
ii
ii
ii
fx
x xx
xx
xx
x x x+′=
⎛⎞
−=
−=
+⎜
⎟⎝
⎠=
⎛⎞
=+
==
⎜⎟
⎝⎠
⎛⎞
=+
=≈
⎜⎟
⎝⎠
Aug
ust 1
2, 2
005
OSU
/CIS
541
34
Find
ing
a sq
uare
-roo
t
•Ex
ampl
e: √
2 =
1.41
4213
5623
7309
5048
8016
8872
4209
7
•N
ote
the
rapi
d co
nver
genc
e
•N
ote,
this
was
don
e w
ith th
e st
anda
rd
Mic
roso
ft ca
lcul
ator
to m
axim
um p
reci
sion
.
3 4 5 6
117
2457
71.
4114
2156
862
1217
408
1.41
4213
5623
746
1.41
4213
5623
7309
5048
8016
896
1.41
4213
5623
7309
5048
8016
8872
4209
7
x x x x
⎛⎞
=+
=≈
⎜⎟
⎝⎠
= = =
((
((
Aug
ust 1
2, 2
005
OSU
/CIS
541
35
Find
ing
a sq
uare
-roo
t
•C
an w
e co
me
up w
ith a
bet
ter i
nitia
l gue
ss?
•Su
re, j
ust d
ivid
e th
e ex
pone
nt b
y 2.
–R
emem
ber t
he b
ias o
ffse
t–
Use
bit-
mas
ks to
ext
ract
the
expo
nent
to a
n in
tege
r, m
odify
and
set t
he in
itial
gue
ss.
•Fo
r √2,
this
will
lead
to x
0=1
(rou
nd d
own)
.
Aug
ust 1
2, 2
005
OSU
/CIS
541
36
Con
verg
ence
Rat
e of
New
ton’
s
•N
ow,
()
2
2
0(
)(
)1
()
()
()
(),
,2
1(
)(
)(
)2
nn
nn
nn
nn
nn
nn
nn
n
nn
nn
n
ex
xor
xx
ef
xf
xe
fx
ef
xe
fx
ef
fors
ome
xx
fx
ef
xe
f
ξξ
ξ
=−
=+
≡=
+
′′′
+=
++
∈
′′′
∴+
=−
11
21
()
()
()
()
()
()
()
()
1 2(
)
nn
nn
nn
nn
nn
n
n
nn
nn
fx
fx
ex
xx
xe
fx
fx
ef
xf
xf
x
fe
ef
xξ
++
+
=−
=−
+=
+′
′′
+=
′
⎛⎞
′′∴
=−
⎜⎟
′⎝
⎠
Aug
ust 1
2, 2
005
OSU
/CIS
541
37
Con
verg
ence
Rat
e of
New
ton’
s
•C
onve
rges
qua
drat
ical
ly.
21
10,
10
kn
kn
ife
then
e
−
−+
≤
≤
Aug
ust 1
2, 2
005
OSU
/CIS
541
38
New
ton’
s Alg
orith
m
•R
equi
res t
he d
eriv
ativ
e fu
nctio
n to
be
eval
uate
d,
henc
e m
ore
func
tion
eval
uatio
ns p
er it
erat
ion.
•A
robu
st so
lutio
n w
ould
che
ck to
see
if th
e ite
ratio
n is
step
ping
too
far a
nd li
mit
the
step
.•
Mos
t use
s of N
ewto
n’s m
etho
d as
sum
e th
e ap
prox
imat
ion
is p
retty
clo
se a
nd a
pply
one
to
thre
e ite
ratio
ns b
lindl
y.
Aug
ust 1
2, 2
005
OSU
/CIS
541
39
Div
isio
n by
Mul
tiplic
atio
n
•N
ewto
n’s m
etho
d ha
s man
y us
es in
co
mpu
ting
basi
c nu
mbe
rs.
•Fo
r exa
mpl
e, c
onsi
der t
he e
quat
ion:
•N
ewto
n’s m
etho
d gi
ves t
he it
erat
ion:
10
ax−
= ()
21
2
1
1
2
kk
kk
kk
k
kka
xx
xx
xax
xx
ax
+
−=
−=
+−
−
=−
Aug
ust 1
2, 2
005
OSU
/CIS
541
40
Reci
proc
al S
quar
e Ro
ot
•A
mor
e us
eful
ope
rato
r may
be
the
reci
proc
al-s
quar
e ro
ot, a
s opp
osed
to th
e sq
uare
-roo
t.–
Nee
ded
to n
orm
aliz
e ve
ctor
s–
Can
be
used
to c
alcu
late
the
squa
re-r
oot.
1a
aa=
Aug
ust 1
2, 2
005
OSU
/CIS
541
41
Reci
proc
al S
quar
e Ro
ot
•N
ewto
n’s i
tera
tion
yiel
ds:
2
3
1(
)0
2(
)
Let
fx
ax
fx
x=−
=
′=−
()3
1
2
22
13
2
kk
kk
kk
xx
xx
xx
+=
+−
=−
Aug
ust 1
2, 2
005
OSU
/CIS
541
42
1/Sq
rt(2
)
•Le
t’s lo
ok a
t the
con
verg
ence
for t
he
reci
proc
al sq
uare
-roo
t of 2
.
()(
)(
) ()
0
21
22 3 4 5 6 7
1 0.5
13
21
0.5
0.5
0.5
32
0.5
0.62
5
0.69
3359
375
0.70
6708
4684
9679
9468
9941
4062
5x
0.70
7106
4446
9590
7075
5117
3067
6593
228
x0.
7071
0678
1186
3073
3592
5435
9312
3773
8x
0.70
7106
7811
8654
7524
4008
4423
97
x x x x x
= =−
⋅=
=−
⋅=
= = = = =(
2481
If w
e co
uld
only
star
t he
re!!
Aug
ust 1
2, 2
005
OSU
/CIS
541
43
1/Sq
rt(x
)
•W
hat i
s a g
ood
choi
ce fo
r the
initi
al se
ed
poin
t?–
Opt
imal
–th
e ro
ot, b
ut it
is u
nkno
wn
–C
onsi
der t
he n
orm
aliz
ed fo
rmat
of t
he n
umbe
r:
–W
hat i
s the
reci
proc
al?
–W
hat i
s the
squa
re-r
oot?
()
127
21
2(1
.)
se
m−
−⋅
⋅
Aug
ust 1
2, 2
005
OSU
/CIS
541
44
1/Sq
rt(x
)
•Th
eore
tical
ly,
•C
urre
nt G
PU’s
prov
ide
this
ope
ratio
n in
as l
ittle
as
2 cl
ock
cycl
es!!
! How
?•
How
man
y si
gnifi
cant
bits
doe
s thi
s est
imat
e ha
ve?
()
()
()
()
11
112
72
22 12
71
22
312
71
127
22
11.
2
1.2
1.2
e
e
e
xm
x
m m
−−
−−
−−
•−
−−
==
•
⎛⎞
=• ⎜
⎟⎝
⎠⎛
⎞=
• ⎜⎟
⎝⎠
New
bit-
patte
rn
for t
he e
xpon
ent
Aug
ust 1
2, 2
005
OSU
/CIS
541
45
1/Sq
rt(x
)
•G
PU’s
such
as n
Vid
ia’s
FX c
ards
pro
vide
a
23-b
it ac
cura
te re
cipr
ocal
squa
re-r
oot i
n tw
o cl
ock
cycl
es, b
y on
ly d
oing
2 it
erat
ions
of
New
ton’
s met
hod.
•N
eed
24-b
its o
f pre
cisi
on =
>–
Prev
ious
iter
atio
n ha
d 12
-bits
of p
reci
sion
–St
arte
d w
ith 6
-bits
of p
reci
sion
Aug
ust 1
2, 2
005
OSU
/CIS
541
46
1/Sq
rt(x
)
•Ex
amin
e th
e m
antis
sa te
rm a
gain
(1.m
).•
Poss
ible
pat
tern
s are
:–
1.00
0…, 1
.100
…, 1
.010
…, 1
.110
…, …
•Pr
e-co
mpu
te th
ese
and
stor
e th
e re
sults
in a
tabl
e.
Fast
and
eas
y ta
ble
look
-up.
•A
6-b
it ta
ble
look
-up
is o
nly
64 w
ords
of o
n ch
ip
cach
e.•
Not
e, w
e on
ly n
eed
to lo
ok-u
p on
m, n
ot 1
.m.
•Th
is y
ield
s a re
cipr
ocal
squa
re-r
oot f
or th
e fir
st
seve
n bi
ts, g
ivin
g us
abo
ut 6
-bits
of p
reci
sion
.
Aug
ust 1
2, 2
005
OSU
/CIS
541
47
1/Sq
rt(x
)
•Sl
ight
pro
blem
:–
The
pr
oduc
es a
resu
lt be
twee
n 1
and
2.–
Hen
ce, i
t rem
ains
nor
mal
ized
, 1.m
’.–
For
,
we
get a
num
ber b
etw
een
½ a
nd 1
.–
Nee
d to
shift
the
expo
nent
.
1.m
1 x
Aug
ust 1
2, 2
005
OSU
/CIS
541
48
Root
Fin
ding
Alg
orith
ms
•C
lose
d or
Bra
cket
ed te
chni
ques
–B
i-sec
tion
–R
egul
a-Fa
lsi
•O
pen
tech
niqu
es–
New
ton
fixed
-poi
nt it
erat
ion
–Se
cant
met
hod
•M
ultid
imen
sion
al n
on-li
near
pro
blem
s–
The
Jaco
bian
mat
rix•
Fixe
d-po
int i
tera
tions
–C
onve
rgen
ce a
nd F
ract
al B
asin
s of A
ttrac
tion
Aug
ust 1
2, 2
005
OSU
/CIS
541
49
Seca
nt M
etho
d
•W
hat i
f we
do n
ot k
now
the
deriv
ativ
e of
f(x
)?
Tang
ent v
ecto
r
x ix i-1
Seca
nt li
ne
Aug
ust 1
2, 2
005
OSU
/CIS
541
50
Seca
nt M
etho
d
•A
s we
conv
erge
on
the
root
, the
seca
nt li
ne
appr
oach
es th
e ta
ngen
t.•
Hen
ce, w
e ca
n us
e th
e se
cant
line
as a
n es
timat
e an
d lo
ok a
t whe
re it
inte
rsec
ts th
e x-
axis
(its
root
).
Aug
ust 1
2, 2
005
OSU
/CIS
541
51
Seca
nt M
etho
d
•Th
is a
lso
wor
ks b
y lo
okin
g at
the
defin
ition
of t
he
deriv
ativ
e:
•Th
eref
ore,
New
ton’
s met
hod
give
s:
•W
hich
is th
e Se
cant
Met
hod.
0
1
1
()
()
()
()
()
()
lim h
kk
kk
k
fx
hf
xf
xh
fx
fx
fx
xx
→
−
−+−
′=
−′
≈−
11
1
()
()
()
kk
kk
kk
k
xx
xx
fx
fx
fx−
+−
⎛⎞
−=
−⎜
⎟−
⎝⎠
Aug
ust 1
2, 2
005
OSU
/CIS
541
52
Con
verg
ence
Rat
e of
Sec
ant
•U
sing
Tay
lor’
s Ser
ies,
it ca
n be
show
n (p
roof
is in
the
book
) tha
t:1
1
11
()
1 2(
)
kk
kk
kk
kk
ex
x
fe
ec
ee
fξ ζ
++
−−
=−
⎛⎞
′′=−
≈⋅
⎜⎟
′′⎝
⎠
Aug
ust 1
2, 2
005
OSU
/CIS
541
53
Con
verg
ence
Rat
e of
Sec
ant
•Th
is is
a re
curs
ive
defin
ition
of t
he e
rror
te
rm. E
xpre
ssed
out
, we
can
say
that
:
•W
here
α=1
.62.
•W
e ca
ll th
is su
per-
linea
r con
verg
ence
.
1k
ke
Ce
α+≤
Aug
ust 1
2, 2
005
OSU
/CIS
541
54
Root
Fin
ding
Alg
orith
ms
•C
lose
d or
Bra
cket
ed te
chni
ques
–B
i-sec
tion
–R
egul
a-Fa
lsi
•O
pen
tech
niqu
es–
New
ton
fixed
-poi
nt it
erat
ion
–Se
cant
met
hod
•M
ultid
imen
siona
l non
-line
ar p
robl
ems
–T
he J
acob
ian
mat
rix
•Fi
xed-
poin
t ite
ratio
ns–
Con
verg
ence
and
Fra
ctal
Bas
ins o
f Attr
actio
n
Aug
ust 1
2, 2
005
OSU
/CIS
541
55
Hig
her-
dim
ensi
onal
Pro
blem
s
•C
onsi
der t
he c
lass
of f
unct
ions
f(x
1,x2,x
3,…,x
n)=0,
w
here
we
have
a m
appi
ng fr
om ℜ
n →ℜ
.•
We
can
appl
y N
ewto
n’s m
etho
d se
para
tely
fo
r eac
h va
riabl
e, x
i, ho
ldin
g th
e ot
her
varia
bles
fixe
d to
the
curr
ent g
uess
.
Aug
ust 1
2, 2
005
OSU
/CIS
541
56
Hig
her-
dim
ensi
onal
Pro
blem
s
•Th
is le
ads t
o th
e ite
ratio
n:
•Tw
o ch
oice
s, ei
ther
I ke
ep o
f com
plet
e se
t of o
ld
gues
ses a
nd c
ompu
te n
ew o
nes,
or I
use
the
new
on
es a
s soo
n as
they
are
upd
ated
.•
Mig
ht a
s wel
l use
the
mor
e ac
cura
te n
ew g
uess
es.
•N
ot a
uni
que
solu
tion,
but
an
infin
ite se
t of
solu
tions
.
()
()
12
12
,,
,,
,,
i
ni
ix
n
fx
xx
xx
fx
xx
→−
K K
Aug
ust 1
2, 2
005
OSU
/CIS
541
57
Hig
her-
dim
ensi
onal
Pro
blem
s
•Ex
ampl
e:•
x+y+
z=3
–So
lutio
ns:
•x=
3, y
=0, z
=0•
x=0,
y=3
, z=0
•…
Aug
ust 1
2, 2
005
OSU
/CIS
541
58
Syst
ems o
f Non
-line
ar E
quat
ions
•C
onsi
der t
he se
t of e
quat
ions
:(
)(
)
()
11
2
21
2
12
,,
,0
,,
,0
,,
,0
n n
nn
fx
xx
fx
xx
fx
xx
= = =
K K
M
K
Aug
ust 1
2, 2
005
OSU
/CIS
541
59
Syst
ems o
f Non
-line
ar E
quat
ions
•Ex
ampl
e:
•C
onse
rvat
ion
of m
ass c
oupl
ed w
ith
cons
erva
tion
of e
nerg
y, c
oupl
ed w
ith
solu
tion
to c
ompl
ex p
robl
em.
22
2
3 5 1x
xy
zx
yz
exy
xz
++
=
++
=
+−
=
Plan
e in
ters
ecte
d w
ith a
sphe
re, i
nter
sect
ed w
ith a
mor
e co
mpl
ex fu
nctio
n.
Aug
ust 1
2, 2
005
OSU
/CIS
541
60
Vect
or N
otat
ion
•W
e ca
n re
writ
e th
is u
sing
vec
tor n
otat
ion:
()
()
12
12
()
,,
,
,,
,n n
ff
f
xx
x
=
= =
fx
0f xr
rr
K K
Aug
ust 1
2, 2
005
OSU
/CIS
541
61
New
ton’
s Met
hod
for
Non
-line
ar S
yste
ms
•N
ewto
n’s m
etho
d fo
r non
-line
ar sy
stem
s ca
n be
writ
ten
as:
()
()
()
1(
1)(
)(
)(
)
()
kk
kk
kw
here
isth
eJac
obia
nmat
rix
−+
⎡⎤
′=
−⎣
⎦′
xx
fx
fx
fx
Aug
ust 1
2, 2
005
OSU
/CIS
541
62
The
Jaco
bian
Mat
rix
•Th
e Ja
cobi
anco
ntai
ns a
ll th
e pa
rtial
der
ivat
ives
of
the
set o
f fun
ctio
ns.
•N
ote,
that
thes
e ar
e al
l fun
ctio
ns a
nd n
eed
to b
e ev
alua
ted
at a
poi
nt to
be
usef
ul.
11
1
12
22
2
12
12
n n
nn
n n
ff
fx
xx
ff
fx
xx
ff
fx
xx
∂∂
∂⎡
⎤⎢
⎥∂
∂∂
⎢⎥
∂∂
∂⎢
⎥⎢
⎥∂
∂∂
=⎢
⎥⎢
⎥⎢
⎥∂
∂∂
⎢⎥
⎢⎥
∂∂
∂⎣
⎦
J
L L
MM
OM
L
Aug
ust 1
2, 2
005
OSU
/CIS
541
63
The
Jaco
bian
Mat
rix
•H
ence
, we
writ
e(
)(
)(
)
()
()
()
()
()
()
()
()
()
11
1
12
()
()
()
22
2(
)1
2
()
()
()
12
()
ii
i
n
ii
ii
n
ii
in
nn n
ff
fx
xx
ff
fx
xx
ff
fx
xx
∂∂
∂⎡
⎤⎢
⎥∂
∂∂
⎢⎥
∂∂
∂⎢
⎥⎢
⎥∂
∂∂
=⎢
⎥⎢
⎥⎢
⎥∂
∂∂
⎢⎥
⎢⎥
∂∂
∂⎣
⎦
xx
x
xx
xJ
x
xx
x
L L
MM
OM
L
Aug
ust 1
2, 2
005
OSU
/CIS
541
64
Mat
rix
Inve
rse
•W
e de
fine
the
inve
rse
of a
mat
rix, t
he sa
me
as th
e re
cipr
ocal
.1
1a
a
−=
⎡⎤
⎢⎥
⎢⎥
==⎢
⎥⎢
⎥⎣
⎦
1
10
00
10
AA
I
00
1
L L
MM
OM
L
Aug
ust 1
2, 2
005
OSU
/CIS
541
65
New
ton’
s Met
hod
•If
the
Jaco
bian
is n
on-s
ingu
lar,
such
that
its
inve
rse
exis
ts, t
hen
we
can
appl
y th
is to
N
ewto
n’s m
etho
d.•
We
rare
ly w
ant t
o co
mpu
te th
e in
vers
e, so
in
stea
d w
e lo
ok a
t the
pro
blem
.
()
()
1(
1)(
)(
)(
)
()
()
ii
ii
ii
−+
⎡⎤
′=
−⎣
⎦=
+
xx
fx
fx
xh
Aug
ust 1
2, 2
005
OSU
/CIS
541
66
New
ton’
s Met
hod
•N
ow, w
e ha
ve a
line
ar sy
stem
and
we
solv
e fo
r h.
•R
epea
t unt
il h
goes
to z
ero.
()
()
()
()
()
(1)
()
()
kk
k
ii
i+
⎡⎤
=−
⎣⎦
=+
Jx
hf
x
xx
h
Aug
ust 1
2, 2
005
OSU
/CIS
541
67
Initi
al G
uess
•H
ow d
o w
e ge
t an
initi
al g
uess
for t
he ro
ot
vect
or in
hig
her-
dim
ensi
ons?
•In
2D
, I n
eed
to fi
nd a
regi
on th
at c
onta
ins
the
root
.•
Stee
pest
Dec
ent i
s a m
ore
adva
nced
topi
c no
t cov
ered
in th
is c
ours
e. It
is m
ore
stab
le
and
can
be u
sed
to d
eter
min
e an
ap
prox
imat
e ro
ot.
Aug
ust 1
2, 2
005
OSU
/CIS
541
68
Root
Fin
ding
Alg
orith
ms
•C
lose
d or
Bra
cket
ed te
chni
ques
–B
i-sec
tion
–R
egul
a-Fa
lsi
•O
pen
tech
niqu
es–
New
ton
fixed
-poi
nt it
erat
ion
–Se
cant
met
hod
•M
ultid
imen
sion
al n
on-li
near
pro
blem
s–
The
Jaco
bian
mat
rix•
Fixe
d-po
int i
tera
tions
–C
onve
rgen
ce a
nd F
ract
al B
asin
s of A
ttrac
tion
Aug
ust 1
2, 2
005
OSU
/CIS
541
69
Fixe
d-Po
int I
tera
tion
•M
any
prob
lem
s als
o ta
ke o
n th
e sp
ecia
lized
fo
rm: g
(x)=
x,w
here
we
seek
, x,t
hat
satis
fies t
his e
quat
ion.
f(x)=
x
g(x)
Aug
ust 1
2, 2
005
OSU
/CIS
541
70
Fixe
d-Po
int I
tera
tion
•N
ewto
n’s i
tera
tion
and
the
Seca
nt m
etho
d ar
e of
cou
rse
in th
is fo
rm.
•In
the
limit,
f(x k
)=0,
hen
ce x
k+1=
x k
Aug
ust 1
2, 2
005
OSU
/CIS
541
71
Fixe
d-Po
int I
tera
tion
•O
nly
prob
lem
is th
at th
at a
ssum
esit
conv
erge
s.•
The
pret
ty fr
acta
l im
ages
you
see
basi
cally
enc
ode
how
man
y ite
ratio
ns it
took
to e
ither
con
verg
e (to
so
me
accu
racy
) or t
o di
verg
e, u
sing
that
poi
nt a
s th
e in
itial
seed
poi
nt.
•Th
e bo
ok a
lso
has a
n ex
ampl
e w
here
the
root
s co
nver
ge to
a fi
nite
set.
By
assi
gnin
g di
ffer
ent
colo
rs to
eac
h ro
ot, w
e ca
n se
e to
whi
ch p
oint
the
initi
al se
ed p
oint
con
verg
ed.
Aug
ust 1
2, 2
005
OSU
/CIS
541
72
Frac
tals
•Im
ages
resu
lt w
hen
we
deal
w
ith 2
-di
men
sion
s.•
Such
as
com
plex
nu
mbe
rs.
•C
olor
in
dica
tes h
ow
quic
kly
it co
nver
ges o
r di
verg
es.
Aug
ust 1
2, 2
005
OSU
/CIS
541
73
Fixe
d-Po
int I
tera
tion
•M
ore
on th
is w
hen
we
look
at i
tera
tive
solu
tions
for l
inea
r sys
tem
s (m
atric
es).
