Upload
doliem
View
232
Download
0
Embed Size (px)
Citation preview
Fixed Points and Contractive Transformations
Ron GoldmanDepartment of Computer ScienceRice University
Applications
Computer Graphics• Fractals• Bezier and B-Spline Curves and Surfaces
Root Finding• Newton’s Method• Trivial Fixed Point Method
Solving Large Systems of Linear Equations -- Relaxation Methods• Jacobi• Gauss-Seidel
Existence and Uniqueness Theorem for Ordinary Differential Equations
Reconstruction of Curves and Surfaces from Point Clouds
Fixed Points by Iteration
Iteration
€
T1 = F(T0 )
T2 = F2 (T0 ) = F(T1)
↓ ↓ ⇒ F(T ) = F∞+1(T0 ) = T ( fixed point)
T = F∞(T0 )
Another Iteration
€
S1 = F(S0 )
S2 = F 2(S0 ) = F(S1)
↓ ↓ ⇒ F(S) = F∞+1(S0 ) = S ( fixed point)
S = F∞(S0 )
Observations• Iteration Might Not Converge• If Iteration Converges → Fixed Point • Different Starting Values May Generate Different Fixed Points
Contractive Transformation
Contractive Map (T)
Dist T(P),T(Q)( ) ≤ sDist(P,Q) 0 < s <1
Examples
Uniform Scaling -- 0 < s <1
Image of Three Points -- Triangle Shrinks on All Sides
Necessary Condition Necessary and Sufficient Conditions
Det(T ) < 1
€
Eigenvalues(T ) <1
Lemma
T Contractive
€
⇒ T Continuous
Iterations and Transformations
Iteration -- Arbitrary Transformations
• Either Converges to a Fixed Point or Diverges
• Possibly Many Distinct Fixed Points
• Starting Point Matters
Iteration -- Contractive Transformations
• Always Converges to a Fixed Point
• Fixed Point is Unique
• Starting Point does not Matter
Fixed Points of Contractive Maps
Theorem 1: Suppose that T is a Contractive and Pn +1 = T(Pn ) for all n ≥ 0.If P = Limn→∞Pn exists, then P is a fixed point of T.
Proof: T(P) = T Limn→∞Pn( ) = Limn→∞T(Pn ) = Limn→∞Pn+1 = P.
Theorem 2: T Contractive ⇒ Fixed Point is Unique.
Proof: If P and Q are both fixed points of T, then
T(P) = P and T(Q) = Q
Dist T(P),T(Q)( ) = Dist(P,Q).
Hence T is not contractive. Contradiction.
Cauchy Sequences
DefinitionA sequence Pn{ } is Cauchy if Dist(Pn +m , Pn ) < ε for all n > N .
IntuitionA sequence of points Pn{ } is Cauchy if the points get closer and closer as n gets larger and larger.
Cauchy’s Theorem (Completeness)Every Cauchy sequence converges.
Triangular Inequality
€
P1
€
P2
€
P3
€
•
€
•
€
•
€
Dist(P1,P3 ) ≤ Dist(P1,P2)+ Dist(P2,P3 )
€
Dist(P1,Pn ) ≤ Dist(P1,P2)+ Dist(P2,P3 )+L+ Dist(Pn−1,Pn )
Contractive Maps and Cauchy Sequences
Theorem 3 Suppose that T is a Contractive Map, and Pn +1 = T(Pn ) for all n ≥ 0 .Then Pn{ } is a Cauchy Sequence for Any Choice of P0 .
Pr oof : Dist Pn+1, Pn( ) = Dist T(Pn ),T(Pn−1 )( ) ≤ sDist Pn , Pn−1( ) = sDist T (Pn−1),T(Pn−2 )( )
M ≤ snDist P1, P0( )
€
Dist Pn+m+1,Pn( ) ≤ Dist Pn+m+1,Pn+m( )+L+ Dist Pn+1,Pn( ) ≤ (sn+m +L+ sn )Dist P1,P0( )
≤ sn Dist P1,P0( )1− s
< ε
Trivial Fixed Point Theorem
Trivial Fixed Point Theorem
Suppose that
T is a Contractive Map
Pn +1 = T(Pn ) for all n ≥ 0 .
Then Limn→∞Pn is the Unique Fixed Point of T for Any Choice of P0 !
Proof: The Sequence Pn{ } is Cauchy. (Theorem 3)
Therefore, Limn→∞Pn Exists. (Cauchy’s Theorem)
Hence, Limn→∞Pn is a Fixed Point of T. (Theorem 1)
Moreover, this Fixed Point is Unique. (Theorem 2)
Equation Solving
Finding Roots of Real Valued Functions
• Guess and Iterate
-- Newton’s Method
-- Trivial Fixed Point Method
Solving Large Systems of Linear Equations
• Relaxation Methods
-- Jacobi
-- Gauss-Seidel
Generating Curves and Surfaces
Ordinary Differential Equations
• Existence and Uniqueness Theorem
• Solutions by Iteration
Fractals
• Iterated Function Systems
• Fractal Algorithm
Root Finding by Newton’s Method
ProblemSolve F(x) = 0
Newton’s Method1. Replace F(x) by
€
G(x) = x − F(x) / ′ F (x)• F(x) has a root
€
⇔ G(x) has a Fixed Point•
€
F(x*) = 0 ⇔ G(x*) = x * 2. Select an initial guess x0 for x *.3. Compute
€
xk+1 = G(xk ) = xk − F(xk ) / ′ F (xk ) for k = 1,K,n .4. Stop when xk +1 − xk < ε
Observations• May Diverge --
€
′ F (xk ) ≈ 0 • May Converge to Different Roots for Different Initial Guesses
Finding a Fixed Point by Iteration
y = x
y = G(x)
x0•
• •
•x1
• •
•x2
• ••
•x3
y = G(x0 )x1 = G(x0 )
Fixed PointG( ˆ x ) = ˆ x
•̂ x
Root Finding by Trivial Fixed Point Method
ProblemSolve F(x) = 0
Fixed Point Method
1. Replace F(x) by
€
G(x) = x + F(x)
• F(x) has a root ⇔ G(x) has a Fixed Point
• F(x*) = 0 ⇔ G(x*) = x *
2. Select any(!) initial guess x0 for x *.
3. Compute xk +1 = G(xk ) for k = 1,K,n .4. Stop when xk +1 − xk < ε
ObservationWorks when G(x) is a Contractive Map.
Root Finding for Differentiable Functions
Mean Value Theorem
G(b) – G(a) ≤ ′ G (c)(b − a) a ≤ c ≤ b
Observations
• ′ G (c) < 1⇒G(x) is Contractive
• ′ G (c) < 1/ 2 ⇒ Fixed Point Method is Faster than Bisection Method
Example
• Find a root of F(x) = cos(x) − x
• Find a fixed point of G(x) = cos(x)
Solving Large Systems of Linear Equations
System of Linear Equations
€
M11x1 + M12x2 +L+ M1nxn = b1M21x1 + M22x2 +L+ M2nxn = b2
M M ⇔ xi =bi
Mii−
MijMiij≠i
∑ x j 1≤ i ≤ n
Mn1x1 + Mn2x2 +L+ Mnnxn = bn
Matrix Form
€
M11 M12 L M1nM21 M22 L M2n
M M O M
Mn1 Mn2 L Mnn
M1 2 4 4 4 4 3 4 4 4 4
x1x2M
xn
X{
=
b1b2M
bn
B{
⇔ M ∗X = B
Relaxation Methods
Jacobi Relaxation
€
xip =
biMii
−MijMiij≠i
∑ x jp−1 1≤ i ≤ n
Gauss-Seidel Relaxation
€
xip =
biMii
−MijMiij=1
i−1
∑ x jp −
MijMiij=i+1
n
∑ x jp−1 1≤ i ≤ n
Jacobi Relaxation
System of Linear Equations
€
M11x1 + M12x2 +L+ M1nxn = b1M21x1 + M22x2 +L+ M2nxn = b2 M M ⇔ M ∗X = B ⇔ B− M ∗X = 0Mn1x1 + Mn2x2 +L+ Mnnxn = bn .
Equivalent System of Linear Equations
€
M11x1 = b1 −M12x2 −L− M1nxnM22x2 = b2 −M21x1 −L− M2nxn M M M ⇔ D∗ X = B − (M − D)∗XMnnxn = bn −Mn1x1 −L
€
D =
M11 0 L 00 M22 L 0M O O M
0 L 0 M NN
Jacobi Relaxation (continued)
Equivalent System of Linear Equations
€
x1 = b1M11
− M12M11
x2 −L− M1nM11
xn
x2 = b2M22
− M21M22
x1 −L− M2nM22
xn
M M M ⇔ X = D−1∗B −D−1∗ (M − D)∗X
xn =bn
Mnn−
Mn1Mnn
x1 −L
Fixed Point
•
€
T(X ) = D−1B + (I −D−1M )X
•
€
T(X ) = X (fixed point)
Gauss-Seidel Relaxation
System of Linear Equations
€
M11x1 + M12x2 +L+ M1nxn = b1M21x1 + M22x2 +L+ M2nxn = b2 M M ⇔ M ∗X = B ⇔ B− M ∗X = 0Mn1x1 + Mn2x2 +L+ Mnnxn = bn .Mn1x1 + Mn2x2 +L+ Mnnxn = bn .
.
Equivalent System of Linear Equations
€
M11x1 = b1 −M12x2 −L− M1nxnM21x1 + M22x2 = b2 − M23x1 −L− M2nxn M M ⇔ L∗ X = B − (M − L)∗ XMn1x1 +L+ Mnnxn = bn
€
L =
M1,1 0 L 0M2,1 M2,2 L 0
M O O M
Mn,1 L Mn−1,n−1 Mn,n
Gauss-Seidel Relaxation (continued)
Equivalent System of Linear Equations
•
€
L∗ X = B − (M − L)∗ X ⇔ X = L−1∗B− L−1∗ (M − L)∗ X
Fixed Point
•
€
T(X ) = L−1B+ L−1(L − M )X
•
€
T(X ) = X (fixed point)
Convergence
Diagonally Dominant
• Both methods converge for any initial guess when M is diagonally dominant
•
€
| Mii |≥ | Mijj≠i∑ | (diagonally dominant)
Fixed Point
•
€
M ∗X = B⇔Q∗X = (Q −M )X + B⇔ X = (I −Q−1M )X +Q−1B
•
€
T(X ) = (I −Q−1M )X +Q−1B
•
€
T(X ) = X (fixed point)
Contractive Map• T is a contractive map when
-- M is diagonally dominant-- Q is the diagonal part of M (Jacobi)-- Q is the lower triangular part of M (Gauss-Seidel)
Ordinary Differential Equations
General Case
•
€
y(n) = an−1(x)y(n−1) +L+ a1(x)y' + a0(x)y +b(x) (Differential Equation)
•
€
y(0) = c0 ,K,y(n−1)(0) = cn−1 (Initial Conditions)
Example 1
€
• y'= −y• y(0) =1
€
⇔ y = e−x
Example 2
€
• y' '= −y• y(0) =1 y'(0) = 0
€
⇔ y = cos(x)
Contractive Map for Ordinary Differential Equations
Ordinary Differential Equation
•
€
y(n) = an−1(x)y(n−1) +L+ a1(x)y' + a0(x)y +b(x) (Differential Equation)
•
€
y(0) = c0 ,K,y(n−1)(0) = cn−1 (Initial Conditions)
Ordinary Differential Equation
•
€
y(n) = F(x,y,y',K,y(n−1)) (Differential Equation)
•
€
y(0) = c0 ,K,y(n−1)(0) = cn−1 (Initial Conditions)
Contractive Map
€
• T ( f )(x) = c0 +c1x +L+cn−1xn−1
(n −1)!
+ L F tn, f (tn ), f ' (tn ),K, f (n −1)(tn )
0tn −1∫0
t1∫0x∫ dtnLdt1
• Fixed Point of T = Solution of ODE
Examples
Example 1
€
• y'= −y
€
• y(0) =1
•
€
T( f )(x) =1+ − f (x)0x∫ dx
•
€
T(e−x ) =1+ −e−t0x∫ dt = e−x
Example 2
•
€
y''= −y
•
€
y(0) =1 y' (0) = 0
•
€
T( f )(x) =1+ − f (u)0t∫0
x∫ dudt
•
€
T cos(x)( ) =1+ −cos(u)0t∫0
x∫ du dt = cos(x)
Solving Ordinary Differential Equations
Ordinary Differential Equation
•
€
y(n) = an−1(x)y(n−1) +L+ a1(x)y' + a0(x)y +b(x) (Differential Equation)
•
€
y(0) = c0 ,K,y(n−1)(0) = cn−1 (Initial Conditions)
Fixed Point Method
1. Select any continuous function
€
f0(x) as the initial guess.
2. Compute
€
fk +1 = T ( fk )
€
k =1,K,n.
3. Stop when
€
fk +1− fk < ε .
Spaces and Functions
Euclidian Space Function Space
Points Continuous Functions on Closed Intervals
Euclidian Distance
€
| f − g |∞= Max | f − g |
Cauchy Sequences of Points Converge Cauchy Sequences of Functions Converge
Contractive Maps Integrals
Fixed Points Solutions of ODE’s
Trivial Fixed Point Theorem Trivial Fixed Point Theorem
Root Finding Algorithm Algorithm for Solving ODE’s• Start with Any Point and Iterate • Start with Any Function and Iterate