Embed Size (px)
Citation preview
Linear Systems of EquationsLinear Systems of EquationsAx = bAx = b
Linear Systems of EquationsLinear Systems of EquationsAx = bAx = b
Marco Lattuada
Swiss Federal Institute of Technology - ETHInstitut für Chemie und BioingenieurwissenschaftenETH Hönggerberg/ HCI F135 – Zürich (Switzerland)
E-mail: [email protected]://www.morbidelli-group.ethz.ch/education/index
Marco Lattuada
Swiss Federal Institute of Technology - ETHInstitut für Chemie und BioingenieurwissenschaftenETH Hönggerberg/ HCI F135 – Zürich (Switzerland)
E-mail: [email protected]://www.morbidelli-group.ethz.ch/education/index
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 2
Definition of the ProblemDefinition of the Problem
We want to solve:
where x is the vector of the unknowns, while A and b are given.
Hypotheses:
1. The number of equations is equal to the number of unknowns
(that is, A is a square matrix)
1. The coefficients of A, b and x are real
2. The solution of the system exists and it is unique
We want to solve:
where x is the vector of the unknowns, while A and b are given.
Hypotheses:
1. The number of equations is equal to the number of unknowns
(that is, A is a square matrix)
1. The coefficients of A, b and x are real
2. The solution of the system exists and it is unique
( , ) ( ,1) ( ,1)n n n n A x b
• A-1 exists• A is not singular• A's columns are linearly independent• A's lines are linearly independent• det(A) is non-zero• rank(A) is equal to n• Ax = 0 only if x is a null vector
Analytical ApproachAnalytical Approach
Cramer’s rule (1750):
The solution of a system of equations:
Is given by:
where Ai is defined as follows:
Cramer’s rule (1750):
The solution of a system of equations:
Is given by:
where Ai is defined as follows:
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 3
det
deti
ix A
A
x b A
1,1 1,2 1, 1 1 1, 1 1,
2,1 2,2 2, 1 2 2, 1 2,
,1 ,2 , 1 , 1 ,
i i n
i i ni
n n n i n n i n n
a a a b a a
a a a b a a
a a a b a a
A
b replaces the ith column
Calculation of the determinantCalculation of the determinant
How to compute the determinant of a square matrix?
Laplace formula (1772):
where Ci,j is the cofactor of element ai,j. The cofactor Ci,j is the determinant of the submatrix obtained by removing the ith row and the jth column of the matrix, multiplied by (-1)i+j:
How to compute the determinant of a square matrix?
Laplace formula (1772):
where Ci,j is the cofactor of element ai,j. The cofactor Ci,j is the determinant of the submatrix obtained by removing the ith row and the jth column of the matrix, multiplied by (-1)i+j:
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 4
1, 1,1
det( )N
i ji
a C
A
, ,1 deti j
i j i jC M
1,1 1,2 1, 1 1, 1 1,
1,1 1,2 1, 1 1, 1 1,
,1,1 1,2 1, 1 1, 1 1,
,1
j j n
i i i j i j i n
i ji i i j i j i n
n
a a a a a
a a a a a
a a a a a
a
M
,2 , 1 , 1 , n n j n j n na a a a
No ith row
No jth column
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 5
Numerical approach: Gauss Elimination MethodNumerical approach: Gauss Elimination Method
Let us consider the system:
Let us consider the following operations:
1. I multiply one line by a constant
2. I substitute one line with a linear combination of the others
3. I operate a permutation of the lines
The result does not change
Let us consider the system:
Let us consider the following operations:
1. I multiply one line by a constant
2. I substitute one line with a linear combination of the others
3. I operate a permutation of the lines
The result does not change
1
2
3
3 2 1 1
1 2 2 5
1 1 1 2
x
x
x
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 6
Gauss Elimination MethodGauss Elimination Method
Numerical ExampleNumerical Example
• Multiply by -3Multiply by -3• Sum it to 1st lineSum it to 1st line• Multiply by -3Multiply by -3
• Sum it to 1st lineSum it to 1st line• Multiply by -4Multiply by -4• Sum it to 2nd lineSum it to 2nd line
Triangular SystemTriangular System
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 7
Gauss Elimination MethodGauss Elimination Method
General CaseGeneral Case
11 12 13
21 22 23
31 32 33
a a a
A a a a
a a a
I want to replace I want to replace aa2121 with a zero with a zero
I define the multiplier I define the multiplier ll2121::( ) ( )21 21 21 11( ) ( )21
21 22 22 21 1211 ( ) ( )
23 23 21 13
0new old
new old
new old
a a l aa
l a a l aa
a a l a
11 12 13 11 12 13( ) ( )
21 21 22 23 22 23
31 32 33 31 32 33
1 0 0
1 0 0
0 0 1
new new
a a a a a a
l a a a a a
a a a a a a
Note that:Note that:
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 8
Gauss Elimination MethodGauss Elimination Method
Gauss Transformation MatrixGauss Transformation Matrix
(0) (0) (0) (0) (0) (0)11 12 13 11 12 13
(0) (0) (1)(0) (0) (0) (0) (1) (1)21 21 22 23 22 23(0) (0) (0) (0) (1) (1)31 31 32 33 32 33
1 0 0
1 0 0
0 1 0
a a a a a a
l a a a a a
l a a a a a
M A A
where:where:( )
( )( )
kijk
ij kjj
al
a
(0) (0) (0) (0) (0) (0)11 12 13 11 12 13
(1) (1) (2)(1) (1) (1) (1)22 23 22 23
(1) (1) (1) (2)21 32 33 33
1 0 0
0 1 0 0 0
0 1 0 0 0
a a a a a a
a a a a
l a a a
M A A
Solution:Solution:( 1) ( 2) (1) (0) (0)
( 1) ( 1)
( 1) ( 2) (1) (0) (0)
n n
n n
n n
A M M M AA x b
b M M M b
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 9
Gauss Elimination MethodGauss Elimination Method
Numerical ExampleNumerical Example
n(n-1) operations (flops)n(n-1) operations (flops)(n-1)(n-2) operations (flops)(n-1)(n-2) operations (flops)
Total number of Total number of operations requiredoperations required
31 12
2 2( 1)
3
n n
j j
nj j j
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 10
Gauss Transformation MethodGauss Transformation Method
Let us change our point of view!
can be used to transform A
Gauss Elimination Method1. Changes the matrix A2. Needs the coefficient vector b3. Must re-run the method if b is changed
( 2) (1) (0)nM M M M
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 11
Gauss Transformation MethodGauss Transformation Method
Properties
• The final matrix A is a right triangular matrix
• The matrix M is a left triangular matrix
• The inverse of M is also a left triangular matrix
• The matrix L = M-1 has the simple form:
Properties
• The final matrix A is a right triangular matrix
• The matrix M is a left triangular matrix
• The inverse of M is also a left triangular matrix
• The matrix L = M-1 has the simple form:
1 (0)21(0) (1)31 31
1 0 0
1 0
1
l
l l
L M
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 12
Consider the following expression:
Let us multiply by L = M-1 both sides:
Consider the following expression:
Let us multiply by L = M-1 both sides:
LR (LU) FactorizationLR (LU) Factorization
( 2) (0)n A MA
Right triangular
(0)R MA
1 (0) (0) LR M MA A A LR
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 13
Starting matrix A is transformed (factorized) as:
Let us solve a linear system with a generic vector b:
Starting matrix A is transformed (factorized) as:
Let us solve a linear system with a generic vector b:
LR FactorizationLR Factorization
A LR
Ly bAx LRx b
Rx y
1. For every vector b, two simple triangular systems must be solved without factorizing again
2. The matrices LR can be stored using the elements of A3. If A is modified, it is often possible to modify L and R
accordingly without factorizing
Marco Lattuada – Statistical and Numerical Methods for Chemical EngineersSolution of Linear Systems of Equations – Page # 14
Starting matrix:
Consider the following system:
Consider the following similar system:
Starting matrix:
Consider the following system:
Consider the following similar system:
Problems of Gaussian Elimination and LRProblems of Gaussian Elimination and LR
(0) (0) (0)11 12 13
(0) (0) (0) (0)21 22 23(0) (0) (0)31 32 33
a a a
a a a
a a a
A Pivot value must be ≠ 0
2
1 2
1
0.4 0.3 0.1
x
x x
a11 = 0 I switch the lines x1 = 1 and x2 = 1
201 2
1 2
2 10 1
0.4 0.3 0.1
x x
x x
Manual LR FactorizationManual LR Factorizationwithout pivotingwithout pivoting
