Upload
pro-skyline
View
221
Download
0
Embed Size (px)
Citation preview
8/8/2019 Applied Linear Chapter 1
1/119
LINEAR ALGEBRA
Dr. Nguyen Ngoc Hai
Department of Mathematics
INTERNATIONAL UNIVERSITY, VNU-HCM
September 28, 2010
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
2/119
References
E. Kreyszig, Advanced Engineering Mathematics, 9th ed., John
Wiley & Sons, 2006. (pp. 271363)
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
Ch MATRICES AND
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
3/119
Chapter 1 MATRICES AND
SYSTEMS OF LINEAR EQUATIONS
1.1 MATRICES
1.1.1 General Concepts of Matrices
Definition 1.1 An m n matrix (read m by n matrix)is a rectangular array of m rows and n columns of the form
a11 a12 a1na21 a22 a2n
......
am1
am2
amn
where each aij is a number (or function) called an entry orelement of the matrix. The numbers m and n are called thedimension or size of the matrix.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
4/119
1.1 MATRICES1.1.1 GENERAL CONCEPTS OF MATRICES
For example,
2 2 3
7
0 1.2
b11 b12 b13 b14b21 b22 b23 b24b31 b32 b33 b34
e2x exsin3x cos5x
1 9 4 02 The first matrix has two rows and three columns.
We denote matrices by capital boldface letters A, B, C, ... In a matrix A, aij denotes the entry that occurs in row i and
column j of A.
Sometimes, we write the m by n matrix A = [aij].
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
5/119
1.1 MATRICES1.1.1 GENERAL CONCEPTS OF MATRICES
Matrices with a single row or column are called vectors.
Matrices having just one row are row vectors.
Matrices having just one column are column vectors.
Its entries are called the components of the vector.
We shall denote vectors by lowercase boldface letters a, b, ...or by its general components in brackets, a = [aj], and so on.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
6/119
1.1 MATRICES1.1.1 GENERAL CONCEPTS OF MATRICES
Definition 1.2 A matrix with n rows and n columns iscalled a square matrix of order n, and the entriesa11, a22, ..., ann form the main diagonal of A.
For instance, the matrix
2 0 91 5 7
4 3 0
is a square matrix of order 3, and 2, 5 and 0 are entries on themain diagonal.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
7/119
1.1 MATRICES1.1.1 GENERAL CONCEPTS OF MATRICES
An m n matrix whose entries are all zeros is called the m nzero matrix and is denoted by O.
For instance, the 2 3 zero matrix is
O =
0 0 00 0 0
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
8/119
1.1 MATRICES1.1.2 MATRICES ADDITION AND SCALAR MULTIPLE
Equality
Definition 1.3 Two matrices are said to be equal if theyhave the same size and the corresponding entries in the twomatrices are equal.
Example 1.1 Two matrices
A =
1 a
1 2
0 b
and B =
1 3x 2
y 5
are equal if and only if a = 3, b = 5, x = 1 and y = 0.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
C S
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
9/119
1.1 MATRICES1.1.2 MATRICES ADDITION AND SCALAR MULTIPLE
Addition of Matrices
Definition 1.4 The sum of two m n matrices A = [aij]and B = [bij] is the m n matrix A + B obtained by addingtogether the corresponding entries of A and B.
A = [aij] and B = [bij] = A + B = [aij + bij]
As a special case, the sum a + b of two row vectors or two columnvectors, which must have the same number of components, isobtained by adding the corresponding components.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
10/119
1.1 MATRICES1.1.2 MATRICES ADDITION AND SCALAR MULTIPLE
Example 1.2 Let
A =
3 0 22 1 4
, B =
5 3 61 2 5
, and C =
4 75 9
0 1
.
Since A and B are the same size (2 3), their sum is defined. Wehave
A + B =
3 + 5 0 + (3) (2) + 62 + 1 (
1) + 2 4 + (
5)
=
8 3 43 1
1
.
The sum A + C is not defined since their sizes are different.
Similarly, since B and C are not the same size, their sum does notexist.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
11/119
1.1 MATRICES1.1.2 MATRICES ADDITION AND SCALAR MULTIPLE
Properties of Matrix Addition
If A, B, C, and O have the same size, then
1. A + B = B + A
2. (A + B) + C = A + (B + C)
3. A + O = O + A = A
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
12/119
1.1 MATRICES1.1.2 MATRICES ADDITION AND SCALAR MULTIPLE
Multiplication of Matrices
Traditionally, when discussing matrices, a number is called ascalar.
Definition 1.5 If A is any matrix and k is any scalar, thenthe product kA is the matrix obtained by multiplying each
entry of A by k.
Example 1.3 If
A = 3 0 22 1 4
then
2A =
6 0 44 2 8
and (1)A =
3 0 22 1 4
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
13/119
1.1 MATRICES1.1.2 MATRICES ADDITION AND SCALAR MULTIPLE
Definition 1.6 If A is any matrix, then A will denote theproduct (1)A. If A and B are two matrices, then A B isdefined to be the sum A + (B).
The matrix A is usually called the negative of A or theadditive inverse for A.
A B is called the difference of A and B.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
14/119
1.1 MATRICES1.1.2 MATRICES ADDITION AND SCALAR MULTIPLE
Properties of Scalar Multiplication
1. k(A + B) = kA + kA
2. (k1 + k2)A = k1A + k2A3. k1(k2A) = (k1k2)A
4. 1A = A
5. 0A = O
6. kO = O
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
15/119
1.1 MATRICES1.1.2 MATRIX MULTIPLICATION
Definition of Matrix Multiplication
Matrix multiplication means multiplication of matrices by matrices.
Definition 1.7 If A is an m n matrix and B is an n pmatrix, then the product AB is the m
p matrix C whose
entry cij in row i and column j is obtained as follows: Sumthe products formed by multiplying each entry in row i of Aby the corresponding entry in column j of B.
cij = ai1b1j + ai2b2j +
+ ainbnj (1)
The only requirement restricting the definition of the product ABis that the number of columns of A must equal the number ofrows of B.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
16/119
1.1 MATRICES1.1.2 MATRIX MULTIPLICATION
Example 1.4 Find AB where
A = 1 3 23 0 5 and B = 4 5 1 31 2 0 12 1 0 2
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
17/119
1.1 MATRICES1.1.2 MATRIX MULTIPLICATION
Example 1.5 A system of linear equations
a11x1 + a12x2 + + a1nxn = b1a21x1 + a22x2 + + a2nxn = b2
......
...
am1x1 + am2x2 + + amnxn = bmcan be written as the matrix equation
a11 a12 a1na21 a22 a2n... ...am1 am2 amn
x1
x2...xn
= b1
b2...bm
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
18/119
1.1 MATRICES1.1.2 MATRIX MULTIPLICATION
orAx = b
where A is the coefficient matrix of the system,
A =
a11 a12 a1na21 a22 a2n
......
am1 am2
amn
, x =
x1x2...
xn
and b =
b1b2...
bm
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
19/119
1.1 MATRICES1.1.2 MATRIX MULTIPLICATION
Properties of Matrix Multiplication
The following equalities hold provided that the operations exist.
1. (AB)C = A(BC)2. (A + B)C = AC + BC
3. A(B + C) = AB + AC
4. k(AB) = (kA)B = A(kB)
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
20/119
1.1 MATRICES1.1.3 TRANSPOSE OF A MATRIX
Definition 1.8 If A = [aij] is an m n matrix, then thetranspose of A, denoted AT, is the n m matrix AT = [bij],where bij = aji for all i and j, 1 j m, and 1 i n.
As a special case, transposition converts row vectors to column
vectors and conversely.
Example 1.6 Find the transpose of
A = 0 21 4
5 9
, B = 13 a 24 1 bc 5 8
, and C = 9 .
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
21/119
1.1 MATRICES1.1.3 TRANSPOSE OF A MATRIX
Transpose are important in their own right, and following are a fewof their important properties.
1. (AT)T = A.2. (A + B)T = AT + BT
3. (AB)T = BTAT
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
22/119
1.1 MATRICES1.1.4 SPECIAL MATRICES
Symmetric Matrices
Symmetric matrices are square matrices whose transpose equalsthe matrix itself.
A symmetric matrix AT
= A aij = aji for all i,jExample 1.7 The matrix
A = 1 5 8
5 0 28 2 3
is symmetric.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.1 MATRICES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
23/119
1.1 MATRICES1.1.4 SPECIAL MATRICES
Triangular Matrices
Upper triangular matrices are square matrices that can havenonzero entries only on and above the main diagonal, whereas any
entry below the diagonal must be zero.
Similarly, lower triangular matrices can have nonzero entriesonly on and below the main diagonal. Any entry on the maindiagonal of a triangular matrix may be zero or not.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
24/119
1.1 MATRICES
8/8/2019 Applied Linear Chapter 1
25/119
1.1.4 SPECIAL MATRICES
Diagonal matrices are square matrices that can have nonzeroentries only on the main diagonal. Any entry above or below themain diagonal must be zero. For example,
4 00 0
and
5 0 0 00 0 0 00 0 7 00 0 0 0
are diagonal matrices.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1 2 INTRODUCTION TO
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
26/119
.
SYSTEMS OF LINEAR EQUATIONS1.2.1 GENERAL LINEAR SYSTEMS
In science, engineering, and the social sciences, one of the mostimportant and frequently occurring mathematical problems isfinding a simultaneous solution to a set of linear equations
involving several unknowns.
In this section we introduce basic terminology and discuss amethod for solving systems of linear equations.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TO
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
27/119
.
SYSTEMS OF LINEAR EQUATIONS1.2.1 GENERAL LINEAR SYSTEMS
Definition 2.1 A solution of a linear equation
a1x1 + a2x2 + + anxn = b
is a sequence of n numbers s1, s2, ..., sn such that if wesubstitute x1 = s1, x2 = s2, ..., xn = sn into the equation, weobtain a true statement, that is, if
a1s1 + a2s2 + + ansn = b.
A solution of an equation is usually written between parentheses as(s1, s2, ..., sn). For instance, (1, 2, 1), (1, 1, 0), (2, 7, 2) aresolutions of the equation 2x1 + x2 + x3 = 1.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TO
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
28/119
.
SYSTEMS OF LINEAR EQUATIONS1.2.1 GENERAL LINEAR SYSTEMS
Definition 2.2 An m n system of linear equations isthe system of m linear equations in n unknowns
a11x1 + a12x2 + + a1nxn = b1a21x1 + a22x2 + + a2nxn = b2
... ... ...am1x1 + am2x2 + + amnxn = bm
In other words, a finite set of linear equations in the variablesx1, x2, ..., xn is called a system of linear equations or more simplya linear system.
A sequence of numbers s1, s2, ..., sn is called a solution of thesystem if x1 = s1, x2 = s2, ..., xn = sn is a solution of everyequation in the system.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
29/119
.
SYSTEMS OF LINEAR EQUATIONS1.2.1 GENERAL LINEAR SYSTEMS
For instance, the system
2x1
x2
3x3 =
1
2x1 + 2x2 + 5x3 = 3has the solution x1 = 1, x2 = 0, and x3 = 1 since these valuessatisfy both equations. However, x1 = 0, x2 = 1, and x3 = 1 isnot a solution since these value satisfy only the second of the two
equations in the system.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
30/119
SYSTEMS OF LINEAR EQUATIONS1.2.1 GENERAL LINEAR SYSTEMS
Not all systems of linear equations have solutions. A system withno solutions is called inconsistent; a system with at least onesolution is called consistent.
Example 2.1 (Geometric Interpretation) If m = n = 2, we havetwo equations in two unknowns x1, x2:
a11x1 + a12x2 = b1
a21x1 + a22x2 = b2
(x1, x2) is a solution of the system if and only if the pointP(x1, x2) lies on both lines. Hence there are three possible cases:
(a) The two lines intersect at a single point, so there is a uniquesolution.
(b) The two lines are coincident, so there are infinitely manysolutions.
(c) The two lines are parallel, so there are no solutions.Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
31/119
SYSTEMS OF LINEAR EQUATIONS1.2.2 MATRIX FORM OF LINEAR SYSTEMS
From the definition of matrix multiplication we see that the system
a11x1 + a12x2 + + a1nxn = b1a21x1 + a22x2 + + a2nxn = b2
......
...am1x1 + am2x2 +
+ amnxn = bm
may be written as a single vector equation
Ax = b (2)
where the coefficient matrix A = [aij] is the m n matrix
A =
a11 a12 a1na21 a22 a2n
......
am1 am2
amn
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
32/119
SYSTEMS OF LINEAR EQUATIONS1.2.2 MATRIX FORM OF LINEAR SYSTEMS
and
x = x1x
2...xn
and b = b1b
2...bm
are column vectors. Note that x has n components, whereas b has
m components.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
33/119
SYSTEMS OF LINEAR EQUATIONS1.2.2 MATRIX FORM OF LINEAR SYSTEMS
The matrix
A =
a11 a12 a1n b1a21 a22
a2n b2
... ...am1 am2 amn bm
is called the augmented matrix for the system (2) and usuallydenoted as [A
|b]. The augmented matrix A = [A|b] determinesthe system (2) completely because it contains all the given
numbers appearing in (2).
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
34/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Elementary Operations
The basic method for solving a system of linear equations is toreplace the given system by a new system that has the samesolution set but which is quicker and easier to solve.
Gauss elimination is a standard elimination method for solvinglinear systems that proceeds systematically irrespective ofparticular features of coefficients.
It is a method of great practical importance and is reasonablewith respect to computing time and storage demand.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
35/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Definition 2.3 Two systems of linear equations in nunknown are equivalent provided that they have the same setof solutions.
A new equivalent system is generally obtained by applying three
types of operations, called elementary operations, to eliminateunknowns systematically.
Theorem 2.1
If one of the following elementary operations is applied to a
system of linear equations, then the resulting system isequivalent:1. Interchange two equations.2. Multiply an equation by a nonzero constant.3. Add a multiple of one equation to another.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
36/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Since the equations of a system correspond to rows of theaugmented matrix, elementary operations on a linear systemcorrespond to the following elementary row operations on the
augmented matrix.
1. Interchange two rows.
2. Multiply a row by a nonzero constant.
3. Add a multiple of one row to another.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
37/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Example 2.2
Linear system Associated augmented matrix
3y + 2z = 7x + 4y
4z = 3
3x + 3y + 8z = 1
0 3 2 71 4
4 3
3 3 8 1
Interchange the first and Interchange the first andsecond equations. second rows.
x + 4y 4z = 33y + 2z = 7
3x + 3y + 8z = 1
1 4 4 30 3 2 73 3 8 1
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
38/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Add 3 times the first Add 3 times the firstequation to the third. row to the third.
x + 4y 4z = 33y + 2z = 7
9y + 20z = 8
1 4 4 30 3 2 7
0 9 20 8
Add 3 times the second Add 3 times the secondequation to the third. row to the third.
x + 4y 4z = 33y + 2z = 7
26z = 13
1 4 4 30 3 2 70 0 26 13
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
39/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Now the system is in triangular form and we can solve it by
back substitution, that is, solve the last equation for z:
26z = 13, z =1
2,
then substitute this value for z into the preceding equation and
solve for y:3y + 2(
1
2) = 7, y = 2.
Finally, substitute the known values for y and z into the firstequation and solve for x:
x + 4(2) 4( 12
) = 3, x = 3.Thus the solution is (3, 2, 1
2).
The method we have used to solve this system of equations is
called Gaussian elimination.Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
40/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Echelon Form
Since a linear system is completely determined by its augmentedmatrix, Gauss elimination can be done by merely considering thematrices.
Definition 2.4 A matrix is said to be in echelon form if(a) All rows that contain only zeros are grouped together atthe bottom of the matrix;
(b) For each row that does not contain only zeros, the firstnonzero number in the row, called the leading entry, appearsstrictly to the right of the first nonzero number in each rowabove it.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
41/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Example 2.3 Determine which of the following matrices are in
echelon form:
A =
2 1 0 3 50 4 3 2 00 0 9 0 80 0 0 1 4
B =
1 4 4 30 3 2 70
9 0
8
C =
1 5 00 3 22 0 10 0 0
D =
0 4 4 32 3 2 70 0 1 80 0 0
5
E =
0 00 0
0 0
F =
0 4 4 30 0 0 70 0 0 00 0 0 0
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
42/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Remark Every matrix can be transformed to matrix in echelonform using elementary operations.
Example 2.4 Solve the system
x + y 2z = 3z y + 3
3z + 4 = 4y2x y + 2z = 1
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
43/119
SYSTEMS OF LINEAR EQUATIONS1.2.3 GAUSSIAN ELIMINATION
Reduced Echelon Form
The goal of Gassian elimination is to obtain a system ofequations that is equivalent to the given system and that has anaugmented matrix in echelon form.
However, the system can be further reduced by proceeding fromright to left. This reduction technique avoids backsolving by usingelementary operations.
Definition 2.5 A matrix M that is in echelon form is inreduced echelon form provided that the first nonzeroelement in each nonzero row is the only nonzero entry in itscolumn.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
44/119
Q1.2.3 GAUSSIAN ELIMINATION
For example, the matrices
C =
2 0 0 10 4 0 30 0 1 0
and D =
1 2 0 1 10 0 3 3 70 0 0 0 0
are in reduced echelon form. However
E =
1 3 0 00 1 0
2
0 0 1 0
is not in reduced echelon form.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
45/119
Q1.2.3 GAUSSIAN ELIMINATION
Example 2.5 Solve the system of linear equations
2x1 3x2 + x3 3x4 + 2x5 = 62x1 3x2 + 5x3 x4 + x5 = 8
4x3 + 3x4 + 2x5 = 3
2x1 + 3x2 + 3x3 + 3x4
9x5 =
6,
Example 2.6 Solve the system
x1 + 3x2 2x3 + 2x5 = 02x1 + 6x2 5x3 2x4 + 4x5 3x6 = 1
5x3 + 10x4 + 15x6 = 52x1 + 6x2 + 8x4 + 4x5 + 18x6 = 6
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
46/119
1.2.3 GAUSSIAN ELIMINATION
From the above examples we see that
Every system of linear equations has either no solutions,exactly one solution, or infinitely many solutions.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
47/119
1.2.4 HOMOGENEOUS SYSTEMS
OF LINEAR EQUATIONS
Definition 2.6 A system of linear equations is said to behomogeneous if all the constant terms are zeros, that is, thesystem has the form
a11x1 + a12x2 +
+ a1nxn = 0a21x1 + a22x2 + + a2nxn = 0
......
...am1x1 + am2x2 + + amnxn = 0
Every homogeneous system is consistent, sincex1 = 0, x2 = 0, ..., xn = 0 is always a solution.
This solution is called the trivial solution; if there are othersolutions, they are called nontrivial solutions.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
48/119
1.2.4 HOMOGENEOUS SYSTEMS
OF LINEAR EQUATIONS
For a homogeneous system of linear equations, exactly one of the
following is true:1. The system has only the trivial solution.
2. The system has infinitely many nontrivial solutions in additionto the trivial solution.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.
2 INTRODUCTION TOSYSTEMS OF LINEAR EQUATIONS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
49/119
1.2.4 HOMOGENEOUS SYSTEMS
OF LINEAR EQUATIONS
Theorem 2.2Let A be the reduced coefficient matrix of a homogeneoussystem of linear equations in n unknowns. If A has exactly knonzero rows, then k n. Moreover,1. if k < n, then the system has infinitely many solutions, and2. if k = n, the system has a unique solution (the trivialsolution).
Corollary 2.1
A homogeneous system of linear equations with fewer
equations than unknowns has infinitely many solutions.
Remark Note that the above theorem and corollary applyonly to homogeneous systems. A nonhomogeneous systemwith more unknowns than equations need not be consistent.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.1 DEFINITION OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
50/119
Cofactor Expansions of Determinants
Definition 3.1 If A = [a] is an 1 1 matrix, the we definethe determinant of A to be the number a. If A = [aij] is an2 2 matrix, the determinant of A is given by
det A = a11a22 a12a21.
The determinant is often expressed by using vertical bars:
det A = a11 a12a21 a22 = a11a22 a12a21.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.1 DEFINITION OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
51/119
Definition 3.2 Let A = [aij] be an n n matrix, and letMrs denote the (n
1)
(n
1) matrix obtained by deleting
the rth row and sth column from A. Then Mrs is called aminor matrix of A, and the number det Mrs is the minor ofthe (r, s)th entry ars. In addition, the numbers
Aij = (
1)i+j det Mrs
are called cofactors (or signed minors).
Example 3.1 Determine the minor matrices M11, M23, and M32for the matrix A given by
A =
2 1 31 2 54 6 2
.Also, calculate the cofactors A11, A23, and A32.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.1 DEFINITION OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
52/119
Definition 3.3 Let A = [aij] be an n n matrix. Then thedeterminant of A is
det A = a11A11 + a12A12 + + a1nA1n
Note that determinants are defined only for square matrices.
Example 3.2 Compute det A, where
A = 3 2 1
2 1 34 0 1
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.1 DEFINITION OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
53/119
Example 3.3Compute det
A, where
A =
1 2 0 21 2 3 13 2 1 0
2 3 2 1
.
Theorem 3.1
If A = [aij] is a lower-triangular matrix, then
det A = a11a22 ann.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.2 PROPERTIES OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
54/119
Theorem 3.2If A is a square matrix then
det(AT) = det A
Corollary 3.1
If A = [aij] is an n n upper or lower triangular matrix, thendet A is the product of the entries on the main diagonal:
det A = a11a22 ann.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.2 PROPERTIES OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
55/119
Theorem 3.3
Let A = [aij] be an n n matrix. Thendet A = ai1Ai1 + ai2Ai2 + + ainAindet A = a1jA1j + a2jA2j + + anjAnj
Example 3.4 Find the determinant
|A| = 1 0 1 3
2 1 1 11 0 3 31 0 8 0
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.2 PROPERTIES OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
56/119
Theorem 3.4
If B is obtained from A by interchanging two rows (or twocolumns), then det B =
det A.
In other words, interchanging two rows (or two columns) of asquare matrix changes the sign of its determinant.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.2 PROPERTIES OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
57/119
Theorem 3.5
If B is obtained from A by multiplying a row (or column) of Aby a scalar k, then det B = kdet A.
In other words, multiplication of a row or column by a constant kmultiplies the value of the determinant by k.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.2 PROPERTIES OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
58/119
Corollary 3.2
If a square matrix contains a row of all zeros, then itsdeterminant is zero.
Corollary 3.3
Let A = [aij] be an n n matrix. Then
det(kA) = kn det A
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.2 PROPERTIES OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
59/119
Theorem 3.6
If A, B, and C are n
n matrices that are equal except that
the sth row of A is equal to the sum of the sth rows of B andC, then det A = det B + det C.
CAUTION In general, det(B + C) = det B + det C
Theorem 3.7
If B is obtained from A by adding a multiple of one row of Ato another row, then det B = det A.
Thus, to obtain the value of a determinant, we can first simplify itsystematically by elementary row operations, similar to those formatrices.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.2 PROPERTIES OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
60/119
Corollary 3.4
(a) If a square matrix has two proportional rows, its determinantis zero.
(b) If two rows of a square matrix are the same, then itsdeterminant is zero.
Example 3.5 Evaluate the determinant of
A =
2a + 1 4 a x2b+ 1 9 b x
2c + 1 5 c x2c + 1 3 d x
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.2 PROPERTIES OF DETERMINANTS
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
61/119
Example 3.6 Evaluate
x 1 1 11 x 1 11 1 x 11 1 1 x
Example 3.7 Compute
2 8 1 1
4 13 3 12 5 3 36 18 1 1
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.3 MATRIX INVERSES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
62/119
Definition 3.4 The identity matrix In is the n n matrixthat has ones on the main diagonal and zeros elsewhere:
In =
1 0 0 00 1 0 0...
...
0 0 0 1
.
That is, the ijth entry of In is 0 when i = j and is 1 when i = j.For example, I2 and I3 are given by
I2 = 1 0
0 1
and I3 =
1 0 00 1 00 0 1
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.3 MATRIX INVERSES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
63/119
Remark The product of a matrix and the identity matrix isthe matrix itself:
AI = A and IB = B
provided that these products exist.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.3 MATRIX INVERSES
3
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
64/119
Definition 3.5 If A is a square matrix and there exists amatrix B such that AB = BA = I, then B is called an inverse
of A, and A is said to be invertible.
Remark If B and C are both inverses for the matrix A,then B = C.
That is,
If A has an inverse, the inverse is unique.
The inverse of A (if exists) is denoted by the symbol A1
. Thus,
A1A = AA1 = I
CAUTION A1
= 1
A
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.3 MATRIX INVERSES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
65/119
Example 3.8 Find the inverse of
A = 1 2
2 3
.
METHOD FOR FINDING AN INVERSE MATRIX
To obtain A1 for any n n matrix A, follow these steps.1. Form the augmented matrix [A|I], where I is the n n
identical matrix.
2. Use elementary row operations to transform [A|I] to the form[I|B].
Matrix B is A1.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.3 MATRIX INVERSES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
66/119
Example 3.9 Find A1 for
A = 1 2 3
2 5 41 1 10
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.4 SOLVING SYSTEMS OF EQUATIONS WITH INVERSES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
67/119
Theorem 3.8
If A is an invertible n n matrix, then for each n 1 matrixb, the system of linear equations Ax = b has exactly onesolution, namely, x = A1b.
Theorem 3.9If A is an n n matrix, then the following statements areequivalent.(a) A is invertible.(b) The system Ax = 0 has only the trivial solution.(c) The system Ax = b has a unique solution for every n 1matrix b.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.4 SOLVING SYSTEMS OF EQUATIONS WITH INVERSES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
68/119
Definition 3.6 A square matrix A is said to benonsingular if the equation Ax = 0 has only the trivialsolution x = 0. A is singular if it is not nonsingular.
By the Theorem 7.2,
A is invertible A is nonsingular.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.5 THE DETERMINANT OF A PRODUCT
Theorem 3.10
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
69/119
Theorem 3.10
If A and B are square matrices of the same size, then
det(AB) = det A det B
Theorem 3.11
A square matrix A is invertible if and only ifdet A = 0.
Combining Theorem 7.3 with Theorem 7.4 we obtain:
A square matrix A is nonsingular if and only if det A = 0.Thus,
A invertible A nonsingular det A = 0
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.5 THE DETERMINANT OF A PRODUCT
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
70/119
Example 3.10 Find all values such that the matrix B issingular, where
B = 2 0 02 3 4
1 2 1
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.5 THE DETERMINANT OF A PRODUCT
Corollary 3.5
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
71/119
Corollary 3.5
If A is invertible, then
det A1 =1
det A.
Corollary 3.6
Suppose that A and B are square matrices of the same size. Ifeither AB = I or BA = I, then B = A1.
Corollary 3.7
Let A, B, C be n n matrices. Then(a) If A is nonsingular and AB = AC, then A = C.(b) If A is nonsingular and AB = O, then B = O.(c) If A is singular, so are AB and BA.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.6 ADJOINT MATRIX AND INVERSE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
72/119
Definition 3.7 If A is any n n matrix and Aij is thecofactor of aij, the the matrix
A11 A12 A1nA21 A22
A2n
... ... ...An1 An2 Ann
is called the matrix of cofactors from A. The transpose ofthis matrix is called the adjoint of A and is denoted byadj(A).
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.6 ADJOINT MATRIX AND INVERSE
Example 3.11 Let
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
73/119
p
A = 2
3 1
4 0 23 1 3 .
The cofactors of A are
A11 = 0 2
1 3 = 2, A12 = 4 23 3 = 6,
A13 =
4 03 1 = 4, A21 =
3 11 3 = 10,
A22 = 2 1
3 3 =
9, A23 =
2 3
3 1 =
7,
A31 =
3 10 2 = 6, A32 =
2 14 2 = 8,
A33 =
2 34 0
= 12
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.6 ADJOINT MATRIX AND INVERSE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
74/119
so that the matrix of cofactors is 2 6 410 9 7
6 8 12
.
Taking the transpose, we obtain the adjoint of A
adj(A) =
2 10 6
6 9 8
4
7 12
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.6 ADJOINT MATRIX AND INVERSE
Theorem 3.12
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
75/119
Theorem 3.12
If A is an square matrix, then
A adj(A) = (det A)I.
Thus if A is invertible, then
A1 = 1det A adj(A).
Example 3.12 Find A1 if
A =
1 1 22 1 34 1 1
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.6 ADJOINT MATRIX AND INVERSE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
76/119
Remark The matrix
A =
a bc d
is invertible if and only if det A = ad bc = 0. Then
A1 =1
ad bc
d bc a
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.7 SOLVING Ax = b WITH CRAMERS RULE
Theorem 3.13 (Cramers Rule)
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
77/119
( )
If Ax = b is a system of n linear equations in n unknowns
such that det A = 0, then the system has a unique solution.This solution is
x1 =det A1det A
, x2 =det A2det A
, . . . , xn =det Andet A
where Aj is the matrix obtained by replacing the entries in thejth column of A by the entries in the matrix
b = b1
b2...bn
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.3 DETERMINANTS1.3.7 SOLVING Ax = b WITH CRAMERS RULE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
78/119
Example 3.13 Use Cramers rule to solve
3x + 2y + 3z = 4
2x 4y + 2z = 122x + 3z = 0.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.1 EUCLIDEAN n-SPACE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
79/119
Definition 4.1 If n is a positive integer, then anordered-n-tuple is a sequence of n real numbers(a1, a2, ..., an). The set of all ordered-n-tuples is calledn-space and is denoted by IRn.
When n = 2 or n = 3, we use the term ordered pair and orderedtriple rather than ordered-2-tuple and 3-tuple.
An ordered-n-tuple (a1, a2, ..., an) can be viewed either as ageneralized point or a generalized vector. We will use bothdescriptions.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.1 EUCLIDEAN n-SPACE
Definition 4.2 Two vectors u = (u1, u2, ..., un) and
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
80/119
( 1, 2, , n)v = (v1, v2, ..., vn) in IR
n are said to be equal if
u1 = v1, u2 = v2, ..., un = vn.
The sum u + v is defined by
u + v = (u1 + v1, u2 + v2, ..., un + vn).
If k is any scalar, the scalar multiple ku is defined by
ku = (ku1, ku2, ..., kun).
The zero (null) vector is defined to be the vector
0 = (0, 0, ..., 0).
It is also called the origin.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.1 EUCLIDEAN n-SPACE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
81/119
Definition 4.3 If u = (u1, u2, ..., un) is any vector in IRn,
then the negative (or additive inverse) of u is denoted byu and is defined by
u = (u1, u2, ..., un).We define subtraction of vectors in IRn by
u v = u + (v) = (u1 v1, u2 v2, ..., un vn).
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.1 EUCLIDEAN n-SPACE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
82/119
Example 4.1 Let x = (2, 1, 0, 5, 8) and y = (2, 4, 9, 1, 3)be in IR5. Then
x + y = (2 + (2), 1 + 4, 0 + 9, 5 + (1), 8 + 3) = (0, 3, 9, 4, 5)7x = (14,
7, 0, 35,
56)
x = (2, 1, 0, 5, 8)x y = (2 (2), 1 4, 0 9, 5 (1), 8 3)
= (4, 5, 9, 6, 11).
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.1 EUCLIDEAN n-SPACE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
83/119
Theorem 4.1
If u = (u1, u2, ..., un), v = (v1, v2, ..., vn), andw = (w1, w2, ..., wn) are vectors in IR
n and k and l are scalars,then(a) u + v = v + u
(b) u + (v + w) = (u + v) + w(c) u + 0 = 0 + u = u(d) u + (u) = 0, that is, u u = 0(e) k(lu) = (kl)u(f) k(u + v) = ku + kv
(g) (k + l)u = ku + lu(h) 1u = u
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.1 EUCLIDEAN n-SPACE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
84/119
Definition 4.4 If u = (u1, u2, ..., un) andv = (v1, v2, ..., vn) are any vectors in IR
n, then the Euclideaninner product or dot product u
v is defined by
u v = u1v1 + u2v2 + + unvn.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.1 EUCLIDEAN n-SPACE
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
85/119
The four main arithmetic properties of the Euclidean inner productare listed in the next theorem.
Theorem 4.2
If u, v, and w are vectors in IRn
and k is any scalar, then(a) u v = v u;(b) (u + v) w = u w + v w;(c) (ku) v = ku v = u (kv);(d) u
u
0. Further, u
u = 0 if and only if u = 0.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.1 EUCLIDEAN n-SPACE
Definition 4.5 The Euclidean norm (or length) of a
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
86/119
( g )vector u = (u1, u2, ..., un) in IR
n, is defined by
u = (u u) 12 =
u21 + u22 + + u2n.
The Euclidean distance between u = (u1, u2, ..., un) andv = (v
1, v
2, ..., v
n) in IRn, is defined y
d(u, v) = uv =
(u1 v1)2 + (u2 v2)2 + + (un vn)2.
By Euclidean n-space we mean IRn, together with these
distance and product.
Example 4.2 Let u = (1, 2
2, 4) and v = (3,
2, 5). Compute(a) u and v (b) d(u, v).
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.1 EUCLIDEAN n-SPACE
N t It i ibl t th t i t ti
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
87/119
Note It is possible to use the matrix notation
u =
u1u2...
un
rather than the horizontal natation u = (u1, u2, ..., un) to denotevectors in IRn. If we use matrix notation for the vectors u and vand omit the brackets on 1 1 matrices, then we have the matrixformula
vTu = v ufor the Euclidean inner product.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
88/119
Definition and Examples of Vector Spaces
Using IRn as a model, we now extends the notion of a vectoreven further to include subjects such as matrices, polynomials,functions continuous on a given intervals, and solutions to certain
differential equations, etc.
We take the basic properties of vectors in IRn as axioms.
The axioms guarantee that one obtains a useful and applicable
theory of those more general situations.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
89/119
Definition 4.6 A nonempty set V of elements is called a
real vector space (or real linear space), and these elementsare called vectors if an addition operation is defined betweenany two elements of V and a scalar multiplication is definedbetween any number and any vector in V as follows.
(1) If u and v are vectors in V, then u + v is a vector in V.
(2) For any two vectors u and v of V, u + v = v + u.(3) For any three vectors u, v and w of V,(u + v) + w = u + (v + w).
(4) There is a vector in V, called the zero vector and denoted by0, such that 0 + u = u + 0 = u for all u in V.
(5) For each u in V, there is a vector in V that is denoted by
u
and is such that u + (u) = (u) + u = 0.(6) If k is any scalar and u is any vector in V, then ku is in V.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
90/119
Definition 4.7
(7) For every scalar k and vectors u and v in V,k(u + v) = ku + kv.
(8) For all scalar k and l and every u in V, (k+ l)u = ku + lu.
(9) For all scalar k and l and every u in V, k(lu) = (kl)u.(10) For every u in V, 1u = u.
A complex vector space is obtained if, instead of real numbers,
we take complex numbers as scalars.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
91/119
Example 4.3 For any positive integer n, IRn
is a real vectorspace.
Example 4.4 Verify that the set of all 2
3 matrices with real
entries is a real vector space.
In general for any m and n the set of all m n matrices with realentries, together with the operations of matrix addition and scalar
multiplication, is a real vector space.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
92/119
Example 4.5 Let Pn denote the set of all real polynomials of
degree n or less. If f and g are two such polynomials and k is anyreal number, define the sum f + g and the scalar multiple kf by
(f + g)(x) = f(x) + g(x)
(kf)(x) = kf(x)Verify that Pn is a linear space.
Example 4.6 Let C[a, b] be the set of all continuous functionsdefined on [a, b]. Verify that C[a, b] is a linear space.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
93/119
Theorem 4.3 If V is a vector space, then1. The zero vector is unique.
2. For each u, the additive inverse u is unique.3. 0u = 0 for every u in V, where0 is the zero number.
4. k0 = 0 for every scalar k.
5. If ku = 0, then k = 0 or u = 0.
6. (1)u = u.7. If u + w = v + w, then v = w.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
Inner Product Spaces
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
94/119
p
Definition 4.8 An inner product on a real vector space Vis a function that associates a real number u, v with eachpair of vectors u and v in V in such a way that the followingaxioms are satisfied for all vectors u, v, and w in V and all
scalars k.(1) u, v = v, u(2) u + v, w = u, w + v, w(3) ku, v = ku, v(4)
u, u
0 and
u, u
= 0 if and only if u = 0.
A real vector with an inner product is called a real innerproduct space.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
Example 4.7 If u = (u1, u2, ..., un) and v = (v1, v2, ..., vn) arevectors in IRn the the formula
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
95/119
vectors in IR , the the formula
u, v = u1v1 + u2v2 + + unvndefines u, v to be the Euclidean inner product on IRn.
More generally, if 1, 2,...,n
are positive real numbers, and if u = (u1, u2, ..., un) andv = (v1, v2, ..., vn) are vectors in IR
n, then it can be shown that theformula
u, v = 1u1v1 + 2u2v2 + + nunvndefines an inner product on IRn; it is called the weightedEuclidean inner product with weights 1, 2,...,n.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
Example 4.8 Let f and g be continuous functions on [a, b], and
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
96/119
define
f, g = ba
f(x)g(x)dx.
Show that this formula defines an inner product on the vectorspace C[a, b].
Theorem 4.4 If u, v and w are vectors in a real inner productspace, and k is any scalar, then
(a)
0, u
= 0;
(b) u, v + w = u, v + u, w;(c) u, kv = ku, v;
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
Length and Distance in Inner Product Spaces
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
97/119
Definition 4.9 If V is an inner product space, then thenorm (or length) of a vector u is denoted by u and definedby
u
=
u, u
1/2.
Definition 4.10 If V is an inner product space, then thedistance between two points (or vectors) u and v is denotedby d(u
v) and is defined by
d(u v) = u v.
Dr Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
Linear Transformations
If V and W are any vector spaces and F is a function that
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
98/119
If V and W are any vector spaces and F is a function that
associates a unique vector in W with each vector in V, we say Fmaps V into W, and write F : V W. Further, if F associatesthe vector w in W with the vector v in V, we write w = F(v) andsay that w is the image of v under F. F is called a mapping (ortransformation or operator) of V into W.
Definition 4.11 Let V and W be vector spaces, and let Fbe a function from V to W. We say that F is a lineartransformation if for all vectors u and v in V and all scalars
k,F(u + v) = F(u) + F(v)
andF(ku) = kF(u).
D N e N H i LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.2 GENERAL VECTOR SPACES
Example 4.9 Let F : IR2 IR3 be the function defined byF(x, y) = (x, x + y, x y).
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
99/119
( , y ) ( , y , y )
Show that F is a linear operator.
Example 4.10 Let A be a fixed m n matrix. Show that thefunction T : IRn IRm defined by
T(x) = Ax.
is a linear operator. This transformation is called multiplicationby A.
Example 4.11 Let T : C[a, b] IR be defined byT(f) =
ba
f(t)dt.
Prove that T is a linear transformation.Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.3 SUBSPACES
Definition 4 12 A nonempty subset W of a vector space
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
100/119
Definition 4.12 A nonempty subset W of a vector space
V is called a subspace of V if W is itself a vector spaceunder the addition and scalar multiplication defined on V.
Theorem 4.5
Let W be a nonempty subset of a vector space V . Then W is asubspace of V if and only if the following conditions are met.
(a) If u and v are vectors in W , then u + v is in W .
(b) If k is any scalar and u is any vector in W , then ku is in W .
In other word, W is a subspace if it is closed under addition andscalar multiplication.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
101/119
8/8/2019 Applied Linear Chapter 1
102/119
1.4 VECTOR SPACES1.4.3 SUBSPACES
Theorem 4 6
8/8/2019 Applied Linear Chapter 1
103/119
Theorem 4.6
If v1, v2, ..., vr are vectors in in a vector space V , then the setof all linear combinations of v1, v2, ..., vr is a subspace of V .
If S ={
v1
, v2
, ..., vr}
is a subset of V, then the subspace Wconsisting of all linear combinations of v1, v2, ..., vr is called thesubspace spanned by S and will denoted by
lin(S) or lin{v1, v2, ..., vr}.
We also say that the set S spans W.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.3 SUBSPACES
Definition 4.14 Let A be an m n matrix. The range of
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
104/119
g
A is the set of vectors in IRm
defined by
R(A) = {y : y = Ax for some x in IRn}.
Theorem 4.7
If A is an m n matrix, then the range of A is a subspace ofIRm. More precisely,
R(A) = lin{
A1, A2..., An}
,
where A1, A2..., An are column vectors of A.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.3 SUBSPACES
Let A be the m n matrix:a11 a12 a1
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
105/119
a11 a12
a1n
a21 a22 a2n...
... ...an1 an2 ann
The rows of A,
r1 = (a11, a12, ..., a1n)r2 = (a21, a22, ..., a2n)...
...
rm = (am1, am2, ..., amn)
can be regarded as vectors in IRn, called the row vectors of A.The row space of A is defined to be lin{r1, r2, ..., rm}.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.3 SUBSPACES
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
106/119
Theorem 4.8
Let A be an m n matrix, and suppose that A is rowequivalent to the m n matrix B. Then A and B have thesame row space.
In other words, elementary row operations do not change the rowspace of a matrix.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Linear Independence
Definition 4 15 A set S = v1 v2 v of p vectors is
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
107/119
Definition 4.15 A set S =
{v1, v2, ..., vp
}of p vectors is
said to be linearly dependent if there are scalarsk1, k2, ..., kp, not all of which are zero, such that
k1v1 + k2v2 + + kpvp = 0.
S is said to be linearly independent if it is not linearlydependent; that is, the only scalars for which
k1v1 + k2v2 + + kpvp = 0are the scalars k1 = k2 =
= kp = 0.
S = {v1, v2, ..., vp} linearly independent k1v1 + k2v2 + + kpvp = 0 = k1 = k2 = = kp = 0
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Example 4.14 (a) The set of vectors {v1, v2, v3}, wherev1 = (2, 1, 6, 5), v2 = (5, 1, 2, 2), and v3 = (0, 1, 4, 3) is linearly
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
108/119
( ,
, , ), ( , , , ), ( ,
, , ) y
dependent, since 5v1 2v2 7v3 = 0.(b) Consider the set of vectors {e1, e2, e3}, wheree1 = (1, 0, 0), e2 = (0, 1, 0), and e3 = (0, 0, 1). The equationk1e1 + k2e2 + k3e3 = 0 is equivalent to (k1, k2, k3) = (0, 0, 0).
Thus, k1 = 0, k2 = 0, and k3 = 0, so the set {e1, e2, e3} is linearlyindependent.
Note (a) If a set contains the zero vector, it is linearly
independent.(b) A set with exactly two vectors is linearly independent if andonly if at least one of the vectors is a scalar multiple of the other.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
109/119
Theorem 4.9
n vectors v1 = (v11, v12, ..., v1n), ..., vn = (vn1, vn2, ..., vnn) inIRn are linearly independent if and only if
v11 v12 ... v1nv21 v22 ... vnn
......
...vn1 vn2 ... vnn
= 0.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Basis
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
110/119
Definition 4.16 A finite set S of vectors in a vector spaceV is called a basis for V if(i) S is linearly independent and(ii) S spans V.
Remark If B = {v1, v2, ..., vp} is a basis for V, then each vectorx in V can be represented uniquely in terms of the basis B. Thatis, there are unique scalars k1, k2, ..., kp such that
x = k1v1 + k2v2 + + kpvp.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Example 4.15 (a) The following systems are bases for IR3:(i) e1 = (1, 0, 0), e2 = (0, 1, 0), and e3 = (0, 0, 1).
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
111/119
(i) e1 (1, 0, 0), e2 (0, 1, 0), and e3 (0, 0, 1).
(ii) v1 = (1, 0, 0), v2 = (1, 1, 0), and v3 = (1, 1, 1).(b) In IRn, the set S of vectors
e1 = (1, 0, ..., 0), e2 = (0, 1, ..., 0), ...., en = (0, 0, ..., 1)
is linearly independent. Since any vector x = (x1, x2, ..., xn) in IRn
can be expressed as x = x1e1 + x2e2 + + xnen, S spans IRn andis therefore a basis. It is called the standard basis for IRn.
Example 4.16 The set S = {1, x, x2, ..., xn} is a basis for thevector space Pn and is called the standard basis for Pn.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Example 4.17 Let
1 0 0 1
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
112/119
M1 = 0 0 , M2 = 0 0 M3 =
0 01 0
, M4 =
0 00 1
.
Then the set S ={
M1
, M2
, M3
, M4}
is a basis for the vector spaceM22 of 2 2 matrices, called the standard basis for M22. Moregenerally, the standard basis for Mmn consists of mn differentmatrices with a single 1 and zero for the remaining entries.
Example 4.18 If S = {v1, v2, ..., vp} is a linearly independent setin a vector space V, then S is a basis for the subspace lin(S) sinceS is dependent, and by definition of lin(S), S spans lin(S).
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Theorem 4.10
If the nonzero matrix A is row equivalent to the matrix B in
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
113/119
q
echelon form, then the nonzero rows of B form a basis for therow space of A.
Example 4.19 Find a basis for the row space of
A =
1 1 2 11 2 1 11 4 1 51 0 4 12 5 0 2
.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Dimension
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
114/119
Definition 4.17 A nonzero vector space V is called finitedimensional if it contains a finite set of vectors{v1, v2, . . . , vp} that forms a basis. If no such set exists, V iscalled infinite dimensional.
Note We shall regard the zero vector space as finite dimensionaleven though it has no linearly independent sets and consequentlyno basis.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
115/119
Theorem 4.11
If S = {v1, v2, ..., vp} is a basis for a vector space V , thenevery set with more than p vectors is linearly dependent.
Corollary 4.1
Let S be a set of p vectors in IRn. If p> n, then S is linearlydependent.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
116/119
Theorem 4.12
Any two bases for a finite-dimensional vector space have thesame number of vectors.
For instance, the standard basis for IRn contains n vectors.Therefore, every basis for IRn contains n vectors.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Definition 4.18 The dimension of a space W is defined
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
117/119
to be the number of vectors in a basis for W and denoteddim(W). In addition, we define the zero vector space to havedimension zero.
For example, since {e1, e2, e3} is a basis for IR3, IR3 hasdimension 3.
In general, IRn has a basis {e1, e2, ..., en} that contains n vectors,so
dim(IRn
) = n
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Theorem 4.13
If A is any matrix, then the row space and the column space
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
118/119
of A have the same dimension.
Definition 4.19 The dimension of the range of a matrix Ais called the rank of A and is denoted by rank(A).
Corollary 4.2
rank(A) = rank(AT)
Note It should be noted that the column space of A is exactlythe range of A.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
1.4 VECTOR SPACES1.4.4 BASIS AND DIMENSION
Theorem 4.14
http://find/http://goback/8/8/2019 Applied Linear Chapter 1
119/119
A system of linear equations Ax = b is consistent if and onlyif the rank of the coefficient matrix is the same as the rank ofthe augmented matrix.
Ax = b consistent
rank(A) = rank([A|b]).
Theorem 4.15
An n
n matrix A is nonsingular if and only if the rank of A
is n.
Dr. Nguyen Ngoc Hai LINEAR ALGEBRA
http://find/http://goback/