Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Linear Algebra
Overviewc© Guan/Linear Algebra/pg 2
1. Vectors
2. Matrices
3. Systems of Linear Equations
4. The Gauss-Jordan Method
5. Linear Independence
6. The Inverse of a Matrix
7. Determinants
Table of Contentsc© Guan/Linear Algebra/pg 3
1. Vectors
2. Matrices
3. Systems of Linear Equations
4. The Gauss-Jordan Method
5. Linear Independence
6. The Inverse of a Matrix
7. Determinants
Algebrac© Guan/Linear Algebra/pg 4
Algebra
I The part of mathematics in which letters and other generalsymbols are used to represent numbers and quantities informulas and equations, e.g., x + 2 = 5.
I Algebra gives methods for writing formulas and solvingequations, as compared to arithmetic approach.
Linear Algebrac© Guan/Linear Algebra/pg 5
Linear Algebra is similar to the algebra, except that in the place ofordinary single numbers, it deals with vectors and linear operations(e.g., addition and scalar multiplication).
Linear Algebra studies
I linear equations such as a1x1 + · · ·+ anxn = b,
I linear maps such as (x1, . . . , xn) 7→ a1x1 + . . .+ anxn,
I and their representations in vector spaces and throughmatrices.
Scalar and Vectorc© Guan/Linear Algebra/pg 6
I Scalar: A scalar is a number. For instance, a magnitude butno “direction,” other than perhaps positive or negative.
I Vector: A vector is a list of numbers.I One way: A vector is a point in a space. This list of numbers
is a way of identifying that point in space.I Another way: A vector is a magnitude and a direction, it is a
directed arrow pointing from the origin to the end point givenby the list of numbers.
I Examples:
Vectorc© Guan/Linear Algebra/pg 7
I Usually written in a bold letter or a small arrow overtop of thesymbol, e.g., a and ~a.
I The total number of the list of numbers in the vector is calledthe dimension of the vector.
I e.g., a1 =
231
is a 3-dimensional vector.
I e.g., a2 = 0 =
0000
is a 4-dimensional vector. It is also a
zero vector.
Vectorc© Guan/Linear Algebra/pg 8
Column vector vs Row vector:
I By default, a vector is referred as a column vector.
I For a (column) vector u, its transpose is a row vector uᵀ.
Vector Operationsc© Guan/Linear Algebra/pg 9
Vectors Addition and SubtractionFor u = [u1, u2, . . . , um]ᵀ and v = [v1, v2, . . . , vm]ᵀ,
I u + v =
I u− v =
Parallelogram and Triangular Method
Scalar Multiplication:
I ku =
Vector Operationsc© Guan/Linear Algebra/pg 10
Scalar Product of Two Vectors (also called dot product)
I The scalar product of two vectors u = [u1, u2, . . . , um]ᵀ andv = [v1, v2, . . . , vm]ᵀ (denoted as u · v) is the numberu1v1 + u2v2 + . . .+ umvm.
Vector Operationsc© Guan/Linear Algebra/pg 11
Scalar Product of Two Vectors
I Two vectors are perpendicular (orthogonal) if and only if theirscalar product equals to 0.
I Also u · v = ||u||||v|| cos θ where ||u|| is the length of thevector u and θ is the angle between vectors u and v.
Vector Space and Axiomsc© Guan/Linear Algebra/pg 12
Vector Space: A vector space is a collection of vectors, which maybe added together and multiplied (“scaled”) by numbers.
1 Associativity of addition: u + (v + w) = (u + v) + w
2 Commutativity of addition: u + v = v + u
3 Identity element of addition: There exists an element 0 ∈ V, called thezero vector, such that v + 0 = v for all v ∈ V
4 Inverse elements of addition: For every v ∈ V, there exists an element−v ∈ V, called the additive inverse of v, such that v + (−v) = 0
5 Compatibility of scalar multiplication with field multiplication:a(bv) = (ab)v
6 Identity element of scalar multiplication: 1v = v
7 Distributivity of scalar multiplication with respect to vector addition:a(u + v) = au + av
8 Distributivity of scalar multiplication with respect to field addition:(a + b)v = av + bv
For example, R2 is a vector space. Note that every vector spacecontains vector 0.
Table of Contentsc© Guan/Linear Algebra/pg 13
1. Vectors
2. Matrices
3. Systems of Linear Equations
4. The Gauss-Jordan Method
5. Linear Independence
6. The Inverse of a Matrix
7. Determinants
Matrixc© Guan/Linear Algebra/pg 14
I A matrix is any rectangular array of numbers.
I If a matrix A has m rows and n columns, we call A a m × nmatrix and m × n is referred as the order of matrix A.
A =
a11 a12 . . . a1na21 a22 . . . a2n
......
...am1 am2 . . . amn
I The number in the ith row and jth column of A is called the
ijth element of A and is written as aij .
Matrix Operationsc© Guan/Linear Algebra/pg 15
I The Scalar Multiple of a Matrix (multiplying each element ofA by the scalar)
I Addition/subtraction of Two Matrices in the same order(add/subtract the corresponding elements)
Matrix Operationsc© Guan/Linear Algebra/pg 16
I The transpose of a Matrix: switch the row and column indicesof the matrix (note (Aᵀ)ᵀ = A)
I Matrix Multiplication: C = ABI Condition: Number of columns in A = number of rows in B.I ij element of C =scalar product of row i of A × column j of B.I If A is m × r and B is r × n, then C is m × n.
Matrix: A linear operatorc© Guan/Linear Algebra/pg 17
I A matrix does things (linear transformation) to Vectors:transform a vector to another vector.
I Linear transformation can be considered a type of function:mapping a vector to another vector.
I Examples:
A =
[0 1−1 0
]rotate 90 degree clockwise.
A =
[cos θ sin θ− sin θ cos θ
]rotate θ degree clockwise.
Properties of Matrix Multiplication and EXCELc© Guan/Linear Algebra/pg 18
I Row i of AB = (row i of A)B
I Column j of AB = A(column j of B)
I Matrix multiplication is associative. That is, A(BC ) = (AB)C
I Matrix multiplication is distributive. That is,A(B + C ) = AB + AC and (B + C )D = BD + CD
Matrix Multiplication with EXCEL: Input Matrices A (D2:F3) andB (D5:E7) and use array function “MMULT(D2:F3, D5:E7)”(Control Shift Enter)
Table of Contentsc© Guan/Linear Algebra/pg 19
1. Vectors
2. Matrices
3. Systems of Linear Equations
4. The Gauss-Jordan Method
5. Linear Independence
6. The Inverse of a Matrix
7. Determinants
The Linear Equation Systemc© Guan/Linear Algebra/pg 20
The linear equation system
a11x1 + a12x2 + . . . + a1nxn = b1a21x1 + a22x2 + . . . + a2nxn = b2
......
... =...
am1x1 + am2x2 + . . . + amnxn = bm.
It can be written as
Ax = b (matrix representation) or A|b (augmented matrix), where
A =
a11 a12 . . . a1na21 a22 . . . a2n
......
...am1 am2 . . . amn
, x =
x1x2...xn
, b =
b1b2...bn
An Examplec© Guan/Linear Algebra/pg 21
The Linear Equation Systemc© Guan/Linear Algebra/pg 22
All three cases for a linear equation system:
I The system has no solution.
I The system has a unique solution.
I The system has in infinite number of solutions.
Table of Contentsc© Guan/Linear Algebra/pg 23
1. Vectors
2. Matrices
3. Systems of Linear Equations
4. The Gauss-Jordan Method
5. Linear Independence
6. The Inverse of a Matrix
7. Determinants
Elementary Row Operations (EROs)c© Guan/Linear Algebra/pg 24
For a given matrix A, we can use EROs to yield a new matrix A′
through one of three procedures:
1: Obtain A′ by multiplying any row of A by a nonzero scalar.
2: Obtain A′ by multiplying a row by a nonzero scalar and addthe result to another row.
3: Obtain A′ by interchanging any two rows of A.
After ERO, A and A′ are row equivalent and the correspondinglinear equation systems are equivalent.
Reduced Row Echelon Formc© Guan/Linear Algebra/pg 25
A matrix is in reduced row echelon form if it satisfies the followingconditions:
I All rows consisting of only zeroes are at the bottom.
I The leading entry (also called the pivot) of a nonzero row isalways strictly to the right of the leading entry of the rowabove it.
I The leading entry in each nonzero row is a 1 (i.e., a leading 1).
I Each column containing a leading 1 has zeros in all its otherentries. 1 0 a1 0 b1
0 1 a2 0 b20 0 0 1 b3
Remark: The leading entry of each row of a matrix is the leftmost
non-zero element of that row.
The Gauss-Jordan Methodc© Guan/Linear Algebra/pg 26
Purpose: Use EROs for A|b to finally transform A into a newmatrix A′|b′ in reduced row echelon form.
1. To solve Ax = b, write down the augmented matrix A|b2. Use ERO type 3 to swap the rows so that all rows with all
zero entries are on the bottom
3. Use ERO type 3 to swap the rows so that the row with theleftmost nonzero entry is on top
4. Use ERO type 1 to multiply the top row by a scalar so thattop row’s leading entry becomes 1.
5. Use ERO type 2 to add/subtract multiples of the top row tothe other rows so that all other entries in the columncontaining the top row’s leading entry are all zero.
6. Repeat steps 3-5 for the next leftmost nonzero entry until allthe leading entries are 1.
7. Double-check if the leading entry of each nonzero row is tothe right of the leading entry of the row above it. If not, useERO type 3 to swap the rows.
The Gauss-Jordan Methodc© Guan/Linear Algebra/pg 27
After the Gauss-Jordan method is applied, the variables aregrouped into two categories:
I Basic variable: a variable that appears with a coefficient of 1in a single equation and a coefficient of 0 in all otherequations.
I Nonbasic variable: any variable that is not a basic variable.
The Gauss-Jordan Methodc© Guan/Linear Algebra/pg 28
I Case 1: A′x = b′ contains at least one row of the form
[0 0 . . . 0|c](c 6= 0).
In this case, Ax = b has no solution.
I Case 2: If Case 1 does not hold and there are no nonbasicvariables, then Ax = b will have a unique solution.
I Case 3: If Case 1 does not hold and there is at least onenonbasic variable, then Ax = b will have an infinite number ofsolutions.
The Gauss-Jordan Methodc© Guan/Linear Algebra/pg 29
Table of Contentsc© Guan/Linear Algebra/pg 30
1. Vectors
2. Matrices
3. Systems of Linear Equations
4. The Gauss-Jordan Method
5. Linear Independence
6. The Inverse of a Matrix
7. Determinants
Linear Independencec© Guan/Linear Algebra/pg 31
I A linear combination of the vectors in a vector space V is anyvector of the form
c1v1 + c2v2 + . . .+ ckvk
where c1, . . . , ck are arbitrary scalars.
I Trivial linear combination: c1 = c2 = . . . ck = 0.
I A set V of vectors is linearly independent if the trivial linearcombination is the only combination to make it equal to 0.Otherwise, it is linearly dependent.
I Example: Any set of vectors containing the 0 vector is alinearly dependent set.
The Rank of a Matrixc© Guan/Linear Algebra/pg 32
I Let A be a m ×m matrix and denote the rows of A byr1, r2, . . . , rm. Define R = {r1, r2, . . . , rm}.
I The rank of A is the number of vectors in the largest linearlyindependently subset of R.
I Examples:
A =
[0 00 0
]B =
[2 23 3
]C =
[0 11 0
]
Gauss-Jordan Method to Find Rank of Matrixc© Guan/Linear Algebra/pg 33
I For a matrix A, apply Gauss-Jordan method to obtain thefinal matrix A.
I It can be shown Rank(A)= Rank(A) = number of nonzerorows in A.
I Method to show if a set of vectors is linearly independent
I Given a set of vectors V = {v1, v2, . . . , vm}I Create a matrix A with the ith row to be vi . Thus, A has m
rows.I Use Gauss-Jordan to find rank A.I If rank A = m (referred as full rank), then V is linearly
independent. Otherwise, i.e., rank A < m, V is linearlydependent.
Table of Contentsc© Guan/Linear Algebra/pg 34
1. Vectors
2. Matrices
3. Systems of Linear Equations
4. The Gauss-Jordan Method
5. Linear Independence
6. The Inverse of a Matrix
7. Determinants
Inverse of a Matrixc© Guan/Linear Algebra/pg 35
I A square matrix
I Diagonal elements of a square matrix
I Identity Matrix
Inverse of a Matrixc© Guan/Linear Algebra/pg 36
I require a square matrix A is (m ×m).
I require full rank, i.e., Rank(A)=m.
I If AB = BA = Im, then B is the inverse of A, denoted asB = A−1.
I Link to reciprocal for real numbers.
I Inverting Matrices with EXCEL (“MINVERSE”)
Inverse of a Matrixc© Guan/Linear Algebra/pg 37
I Gauss-Jordan method to get A−1:I Step 1: Create the m × 2m matrix A|Im.I Step 2: Use EROs to transform A|Im to Im|B. Then B = A−1.
If Rank(A) < m, then A has no inverse.
Table of Contentsc© Guan/Linear Algebra/pg 38
1. Vectors
2. Matrices
3. Systems of Linear Equations
4. The Gauss-Jordan Method
5. Linear Independence
6. The Inverse of a Matrix
7. Determinants
Determinantc© Guan/Linear Algebra/pg 39
I require a square matrix A is (m ×m).
I the determinant of A is denoted as detA or |A|.I For a 1× 1 matrix, detA = a11.
I For a 2× 2 matrix, detA = a11a22 − a21a12.
Determinantc© Guan/Linear Algebra/pg 40
I For a general m ×m matrix, use
detA = (−1)i+1ai1(detAi1)+(−1)i+2ai2(detAi2)+. . .+(−1)i+maim(detAim)
Here Aij is the ijth minor of a, which is the (m− 1)× (m− 1)matrix obtained from A after deleting the ith row and jthcolumn of A.
Determinantc© Guan/Linear Algebra/pg 41
Geometrically, determinant can be viewed as the signed volumescaling factor of the linear transformation described by the matrix.
Q & A