De�nitions of Linear Algebra Terms

In order to learn and understand mathematics, it is necessary to understandthe meanings of the terms (vocabulary words) that are used. This documentcontains de�nitions of some of the important terms used in linear algebra.All of these de�nitions should be memorized (and not just memorized butunderstood). In addition to the de�nitions that are given here, we also includestatements of some of the most important results (theorems) of linear algebra.The proofs of these theorems are not included here but will be given in class.There will be a question on each of the exams that asks you to state

some de�nitions. It will be required that you state these de�nitions veryprecisely in order to receive credit for the exam question. (Seemingly verysmall things such as writing the word �or� instead of the word �and�, forexample, can render a de�nition incorrect.) The de�nitions given here areorganized according to the section number in which they appear in DavidLay�s textbook.Here is a suggestion for how to study linear algebra (and other subjects

in higher level mathematics):

1. Carefully read a section in the textbook and also read your class notes.Pay attention to de�nitions of terms and examples and try to under-stand each concept along the way as you read it. You don�t have tomemorize all of the de�nitions upon �rst reading. You will memorizethem gradually as you work the homework problems.

2. Work on the homework problems. As you work on the homework prob-lems, you should �nd that you will need to continuously refer back tothe de�nitions and to the examples given in the textbook. (This ishow you will naturally come to memorize the de�nitions.) You will�nd that you have to spend a considerable amount of time working onhomework. When you are done, you will probably have read throughthe section and your course notes several times and you will probablyhave memorized the de�nitions. (There are some very good True�Falsequestions among the homework problems that are good tests of yourunderstanding of the basic terms and concepts. Such True�False ques-tions will also appear on the exams given in this course.)

3. Test yourself. After you are satis�ed that you have a good under-standing of the concepts and that can apply those concepts in solving

1

problems, try to write out the de�nitions without referring to your bookor notes. You might then also try to work some of the supplementaryproblems that appear at the end of each chapter of the textbook.

4. One other tip is that it is a good idea to read a section in the textbookand perhaps even get started on working on the homework problemsbefore we go over that section in class. This will better prepare you totake notes and ask questions when we do go over the material in class.It will alert you to certain aspects of the material that you might notquite understand upon �rst reading and you can then ask questionsabout those particular aspects during class.

1.1: Systems of Linear Equations

De�nition 1 A linear equation in the variables x1; x2; : : : ; xn is an equa-tion of the form

a1x1 + a2x2 + : : :+ anxn = b

where a1; a2; : : : ; an and b are real or complex numbers (that are often givenin advance).

Example 24x+ 4y � 7z + 8w = 8

is a linear equation in the variables x; y; z; and w.

3x+ 5y2 = 6

is not a linear equation (because of the appearance of y2).

De�nition 3 Suppose that m and n are positive integers. A system oflinear equations in the variables x1; x2; : : : ; xn is a set of equations of theform

a11x1 + a12x2 + : : :+ a1nxn = b1

a21x1 + a22x2 + : : :+ a2nxn = b2...

am1x1 + am2x2 + : : :+ amnxn = bm

2

where aij (i = 1; : : : ;m, j = 1; : : : n) and bi (i = 1; : : : ;m) are real or complexnumbers (that are often given in advance). This system of linear equationsis said to consist of m equations with n unknowns.

Example 4

2x1 + 2x2 � 5x3 + 4x4 � 2x5 = 4�4x1 + x2 � 5x4 = �1

3x1 � 2x2 + 4x4 � 2x5 = �4

is a system of three linear equations in �ve unknowns. (The �unknowns�or�variables�of the system are x1; x2; x3; x4; and x5.)

De�nition 5 A solution of the linear system

a11x1 + a12x2 + : : :+ a1nxn = b1

a21x1 + a22x2 + : : :+ a2nxn = b2...

am1x1 + am2x2 + : : :+ amnxn = bm

is an ordered n�tuple of numbers (s1; s2; : : : ; sn) such that all of the equationsin the system are true when the substitution x1 = s1; x2 = s2; : : : ; xn = sn ismade.

Example 6 The 5�tuple�0;�1;�12

5; 0; 3

�is a solution of the linear system

2x1 + 2x2 � 5x3 + 4x4 � 2x5 = 4�4x1 + x2 � 5x4 = �1

3x1 � 2x2 + 4x4 � 2x5 = �4

because all equations in the system are true when we make the substitutions

x1 = 0

x2 = �1

x3 = �12

5x4 = 0

x5 = 3.

The 5�tuple�1; 3; 3

5; 0; 1

2

�is also a solution of the system (as the reader can

check).

3

De�nition 7 The solution set of a system of linear equations is the set ofall solutions of the system.

De�nition 8 Two linear systems are said to be equivalent to each other ifthey have the same solution set.

De�nition 9 A linear system is said to be consistent if it has at least onesolution and is otherwise said to be inconsistent.

Example 10 The linear system

2x1 + 2x2 � 5x3 + 4x4 � 2x5 = 4�4x1 + x2 � 5x4 = �1

3x1 � 2x2 + 4x4 � 2x5 = �4

is consistent because it has at least one solution. (It in fact has in�nitelymany solutions.) However, the linear system

x+ 2y = 5

x+ 2y = 9

is obviously inconsistent.

De�nition 11 Suppose thatm and n are positive integers. Anm�nmatrixis a rectangular array of numbers with m rows and n columns. Matrices areusually denoted by capital letters. If m = 1 or n = 1, then the matrix isreferred to as a vector. In particular, if m = 1, then the matrix is a rowvector of dimension n, and if n = 1 then the matrix is a column vector ofdimension m. Vectors are usually denoted by lower case boldface letters or(when hand�written) as lower case letters with arrows above them.

Example 12

A =

2666648 10 �6 �8�5 7 6 �60 5 �10 11 �3 �10 5�4 �8 �1 �10

377775is a 5� 4 matrix.

r = �!r =�2 3 �1 �9 8

�4

is a row vector of dimension 5.

c = �!c =

24 4�510

35is a column vector of dimension 3.

Remark 13 If A is an m � n matrix whose columns are made up of thecolumn vectors c1; c2; : : : ; cn, then A can be written as

A = [c1 c2 � � � cn] .

Likewise, if A is made up of the row vectors r1; r2; : : : ; rm, then A can bewritten as

A =

26664r1r2...rm

37775 .Example 14 The 5� 4 matrix

A =

2666648 10 �6 �8�5 7 6 �60 5 �10 11 �3 �10 5�4 �8 �1 �10

377775can be written as

A = [c1 c2 c3 c4]

where

c1 =

2666648�501�4

377775 , c2 =

2666641075�3�8

377775 , c3 =

266664�66�10�10�1

377775 , c4 =

266664�8�615�10

377775

5

or can be written as

A =

266664r1r2r3r4r5

377775where

r1 =�8 10 �6 �8

�r2 =

��5 7 6 �6

�r3 =

�0 5 �10 1

�r4 =

�1 �3 �10 5

�r5 =

��4 �8 �1 �10

�.

De�nition 15 For a given linear system of equations

a11x1 + a12x2 + : : :+ a1nxn = b1

a21x1 + a22x2 + : : :+ a2nxn = b2...

am1x1 + am2x2 + : : :+ amnxn = bm,

the m� n matrix

A =

26664a11 a12 � � � a1na21 a22 � � � a2n...

.... . .

...am1 am2 � � � amn

37775is called the coe¢ cient matrix of the system and the m� (n+ 1) matrix

[A b] =

26664a11 a12 � � � a1n b1a21 a22 � � � a2n b2...

.... . .

......

am1 am2 � � � amn bm

37775is called the augmented matrix of the system.

6

Example 16 The coe¢ cient matrix of the linear system

2x1 + 2x2 � 5x3 + 4x4 � 2x5 = 4�4x1 + x2 � 5x4 = �1

3x1 � 2x2 + 4x4 � 2x5 = �4

is

A =

24 2 2 �5 4 �2�4 1 0 �5 03 �2 0 4 �2

35and the augmented matrix of this system is

[A b] =

24 2 2 �5 4 �2 4�4 1 0 �5 0 �13 �2 0 4 �2 �4

35 .De�nition 17 An elementary row operation is performed on a matrixby changing the matrix in one of the following ways:

1. Interchange: Interchange two rows of the matrix. If rows i and j areinterchanged, then we use the notation ri ! rj.

2. Scaling: Multiply all entries in a single row by a non�zero constant,k. If the ith row is multiplied by k, then we use the notation k �ri ! ri.

3. Replacement: Replace a row with the sum of itself and a non�zeromultiple of another row. If the ith row is replaced with the sum of itselfand k times the jth row, then we use the notation ri + k � rj ! ri.

Example 18 Here are some examples of elementary row operations per-formed on the matrix

A =

24 �2 5 7 7�7 4 �8 28 7 8 �2

35 .1. Interchange rows 1 and 3:24 �2 5 7 7

�7 4 �8 28 7 8 �2

35 r1 !r3�!

24 8 7 8 �2�7 4 �8 2�2 5 7 7

357

2. Scale row 1 by a factor of �5.24 �2 5 7 7�7 4 �8 28 7 8 �2

35 �5�r1!r1�!

24 10 �25 �35 �35�7 4 �8 28 7 8 �2

353. Replace row 2 with the sum of itself and 2 times row 3.24 �2 5 7 7

�7 4 �8 28 7 8 �2

35 r2+2�r3!r2�!

24 �2 5 7 79 18 8 �28 7 8 �2

35De�nition 19 An m�n matrix, A, is said to be row equivalent (or simplyequivalent) to an m � n matrix, B, if A can be transformed into B by aseries of elementary row operations. If A is equivalent to B, then we writeA � B.

Example 20 As can be seen by studying the previous example24 �2 5 7 7�7 4 �8 28 7 8 �2

35�24 8 7 8 �2�7 4 �8 2�2 5 7 7

35 ,24 �2 5 7 7�7 4 �8 28 7 8 �2

35�24 10 �25 �35 �35�7 4 �8 28 7 8 �2

35 ,and 24 �2 5 7 7

�7 4 �8 28 7 8 �2

35�24 �2 5 7 79 18 8 �28 7 8 �2

35 .Example 21 The sequence of row operations24 3 6�1 6�7 �1

35 �2�r1!r1�!

24 �6 �12�1 6�7 �1

35 �4�r3!r3�!

24 �6 �12�1 628 4

35 r1 !r3�!

24 28 4�1 6�6 �12

35shows that 24 3 6

�1 6�7 �1

35�24 28 4�1 6�6 �12

35 .8

Remark 22 (optional) Row equivalence of matrices is an example of whatis called an equivalence relation. An equivalence relation on a non�emptyset of objects, X, is a relation, �, that is re�exive, symmetric, and transitive.The re�exive property is the property that x � x for all x 2 X. Thesymmetric property is the property that if x 2 X and y 2 X with x � y,then y � x. The transitive property is the property that if x 2 X, y 2 X,and z 2 X with x � y and y � z, then x � z. If we take X to be the setof all m� n matrices, then row equivalence is an equivalence relation on Xbecause

1. A � A for all A 2 X.

2. If A 2 X and B 2 X and A � B, then B � A.

3. If A 2 X, B 2 X, and C 2 X and A � B and B � C, then A � C.

1.2: Row Reduction and Echelon Forms

De�nition 23 The leading entry in a row of a matrix is the �rst (fromthe left) non�zero entry in that row.

Example 24 For the matrix2666640 1 4 80 0 �4 35 0 0 100 0 0 �19 �7 2 8

377775 ,the leading entry in row 1 is 1, the leading entry in row 2 is �4, the leadingentry in row 3 is 5, the leading entry in row 4 is �1, and the leading entryin row 5 is 9. (Note that if a matrix happens to have a row consisting of allzeros, then that row does not have a leading entry.)

De�nition 25 A zero row of a matrix is a row consisting entirely of zeros.A nonzero row is a row that is not a zero row.

De�nition 26 An echelon form matrix is a matrix for which

1. No zero row has a nonzero row below it.

9

2. Each leading entry is in a column to the right of the leading entry inthe row above it.

De�nition 27 A reduced echelon form matrix is an echelon form matrixwhich also has the properties:

3. The leading entry in each nonzero row is 1.

4. Each leading 1 is the only nonzero entry in its column.

Example 28 The matrix 2664�6 4 �101 2 �104 4 �20 10 �1

3775is not an echelon form matrix.The matrix �

0 0 0 4 80 0 0 0 3

�is an echelon form matrix but not a reduced echelon form matrix.The matrix 266664

0 0 1 00 0 0 10 0 0 00 0 0 00 0 0 0

377775is a reduced echelon form matrix.

Theorem 29 (Uniqueness of the Reduced Echelon Form) Each ma-trix, A, is equivalent to one and only one reduced echelon form matrix. Thisunique matrix is denoted by rref(A).

Example 30 For

A =

2664�6 4 �101 2 �104 4 �20 10 �1

3775 ;10

we can see (after some work) that

rref (A) =

26641 0 00 1 00 0 10 0 0

3775 .For

A =

�0 0 0 4 80 0 0 0 3

�,

we have

rref (A) =�0 0 0 1 00 0 0 0 1

�.

For

A =

2666640 0 1 00 0 0 10 0 0 00 0 0 00 0 0 0

377775 ,we have

rref (A) =

2666640 0 1 00 0 0 10 0 0 00 0 0 00 0 0 0

377775 .De�nition 31 A pivot position in a matrix, A, is a position in A thatcorresponds to a row�leading 1 in rref(A). A pivot column of A is columnof A that contains a pivot position.

Example 32 By studying the preceding example, we can see that the pivotcolumns of

A =

2664�6 4 �101 2 �104 4 �20 10 �1

3775are columns 1, 2, and 3. (Thus all columns of A are pivot columns).

11

The pivot columns of

A =

�0 0 0 4 80 0 0 0 3

�are columns 4 and 5.The pivot columns of

A =

2666640 0 1 00 0 0 10 0 0 00 0 0 00 0 0 0

377775are columns 3 and 4.

De�nition 33 Let A be the coe¢ cient matrix of the linear system of equa-tions

a11x1 + a12x2 + : : :+ a1nxn = b1

a21x1 + a22x2 + : : :+ a2nxn = b2...

am1x1 + am2x2 + : : :+ amnxn = bm.

Each variable that corresponds to a pivot column of A is called a basic vari-able and the other variables are called free variables.

Example 34 The coe¢ cient matrix of the linear system

2x1 + 2x2 � 5x3 + 4x4 � 2x5 = 4�4x1 + x2 � 5x4 = �1

3x1 � 2x2 + 4x4 � 2x5 = �4

is

A =

24 2 2 �5 4 �2�4 1 0 �5 03 �2 0 4 �2

35 .Since

rref (A) =

24 1 0 0 65

25

0 1 0 �15

85

0 0 1 �25

65

35 ,12

we see that the pivot columns of A are columns 1, 2, and 3. Thus the basicvariables of this linear system are x1, x2; and x3. The free variables are x4and x5.

Theorem 35 (Existence and Uniqueness Theorem) A linear system ofequations is consistent if and only if the rightmost column of the augmentedmatrix for the system is not a pivot column. If a linear system is consistent,then the solution set of the linear system contains either a unique solution(when there are no free variables), or in�nitely many solutions (whenthere is at least one free variable).

Example 36 The augmented matrix of the system

2x1 + 2x2 � 5x3 + 4x4 � 2x5 = 4�4x1 + x2 � 5x4 = �1

3x1 � 2x2 + 4x4 � 2x5 = �4

is

[A b] =

24 2 2 �5 4 �2 4�4 1 0 �5 0 �13 �2 0 4 �2 �4

35 .Since

rref ([A b]) =

24 1 0 0 65

25

65

0 1 0 �15

85

195

0 0 1 �25

65

65

35 ,we see that the rightmost column of [A b] is not a pivot column. This meansthat the system is consistent. Furthermore, since the system does have freevariables, there are in�nitely many solutions to the system.

Here is a general procedure for determining whether a given linear sys-tem of equations is consistent or not and, if it is consistent, whether it hasin�nitely many solutions or a unique solution:

1. Compute rref([A b]) to determine whether or not the rightmost col-umn of [A b] is a pivot column. If the rightmost of [A b] is a pivotcolumn, then the system is inconsistent.

13

2. Assuming that the rightmost column of [A b] is not a pivot column,take a look at rref(A). (This requires no extra computation.) If allcolumns of A are pivot columns, then the linear system has a uniquesolution. If not all columns of A are pivot columns, then the systemhas in�nitely many solutions.

1.3: Vector Equations

De�nition 37 Suppose that

u =

26664u1u2...un

37775 and v =

26664v1v2...vn

37775are column vectors of dimension n. We de�ne the sum of u and v to be thevector

u+ v =

26664u1 + v1u2 + v2...

un + vn

37775 .De�nition 38 Suppose that

u =

26664u1u2...un

37775is a column vectors of dimension n and suppose that c is a real number (alsocalled a scalar). We de�ne the scalar multiple cu to be

cu =

26664cu1cu2...cun

37775 .

14

De�nition 39 The symbol Rn denotes the set of all ordered n�tuples of num-bers. The symbol Vn denotes the set of all column vectors of dimension n.(The textbook author does not make this distinction. He uses Rn to denoteboth things. However, there really is a di¤erence. Elements of Rn shouldbe regarded as points whereas elements of Vn are matrices with one column.)If x = (x1; x2; : : : ; xn) is a point in Rn, then the position vector of x is thevector

x =

26664x1x2...xn

37775 .Example 40 x = (1; 2) is a point in R2. The position vector of this point(which is an element of V2) is

x =

�12

�.

De�nition 41 Suppose that u1;u2; : : : ;un are vectors in Vm. (Note thatthis is a set of n vectors, each of which has dimension m). Also suppose thatc1; c2; : : : ; cn are scalars. Then the vector

c1u1 + c2u2 + � � �+ cnunis called a linear combination of the vectors u1;u2; : : : ;un.

Example 42 Suppose that

u1 =

�12

�, u2 =

�0�5

�, u3 =

�14

�.

Then the vector

4u1 � 2u2 + 3u3 = 4�12

�� 2

�0�5

�+ 3

�14

�=

�730

�is a linear combination of u1,u2, and u3.

De�nition 43 Suppose that u1;u2; : : : ;un are vectors in Vm. The span ofthis set of vectors, denoted by Spanfu1;u2; : : : ;ung, is the set of all vectorsthat are linear combinations of the vectors u1;u2; : : : ;un. Formally,

Span fu1;u2; : : : ;ung= fv 2 Vm j there exist scalars c1; c2; : : : ; cn such that v = c1u1 + c2u2 + � � �+ cnung .

15

De�nition 44 The zero vector in Vm is the vector

0m =

2666400...0

37775 .Remark 45 If u1;u2; : : : ;un is any set of vectors in Vm, then 0m 2 Spanfu1;u2; : : : ;ungbecause

0m = 0u1 + 0u2 + � � �+ 0un.

De�nition 46 A vector equation in the variables x1, x2,. . . ,xn is an equa-tion of the form

x1a1 + x2a2 + � � �+ xnan = bwhere a1,a2,. . . ,an and b are vectors in Vm (which are usually known before-hand). An ordered n�tuple (s1; s2; : : : ; sn) is called a solution of this vectorequation if the equation is satis�ed when we set x1 = s1, x2 = s2,. . . , xn = sn.The solution set of the vector equation is the set of all of its solutions.

Example 47 An example of a vector equation is

x1

�12

�+ x2

�0�5

�+ x3

�14

�=

��4�2

�.

The ordered triple (x1; x2; x3) = (�9; 4=5; 5) is a solution of this vector equa-tion because

�9�12

�+4

5

�0�5

�+ 5

�14

�=

��4�2

�.

Remark 48 The vector equation

x1a1 + x2a2 + � � �+ xnan = b

is consistent if and only if b 2 Spanfa1; a2; : : : ; ang.

Remark 49 The vector equation

x1a1 + x2a2 + � � �+ xnan = b

16

where

a1 =

26664a11a21...am1

37775 , a2 =26664a12a22...am2

37775 , . . . ,an =26664a1na2n...amn

37775and

b =

26664b1b2...bm

37775is equivalent to the system of linear equations

a11x1 + a12x2 + : : :+ a1nxn = b1

a21x1 + a22x2 + : : :+ a2nxn = b2...

am1x1 + am2x2 + : : :+ amnxn = bm.

This means that the vector equation and the system have the same solutionsets. Each can be solved (or determined to be inconsistent) by row�reducingthe augmented matrix �

a1 a2 � � � an b�.

Example 50 Let us solve the vector equation

x1

�12

�+ x2

�0�5

�+ x3

�14

�=

��4�2

�.

This equation is equivalent to the linear system

x1 + x3 = �42x1 � 5x2 + 4x3 = �2.

Since

[A b] =

�1 0 1 �42 �5 4 �2

�and

rref ([A b]) =

�1 0 1 �40 1 �2

5�65

�,

17

we see that the vector equation is consistent (because the rightmost column of[A b] is not a pivot column) and that it in fact has in�nitely many solutions(because not all columns of A are pivot columns). We also see that the generalsolution is

x1 = �4� x3

x2 = �6

5+2

5x3

x3 = free.

Note that if we set x3 = 5, then we get the particular solution

x1 = �9

x2 =4

5x3 = 5

that was identi�ed in Example 47.

1.4: The Matrix Equation Ax = b

De�nition 51 For a matrix of size m� n,

A =�a1 a2 � � � an

�=

26664a11 a12 � � � a1na21 a22 � � � a2n...

.... . .

...am1 am2 � � � amn

37775 ,and a vector of dimension n,

x =

26664x1x2...xn

37775 ,we de�ne the product Ax to be

Ax = x1a1 + x2a2 + � � �+ xnan.

18

Example 52 Suppose

A =

��2 3 44 �3 �3

�and

x =

24 4�10

35 .Then

Ax = 4

��24

�� 1

�3�3

�+ 0

�4�3

�=

��1119

�.

De�nition 53 Suppose that

A =�a1 a2 � � � an

�=

26664a11 a12 � � � a1na21 a22 � � � a2n...

.... . .

...am1 am2 � � � amn

37775is an m� n matrix and that

b =

26664b1b2...bm

37775 2 Vm.Then

Ax = b

is called a matrix equation.

Remark 54 The system of linear equations

a11x1 + a12x2 + : : :+ a1nxn = b1

a21x1 + a22x2 + : : :+ a2nxn = b2...

am1x1 + am2x2 + : : :+ amnxn = bm,

19

the vector equation

x1a1 + x2a2 + � � �+ xnan = b,

and the matrix equationAx = b

are all equivalent to each other! The only subtle di¤erence is that solutionsof the system and of the vector equation are ordered n�tuples

(x1; x2; : : : ; xn) = (s1; s2; : : : ; sn) 2 Rn;

whereas solutions of the matrix equation are the corresponding vectors26664x1x2...xn

37775 =26664s1s2...sn

37775 2 Vn.

1.5: Solution Sets of Linear Systems

De�nition 55 A homogeneous linear system is one of the form

Ax = 0m

where A is an m� n matrix.A non�homogeneous linear system is a linear system that is not ho-

mogeneous.

Remark 56 Homogenous systems are consistent! That�s because they havethe solution x = 0n.

De�nition 57 The solution x = 0n is called the trivial solution of thehomogeneous system Ax = 0m. A non�trivial solution of the homogeneoussystem Ax = 0m is a solution x 2 Vn such that x 6= 0n.

Example 58 The system

5x+ 2y � 3z + w = 03x+ y � 5z + w = 0x� 3y � 5z + 5w = 0

is homogeneous. A non�trivial solution of this system (as the reader cancheck) is (x; y; z; w) = (�18; 34; 1; 25).

20

Theorem 59 If every column of A is a pivot column, then the homogeneoussystem Ax = 0m has only the trivial solution. If not every column of A isa pivot column, then the homogeneous system Ax = 0m has in�nitely manynon�trivial solutions (in addition to the trivial solution).

Example 60 The system given in Example 58 has coe¢ cient matrix

A =

24 5 2 �3 13 1 �5 11 �3 �5 5

35 .Without doing any computations, we can see that it is not possible that everycolumn of A can be a pivot column. (Why?) Therefore the homogeneoussystem Ax = 0m has in�nitely many non�trivial solutions.

Theorem 61 Suppose that A is an m� n matrix and suppose that b 2 Vm.Also suppose that v 2 Vn is a solution of the homogeneous system Ax = 0mand that w 2 Vn is a solution of the system Ax = b. Then v +w is also asolution of Ax = b.

1.7: Linear Independence

De�nition 62 A set of vectors fv1;v2; : : : ;vng in Vm is said to be linearlyindependent if the vector equation

x1v1 + x2v2 + � � �+ xnvn = 0m

has only the trivial solution. Otherwise the set of vectors is said to be linearlydependent.

Example 63 Let v1 and v2 be the vectors

v1 =

24 4�1�4

35 , v2 =

24 �1�1�4

35 .We solve the vector equation

x1v1 + x2v2 = 03

21

by observing that

A =

24 4 �1�1 �1�4 �4

35�24 1 00 10 0

35 .Since each column of A is a pivot column, the vector equation has only thetrivial solution. Therefore the set of vectors fv1;v2g is linearly independent.De�nition 64 Suppose that fv1;v2; : : : ;vng is a linearly dependent set ofvectors and suppose that (c1; c2; : : : ; cn) is a non�trivial solution of the vec-tor equation

x1v1 + x2v2 + � � �+ xnvn = 0m.Then the equation

c1v1 + c2v2 + � � �+ cnvn = 0mis called a linear dependence relation for the set of vectors fv1;v2; : : : ;vng.Example 65 Consider the set of vectors

v1 =

24 4�1�4

35 , v2 =

24 �1�1�4

35 , v3 =

24 �234

35 , v4 =

24 4�3�3

35 .We solve the vector equation

x1v1 + x2v2 + x3v3 + x4v4 = 03

by observing that24 4 �1 �2 4�1 �1 3 �3�4 �4 4 �3

35�24 1 0 0 11

40

0 1 0 �1320

0 0 1 �98

35 .Our conclusion is that the vector equation has non�trivial solutions and thatthe general solution is

x1 = �11tx2 = 26t

x3 = 45t

x4 = 40t

where t can be any real number. By choosing t = 1, we see that a lineardependence relation for the set of vectors fv1,v2,v3,v4g is

�11v1 + 26v2 + 45v3 + 40v4 = 03.

22

1.8: Introduction to Linear Transformations

De�nition 66 A transformation (or function or mapping), T , from Vnto Vm is a rule that assigns each vector x 2 Vn to a vector T (x) 2 Vm. Theset Vn is called the domain of T and the set Vm is called the codomain (ortarget set) of T . If T is a transformation from Vn to Vm, then we writeT : Vn �! Vm.

De�nition 67 Suppose that T : Vn �! Vm. For each x 2 Vn, the vectorT (x) 2 Vm is called the image of x under the action of T . The set of allimages of vectors in Vn under the action of T is called the range of T:In setnotation, the range of T is

fy 2 Vm j y = T (x) for at least one x 2 Vng .

Example 68 This example illustrates the terms de�ned in the preceding twode�nitions: Consider the transformation T : V2 �! V3 de�ned by

T

��xy

��=

24 x2

4y � x

35 .The domain of T is V2 and the codomain of T is V3.The image of the vector

x =

�4�1

�is

T (x) =

24 164�5

35 .The range of T consists of all vectors in V3 that have the form24 s4

t

35where s can be any non�negative real number and t can be any real number.For example, the vector

y =

24 64�9

3523

is in the range of T because

y = T

�� p6

�9 +p6

��.

Note that it is also true that

y = T

���p6

�9�p6

��.

Are there any other vectors x 2 V2 such that T (x) = y? The answer to thisdoes not matter as far as determining whether or not y is in the range ofT is concerned. We know that y is in the range of T because we know thatthere exists at least one vector x 2 V2 such that T (x) = y.

De�nition 69 A transformation T : Vn �! Vm is said to be a linear trans-formation if both of the following conditions are satis�ed:

1. T (u+ v) = T (u) + T (v) for all vectors u and v in Vn.

2. T (cu) = cT (u) for all vectors u in Vn and for all scalars c.

Example 70 Let T : V3 �! V2 be de�ned by

T

0@24 xyz

351A =

�x

x+ y + 2z

�.

We will show that T is a linear transformation: Let

u =

24 x1y1z1

35 and v =

24 x2y2z2

35

24

be two vectors in V3. Then

T (u+ v) = T

0@24 x1 + x2y1 + y2z1 + z2

351A=

�x1 + x2

(x1 + x2) + (y1 + y2) + 2 (z1 + z2)

�=

�x1 + x2

(x1 + y1 + 2z1) + (x2 + y2 + 2z2)

�=

�x1

x1 + y1 + 2z1

�+

�x2

x2 + y2 + 2z2

�= T (u) + T (v) :

Now let

u =

24 xyz

35be a vector in V3 and let c be a scalar. Then

T (cu) = T

0@24 cxcycz

351A=

�cx

cx+ cy + 2 (cz)

�= c

�x

x+ y + 2z

�cT (u) .

We have shown that T is a linear transformation.

De�nition 71 The zero transformation is the linear transformation T :Vn �! Vm de�ned by T (u) = 0m for all u 2 Vn.

De�nition 72 The identity transformation is the linear transformationT : Vn �! Vn de�ned by T (u) = u for all u 2 Vn.

We leave it to the reader to prove that the zero transformation and theidentity transformation are both linear transformations.

25

Example 73 The transformation T : V2 �! V3 de�ned by

T

��xy

��=

24 x2

4y � x

35is not a linear transformation. We leave it to the reader to explain why not.

1.9: The Matrix of a Linear Transformation

De�nition 74 A transformation T : Vn �! Vm is said to be onto Vm if therange of T is Vm.

De�nition 75 (Alternate De�nition) A transformation T : Vn �! Vmis said to be onto Vm if each vector y 2 Vm is the image of at least onevector x 2 Vn.

De�nition 76 A transformation T : Vn �! Vm is said to be one�to�oneif each vector y 2 Vm is the image of at most one vector x 2 Vn.

De�nition 77 A transformation T : Vn �! Vm that is both onto Vm andone�to�one is called a bijection.

Remark 78 In reference to the �dart� analogy that we have been using inclass: Supposing that Vn is our box of darts and Vm is our dartboard, atransformation T : Vn �! Vm is onto Vm if every point on the dartboardgets hit by at least one dart; whereas T is one�to�one if every point on thedartboard either does not get hit or gets hit by exactly one dart. If T is abijection, then every point on the dartboard gets hit by exactly one dart.

2.2: The Inverse of a Matrix

De�nition 79 Suppose that A =�a bc d

�is a 2 � 2 matrix. The quantity

ad� bc is called the determinant of A. It is denoted either by det (A) or byjAj.

De�nition 80 The n � n identity matrix, denoted by In, is the matrixthat has entries of 1 at all positions along its main diagonal and entries of 0elsewhere.

26

Example 81 The 3� 3 identity matrix is

I3 =

24 1 0 00 1 00 0 1

35 .De�nition 82 An n � n matrix, A, is said to be invertible if there existsan n� n matrix, B, such that BA = AB = In.

Proposition 83 If A is an invertible n� n matrix and B and C are n� nmatrices such that BA = AB = In and CA = AC = In, then B = C. Inother words, if A has an inverse, then this inverse is unique.

De�nition 84 Suppose that A is an invertible n � n matrix. The inverseof A is de�ned to be the unique matrix, A�1, such that A�1A = AA�1 = In.

Example 85 The inverse of the matrix

A =

�3 01 3

�is

A�1 =

�13

0�19

13

�.

(The reader should perform the computations to check that A�1A = AA�1 =I2.)

De�nition 86 An elementary matrix is a matrix is a matrix that canbe obtained by performing a single elementary row operation on an identitymatrix.

Example 87 The matrix

E =

24 1 0 00 1 0�9 0 1

35is an elementary matrix because it is obtained from I3 by the elementary rowoperation �9 �R1 +R3 ! R3.

27

2.3: Characterizations of Invertible Matrices

De�nition 88 A transformation T : Vn �! Vn is said to be invertible ifthere exists a transformation S : Vn �! Vn such that S (T (x)) = x for allx 2 Vn and T (S (x)) = x for all x 2 Vn.

Proposition 89 If T : Vn �! Vn is invertible, then there is a unique trans-formation S : Vn �! Vn such that S (T (x)) = x for all x 2 Vn andT (S (x)) = x for all x 2 Vn. This unique transformation is called theinverse of T and is denoted by T�1.

Example 90 If T : V2 �! V2 is the linear transformation that rotates allvectors in V2 counterclockwise about the origin through an angle of 90�, thenT is invertible and T�1 : V2 �! V2 is the linear transformation that rotatesall vectors in V2 clockwise about the origin through an angle of 90�.

Theorem 91 Suppose that A is an n � n matrix and let T : Vn �! Vn bethe linear transformation de�ned by T (x) = Ax for all x 2 Vn. Then thefollowing statements are all equivalent to each other (meaning that all of thestatements are true or all of them are false).

1. A is invertible.

2. T is invertible.

3. T is onto Vn.

4. T is one�to�one.

5. A � In.

6. The columns of A span Vn.

7. The homogeneous equation Ax = 0n has only the trivial solution.

8. For every b 2 Vn, the equation Ax = b has a unique solution.

28

4.1: Vector Spaces and Subspaces

De�nition 92 A vector space is a non-empty set of objects, V , called vec-tors, on which are de�ned two operations, called addition and multiplicationby scalars (real numbers), subject to the ten axioms listed below. The axiomsmust hold for all vectors u; v; and w in V and for all scalars c and d.

1. The sum of u and v, denoted by u+ v is in V .

2. The scalar multiple of u by c, denoted by cu, is in V .

3. u+ v = v + u.

4. (u+ v) +w = u+ (v +w).

5. There is a zero vector, 0, in V such that u+ 0 = u.

6. There is a vector �u in V such that u+ (�u) = 0.

7. c (u+ v) = cu+ cv.

8. (c+ d)u = cu+ du.

9. c (du) = (cd)u.

10. 1u = u.

De�nition 93 If V is a vector space and W is a non�empty subset of V ,then W is said to be a subspace of V if W is itself a vector space under thesame operations of addition and multiplication by scalars that hold in V .

De�nition 94 Suppose that v1,v2,. . . ,vn are vectors in a vector space, V ,and suppose that c1,c2,. . . ,cn are scalars. Then

c1v1 + c2v2 + � � �+ cnvnis said to be a linear combination of the vectors v1,v2,. . . ,vn.

De�nition 95 Suppose that fv1;v2; : : : ;vng is a set of vectors in a vectorspace, V . This set of vectors is said to be linearly independent if theequation

c1v1 + c2v2 + � � �+ cnvn = 0has only the trivial solution (c1 = c2 = � � � = cn = 0). Otherwise the set ofvectors is said to be linearly dependent.

29

De�nition 96 Suppose that S = fv1;v2; : : : ;vng is a set of vectors in avector space, V . The span of S, denoted by Span (S), is de�ned to be theset of all possible linear combinations of the vectors in S.

4.2: Linear Transformations

De�nition 97 Suppose that V and W are vector spaces. A transformationT : V �! W is said to be a linear transformation if

T (u+ v) = T (u) + T (v) for all u and v in V

andT (cu) = cT (u) for all u in V and all scalars c.

Remark 98 As in our previous discussions of transformations, the vectorspace V is called the domain of T and the vector space W is called thecodomain (or target set) of T .

Remark 99 In the above de�nition, since V and W might be two di¤erentvectors spaces and hence might have two di¤erent operations of addition andmultiplication by scalars, we might want to use di¤erent symbols to denotethese di¤erent operations. For example we might use + and � to denoteaddition and scalar multiplication in V and use � and � to denote additionand scalar multiplication in W . With these notations, the conditions for Tto be a linear transformation would appear as

T (u+ v) = T (u)� T (v) for all u and v in V

andT (c � u) = c� T (u) for all u in V and all scalars c.

In most cases it is not necessary to use di¤erent notations such as this sincethe operations that are intended are clear from the context of whatever isbeing discussed.

De�nition 100 The nullspace (also called the kernel) of a linear trans-formation T : V �! W is the set of all vectors in V that are transformedby T into the zero vector in W . This space (which is a subspace of V ) isdenoted by ker (T ). Thus

ker (T ) = fv 2 V j T (v) = 0Wg .

30

De�nition 101 The range of a linear transformation T : V �! W is theset of all vectors in W that are images under T of at least one vector in V .This space (which is a subspace of W ) is denoted by range(T ). Thus

range (T ) = fw 2 W j T (v) = w for at least one v 2 V g .

Another (shorter) way to write this is

range (T ) = fT (v) j v 2 V g .

De�nition 102 In the case of a linear transformation T : Vn �! Vm whosestandard matrix is the m � n matrix A, the nullspace of A, denoted byNul(A), is de�ned to be the nullspace (or kernel) of T . Thus Nul(A) =ker (T ).

Remark 103 Nul(A) is the solution set of the matrix equation Ax = 0m.

De�nition 104 In the case of a linear transformation T : Vn �! Vm whosestandard matrix is the m� n matrix A, the column space of A, denoted byCol(A), is de�ned to be the range of T . Thus Col(A) =range(T ).

Remark 105 If A = [a1 a2 � � � an], then Col(A) = Span (fa1; a2; : : : ; ang).

4.3: Bases

De�nition 106 A basis for a vector space, V , is a �nite indexed set ofvectors, B = fv1;v2; : : : ;vng, in V that is linearly independent and alsospans V (meaning that Span (B) = V ).

Remark 107 Some vector spaces have �standard� bases � meaning basesthat are rather obvious without having to do to much checking. For example,the standard basis for Vn is B = fe1; e2; : : : ; eng where

e1 =

2666410...0

37775 , e2 =

2666401...0

37775 , � � � , en =

2666400...1

37775 .

31

The standard basis for Pn (R) (the set of all polynomial functions with domainR and degree less than or equal to n) is B = fp0; p1; p2; : : : ; png where

p0 (x) = 1 for all x 2 Rp1 (x) = x for all x 2 Rp2 (x) = x

2 for all x 2 R...

pn (x) = xn for all x 2 R.

The vector space F (R) (the set of all functions with domain R) does nothave a basis because no �nite set of functions spans this space.

4.4: Coordinate Systems

De�nition 108 Suppose that V is a vector space, that B = fv1;v2; : : : ;vngis a basis for V , and that v is a vector in V . The coordinates of v relativeto the basis B are the unique scalars c1; c2; � � � ; cn such that c1v1 + c2v2 +� � � + cnvn = v. The coordinate vector of v relative to the basis B is thevector

[v]B =

26664c1c2...cn

37775 2 Vn.The transformation T : V �! Vn de�ned by T (v) = [v]B is a linear trans-formation that is called the coordinate transformation (or coordinatemapping) on V determined by the basis B.

De�nition 109 If V and W are vector spaces and T : V �! W is a lineartransformation that is one�to�one and maps V onto W , then T is said to bean isomorphism of V and W .

De�nition 110 If V and W are vector spaces and if there exists an isomor-phism T : V �! W , then V is said to be isomorphic to W .

Remark 111 Isomorphism is an equivalence relation on the set of all vectorspaces. This is because:

32

1. Every vector space is isomorphic to itself. (Why?)

2. If V is isomorphic to W , then W is isomorphic to V . (Why?)

3. If V is isomorphic to W and W is isomorphic to X, then V is isomor-phic to X. (Why?)

4.5: The Dimension of a Vector Space

De�nition 112 If V is a vector space that is spanned by a �nite set ofvectors, then V is said to be �nite�dimensional and the dimension of V ,denoted by dim (V ), is de�ned to be the number of vectors in a basis for V .The dimension of the zero vector space, V = f0g, is de�ned to be 0. (Notethat the zero vector space has no basis.) If V 6= f0g and V is not spanned byany �nite set of vectors, then we say that V is in�nite�dimensional.

4.6: The Rank of a Matrix

De�nition 113 If A is an m � n matrix, then the rank of A, denoted byrank (A), is de�ned to be the dimension of the column space of A. Thusrank (A) = dim (Col (A)).

Remark 114 Since we know that the column space and the row space of anymatrix have the same dimension, it is also true that rank (A) = dim (row (A)).

Direct Sums of Subspaces

De�nition 115 If V is a vector space and H and K are subspaces of V suchthat H \K = f0g, then the direct sum of H and K, which is denoted byH �K and which is also a subspace of V , is de�ned to be

H �K = fu+ v j u 2 H and v 2 Kg .

Remark 116 In order for the direct sum, H � K, to be de�ned, it is nec-essary that H \ K = f0g. (This is part of the de�nition of direct sum.)Whether or not H \K = f0g, the set

H +K = fu+ v j u 2 H and v 2 Kg

33

is called the sum of H and K. H + K is a subspace of V whether or notH\K = f0g. The di¤erence is that when H\K = f0g (and hence H+K =H � K), each vector, w, in H � K can be written uniquely in the formw = u + v where u 2 H and v 2 K; whereas this unique representationmight not be possible for all vectors in H +K if H \K 6= f0g.

De�nition 117 Two subspaces, H and K, of a vector space V are said tobe complementary to each other if H �K = V .

De�nition 118 Two vectors, u and v, in Vn are said to be orthogonal toeach other if uTv = vTu = [0].

Remark 119 If u and v are any two vectors in Vn, then uTv = vTu. Thuswe could simply state the above de�nition by saying that u and v are orthog-onal to each other if uTv = [0].

Remark 120 If [a] is a 1 � 1 matrix, then we often adopt the conventionto just view [a] as a scalar and hence just write a instead of [a]. With thisconvention, we can say that u and v are orthogonal to each other if uTv = 0(without writing the brackets around 0). Also, with this convention, uTv (forany two vectors, u and v, in Vn) is called the dot product of u and v and isdenoted by u � v. For example, if

u =

24 4�1�4

35 and v =24 �1�1�4

35 ,then

u�v = uTv =�4 �1 �4

� 24 �1�1�4

35 = (4) (�1)+(�1) (�1)+(�4) (�4) = 13.De�nition 121 Two subspaces, H and K, of Vn are said to be orthogonalto each other if uTv = 0 for all u 2 H and v 2 K.

De�nition 122 Two subspaces, H and K, of Vn are said to be orthogonalcomplements of each other if H and K are orthogonal to each other andH �K = Vn.

34

Fundamental Subspaces of a Matrix

Let

A = [a1 a2 � � � an] =

26664b1b2...bm

37775be an m� n matrix (where a1, a2, . . . , an are the column vectors of A (eachof which has size m� 1) and b1, b2, . . . , bm are the row vectors of A (eachof which has size 1 � n). The following four subspaces (two of which aresubspaces of Vn and the other two of which are subspaces of Vm are calledthe fundamental subspaces of the matrix A:

1. The row space of A,

row (A) = Span�bT1 ;b

T2 ; : : : ;b

Tm

�is a subspace of Vn.

2. The null space of A,

Nul (A) = fx 2 Vn j Ax = 0mg

is a subspace of Vn.

3. The column space of A,

Col (A) = Span (a1; a2; : : : ; an)

is a subspace of Vm.

4. The null space of AT ,

Nul�AT�=�x 2 Vm j ATx = 0n

is a subspace of Vm.

35