These notes closely follow the presentation of the material given in David C. Lay’stextbook Linear Algebra and its Applications (3rd edition). These notes are intendedprimarily for in-class presentation and should not be regarded as a substitute forthoroughly reading the textbook itself and working through the exercises therein.

The Dot Product of Two Vectors in n

Suppose that

u u1u2

and v v1v2

are two vectors in 2.We define the dot product (also called the inner product or the scalar product)

of u and v to be u v uTv. Thus,

u v uTv u1 u2v1v2

u1v1 u2v2 .

Note that u v is actually a 1 1 matrix. However, we will adopt the conventionthat u v is a scalar (equal to the single entry in this 1 1 matrix). For example, if

u 13

and v 73

,

thenu v 17 33 2.

Clearly, this definition can be extended to n for any positive integer n: If

u

u1u2

un

and v

v1v2

vnare any two vectors in n, then we define

u v u1v1 u2v2 unvn.When convenient, we will simply write u v uTv (even though, technically, u v

is a scalar and uTv is a 1 1 matrix).

1

Example Let

u 168

, v 341

, and w

550

.

Compute u v, v u, 3u v, u 3v, 3u v, u u, u v w, andu v u w.

2

Basic Properties of the Dot Product1. If u and v are any two vectors in n, then u v v u.2. If u and v are any two vectors in n and c is any scalar, then

cu v u cv cu v.3. If u, v, and w are any three vectors in n, then u v w u v u w.4. If u is any vector in n, then u u 0. Also, u u 0 if and only if u 0n.

3

Length and DistanceFor any vector u n, we define the length (also called the norm or the

magnitude) of u to be u u u . Thus, if

u

u1u2

un

,

thenu u u u12 u22 un2 .

Example Find the length of the vector

u 15

.

4

If u and v are any two vectors in n, then we define the distance between u and vto be u v.

Example Find the distance between the vectors

u 18

and v 39

.

5

Basic Properties of Length and Distance1. If u is any vector in n, then u 0. Also, u 0 if and only if u 0n.2. If u is any vector in n and c is any scalar, then cu |c|u.3. If u and v are any two vectors in n, then u v v u.4. If u and v are any two vectors in n, then |u v| uv. (This is called the

Cauchy–Schwarz Inequality.)5. If u and v are any two vectors in n, then u v u v. (This is

called the Triangle Inequality.)

6

A unit vector is a vector of length 1. If v is any vector in n with v 0n, then it iseasy to find a unit vector that points in the same direction as v: In fact, the vector

u 1v v

has length 1 and is a positive scalar multiple of v.

Example Find a unit vector that points in the same direction as the vector

u 168

.

7

Orthogonality of VectorsTwo vectors, u and v, in n are said to be orthogonal (or perpendicular) to each

other if u v 0. This definition of orthogonality is inspired by what we can visualize in2 or 3: Two non–zero vectors, u and v, in 2 (or 3) are perpendicular to eachother if and only if u v u v, and this is true if and only if u v 0.

Example Show that the vectors

u 94

and v 2 92

are orthogonal to each other, and show that the vectors

u 26

and v 14

are not orthogonal to each other.

8

Example Describe the set of all vectors in 2 that are orthogonal to the vector

u 94

.

9

Example Describe the set of all vectors in 3 that are orthogonal to the vector

u 447

.

10

Example Let

W Span116

,305

.

Note that W is a two–dimensional subspace of 3 (a plane in 3 that passesthrough the origin in 3).

Describe the set of all vectors in 3 that are orthogonal to every vector in W.

11

Definition If W is a subspace of n, we define the orthogonal complement of W, denotedby W, to be the set of all vectors in n that are orthogonal to all of the vectors inW.

Theorem Suppose that W is a subspace of n. Then:1. W is a subspace of n.2. W W 0n.3. dimW dimW n and if BW is a basis for W and BW is a basis for W,

then BW BW a basis for n.

Remark If W and U are any two subspaces of n such that W U 0n and such thatthe union of a basis for W and a basis for U is a basis for n, then we say that n

is the direct sum of W and U and we write n W U. Thus, the above theoremtells us that if W is any subspace of n, then n W W.

We remark thatn W U U W

but thatU W n W U.

Example If

W Span94

,

then W is a subspace of 2. In fact, W is a line passing through the origin in 2.The subspace W consists of all vectors

x x1x2

2

such that9x1 4x2 0,

or, to write this in a different way,

9 4x1x2

0.

Since

9 4 ~ 1 49 ,

12

we see that

x x1x2

49 t

t t

49

1 t

49

.

Therefore

W Span49

.

Note that

W W Span94

,49

2.

Example If

W Span447

,

then W is a subspace of 3. In fact, W is a line passing through the origin in 3.The subspace W consists of all vectors

x x1x2x3

2

13

such that 4x1 4x2 7x3 0,

or, to write this in a different way,

4 4 7x1x2x3

0.

Since

4 4 7 ~ 1 1 74 ,

we see that

x x1x2x3

t 74 s

ts

t110

s

74

01

t110

s704

.

Therefore

W Span110

,704

.

Note that

W W Span447

,110

,704

3.

Example If

W Span116

,305

,

then W is a subspace of 3. In fact, W is a plane passing through the origin in 3.The subspace W consists of all vectors

x x1x2x3

2

such that

14

x1 x2 6x3 0 3x1 5x3 0

or, to write this in a different way,

1 1 63 0 5

x1x2x3

00

.

Since

1 1 63 0 5

~1 0 530 1 233

,

we see that

x x1x2x3

53 t233 t

t

t

53233

1

t5233

.

Therefore

W Span5233

.

Note that

W W Span116

,305

,5233

3.

Theorem If A is any m n matrix, then rowA nulA andn rowA nulA. Also, colA nulAT and m colA nulAT.

Proof Suppose that r rowA and n nulA. Then r ATx for some vector x m

and An 0m. Now note thatr n rTn ATxTn xTAn xTAn xT0m 0.

This shows that every vector in rowA is orthogonal to every vector in nulA. andhence that rowA nulA.Now, suppose that x rowA. Then r x 0 for all vectors r rowA.

Thus,

15

Ax

r1r2

rm

x

r1 xr2 x

rm x

0m,

which shows that x nulA, and hence that rowA nulA.We conclude that rowA nulA and hence that n rowA nulA.To verify the second assertion of the theorem, note that

colA rowAT nulAT.

Example In order to illustrate the above theorem, suppose that A is the matrix

A

2 6 90 5 14 7 19

.

We will show that 3 rowA nulA and that 3 colA nulAT.

First, note that

A

2 6 90 5 14 7 19

~1 0 51100 1 150 0 0

~10 0 510 5 10 0 0

which shows that

rowA Span10051

,051

.

We also see that nulA consists of all vectors x 3 such that

x x1x2x3

5110 t15 t

t

t

511015

1

t51210

.

Therefore,

nulA Span51210

.

Now, observe that 3 rowA nulA.

16

Next, note that

AT 2 0 46 5 79 1 19

~1 0 20 1 10 0 0

which shows that

colA rowAT Span102

,011

and

nulAT Span211

.

We observe that 3 colA nulAT.

As a double–check, observe that the row reduction that was performed earlier:

A

2 6 90 5 14 7 19

~1 0 51100 1 150 0 0

shows that

colA Span204

,657

from which it can still be observed that 3 colA nulAT.

17

Extending the Idea of Orthogonality to Vector SpacesOther Than n

Definition Let V be a vector space. An inner product for V is a binary function, : V such that

1. If u and v are any two vectors in V, then u,vv,u.2. If u and v are any two vectors in V and c is any scalar, then

cu,vu, cvcu,v.3. If u, v, and w are any three vectors in V, then u,v w u,v u,w.4. If u is any vector in V, then u,u 0, and u,u 0 if and only if u 0V.

Example The dot product is an inner product for n.

Example For the vector space C0, (consisting of all continuous functionsf : , ), we can define an inner product as follows:

f,g

fxgxdx.

To verify that this is indeed an inner product for C0,, note that:1. If f and g are any two functions in C0,, then

f,g

fxgxdx

gxfxdx g, f.

2. If f and g are any two functions in C0, and c is any scalar, then

cf,g

cfxgxdx c

fxgxdx

f,cg

fxcgxdx c

fxgxdx

cf,g c

fxgxdx.

Thus cf,g f,cg cf,g.3. If f , g, and h are any three functions in C0,, then

f,g h

fxgx hxdx

fxgx fxhxdx

fxgxdx

fxhxdx

f,g f,h.4. If f is any function in C0,, then

f, f

fxfxdx

fx2 dx 0

18

(because fx2 0 for all x ,.Also,

fx2 dx 0 if and only if fx 0 for all x ,

which means that f, f 0 if and only if f is the zero vector (that is, the zerofunction) in C0,.

Definition If V is a vector space with an inner product , , then we define the norm ofany vector v V to be

v v,v .If u 1, then we say that u is a unit vector.

As before, if we are given a vector v V, then a unit vector in the same “direction”as v is

u 1v v.

Example Let V be the function space C0, with inner product as defined above.Consider the following functions which are all members of C0,:

f0t 1f1t costg1t sintf2t cos2tg2t sin2tf3t cos3t

Find the norms of these functions and find a “unit function” corresponding to eachof them. (In other words, for each of the above functions, fi, find a function i, suchthat i 1 and i is a scalar multiple of fi, and for each of the functions, gi,find a function i, such that i 1 and i is a scalar multiple of gi.)

Solutionf02 f0, f0

f0x2 dx

1dx

2

19

shows thatf0 2 .

Therefore, a unit function in the same direction as f0 is0 1

2f0.

This is the constant function defined by0t 1

2for all t ,.

Next, we find 1:f12 f1, f1

f1x2 dx

cos2xdx

shows that

f1 .Therefore, a unit function in the same direction as f1 is

1 1f1.

This is the function defined by1t 1

cost for all t ,.

Next, we find 1:g12 g1,g1

g1x2 dx

sin2xdx

shows that

g1 .Therefore, a unit function in the same direction as g1 is

1 1g1.

This is the function defined by1t 1

sint for all t ,.

Next, we find 2:

20

f22 f2, f2

f2x2 dx

cos22xdx

shows that

f2 .Therefore, a unit function in the same direction as f2 is

2 1f2.

This is the function defined by2t 1

cos2t for all t ,.

Does there seem to be a pattern here? There is. In fact, it can be shown that0t 1

2

nt 1cosnt, n 1

nt 1sinnt, n 1.

Definition If V is a vector space with an inner product , and B is a basis for V, then Bis called an orthonormal basis if every vector in B is a unit vector and any twovectors in B are orthogonal to each other.

Example The standard basis E e1,e2, ,en is an orthonormal basis for n.

Example The basis

B 3545

, 4535

is an orthonormal basis for 2.

21

Theorem If V is a vector space with an inner product , and with orthonormal basisB b1,b2, ,bn, and if v is any vector in V, then

vB

v,b1 v,b2

v,bn

.

In other words,v v,b1 b1 v,b2 b2 v,bn bn.

Proof Since B is a basis for V, we know that v can be written asv c1b1 c2b2 cnbn.

Note thatv,b1 b1,v

b1,c1b1 c2b2 cnbn b1,c1b1 b1,c2b2 b1,cnbn c1b1,b1 c2b1,b2 cnb1,bn c1b12

c1Likewise, it can be shown that c2 v,b2 ,,cn v,bn .

Example The basis

B b1,b2 3545

, 4535

is an orthonormal basis for 2. Find the coordinates of the vector

v 105

relative to this basis.

Solutionv,b1 10 3

5 5 45 2

and

v,b2 10 45 5 35 11.

Thus,v 2b1 11b2.

22

The most interesting and powerful applications of these ideas come into play whenwe are dealing with function spaces, which are infinite–dimensional vector spaces. Wecannot study this in great detail in this course. However, we can get a feel for thebasic ideas.

Here is our main example: As we saw in an earlier example, every one of thefunctions

0t 12

nt 1cosnt, n 1

nt 1sinnt, n 1.

has norm 1. It is also true that any two functions in this set are orthogonal to eachother. For example,

1,3

1t3tdt

1cost 1

sin3t dt

0 (because the integrand is an odd function),and

1,3

1t3tdt

1cost 1

cos3t dt

1

costcos3tdt

0 (is seen by using integration by parts).Thus, the set of functions 0,1,1,2,2,3, is an orthonormal set of

functions in the function space C0,.Motivated by the previous theorem (which pertains to finite–dimensional vector

spaces), we are led to ask the following question: Given any function f C0,, isit true that

f f,0 0 f,1 1 f,1 1 f,2 2 f,2 2?In other words, is it true that

ft f,0 2

f,1

cost f,1

sint f,2

cos2t f,2

sin2t

for all t ,?Under certain (rather small) restrictions on the function f, it can be proved that the

answer to this question is “Yes”. We cannot delve into all of the details, which wouldrequire a course in advanced calculus at the minimum. Let us just point out that the

23

infinite trigonometric series on the right is called the Fourier series of the function fand that it is often true that f is in fact equal to its Fourier series. The coefficients

f,0 2

, f,1 , f,1

, f,2

, f,2

,

that appear in the Fourier Series are called the Fourier coefficients of f.

Example The function fx x2 is a function in C0,. Find the first ten Fouriercoefficients of f and compare the graphs of f with the graphs of the “truncated”Fourier series of f consisting of the first ten terms of the series.

Solution

f,0 2

12

x2 1

2dx 2

3

f,1

1

x2 1

cosx dx 4

f,1

1

x2 1

sinx dx 0

f,2

1

x2 1

cos2x dx 1

f,2

0

f,3

1

x2 1

cos3x dx 49

f,3

0

f,4

1

x2 1

cos4x dx 1

4f,4

0

f,5

1

x2 1

cos5x dx 425 .

A truncated Fourier series for fx x2 is thus23 4cosx cos2x 49 cos3x

14 cos4x

425 cos5x.

The graph of this function and the graph of fx x2 are shown below.

24

0

24681012141618202224

-4 -2 2 4x

25