1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Chapter 8 Orthogonality8.2 Projections and the Gram-Schmidt Process

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Projections

DefinitionThe projection of a vector u onto another vector v, denotedprojvu, is the vector that is parallel to v such thatu− projvu makes a right angle with v.

FormulaUsing dot product notation, this vector can be written as

projvu =u · v‖v‖2

v

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Projections

DefinitionThe projection of a vector u onto another vector v, denotedprojvu, is the vector that is parallel to v such thatu− projvu makes a right angle with v.

FormulaUsing dot product notation, this vector can be written as

projvu =u · v‖v‖2

v

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Visualization

v

u

{u, v} is a basis for R2

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Visualization

v

u

projvu

u− projvu

Think of projvu as the component of u in the direction of v.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Visualization

v

u

projvu

u− projvu

Think of projvu as the component of u in the direction of v.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Visualization

v

u

projvu

u− projvu

Think of projvu as the component of u in the direction of v.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(a) projvu ∈ span {v}

Why?

projvu =u · v‖v‖2

v

is a multiple of v, so ...

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(a) projvu ∈ span {v}

Why?

projvu =u · v‖v‖2

v

is a multiple of v, so ...

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(a) projvu ∈ span {v}

Why?

projvu =u · v‖v‖2

v

is a multiple of v, so ...

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(b) u− projvu is orthogonal to v

If we claim they are orthogonal, what do we know?

v · (u− projvu) = v ·(u− u · v‖v‖2

v

)= v · u− u · v

‖v‖2v · v

= v · u− u · v‖v‖2

‖v‖2

= 0

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(b) u− projvu is orthogonal to v

If we claim they are orthogonal, what do we know?

v · (u− projvu) = v ·(u− u · v‖v‖2

v

)= v · u− u · v

‖v‖2v · v

= v · u− u · v‖v‖2

‖v‖2

= 0

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(b) u− projvu is orthogonal to v

If we claim they are orthogonal, what do we know?

v · (u− projvu)

= v ·(u− u · v‖v‖2

v

)= v · u− u · v

‖v‖2v · v

= v · u− u · v‖v‖2

‖v‖2

= 0

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(b) u− projvu is orthogonal to v

If we claim they are orthogonal, what do we know?

v · (u− projvu) = v ·(u− u · v‖v‖2

v

)

= v · u− u · v‖v‖2

v · v

= v · u− u · v‖v‖2

‖v‖2

= 0

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(b) u− projvu is orthogonal to v

If we claim they are orthogonal, what do we know?

v · (u− projvu) = v ·(u− u · v‖v‖2

v

)= v · u− u · v

‖v‖2v · v

= v · u− u · v‖v‖2

‖v‖2

= 0

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(c) If u ∈ span{v} then u = projvu

What does it mean for u ∈ span{v}? There exists c ∈ Rsuch that u = cv.

projvu =v · cv‖v‖2

v

= cv · v‖v‖2

v

= c‖v‖2

‖v‖2v

= cv

= u

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(c) If u ∈ span{v} then u = projvu

What does it mean for u ∈ span{v}?

There exists c ∈ Rsuch that u = cv.

projvu =v · cv‖v‖2

v

= cv · v‖v‖2

v

= c‖v‖2

‖v‖2v

= cv

= u

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(c) If u ∈ span{v} then u = projvu

What does it mean for u ∈ span{v}? There exists c ∈ Rsuch that u = cv.

projvu =v · cv‖v‖2

v

= cv · v‖v‖2

v

= c‖v‖2

‖v‖2v

= cv

= u

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(c) If u ∈ span{v} then u = projvu

What does it mean for u ∈ span{v}? There exists c ∈ Rsuch that u = cv.

projvu =v · cv‖v‖2

v

= cv · v‖v‖2

v

= c‖v‖2

‖v‖2v

= cv

= u

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(c) If u ∈ span{v} then u = projvu

What does it mean for u ∈ span{v}? There exists c ∈ Rsuch that u = cv.

projvu =v · cv‖v‖2

v

= cv · v‖v‖2

v

= c‖v‖2

‖v‖2v

= cv

= u

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(c) If u ∈ span{v} then u = projvu

What does it mean for u ∈ span{v}? There exists c ∈ Rsuch that u = cv.

projvu =v · cv‖v‖2

v

= cv · v‖v‖2

v

= c‖v‖2

‖v‖2v

= cv

= u

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(c) If u ∈ span{v} then u = projvu

What does it mean for u ∈ span{v}? There exists c ∈ Rsuch that u = cv.

projvu =v · cv‖v‖2

v

= cv · v‖v‖2

v

= c‖v‖2

‖v‖2v

= cv

= u

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(c) If u ∈ span{v} then u = projvu

What does it mean for u ∈ span{v}? There exists c ∈ Rsuch that u = cv.

projvu =v · cv‖v‖2

v

= cv · v‖v‖2

v

= c‖v‖2

‖v‖2v

= cv

= u

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(d) projvu = projcvu

What does this statement say?

projcvu =cv · u‖cv‖2

(cv)

=c2v · uc2‖v‖2

v

=v · u‖v‖2

v

= projvu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(d) projvu = projcvu

What does this statement say?

projcvu =cv · u‖cv‖2

(cv)

=c2v · uc2‖v‖2

v

=v · u‖v‖2

v

= projvu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(d) projvu = projcvu

What does this statement say?

projcvu =cv · u‖cv‖2

(cv)

=c2v · uc2‖v‖2

v

=v · u‖v‖2

v

= projvu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(d) projvu = projcvu

What does this statement say?

projcvu =cv · u‖cv‖2

(cv)

=c2v · uc2‖v‖2

v

=v · u‖v‖2

v

= projvu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(d) projvu = projcvu

What does this statement say?

projcvu =cv · u‖cv‖2

(cv)

=c2v · uc2‖v‖2

v

=v · u‖v‖2

v

= projvu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

TheoremLet u, v ∈ Rn with v non-zero and let c ∈ R.(d) projvu = projcvu

What does this statement say?

projcvu =cv · u‖cv‖2

(cv)

=c2v · uc2‖v‖2

v

=v · u‖v‖2

v

= projvu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 1

Example

Consider u1 =

[2−1

]and u2 =

[42

]. Find proju1u2.

proju1u2 =

(u1 · u2‖u1‖2

)u1

=

(8− 2

22 + (−1)2

)[2−1

]=

6

5

[2−1

]

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 1

Example

Consider u1 =

[2−1

]and u2 =

[42

]. Find proju1u2.

proju1u2 =

(u1 · u2‖u1‖2

)u1

=

(8− 2

22 + (−1)2

)[2−1

]=

6

5

[2−1

]

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 1

Example

Consider u1 =

[2−1

]and u2 =

[42

]. Find proju1u2.

proju1u2 =

(u1 · u2‖u1‖2

)u1

=

(8− 2

22 + (−1)2

)[2−1

]

=6

5

[2−1

]

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 1

Example

Consider u1 =

[2−1

]and u2 =

[42

]. Find proju1u2.

proju1u2 =

(u1 · u2‖u1‖2

)u1

=

(8− 2

22 + (−1)2

)[2−1

]=

6

5

[2−1

]

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 2

Example

Consider v1 =

02−1

and v2 = 2−1

2

. Find projv1v2

projv1v2 =

(v1 · v2‖v1‖2

)v1

=

(0− 2− 2

02 + 22 + (−1)2

) 02−1

= −4

5

02−1

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 2

Example

Consider v1 =

02−1

and v2 = 2−1

2

. Find projv1v2projv1v2 =

(v1 · v2‖v1‖2

)v1

=

(0− 2− 2

02 + 22 + (−1)2

) 02−1

= −4

5

02−1

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 2

Example

Consider v1 =

02−1

and v2 = 2−1

2

. Find projv1v2projv1v2 =

(v1 · v2‖v1‖2

)v1

=

(0− 2− 2

02 + 22 + (−1)2

) 02−1

= −45

02−1

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 2

Example

Consider v1 =

02−1

and v2 = 2−1

2

. Find projv1v2projv1v2 =

(v1 · v2‖v1‖2

)v1

=

(0− 2− 2

02 + 22 + (−1)2

) 02−1

= −4

5

02−1

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Projection onto a Subspace

What could we be talking about when we have anorthogonal basis?

DefinitionIf {v1, v2, . . . , vk} is an orthogonal basis for a subspace S ofRn, then the projection of u onto S is defined by

projSu = projv1u + projv2u + . . . + projvku

=u · v1‖v1‖2

v1 +u · v2‖v2‖2

v2 + . . . +u · vk‖vk‖2

vk

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Projection onto a Subspace

What could we be talking about when we have anorthogonal basis?

DefinitionIf {v1, v2, . . . , vk} is an orthogonal basis for a subspace S ofRn, then the projection of u onto S is defined by

projSu = projv1u + projv2u + . . . + projvku

=u · v1‖v1‖2

v1 +u · v2‖v2‖2

v2 + . . . +u · vk‖vk‖2

vk

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Visual Representation

S

u

projSu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Visual Representation

S

u

projSu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Visual Representation

S

u

projSu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Visual Representation

S

u

projSu

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 3

Example

Consider v1 =

02−1

, v2 = 2−1

2

and v3 = 22−1

. FindprojSv1 where S = span{v2, v3}.

Notice that v2 and v3 form an orthogonal basis for S.

projSv1 = projv2v1 + projv3v1

=v1 · v2‖v2‖2

v2 +v1 · v3‖v3‖2

v3

= −49

2−12

+ 59

22−1

=

1

9

214−13

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 3

Example

Consider v1 =

02−1

, v2 = 2−1

2

and v3 = 22−1

. FindprojSv1 where S = span{v2, v3}.Notice that v2 and v3 form an orthogonal basis for S.

projSv1 = projv2v1 + projv3v1

=v1 · v2‖v2‖2

v2 +v1 · v3‖v3‖2

v3

= −49

2−12

+ 59

22−1

=

1

9

214−13

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 3

Example

Consider v1 =

02−1

, v2 = 2−1

2

and v3 = 22−1

. FindprojSv1 where S = span{v2, v3}.Notice that v2 and v3 form an orthogonal basis for S.

projSv1 = projv2v1 + projv3v1

=v1 · v2‖v2‖2

v2 +v1 · v3‖v3‖2

v3

= −49

2−12

+ 59

22−1

=

1

9

214−13

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 3

Example

Consider v1 =

02−1

, v2 = 2−1

2

and v3 = 22−1

. FindprojSv1 where S = span{v2, v3}.Notice that v2 and v3 form an orthogonal basis for S.

projSv1 = projv2v1 + projv3v1

=v1 · v2‖v2‖2

v2 +v1 · v3‖v3‖2

v3

= −49

2−12

+ 59

22−1

=

1

9

214−13

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 3

Example

Consider v1 =

02−1

, v2 = 2−1

2

and v3 = 22−1

. FindprojSv1 where S = span{v2, v3}.Notice that v2 and v3 form an orthogonal basis for S.

projSv1 = projv2v1 + projv3v1

=v1 · v2‖v2‖2

v2 +v1 · v3‖v3‖2

v3

= −49

2−12

+ 59

22−1

=1

9

214−13

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Example 3

Example

Consider v1 =

02−1

, v2 = 2−1

2

and v3 = 22−1

. FindprojSv1 where S = span{v2, v3}.Notice that v2 and v3 form an orthogonal basis for S.

projSv1 = projv2v1 + projv3v1

=v1 · v2‖v2‖2

v2 +v1 · v3‖v3‖2

v3

= −49

2−12

+ 59

22−1

=

1

9

214−13

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

The following theorem is analagous to the one we previouslyproved, so we omit the proof. It is in the text.

TheoremIf S is a non-zero subspace of Rn, u is a vector in Rn and cis a real scalar, then

(a) projSu is in S

(b) u− projSu is orthogonal to S, i.e. in S⊥

(c) If u is in S, then u = projSu(d) projSu does not depend on the choice of orthogonal

basis.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

The following theorem is analagous to the one we previouslyproved, so we omit the proof. It is in the text.

TheoremIf S is a non-zero subspace of Rn, u is a vector in Rn and cis a real scalar, then

(a) projSu is in S(b) u− projSu is orthogonal to S, i.e. in S⊥

(c) If u is in S, then u = projSu(d) projSu does not depend on the choice of orthogonal

basis.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

The following theorem is analagous to the one we previouslyproved, so we omit the proof. It is in the text.

TheoremIf S is a non-zero subspace of Rn, u is a vector in Rn and cis a real scalar, then

(a) projSu is in S(b) u− projSu is orthogonal to S, i.e. in S⊥

(c) If u is in S, then u = projSu

(d) projSu does not depend on the choice of orthogonalbasis.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Properties

The following theorem is analagous to the one we previouslyproved, so we omit the proof. It is in the text.

TheoremIf S is a non-zero subspace of Rn, u is a vector in Rn and cis a real scalar, then

(a) projSu is in S(b) u− projSu is orthogonal to S, i.e. in S⊥

(c) If u is in S, then u = projSu(d) projSu does not depend on the choice of orthogonal

basis.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

The Gram-Schmidt Process

We will see the use of these orthonormal basis of a vectorspace later in this chapter. Right now, we will be concernedwith finding this orthonormal basis.

What we need to be able to do is use the idea of aprojection, but find the projection to increasingly largerorthogonal subspaces until we have the entire basis.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

The Gram-Schmidt Process

We will see the use of these orthonormal basis of a vectorspace later in this chapter. Right now, we will be concernedwith finding this orthonormal basis.

What we need to be able to do is use the idea of aprojection, but find the projection to increasingly largerorthogonal subspaces until we have the entire basis.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

The Gram-Schmidt Process

The ProcessGiven any basis of a {s1, s2, . . . , sk} subspace S of Rn, wecan produce the orthogonal basis {v1, v2, . . . , vk} as follows:

v1 = s1

v2 = s2 − projv1s2v3 = s3 − projv1s3 − projv2s3

...

vk = sk − projv1sk − projv2sk − projv3sk − projvk−1sk

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

The Gram-Schmidt Process

The ProcessGiven any basis of a {s1, s2, . . . , sk} subspace S of Rn, wecan produce the orthogonal basis {v1, v2, . . . , vk} as follows:

v1 = s1

v2 = s2 − projv1s2

v3 = s3 − projv1s3 − projv2s3...

vk = sk − projv1sk − projv2sk − projv3sk − projvk−1sk

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

The Gram-Schmidt Process

The ProcessGiven any basis of a {s1, s2, . . . , sk} subspace S of Rn, wecan produce the orthogonal basis {v1, v2, . . . , vk} as follows:

v1 = s1

v2 = s2 − projv1s2v3 = s3 − projv1s3 − projv2s3

...

vk = sk − projv1sk − projv2sk − projv3sk − projvk−1sk

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

The Gram-Schmidt Process

The ProcessGiven any basis of a {s1, s2, . . . , sk} subspace S of Rn, wecan produce the orthogonal basis {v1, v2, . . . , vk} as follows:

v1 = s1

v2 = s2 − projv1s2v3 = s3 − projv1s3 − projv2s3

...

vk = sk − projv1sk − projv2sk − projv3sk − projvk−1sk

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

An Example

Example

Apply the Gram-Schmidt process to find an orthogonal basisfor

S = span{[

12

],

[−11

]}

Since we are looking for the vectors to be orthogonal, we

cans start with the first one, so v1 =

[12

].

How many basis vectors do we need to have?

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

An Example

Example

Apply the Gram-Schmidt process to find an orthogonal basisfor

S = span{[

12

],

[−11

]}

Since we are looking for the vectors to be orthogonal, we

cans start with the first one, so v1 =

[12

].

How many basis vectors do we need to have?

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

An Example

Example

Apply the Gram-Schmidt process to find an orthogonal basisfor

S = span{[

12

],

[−11

]}

Since we are looking for the vectors to be orthogonal, we

cans start with the first one, so v1 =

[12

].

How many basis vectors do we need to have?

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

An Example

We need to find the projection to find the other basis vectorwe need.

v2 =

[−11

]− projv1

([−11

])=

[−11

]− 1

5

[12

]=

1

5

[−63

]So, the orthogonal basis is therefore{[

12

],

1

5

[−63

]}

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

An Example

We need to find the projection to find the other basis vectorwe need.

v2 =

[−11

]− projv1

([−11

])

=

[−11

]− 1

5

[12

]=

1

5

[−63

]So, the orthogonal basis is therefore{[

12

],

1

5

[−63

]}

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

An Example

We need to find the projection to find the other basis vectorwe need.

v2 =

[−11

]− projv1

([−11

])=

[−11

]− 1

5

[12

]

=1

5

[−63

]So, the orthogonal basis is therefore{[

12

],

1

5

[−63

]}

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

An Example

We need to find the projection to find the other basis vectorwe need.

v2 =

[−11

]− projv1

([−11

])=

[−11

]− 1

5

[12

]=

1

5

[−63

]

So, the orthogonal basis is therefore{[12

],

1

5

[−63

]}

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

An Example

We need to find the projection to find the other basis vectorwe need.

v2 =

[−11

]− projv1

([−11

])=

[−11

]− 1

5

[12

]=

1

5

[−63

]So, the orthogonal basis is therefore{[

12

],

1

5

[−63

]}

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

Example

Apply the Gram-Schmidt process to find an orthogonal basisfor

S = span

1201

,

2011

,

1221

We begin as before, with v1 =

1201

. Now what?

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

Example

Apply the Gram-Schmidt process to find an orthogonal basisfor

S = span

1201

,

2011

,

1221

We begin as before, with v1 =

1201

. Now what?

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

Example

Apply the Gram-Schmidt process to find an orthogonal basisfor

S = span

1201

,

2011

,

1221

We begin as before, with v1 =

1201

. Now what?

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v2 =

2011

− projv1

2011

=

2011

− 36

1201

=

32−1112

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v2 =

2011

−

projv1

2011

=

2011

− 36

1201

=

32−1112

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v2 =

2011

− projv1

2011

=

2011

− 36

1201

=

32−1112

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v2 =

2011

− projv1

2011

=

2011

− 36

1201

=

32−1112

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v2 =

2011

− projv1

2011

=

2011

− 36

1201

=

32−1112

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v3 =

1221

− projv1

1221

− projv2

1221

=

1221

− 66

1201

− 292

32−1112

=

1221

−

1201

−

23−494929

=−2349149−29

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v3 =

1221

−

projv1

1221

− projv2

1221

=

1221

− 66

1201

− 292

32−1112

=

1221

−

1201

−

23−494929

=−2349149−29

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v3 =

1221

− projv1

1221

− projv2

1221

=

1221

− 66

1201

− 292

32−1112

=

1221

−

1201

−

23−494929

=−2349149−29

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v3 =

1221

− projv1

1221

− projv2

1221

=

1221

− 66

1201

− 292

32−1112

=

1221

−

1201

−

23−494929

=−2349149−29

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v3 =

1221

− projv1

1221

− projv2

1221

=

1221

− 66

1201

− 292

32−1112

=

1221

−

1201

−

23−494929

=−2349149−29

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v3 =

1221

− projv1

1221

− projv2

1221

=

1221

− 66

1201

−

292

32−1112

=

1221

−

1201

−

23−494929

=−2349149−29

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v3 =

1221

− projv1

1221

− projv2

1221

=

1221

− 66

1201

− 292

32−1112

=

1221

−

1201

−

23−494929

=−2349149−29

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v3 =

1221

− projv1

1221

− projv2

1221

=

1221

− 66

1201

− 292

32−1112

=

1221

−

1201

−

23−494929

=

−2349149−29

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Another Example

v3 =

1221

− projv1

1221

− projv2

1221

=

1221

− 66

1201

− 292

32−1112

=

1221

−

1201

−

23−494929

=−2349149−29

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization

DefinitionA set {v1, v2, . . . , vk} of vectors in Rn is orthonormal if it isan orthogonal set and every vector in the set has length one.

If we think about this in a geometric sense, we have toconsider that vectors need not intersect to be orthogonal. Itis the orientation that matters. So, the question is, how canwe guarantee that all of the vectors are of length one?

OrthonormalizationBy replacing the output of the Gram-Schmidt process{v1, v2, . . . , vk} with {w1,w2, . . . ,wk} where wi = 1‖vi‖vi ,one obtains an orthonormal basis for the given subspace.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization

DefinitionA set {v1, v2, . . . , vk} of vectors in Rn is orthonormal if it isan orthogonal set and every vector in the set has length one.

If we think about this in a geometric sense, we have toconsider that vectors need not intersect to be orthogonal. Itis the orientation that matters. So, the question is, how canwe guarantee that all of the vectors are of length one?

OrthonormalizationBy replacing the output of the Gram-Schmidt process{v1, v2, . . . , vk} with {w1,w2, . . . ,wk} where wi = 1‖vi‖vi ,one obtains an orthonormal basis for the given subspace.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization

DefinitionA set {v1, v2, . . . , vk} of vectors in Rn is orthonormal if it isan orthogonal set and every vector in the set has length one.

If we think about this in a geometric sense, we have toconsider that vectors need not intersect to be orthogonal. Itis the orientation that matters. So, the question is, how canwe guarantee that all of the vectors are of length one?

OrthonormalizationBy replacing the output of the Gram-Schmidt process{v1, v2, . . . , vk} with {w1,w2, . . . ,wk} where wi = 1‖vi‖vi ,one obtains an orthonormal basis for the given subspace.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

Example

Apply the Gram-Schmidt orthonormalization process to thegiven basis for R3.

S =

11

0

,12

0

,01

2

First, let’s find the orthogonal basis.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

Example

Apply the Gram-Schmidt orthonormalization process to thegiven basis for R3.

S =

11

0

,12

0

,01

2

First, let’s find the orthogonal basis.

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v1 =

s1 =

110

v2 = s2 − projv1s2

=

120

− 32

110

=

−12120

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v1 = s1 =

110

v2 = s2 − projv1s2

=

120

− 32

110

=

−12120

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v1 = s1 =

110

v2 =

s2 − projv1s2

=

120

− 32

110

=

−12120

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v1 = s1 =

110

v2 = s2 − projv1s2

=

120

− 32

110

=

−12120

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v1 = s1 =

110

v2 = s2 − projv1s2

=

120

− 32

110

=

−12120

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v1 = s1 =

110

v2 = s2 − projv1s2

=

120

− 32

110

=

−12120

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v3 =

s3 − projv1s3 − projv2s3

=

012

− 12

110

− 1212

−12120

=

002

So, the orthogonal basis is11

0

,−121

20

,00

2

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v3 = s3 − projv1s3 − projv2s3

=

012

− 12

110

− 1212

−12120

=

002

So, the orthogonal basis is11

0

,−121

20

,00

2

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v3 = s3 − projv1s3 − projv2s3

=

012

− 12

110

− 1212

−12120

=

002

So, the orthogonal basis is11

0

,−121

20

,00

2

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v3 = s3 − projv1s3 − projv2s3

=

012

− 12

110

− 1212

−12120

=

002

So, the orthogonal basis is11

0

,−121

20

,00

2

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

v3 = s3 − projv1s3 − projv2s3

=

012

− 12

110

− 1212

−12120

=

002

So, the orthogonal basis is11

0

,−121

20

,00

2

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

Now we normalize ...

w1 =1

‖v1‖v1

=1√2

110

=

1√21√2

0

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

Now we normalize ...

w1 =1

‖v1‖v1

=1√2

110

=

1√21√2

0

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

Now we normalize ...

w1 =1

‖v1‖v1

=1√2

110

=

1√21√2

0

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

w2 =1

‖v2‖v2

=1√12

−12120

=

−√22√220

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

w2 =1

‖v2‖v2

=1√12

−12120

=

−√22√220

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

w3 =1

‖v3‖v3

=1√4

002

=

001

So, the orthonormal basis we seek is

1√21√2

0

,−

√22√220

,00

1

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

w3 =1

‖v3‖v3

=1√4

002

=

001

So, the orthonormal basis we seek is

1√21√2

0

,−

√22√220

,00

1

1 Projections

2 The Properties

3 Example 1

4 Example 2

5 Back toProjections

6 TheGram-SchmidtProcess

7 G-S Example 1

8 G-S Example 2

9 Orthonormaliza-tion

Orthonormalization Example

w3 =1

‖v3‖v3

=1√4

002

=

001

So, the orthonormal basis we seek is

1√21√2

0

,−

√22√220

,00

1

First SectionSecond SectionThird SectionFourth SectionFifth SectionSixth SectionSeventh SectionEighth SectionNinthSection