Bootcamp Math Workbook v1_0

7/28/2019 Bootcamp Math Workbook v1_0

1/46

Math Workbook v1.0


2/46

Completely Voluntary Office Hours7:30-9:30pm @ Steele 214

During the first week of classes

Monday'

9/26/2011'

Tuesday'

9/27/2011'

Wednesday'

9/28/2011'

Thursday'

9/29/2011'

Friday'

9/30/2011'

Calculus' Linear'

Algebra'I'

Linear'

Algebra'II'

ODEs' Complex'

Analysis'

FREEFOO

D

FREEDRINKSONFRIDAY


3/46


4/46

Gaussian'(normal)'distribu1on'P(x)=

1

22e

(x)2

22

Bell$curve$about$mean$$with$variance$2$

Binomial'distribu1on'

P(k;n, p)= kn( )pk(1 p)nkProbability$of$k$occurrences$for$n$trials,$

each$with$probability$of$success$p$

PROBABILITY'

Permuta1ons$(order&ma)ers&&&&

Combina1ons'(order&d/n&ma)er&

Repeats&allowed& Repeats&not&allowed&

nk n!

(n k)!

n

k

!

"#

$

%&=

n

n k

!

"#

$

%&=

n!

k! n k( )!n+ k1

k

"

#$

%

&'=

n+ k1

n1

"

#$

%

&'=

n+ k1( )!

k! n1( )!

or$n!

ni!

(x+y)n= k

n( )k=0

n

xnkyk = kn( )k=0

n

xkynk

(x1+ x

2+...+ x

m)n=

k1,k2 ,...,km

n( )k1,k2 ,...,km

x1k1x2k2xmkm

Binomial&theorem&

Mul8nomial&theorem&

LINEAR'ALGEBRA'

inver8ble&matrices&

$$ $ $$$$$$always$has$a$unique$soln$ zero$is$not$an$eigenvalue$ matrix$is$square$and$has$full$rank$

Ax = b

detA 0

FIELDS'AND'FLOWS'

f =f

xx+

f

yy+

f

zz =

f

xiGrad'

Div'

rate$&$direcon$of$change$in$scalar$field$f$

v=

vx

x+

vy

y+

vz

z=

vi

xi

v =

x y z

x

y

z

vx vy vz

=ijk

vk

xjCurl'

magnitude$of$vector$

field$vs$source/sink$

rotaon$of$vector$field$v$

Divergence'(Gausss)'theorem'

dV=

V

S

n dSdiv$a ""$$$$$$$$$$$a"Net$sources/sinks$in$a

volume$V$==$net$flow$

across$a$surface$S$$

Con1nuity'equa1on' f

t+

v = s,

s = 0 if$f$is$conserved

GSC'BOOTCAMP!'

Dot'(inner)'product:'

a

b = a1b1+ a

2b2+

=

a

b cos

Cross'product:'

a b =

x y z

a1

a2

a3

b1

b2

b3

Triple'product:'

projec8on$of$a$onto$b$

volume$of$parallelepiped'

a!b'

b'

a'

magnitude$=$area$

a

b

c( ) =a1

a2

a3

b1

b2

b3

c1

c2

c3

More&informa8on&on&the&wiki:&www.bootcatmp.calech.edu&

Join&our&mailing&list:&[email protected]&

a =

a

a = a1

2+a

2

2++ a

n

2

magnitude&of&a&vector&

determinant&of&a&matrix&

detA = A =

a11

a12

a13

a21

a22

a23

a31

a32

a33

= a11

a22a33 a

23a32( )

a12

a21a33a

23a31( )

+ a13

a21a32 a

22a31( )

Useful'defini1ons'

A1=

1

A

a22

a12

a21

a11

"

#

$$

%

&

''

Worked'example'problem:'Markov$(stochasc)$matrix$Each$year,$$6%$of$the$urban$populaon$

move$to$rural$areas$while$9%$of$the$rural$

populaon$move$to$urban$areas:$

urban$(node$1)$

rural$(node$2)$

0.06$

0.09$

0.91$0.94$

system&of&equa8ons:'

uk+1= 0.94u

k+0.09r

k

rk+1

= 0.06uk+0.91r

k

matrix&form:'

uk+1

rk+1

!

"

##

$

%

&&=

0.94 0.09

0.06 0.91

!

"#

$

%&

uk

rk

!

"

##

$

%

&&

Each$aij$describes$the$interacon$

btwn$nodes$i$and$j&

find&the&eigenvalues&:'

0.94 0.09

0.06 0.91

= 0.94( ) 0.91( ) 0.09( ) 0.06( ) = 1( ) 0.85( )

find&the&eigenvector&for&1:'

1 =1, 2 = 0.85

0.94 1

0.09

0.06 0.911

"

#

$$

%

&

''

v1

v2

"

#

$$

%

&

''=

0

0

"

#$

%

&'

0.06v1+0.09v

2= 0

0.06v10.09v

2= 0

v2=

2

3v1

1

23

!

"

##

$

%

&&or

0.6

0.4

!

"#

$

%&

det AI( ) = 0

eigenvalue&equa8on&

steady&state&

Av = v

eigenvalues$$and$eigenvectors$v& find$eigenvalues$by$solving$$ find$eigenvector$for$each$eigenvalue$

by$solving$ AiI( )vi = 0unit&(normalized&vector& a =

a

a

Laplacian'

f =2f =f =2f

x2+

2f

y2=

2f

xi2

flux$density$of$the$gradient$flow$of$a$scalar$field$f&

Greens'theorem'

L dx +Mdy( )C =

M

xL

y

$

%&

'

()

D dxdy

Net$fluid$ouXlows$in$a$volume$$==$total$ouXlow$about$an$enclosing$surface$area$

Rate$of$change$of$fluid$density$f$and$divergence$of$flux$v$==$

net$sources/sinks$s$

2x2&ex:&


5/46

Bootcamp Workbook: Calculus and Dierential

Equations

September 6, 2011

1 Introduction

Hey! This is in case its been a while since you learned the basics of calculus anddierential equations. We assume youve seen most of this stu before and thatsome of it is laughably easy for you. Other parts of it may jog your memoryso that when the first few problem sets hit, youll have the basic tools ready tohand. A few things may have been unaccountably left out of your education sofar, in which case, dont worry, just look it up on Wikipedia or something.

None of this is compulsory. Its here for students who think they might haveforgotten some of the basics, and/or students who want to reassure themselvesprior to their upcoming classes.

Welcome to Caltech!

2 Calculus2.1 Dierentiation Techniques

1. Use the chain rule to evaluate

d

dxexp(sin(x))

2. Use the product rule to evaluate

d

dx

x3exp(x)

3. Use the quotient rule to evaluate

d

dx

x3 + 3

cos(x)

1


6/46

2.2 Integration Techniques

1. Use integration by parts to evaluate the following indefinite integral:Zxexdx

2. Use the substitution x = cos() to evaluate the following integral:

Z11

1p

1 x2dx

3. Use partial fractions to evaluate the following integral:

Z1

x2

+ 3x + 2

dx

2.3 Taylor Series

1. Find the Taylor series for f(x) = 1p2

ex2/2 about zero up to O(x5).

Integrate it to estimate the probability that a given standard-normally-distributed event falls within 0.05 of the mean.

2. Give the Taylor series for ln(x) about x = 1 up to O((x 1)3).

2.4 Partial Derivatives

1. Iff(x, y) = x2y + exy, calculate @f@x .

2. Let z = x + y and suppose that u = z x. First calculate the partialderivative ofu with respect to x with y held constant @u

@x

y

. Then calculate

the partial derivative of u with respect to x with z held constant @u@x

z

.Are these two quantities the same? Why, or why not?

2.5 The Multivariable Chain Rule

1. Let u = x2 + xy, with dxdt = 1 anddydt = 2. Write

dudt in terms ofx and y.

2. Suppose that@F

@x= y

and

@F@y

= x + 2

Use the chain rule to rewrite these equations in terms of the new variablesu = x + y and v = x2.

2


7/46

2.6 Line Integrals1. A unit mass moves through a gravitational field. The gravitational field

exerts a force F of 10N on this mass, in the y direction. The positionof the mass at a time t is given by s = (x, y) = (t, 5 t2). Find the workdone on the mass by the gravitational field between t = 0 and t = 2, usingthe line integral Z

F ds

2. A straight wire stretches between the points (0, 0) and (1, 1). The massper unit length of the wire is given by (x, y) = yex + x. Find the totalmass of the wire.

2.7 Lagrange Multipliers

A positively charged particle is constrained to lie on the circle x2 + y2 = 1. Itreacts to the electric potential generated by another positively charged particlefixed at (2, 2), which is given by 14

1p(x2)2+(y2)2

. Use Lagrange multipliers

to find the point of lowest potential energy on the unit circle, towards whichthe constrained particle will travel. Check that the answer accords with yourphysical intuition.

3 Dierential Equations

3.1 Separation of Variables

Find the general solution to the following dierential equation using separationof variables:

dy

dx= x2y + y

3.2 Integrating Factors

Consider a circuit involving a resistance R and an inductance L. After theswitch is closed at time t = 0, the current i in this circuit obeys the followingdierential equation:

L

di

dt + Ri = E(t)

where E(t) = t is the time-dependent voltage, L = 1 and R = 2.Solve for the current as a function of time, using an integrating factor. If

youve forgotten, an integrating factor is that thing where you take e to thepower of a particular function...

3


8/46

3.3 Second-Order Linear Homogeneous ODEs

A unit mass is attached to a spring. Let y be the displacement of the mass fromits equilibrium position. The force of the spring acting on the mass is given byky where k is the spring constant. Assume that the drag force acting on themass is given by FD = 4y.

At time t=0, the mass is released from where it has been held steady aty = 1. Calculate the resulting motion of the mass in the following cases:

1. k = 3

2. k = 4

3. k = 5

Graph the behaviour of the resulting solutions. (You can use software like

Mathematica if you like. Obviously. I mean, all of this is voluntary to beginwith.)

3.4 Systems of dierential equations

1. Write the following second-order dierential equation as a pair of coupledfirst-order dierential equations:

y + 6y + 9 = 0

Hint: Start by setting y = v.

2. Solve the following pair of coupled first-order dierential equations

x = x + 2y

y = x + 4y

Hint: Eigenvalues and eigenvectors may be useful here.

4


9/46

ACM 100 Bootcamp: Vectors

Adapted from Appendix A of

Analysis of Transport Phenomena by William Deen

September 1, 2011

Vector Representation

Components

You are probably already familiar with the concepts of scalars and vectors. A scalar is a quantity with solelya magnitude, while a vector is a quantity with magnitude and direction. For example, the speedometerin your car tells you your speed (a scalar), while your velocity (a vector) is the speed you are traveling in acertain direction. A vector in 2-dimensional space, i.e. in the xy-plane, can be represented by an orderedpair of scalars called components. There are several different ways to represent a vector. One may useboldface, as follows:

v = vx1x + vy1y. (1)

In Equation 1, v is a vector in 2-D space, and vx & vy are the scalar components in x and y, respectively.The terms 1x and 1y are unit vectors in the x and y directions. Unit vectors have a magnitude of 1.Other representations of unit vectors in 2-D space include x & y, ex & ey, e1 & e2, and i & j. We couldalso arrange the scalar components into row or column arrays, which is useful when performing operationsbetween vectors and matrices (two-dimensional arrays):

v =

vx vy

(row form), (2)

v =

vxvy

(column form). (3)

This notation can be extended to 3-D space by adding a component in the third direction:

v = vx1x + vy1y + vz1z, (4)

or to N-D space by having a component for each of the N directions:

v = v1e1 + v2e2 + + vNeN =NXi=1

viei. (5)

Magnitude and Unit Vectors

The length, magnitude, or norm of a vector v is denoted by v. We can take the Euclidean norm of thevector to find the length. For example, the norm of a vector in 3-D space is:

v =q

v21 + v22 + v

23. (6)

As mentioned above, unit vectors have a magnitude of one. To make any vector into a unit vector, justdivide each component by the magnitude:

v = v/ v . (7)

1


10/46

Tensors

Each component in a vector is associated with one direction. For example, the vector v in 3-D space hasthree components:

v = v1e1 + v2e2 + v3e3. (8)

The component v1 is associated with the e1 direction, etc. Vectors are the appropriate representation for

magnitudes are associated with one direction, but what if we have a quantity associated with two or moredirections? For example, each component of stress is associated with two directions: one direction describesthe unit normal of a surface or plane, while the second direction describes the direction of the force actingon that plane. With three possible orientations of a plane and three directions of force, there are nine totalcomponents of the stress tensor. Much like each component of a vector is associated with a unit vector,each component of a second-order tensor is associated with a unit dyad.

= 11e1e1 + 12e1e2 + 13e1e3 + 21e2e1 + 22e2e2 + 23e2e3 + 31e3e1 + 32e3e2 + 33e3e3. (9)

Tensors can be even higher order than second-order! For example, the most general linear relation betweentwo second-order tensors (e.g. the stress and strain in a continuum) is a fourth-order tensor. The numberof components required to fully specify an Nth order tensor in D-dimensional space is DN. The arrow-in-space representation that works so well for vectors ceases to be useful for higher tensorial order than one.

Fortunately, there are other ways to represent tensors. Two-dimensional arrays can faithfully represent thecomponents of a second-order tensor:

=

24 11 12 1321 22 23

31 32 33

35 . (10)

Index Notation

A convenient way to represent tensorial quantities compactly is using index notation. To explain indexnotation, we will use the orthonormal basis {e1, e2, e3}. The ortho- prefix in orthonormal represents thefact that basis vectors are orthogonal (mutually perpendicular) to one another. The -normal suffix representsthe fact that the vectors in the basis are normalized, or are unit vectors. Here are the steps to convert a

vector from Gibbs (boldface) notation to index notation:

v(1)= v1e1 + v2e2 + v3e3

(2)=

3Xi=1

viei(3)= vi.

1. Write the vector v in terms of its components.

2. Sum over a dummy index (in this case, i) that iterates through all components.

3. Summation symbols and unit vectors are made implicit.

You dont need to go through all these steps every time you convert from Gibbs notation to index notation -just recognize that a quantity with one subscript index is a vector. We may do the same for a second-ordertensor:

(1)= 11e1e1 + 12e1e2 + + 33e3e3 (2)=

3Xi=1

3Xj=1

ijeiej (3)= ij.

1. Write the tensor in terms of its components.

2. Sum over dummy indices (in this case, i and j) that iterate through all components.

3. Summation symbols and unit dyads are made implicit.

Note that we needed two distinct dummy indices for a second-order tensor. For an Nth-order tensor, wewould need N distinct dummy indices. Later, we will show conventions for repeated indices in index notation.

2


11/46

Vector Operations

Vector Addition

Vectors are added in a componentwise fashion. For example, to add the 2-D vectors a and b in the xy-plane,we simply sum their x components together and sum their y components together:

a + b = (ax + bx)1x + (ay + by)1y. (11)

The addition of vectors is a commutative operation, i.e.

a + b = b + a (addition is commutative). (12)

Vector addition is also associative, i.e. the order of operations for successive vector addition does notmatter:

(a + b) + c = a + (b + c) (addition is associative). (13)

The additive identity for vector addition is the zero vector, or the vector with all scalar componentsequal to zero. In 2-D, the zero vector is:

0 = 0 1x + 0 1y (zero vector). (14)

The additive inverse for a vector v is the vector where each component is the negative of that in v:

v = (vx)1x + (vy)1y (additive inverse). (15)

Using the arrow in space representation of vectors, addition of two vectors a and b is possible by translatingb such that the tail of vector b is coincident with the tip of vector a. The vector sum a + b is simply thevector from the tail of vector a to the tip of the translated vector b (See Figure 1). Note that the sameresultant vector is achieved from b + a by translating a such that the tail of vector a is coincident with thetip of vector b.

1x

1y

a

b

a+ b

Figure 1: Vectors a =

2 1

and b =

1 3

are added to form the vector sum a + b =

3 4

.

3


12/46

Scalar Multiplication

Multiplying a vector v by a scalar s is done by multiplying each scalar component of v by the scalar s.Multiplying a vector by a scalar returns a vector:

sv = svx1x + svy1y (scalar multiplication). (16)

Scalar multiplication is distributive over addition of vectors:

s (a + b) = sa + sb (distributive), (17)

and over addition of scalars:(s + t)a = sa + ta (distributive). (18)

The multiplicative identity for scalar multiplication is simply the scalar 1:

1v = v (multiplicative identity). (19)

Using the arrow in space representation of vectors, scalar multiplication by s is a dilation, lengthening (orshortening, for 0 < |s| < 1) the vector v by a factor s. Negative scalars reflect the arrow through the origin.

1x

1y aa/2

a

2a

Figure 2: Scalar multiplication of the vector a =

2 1

by 2, 1/2, and 1.

4


13/46

Dot Product

The dot product (also known as the inner product or scalar product) of two vectors is given by theformula:

a b = a b cos ab (dot product), (20)where ab is the angle ( 180) between vectors a and b. We see right away that the dot product fororthogonal (perpendicular) vectors must be identically zero, as the cosine of 90

is zero. We also see thatthe dot product of a vector with itself is simply the magnitude squared. Thus, another way to define thenorm of a vector uses the dot product:

a = a a (norm of a vector). (21)

The dot product of any two different orthonormal basis vectors {e1, e2, e3} is zero, as they are mutuallyperpendicular. The dot product of an orthonormal basis vector with itself is one. We can write these twofacts compactly using the Kronecker delta:

ei ej = ij, (22)

where the Kronecker delta is defined as:

ij =(

1, i = j,0, i = j (Kronecker Delta). (23)

The dot product of vectors is commutative and distributive:

a b = b a; a (b + c) = a b + a c. (24)

Dot Product as Projection

Suppose we have some vector a that we would like to project onto another vector b. If we think of the vectora as the hypotenuse of a right triangle, the length of the projection of a onto b is given by the leg of theright triangle parallel to b. This projection has length a cos ab , which is equal to the dot product a b(See Figure 3). We now have a geometric interpretation of the dot product of vectors a and b: the product

of the magnitudes ofb and the projection ofa onto b. Conversely, a b is also equal to the product of themagnitudes ofa and the projection ofb onto a.

a

1x

1y

b

a cos

Figure 3: Projecting a vector a onto another vector b is easily visualized with a as the hypotenuse of a righttriangle. The projection is the leg parallel to b. The length of the projection is equal to a b.

5


14/46

Cross Product

The cross product (or vector product) of two vectors in 3 dimensions is defined as:

a b a b sin ab eab (cross product), (25)where eab is a unit vector perpendicular to the plane passing through vectors a and b. Note that if the vectorsa and b are parallel or antiparallel, there are infinitely many planes passing through a and b. However, the

sin ab results in parallel or antiparallel vectors having a zero cross product. The convention for the crossproduct is for the set of vectors {a, b, a b} to be a right-handed set: when you curl the fingers of yourright hand from the vector a to the vector b, the thumb of your right hand will point in the direction ofa b. Figure 4 depcits this right-hand rule.

a

1x

1y

bab

ba

Figure 4: For the above example, a b points in the z-direction (into the page), while ba points in the+z-direction (out of the page).

Using our orthonormal basis {e1, e2, e3} for 3-dimensional space, we see that, as long as we prescribethe basis to be a right-handed set of unit vectors, the cross product of any two basis vectors is given by thecompact expression:

ei

ej

=3

Xk=1

ijkek

. (26)

In Equation 26, we introduced the permutation symbol ijk defined as follows:

ijk

8>:

1 ijk = 123, 231, or 312,

1 ijk = 132, 213, or 321,0, otherwise

(permutation symbol). (27)

The cross product of two vectors is anticommutative. Switching the order of vectors negates the result:

a b = b a. (28)The cross product is distributive over vector addition:

a (b + c) = a b + a c. (29)

Using the fact that the cross product is distributive over vector addition, we can write the cross product outin terms of vector components:

a b = (a2b3 a3b2) e1 + (a3b1 a1b3)e2 + (a1b2 a2b1) e3. (30)One way to remember the cross product is to take the determinant of a 3 3 matrix with rows that are thecomponents ofa, the components ofb, and the unit basis vectors respectively:

a b = det24 a1 a2 a3b1 b2 b3

e1 e2 e3

35 . (31)

6


15/46

Cross Product as Area

Consider a parallelogram whose two sides are defined by the vectors a and b (See Figure 5). The area ofthis parallelogram is given by a b sin ab, which is the magnitude of the cross product a b. Thus, wehave a geometric interpretation of the cross product.

a

1x

1y

b

a sin

a

b

Figure 5: A parallelogram formed by two vectors (a and b) has an area equal to a b sin ab, which isprecisely the magnitude ofa b

Scalar Triple Product as Volume

Suppose we wanted to find the volume of a parallelepiped, or a 3-dimensional figure with six parallelogramfaces. We can use the cross and dot products of vectors a, b, and c defining three of the edges (See Figure6). The absolute value of the scalar triple product will give us the volume of the parallelepiped:

(a b) c = a b sin ab c cos ab,c (scalar triple product). (32)

Cyclic rearrangements of the arguments do not change the value of the product:

(a b) c = (b c) a = (c a) b = c (a b) = a (b c) = b (c a) . (33)

b

ab

a b

a

c

a b = a b sin ab

(a b) c = a b sin ab c cos ab,c

ab,c

cc

osa

b,

c

Figure 6: The scalar triple product gives the volume of the parallelepiped formed by a, b, and c.

7


16/46

Dyadic Product

A third method of vector multiplication, the dyadic product, results in a second-order tensor. The threecomponents of both vectors are multiplied together in all nine possible combinations, and each product isassociated with a unit dyad:

ab = a1b1e1e1 + a1b2e1e2 + + a3b2e3e2 + a3b3e3e3 (dyadic product). (34)

The product is easy to visualize using row and column arrays:

ab =

24 a1a2

a3

35 b1 b2 b3 =

24 a1b1 a1b2 a1b3a2b1 a2b2 a2b3

a3b1 a3b2 a3b3

35 . (35)

The dyadic product is not necessarily commutative. The dyadic product commutes if and only if b is ascalar multiple ofa. The resulting dyad is symmetric:

ab = ba b = ca. (36)

The dyadic product is distributive over vector addition:

c (a + b) = ca + cb. (37)

Extensions to Tensors

The definition of the dot product can be extended to cases where either or both arguments are dyads.Consider the unit vectors ei, ej , ek and el selected from the orthonormal basis {e1, e2, e3}. The followingoperations hold:

eiej ek = jkei, (38)

ej ekel = jkel, (39)

eiej ekel = jkeiel, (40)

eiej : ekel = jkil. (41)

8


17/46

Index Notation of Dot, Cross, and Dyadic Products

Using the Kronecker delta, the index notation representation of the dot product can be determined:

a b(1)=

3Xi=1

aiei 3X

j=1

bjej(2)=

3Xi=1

3Xj=1

aibjei ej(3)=

3Xi=1

3Xj=1

aibjij(4)=

3Xi=1

Xj=i

aibj(5)=

3Xi=1

aibi(6)= aibi. (42)

1. The vectors a and b are written as sums over distinct dummy indices.

2. The distributive properties of scalar multiplication and dot products are used to bring all terms insidethe summations.

3. The definition of the dot product of orthonormal basis vectors is applied

4. The definition of the Kronecker delta is applied to the innermost summation

5. Summing over j = i is equivalent to changing every instance of j to i.

6. Summation symbols are made implicit.

The index notation for a b is thus aibi. We may do the same for the cross product:

ab (1)=3X

i=1

aiei3X

j=1

bjej(2)=

3Xi=1

3Xj=1

aibjeiej (3)=3X

i=1

3Xj=1

aibj

3Xk=1

ijkek(4)=

3Xi=1

3Xj=1

3Xk=1

aibjijkek(5)= aibjijk

(43)


2. The distributive properties of scalar multiplication and cross products are used to bring all terms insidethe summations.

3. The definition of the cross product of orthonormal basis vectors is applied

4. The distributive property of scalar multiplication is used again to bring all terms inside the summation

over k.5. Summation symbols and unit vectors are made implicit.

The dyadic product also has a simple representation in index notation:

ab(1)=

3Xi=1

aiei

3Xj=1

bjej(2)=

3Xi=1

3Xj=1

aibjeiej(3)= aibj. (44)


2. The distributive properties of the dyadic product is used to bring all terms inside the summations.

3. Summation symbols and the unit dyad are made implicit.

This brings up important points about dummy indices:

A dummy index can be repeated no more than twice in an index notation expression. For example,the expression aibici is ill-defined in index notation.

Summation symbols are implicit for each distinct dummy index.

Unit basis vectors are implicit for non-repeated or free indices.

You can determine the tensorial order of an expression by counting the number of free indices.

9


18/46

Vector Rotation

Rotation in a plane

Consider a vector a in the xy-plane. The x and y components ofa can be expressed in terms of the magnitudea and angle from the x-axis:

ax = a cos ; ay = a sin . (45)Suppose we want to rotate the vector a about the origin by some angular increment (see Figure 7). Letscall the rotated vector a. Because were rotating and not dilating the vector, the magnitude of the vectorremains unchanged:

a = a . (46)The new angle from the x-axis is incremented by :

+. (47)

We can use Equation 45 to write the components of the rotated vector a in terms of its magnitude anddirection:

ax = a cos ; ay = a sin . (48)Using Equations 46 and 47, and using trig sum identities, the rotated components may be expressed as:

ax = a (cos cos() sin sin()) ; ay = a (sin cos() + cos sin()) . (49)

Recognizing that we can use Equation 45 to factor out the components from the reference frame, the rotatedcomponents may be expressed as:

ax = ax cos() ay sin() ; ay = ax sin() + ay cos() . (50)

We can write the previous pair of equations in vector form:

a = R() a,

where the tensor R is the rotation tensor:

R

cos() sin()sin() cos ()

.

1x

1y

a

a

Figure 7: Rotating the vector a from the angle to the angle .

10


19/46

Rotation in 3-D space

Suppose the vector a is now in three-dimensional space and we want to rotate it by the same angularincrement about the z-axis. If we project this problem of rotation onto the xy-plane, we recover exactlywhat was just shown in the previous section. The z-component ofa is unchanged by the rotation deformation.Thus, in 3-D, the rotation tensor is given by:

R24 cos() sin() 0sin() cos () 0

0 0 1

35 (rotation about z-axis).Not all rotations must occur about a coordinate axis. What if we wanted to rotate a by the angular increment about some arbitrary vector b? It would help for us to transform the vector a from components in theorthonormal Cartesian basis B {1x,1y,1z} to components in a new orthonormal basis B

1x,1

y,1

z

This transformation of bases is given by the matrix B, defined as:

B 24 1x 1x 1y 1x 1z 1x1x 1y 1y 1y 1z 1y1x 1

z 1y 1

z 1z 1

z

35 .

The transpose of this tensor BT

transforms a vector from the new orthonormal basis B

to the original basisB. If we select the new orthonormal basis B such that the vector about which to rotate (b) is coincidentwith 1z, then we can use the rotation matrix R to rotate vectors about b when they are expressed in thebasis B! Thus, the rotation transformation is:

a = BT RB a.

11


20/46

Vector Differential Operators

Gradient

The gradient operator is denoted by the symbol r (nabla). The gradient of a scalar quantity returns avector. In cartesian coordinates, we can write the gradient of some scalar field (x,y,z) as:

r = x

1x + y

1y + z

1z.

An example of gradient with which you may already be familiar is elevation. Suppose h (x, y) gives us theelevation of an area with latitude and longitude specified by x and y. We can find the gradient of theelevation as:

rh hx

1x +h

y1y.

This gradient vectorrh tells us the vector direction of the steepest ascent at any point. The gradient vectoris larger in regions of steeper ascent. Lastly, note that the gradient is everywhere normal to the iso-elevationlines. Thus, if you travel perpendicular to the gradient of a scalar field, you should stay on a surface ofconstant scalar.

The gradient has a representation in index notation, as shown:

r(1)=

x1e1 +

x2e2 +

x3e3

(2)=

3Xi=1

xiei

(3)=

xi.

In step (1), we write out the gradient component by component. In step (2), we condense the three termsinto a sum over a dummy index i. In step (3), the summation symbol and the unit vector are made implicit.

Not only can we take the gradient of a scalar, but we can also take the gradient of a vector:

rv =v1x1

e1e1 +v2x1

e1e2 +v3x1

e1e3 +v1x2

e2e1 + +v2x3

e3e2 +v3x3

e3e3.

Taking the gradient of a vector quantity returns a second-order tensor. We see now that the gradient operatorincreases the tensorial order of a quantity by 1. The index notation for the gradient of a vector is:

rv(1)

=v1x1 e

1e1 + +v3x3 e

3e3(2)

=

3Xi=1

3Xj=1

vjxi e

iej(3)

=vjxi .

In step (1), we write out the gradient component by component. In step (2), we condense the nine termsinto a double sum over the dummy indices i and j. In step (3), the summation symbols and unit dyads aremade implicit.

Divergence

The divergence operator is denoted by r or div (). It is somewhat like a dot product between a gradientoperator and a tensor. It reduces the tensorial order of a quantity by one. For example, we can take thedivergence of a vector to yield a scalar:

r

v

=

vx

x +

vy

y +

vz

z .

The divergence of a second-order tensor results in a vector:

r =

xx

x+

yx

y+

zx

z

1x +

xy

x+

yy

y+

zy

z

1y +

xz

x+

yz

y+

zz

z

1z.

The index notation form of the divergence is as follows:

r v(1)=

x1e1 +

x2e2 +

x3e3

(v1e1 + v2e2 + v3e3)

(2)=

3Xi=1

3Xj=1

vjxi

ei ej(3)=

3Xi=1

3Xj=1

vjxi

ij

12


21/46

(4)=

3Xi=1

Xj=i

vjxi

(5)=

3Xi=1

vixi

(6)=

vixi

.

In step (1), we write out r and v component by component. In step (2), we condense the terms into adouble sum notation over the dummy indices i and j. In step (3), we apply the definition of the dot productof orthonormal unit vectors to result in the Kronecker delta. In step (4), we apply the definition of the

Kronecker delta. In step (5) we remove the redundant summation over j = i and change each instance of jto i within the summation. In step (6), the summation over the dummy index i is made implicit. Note thatwe have a repeated dummy index i, so the summation is implicit but there is no implicit unit vector.

The index notation form for the divergence of a second-order tensor can be found in the same manner:

r(1)=

x1e1 +

x2e2 +

x3e3

(11e1e1 + + 33e3e3)

(2)=

3Xi=1

3Xj=1

3Xk=1

jk

xieiejek

(3)=

3Xi=1

3Xj=1

3Xk=1

jk

xiijek

(4)=

3Xi=1

Xj=i

3Xk=1

jk

xiek

(5)=

3Xi=1

3Xk=1

ik

xiek

(6)=

ik

xi.

In step (1), we write outr and component by component. In step (2), we condense the terms into a triplesum notation over the dummy indices i, j, and k. In step (3), we apply the definition of the dot product

of an orthonormal unit vector and unit dyad to result in the Kronecker delta. In step (4), we apply thedefinition of the Kronecker delta. In step (5) we remove the redundant summation over j = i and changeeach instance of j to i within the summation. In step (6), the summations over the dummy indices i & kand the unit vector ek are made implicit.

Curl

The curl operator is denoted by r or curl () . It is somewhat like a cross product between a gradientoperator and a vector. It maintains the tensorial order of a quantity. For example, the curl of a vector is asfollows:

r v =vzy

vyz

1x +

vxz

vzx

1y +

vyx

vxy

1z.

Cross products introduce a chirality or handedness to a quantity thus, the curl of a true vector is apseudovector and the curl of a pseudovector is a true vector.

The representation of the curl in index notation is:

r v (1)=

3Xi=1

xiei

!0@ 3X

j=1

vjej

1A (2)= 3X

i=1

3Xj=1

vjxi

ei ej

(3)=

3Xi=1

3Xj=1

vjxi

3Xk=1

ijk ek

!

(4)=

3Xi=1

3Xj=1

3Xk=1

vjxi

ijk ek

(5)=

vjxi

ijk

Steps (1) and (2) are the same as for the dot product. In step (3), we apply the definition of the crossproduct to obtain the summation over a third dummy index k of the permutation symbol. In step (4), webring all terms inside of the three summations over i, j, and k. This is the same principle used in step (2).

Lastly, the summation symbols and the unit vector ek are made implicit in step (5).The curl of the fluid velocity vector is known as the vorticity, a measure of the local angular rate of

rotation in the fluid. Flows with zero vorticity are known as irrotational.

Position Vector

The position vector is often denoted by x. It denotes the position of a point in space with respect to somereference point or origin. In cartesian component representation, the position vector is:

x = x1x + y1y + z1z.

13


22/46

We may also write the position vector in index notation:

x = x1e1 + x2e2 + x3e3 =

3Xi=1

xiei = xi.

The position vector is first expressed component by component, then the terms are combined into a sum-

mation over all components using the dummy index i. The unit vectors and summation symbol are lastlymade implicit.

If we take the gradient of the position vector, we find:

rx =x1x1

e1e1 +x2x1

e1e2 +x3x1

e1e3 +x1x2

e2e1 + +x2x3

e3e2 +x3x3

e3e3

The three coordinates x1, x2, and x3 vary independently of one another. Thus, the derivatives x/x areequal to unity when = and are otherwise equal to zero.

rx = 1e1e1 + 0e1e2 + 0e1e3 + 0e2e1 + + 0e3e2 + 1e3e3

We already have a method of representing this quantity it is the second-order identity tensor:

rx = I.

In index notation, we can write the gradient of the position vector as:

xjxi

= ij .

The divergence of the position vector may be expressed as:

r x =x1x1

+x2x2

+x3x3

= 3.

This result may make more sense if we think about it in index notation. First, we write the divergence interms of a dummy index:

r x = xixi

= ii.

The quantity ii has an implicit summation over all three coordinates, i.e. we want the trace of the identitytensor, which is three in three-dimensional space.

Intuitively, one can guess that the curl of the position vector will be zero since it always radiates outwardfrom the origin:

r x =x3x2

x2x3

e1 +

x1x2

x3x1

e2 +

x2x1

x1x2

e3 = 0.

We can see this also using index notation:

r

x =

xj

xiijk = ijijk = iik = 0.

In the first step, we write the curl in index notation. We apply the definition of xj/xi in the next step,introducing the Kronecker delta. Next, we apply the definition of the Kronecker delta, setting each instanceof the dummy index j to i. However, the permutation symbol is equal to zero should any of the indices beequal. Thus, the curl of the position vector is zero.

The magnitude of the position vector is represented by r =x x =

px2 + y2 + z2, or the radial

distance from the origin to the end of the position vector. Thus, if we want to normalize the position vector,we divide it by r:

n = x/ |x| = x/r.

14


23/46

The unit normal n represents the outward-facing unit normal vector to the surface of a sphere centered atthe origin.

How would we find the gradient of a function which only depends on the magnitude r?

rf(r) =f(r)

xi=

r

xi

df

dr.

We used the chain rule of differentiation to break up the gradient, but how do we evaluate the quantityr/xi? Remember that r represents the magnitude of the position vector:

r

xi=

xi(xjxj)

1/2=

1

2 (xjxj)1/2

xi(xjxj) =

1

2r(ijxj + xjij) =

xir

.

Thus, we have two important formulas for taking gradients of functions which contain the position vectorand the magnitude of the position vector:

rx = I; rf(r) =x

r

df

dr.

In index notation, these two formulas are written as:

xjxi

= ij;

xif(r) =

xir

df

dr.

15


24/46

Sample Problems: Exercises

Intro to Index Notation. (2-4 in Leal) Write the following expressions using index notation

1. A =r u (A is a scalar, u is a vector)

2. A =ru (A is a second-order tensor, u is a vector)

3. =r u ( and u are vectors)4. C = (x y) z (C, x, y, and z are vectors)

5. AT A x = AT b (A is a second-order tensor, x and b are vectors)

Vector Differential Operator Identities. (2-1 in Leal) Prove the following identities, where is ascalar, u and v are vectors:

1. r (v) = r v + v r

2. r (v) = r v +r v3. r (u v) = v r u u r v4. r (u v) = v ru u rv + ur v vr u

Note: The following permutation symbol identity will prove useful:

jkijmn = kmin knim. (51)

Validity of Expressions in Index Notation. Are the following index notation expressions valid? If so,what is the corresponding Gibbs notation form? If not, whats wrong?

1. d = aiCijbj

2. xi = ijkajbk + aibjckdk

3. Cim = ijkjklklm

4. Ail = CijDjkEkl + ClmEmnDni

16


25/46

Sample Problems: Applications

Hinged Gate

A static fluid of density is contained behind a gate of height h and width W (in the z direction). The fluiddepth is also h. The free surface of the fluid is open to the atmosphere. We wish to determine the torqueper unit width that the fluid exerts on the hinge (See Figure 8).

1x

1y

h

g = g1y

Figure 8: Fluid of density is contained by a hinged gate. The depth of fluid and the height of the gate areh. The origin of the coordinate system is at the hinge.

1. Determine the pressure profile p (x,y,z) in the fluid. For hydrostatics, the governing equation for thepressure is:

0 = rp + g. (52)

2. Determine the net traction vector t (n) at any point on the gate. The traction vector is the force perunit area, and is given by:

t (n) n

. (53)In Equation, n is the outward-facing unit normal on the object, and is the fluid stress tensor. For astatic fluid, the fluid stress tensor is given by:

pI, (54)

where p is the fluid pressure and I is the second-order identity tensor.

3. The torque L on a body is given by:

L s

x t (n) dS, (55)

where x is the moment arm measured from the point of rotation. What is the torque per unit width

on the hinge?

17


26/46

4. Suppose the gate were hinged from the top rather than the bottom (See Figure 9). Before doing anywork, would you expect the torque per unit width to be larger or smaller than in the previous case?What is the torque per unit width in this case?

1x

1y

h

g = g1y

Figure 9: Gate is hinged from the top rather than the bottom. All other parameters are the same.

18


27/46

Crash Course in Linear Algebra

Jim Fakonas

August 28, 2011

1 Definitions

The following terms and concepts are fundamental to linear algebra. They are

arranged roughly in the order in which they would be introduced in a typicalintroductory course.

Matrix Product The matrix product of an m n matrix A and an n pmatrix B is the mp matrix AB whose (i, j)th element is the dot productof the ith row of A with the jth column of B:0

BBB@ A1 A2

... Am

1CCCA

| {z }mn

0BBBB@

| | | |

(BT)1 (BT)2 . . . (B

T)n

| | | |

1CCCCA

| {z }np=

0BBB@A1 (BT)1 A1 (BT)2 . . . A1 (BT)nA2 (BT)1 A2 (BT)2 . . . A2 (BT)n

.... . .

...Am (B

T)1 Am (BT)2 . . . Am (B

T)n

1CCCA| {z }

mp

.

Here, Ai is the ith row of A and (BT)j is the j

th column of B (i.e., thejth row of B transpose). More compactly,

(AB)ij =nX

k=1

AikBkj .

A special case of matrix multiplication is the matrix-vector product. Theproduct of an m n matrix A with an n-element column vector x is them-element column vector Ax whose ith entry is the dot product of the ith

row of A with x.

Vector Space A vector space is any set Vfor which two operations are defined:

1


28/46

Vector Addition: Any two vectors v1 and v2 in V can be added toproduce a third vector v1 + v2 which is also in V.

Scalar Multiplication: Any vector v in V can be multiplied (scaled)by a real number1 c 2 R to produce a second vector cv which is alsoin V.

and which satisfies the following axioms:

1. Vector addition is commutative: v1 + v2 = v2 + v1.

2. Vector addition is associative: (v1 + v2) + v3 = v1 + (v2 + v3).

3. There exists an additive identity element 0 in V such that, for anyv 2 V, v + 0 = v.

4. There exists for each v 2 V an additive inverse v such that v +(v) = 0.

5. Scalar multiplication is associative: c(dv) = (cd)v for c, d 2 R andv 2 V.

6. Scalar multiplication distributes over vector and scalar addition: forc, d 2 R and v1, v2 2 V, c(v1+v2) = cv1+cv2 and (c+d)v1 = cv1+cv2.

7. Scalar multiplication is defined such that 1v = v for all v 2 V.Any element of such a set is called a vector; this is the rigorous definitionof the term.

Function Space A function space is a set of functions that satisfy the aboveaxioms and hence form a vector space. That is, each vector in the spaceis a function, and vector addition is typically defined in the obvious way:for any functions f and g in the space, their sum (f + g) is defined as

(f + g)(x) f(x) + g(x). Common function spaces are Pn, the space ofnth-degree polynomials, and Cn, the space of n-times continuously dier-entiable functions.

Inner Product An inner product h , i on a real vector space V is a map thattakes any two elements of V to a real number. Additionally, it satisfiesthe following axioms:

1. It is symmetric: for any v1, v2 2 V, hv1, v2i = hv2, v1i.2. It is bilinear: for any v1, v2, v3 2 V and a, b 2 R, hav1 + bv2, v3i =

ahv1, v3i + bhv2, v3i and hv1, av2 + bv3i = ahv1, v2i + bhv1, v3i.3. It is positive definite: for any v 2 V, hv, vi 0, where the equality

holds only for the case v = 0.

These axioms can be generalized slightly to include complex vector spaces.An inner product on such a space satisfies the following axioms:

1More general definitions of a vector space are possible by allowing scalar multiplication

to be defined with respect to any arbitrary field, but the most common vector spaces take

scalars to be real or complex numbers.

2


29/46

1. It is conjugate symmetric: for any v1, v2 2 V, hv1, v2i = hv2, v1i,where the overbar denotes the complex conjugate of the expression

below it.

2. It is sesquilinear (linear in the first argument and conjugate-linearin the second): for any v1, v2, v3 2 V and a, b 2 C, hav1 + bv2, v3i =ahv1, v3i + bhv2, v3i and hv1, av2 + bv3i = ahv1, v2i + bhv1, v3i.

3. It is positive definite, as defined above.

The most common inner product on Rn is the dot product, defined as

hu, vi nXi=1

uivi for u, v 2 Rn.

Similarly, a common inner product on Cn is defined as

hu, vi nXi=1

uivi f or u, v 2 Cn

Note, however, that in physics it is often the first vector in the angledbrackets whose complex conjugate is used, and the second axiom above ismodified accordingly. In function spaces, the most common inner productsare integrals. For example, the L2 norm on the space Cn[a, b] of n-timescontinuously dierentiable functions on the interval [a, b] is defined as

hf, gi =Zba

f(x)g(x) dx

for all f, g

2Cn[a, b].

Linear Combination A linear combination of k vectors, v1, v2, . . . , vk, is thevector sum

S = c1v1 + c2v2 + + ckvk

for some set of scalars {ci}.

Linear Independence A set of vectors is linearly independent if no vector init can be written as a linear combination of the others. For example, thevectors (1, 1) and (1, 1) are linearly independent, since there is no way towrite (1, 1) as a scalar multiple of (1, 1). The vectors (1, 1), (1, 1), and(1, 0) are not linearly independent, however, since any one can be writtenas a linear combination of the other two, as in (1, 1) = 2 (1, 0) (1, 1).

Span The span of a set of vectors {v1, v2, . . . , vk}, denoted span(v1, v2, . . . , vk)is the set of all possible linear combinations of the vectors. Intuitively,the span of a collection of vectors is the set of all points that are reach-able by some linear combination of the vectors. For example, the vectorsv1 = (1, 0, 0), v2 = (0, 2, 0), and v3 = (0, 5, 5) span R3, since any three-component vector can be written as a sum of these three vectors using

3


30/46

the proper coecients. In contrast, the vectors v1, v2, and v4 = (1, 1, 0)span only the x-y plane and not all ofR3, since none of these vectors has

a component along the z direction.

Basis and Dimension A basis of a vector space is any set of vectors whichare linearly independent and which span the space. The vectors v1, v2,and v3 in the previous example form a basis for R

3, while the vectors v1,v2, and v4 do not form a basis of either R

3 (they do not span the space) orR2 (they are not linearly independent). There are infinitely many bases

for any given space (except for the trivial space, consisting only of thenumber 0), and all of them contain the same number of basis vectors.This number is called the dimension of the space.

Range/Image/Column Space The range of a matrix, also known as its im-age or column space, is the space spanned by its columns. Equivalently, it

is the set of all possible linear combinations of the columns of the matrix.For example, if one views an m n matrix as a linear transformationoperating on Rn, then the range of the matrix is the subspace ofRm intowhich it maps Rn.

Rank The rank of a matrix is the dimension of its column space (or that ofits row space, since it turns out that these dimensions are equal). Equiv-alently, it is the number of linearly independent columns (or rows) in thematrix.

Kernel/Null Space The kernel or null space of an m n matrix A, denotedker(A), is the set of all vectors that the matrix maps to the zero vector.In symbols,

ker(A) = {x

2Rn|Ax = 0}.

The following terms relate only to square matrices. They are arranged roughlyin order of conceptual importance.

Eigenvalues and Eigenvectors An eigenvector of a matrix A is a vector vwhich satisfies the equation

Av = v

for some complex number , which is called the corresponding eigenvalue.Note that might have a nonzero imaginary part even if A is real. (Seebelow for more information about eigenvalues and eigenvectors.)

Invertible An invertible matrix is any matrix A for which there exists a matrixB such that AB = BA = I, where I is the identity matrix of the samedimension as A and B. The matrix B is said to be the inverse of A,and is usually denoted A1: AA1 = A1A = I. A matrix which is notinvertible is called singular.

4


31/46

Diagonalizable A diagonalizablematrix A is any matrix for which there existsan invertible matrix Ssuch that S1AS = D, where D is a diagonal matrix

(i.e. all of its o-diagonal elements are zero). A square matrix which isnot diagonalizable is called defective.

Orthogonal (Unitary) An orthogonal matrix is any square matrix A withreal elements that satisfies AT = A1, so ATA = AAT = I. Equivalently,a real, square matrix is orthogonal if its columns are orthonormal withrespect to the dot product. A unitary matrix is the complex equivalent;a complex, square matrix is unitary if it satisfies A = A1 (where A AT), so AA = AA = I.

Symmetric (Hermitian) A symmetric matrix is any real matrix that satis-fies AT = A. Similarly, a Hermitian matrix is any complex matrix thatsatisfies A = A.

Normal A normal matrix is any matrix for which ATA = AAT (or AA =AA for complex matrices). For example, all symmetric and orthogonalmatrices are normal.

Positive and Negative Definite and Semidefinite A positive definite ma-trix is any symmetric or Hermitian matrix A for which the quantityvTAv 0 for all v, with the equality holding only for the case v = 0.If the inequality holds but there is at least one nonzero vector v for whichvTAv = 0, then the matrix is called positive semidefinite. Likewise, a neg-ative definite matrix is any symmetric or Hermitian matrix A for whichvTAv 0 for all v, with the equality holding only for v = 0. IfvTAv 0for all v but vTAv = 0 for at least one nonzero v, the matrix A is callednegative semidefinite.

2 Important Concepts

This section contains brief explanations of several essential concepts in linearalgebra. It is intended for students who are familiar with the mechanics of linearalgebra but who might not have had a full course in the subject.

2.1 The Meaning ofAx = b

One of the most important equations in linear algebra is the simple matrixequation Ax = b. In this equation, A is a given m n matrix, b is a givenvector in Rm, and the problem is to solve for the unknown vector x in Rn. This

equation arises wherever one must solve m linear equations for n unknowns.Notice that the matrix-vector product Ax on the left-hand side is nothing

other than a linear combination of the columns of A with coecients given by

5


32/46

the elements of x:

0BBB@A11 A12 . . . A1nA21 A22 . . . A2n

.... . .

...Am1 Am2 . . . Amn

1CCCA0BBB@

x1x2...

xn

1CCCA =0BBB@

A11x1 + A12x2 + + A1nxnA21x1 + A22x2 + + A2nxn

...Am1x1 + Am2x2 + + Amnxn

1CCCA

= x1

0BBB@

A11A21

...Am1

1CCCA + x2

0BBB@

A12A22

...Am2

1CCCA + + xn

0BBB@

A1nA2n

...Amn

1CCCA .

That is, the question of whether or not Ax = b has a solution is essentially thequestion of whether or not the vector b lies in the space spanned by the columnsofA. There are three possibilities: 1) it does not, in which case there is no x for

which Ax = b; 2) it does, and there is one and only one x that solves Ax = b;or 3) it does, and there are infinitely many vectors that solve Ax = b. In case(1), no matter how the columns of A are weighted, they cannot sum to give thevector b. In case (2) the columns ofA are linearly independent and b lies in theirspan. Finally, in case (3), the columns of A are linearly dependent and b lies intheir span, so there are infinitely many linear combinations of them which sumto give b.

The conceptual power of this interpretation is that it lends a geometricsignificance to the algebraic equation Ax = b. One can picture the n columns ofA as vectors in Rm. Together, they span some space, which could be all ofRm

or only some proper subspace. In the former case, the vectors can be combined,with an appropriate choice of scalar coecients, to produce any vector in Rm.In the latter case, in contrast, there are some vectors Rm that lie outside of thespan of the columns of A. If the vector b in Ax = b happens to be one of thosevectors, then no possible linear combination of the columns of A can reach it,and the equation Ax = b has no solution.

2.2 The Eigenvalue Equation, Av = v

Another ubiquitous and fundamentally important equation in linear algebra isthe relation Av = v, where A is an n n matrix (notice that it must be squarein order for the left- and right-hand sides to have the same dimension), v is ann-element vector, and is a constant. The task is to solve for all eigenvectorsv and corresponding eigenvalues that satisfy this relation.

Rearranging the eigenvalue equation gives

(A I)v = 0, (1)where I is the n n identity matrix. The only way for this equation to havea nontrivial solution is for the matrix (A I) to be singular, in which caseits determinant is zero. This fact provides a useful method for finding the

6


33/46

eigenvalues of small matrices. For example, to find the eigenvalues of

A = 1 3

3 1

,

first construct the matrix (AI) and then require that its determinant vanish:

det(A I)= det

1 33 1

= (1 )2 9 = 0

! ( 2)( 4) = 0

The eigenvalues ofA are therefore 1 = 2 and 2 = 4. To find the eigenvectors,substitute these numbers into (1). For example,

1 1 3

3 11 v1 = 0

3 33 3

v(1)1

v(2)1

!=

00

! v(2)1 = v(1)1 ! v1 = v(1)1

11

Thus, v(2)1 = v(1)1 and therefore the eigenvector v1 corresponding to 1 is

v1 = v(1)1

1

1

,

where v(1)1 is an arbitrary scalar (note that A(cv1) = 1(cv1), so eigenvectors

are only unique up to a multiplicative constant).But what does it mean for a vector to be an eigenvector of a matrix? Is therea geometric interpretation of this relationship similar to that given for Ax = b inthe previous section? It turns out that there is. It is helpful to think of squarematrices as operators on vector spaces, as any linear transformation on Rn (orCn) can be represented by an n n matrix. Examples of such transformations

include rotations, reflections, shear deformations, and inversion through theorigin. The eigenvectors of such a matrix, then, are special directions in thatspace that are unchanged by the transformation that the matrix encodes. Avector that points in one of these directions is scaled by a constant factor (theeigenvalue, ) upon multiplication by the matrix, but it points along the sameaxis as it did before the matrix operated on it.

This geometric interpretation is especially useful when zero is an eigenvalue

of a matrix. In that case, there is at least one vector (there can be more) that,when multiplied by the matrix, maps to the zero vector. Let A be such a matrix,and let v0 be an eigenvector ofA whose eigenvalue is zero. Then, Av0 = 0v0 = 0.Likewise, any scalar multiple cv0 also maps to zero: Acv0 = cAv0 = c0 = 0. Asa result, an entire axis through Rn (here A is taken to be n n and real, asan example) collapses to the zero vector under the transformation represented

7


34/46

by A, so the image of A is a lower-dimensional subspace ofRn. For example,if A is a 3

3 real matrix with one eigenvector whose eigenvalue is zero, then

there is a line in R3 that collapses to zero under multiplication by A, and theimage of A is therefore a two-dimensional plane in R3. As a result, any vectorb that lies outside of this plane lies outside the span of A, so there is no vectorx that solves Ax = b. This is why a matrix is singular if one of its eigenvaluesis zero; if it were invertible (i.e. not singular), there would always exist such anx (equal to A1b) for every b.

3 Exercises

1. Which of the following sets are vector spaces?

(a) C3, the set of all ordered triples of complex numbers, with scalar

multiplication defined over the complex numbers(b) C3, with scalar multiplication defined over the real numbers

(c) R3, the set of all ordered triples of real numbers, with scalar multi-plication defined over the complex numbers

(d) The subset ofR2 enclosed by the unit circle, with scalar multiplica-tion defined over the real numbers

(e) The line y = 4x (i.e. the subset ofR2 comprised by the points onthis line), with scalar multiplication over R

(f) The line y = 4x + 2, with scalar multiplication over R

(g) The subset ofR3 bounded above by the plane z = 10, with scalarmultiplication over R

(h) The functions f(x) = x3, g(x) = cos(px), and h(x) = 1, wherex 2 [0, 1], and all linear combinations of these functions with realcoecients, with function addition and scalar multiplication definedas usual

2. Consider the vectors

v1 =

0@ 11

2

1A , v2 =

0@ 30

1

1A , and v3 =

0@ 12

3

1A .

Are these vectors linearly independent? What is the dimension of thespace they span? Use two methods to answer these questions:

(a) Recall that if these vectors are linearly independent, then none canbe written as a linear combination of the others. As a direct result,the only solution to the equation

c1v1 + c2v2 + c3v3 = 0 (2)

8


35/46

is the trivial solution, c1 = c2 = c3 = 0. Write (2) in matrix formand solve it using substitution, row reduction, or MATLAB (make

sure you are at least familiar with the other methods before you useMATLAB, though). Is the trivial solution the only solution?

(b) Notice that the matrix that you constructed in (2a) is square. Findits eigenvalues. What do they tell you about the vectors v1, v2, andv3? (Hint: What is the null space of this matrix?)

3. Give an example of a basis for each of the following vector spaces. Whatis the dimension of each?

(a) C4, the space of all ordered quadruples of complex numbers

(b) P3, the space of all polynomials of degree less than or equal to threethat have real coecients

(c) M22

, the space of all 2 2 matrices with real elements(d) C2[0, 1], the space of all twice-dierentiable functions defined on the

interval [0, 1]

4. Find the eigenvalues and eigenvectors of the following matrices.

(a) 0@ 3 0 20 0 1

2 1 2

1A

(b)

cos sin sin cos

5. Put the following system of equations into matrix form and solve for x, y,and z.

2x + 3y + z = 4

x + 3y + z = 1

x + 2z = 2

9


36/46

4 Appendix: Venn Diagram of Matrix Types

1. The following statements about a matrix A are equivalent:

(a) A is invertible.

(b) The equation Ax = b always has a unique solution.

(c) Zero is not an eigenvalue ofA.

(d) detA 6= 0(e) A is square and has full rank.

2. PSD: Positive Semidefinite

PD: Positive DefiniteNSD: Negative SemidefiniteND: Negative Definite

10


37/46

COMPLEX ANALYSIS EXAMPLES

1. Introduction

Complex analysis is the study of functions f so that f(z) is a complex number for everycomplex number z. We will generally restrict our attention to functions f that are complexdierentiable; also called analytic functions or holomorphic functions. Such functions havemany nice properties, some of which are listed in Section 3. The power of complex analysis asa tool in the physical sciences comes from using the methods of complex analysis to calculatedefinite integrals (see Section 6).

As another motivation, consider the problem of the harmonic oscillator. In the under-damped case, the motion can be described as

x(t) = Aet cos(t + ) or x(t) = Aet sin(t + )

for an appropriate choice of constants. It is often more helpful to think about this as the

real or imaginary parts of the expression

e+(+i)t

for an appropriate choice of constants. Indeed, the way we solve the harmonic oscillator isby guessing a solution of the form Aeat and adjusting the constants A and a so that thedierential equation is solved. This is impossible without complex numbers. We see thenthat even though the solution is real valued, we must use complex functions to arrive at thesolution.

As another example, in quantum mechanics, one solves a dierential equation for a wavefunction in either position space or momentum space. To go between the two, one uses theFourier Transform, often written as

f(x) =

Z1

1

e2ixtf(t)dt.

Such integrals are often dicult to compute, but the hardest examples require moving thepath of integration from the real axis to another curve in the plane. Why can we get awaywith this? Unfortunately the answer is not simple, but it can be understood once we knowsomething about analytic functions. A challenging example is given in the exercises section.

As a final motivation example, consider the problem of finding the equilibrium state of aninsulated heated metal plate. More specifically, suppose we are given a metal plate with areasonable boundary (i.e. no fractal boundaries) as in the picture.

Suppose also we have set up a very elaborate system of heaters an coolers so that we cancontrol the temperature on the boundary of the plate. If we assume the plate is insulated(i.e. no heat is lost), then as time tends to infinity, the temperature distribution insidethe plate will approach an equilibrium state. Thinking of the plate as a region in R2 andthe equilibrium distribution of temperature as a function f(x, y), then f satisfies the HeatEquation

@2f

@x2+

@2f

@y2= 0.

1


38/46

2 COMPLEX ANALYSIS EXAMPLES

Figure 1. A metal platewith smooth boundary.

Figure 2. A metal platewith non-smooth (but stillreasonable) boundary.

Of course the function f depends on how we set the temperature at the boundary. Thefunction f is known as the solution to the Dirichlet Problem with given boundary data.

Solving this problem when the plate is the shape of a disk with radius 1 is easy. If theboundary temperature is a fixed function g(ei) then the equilibrium temperature at a pointz inside the disk is

f(z) =

Z20

g(ei)1 |z|2

|ei z|2d

2.

Notice that the function g does not even have to be continuous for this integral to makesense.

Now the obvious question arises: what if the plate is not in the shape of a disk with radius1? It turns out that complex analysis provides the right tool for this problem. We mustappeal to the Riemann Mapping Theorem. This result essentially asserts the existence of afunction that allows us to start with our plate, deform it until it is a disk of radius 1, solve

the problem on the disk of radius 1 as above, and then deform the disk back to the shapeof the original plate. Such a function is called a conformal map from the plate to the disk.Proving this function exists is not easy, but is a standard part of a first course in complexanalysis. Actually finding a formula for the map is very dicult (for a general plate), thoughthere are methods available to estimate it.

The remainder of these notes are meant to serve as a reminder of some of the basicterminology, notation, and background necessary to hit the ground running when you firstencounter complex analysis at Caltech (whether in a course or in research). This set of notesis not a prerequisite for any course at Caltech; it is meant only to serve as a reminder of whatyou might have forgotten or as a primer for what you are about to learn. The fourth sectionof the Practice Problems requires more than the material presented here to solve. They are

included as examples of the kinds of things one can calculate with complex analysis tools.

2. Useful Formulas

Let us begin with a quick review of some basic concepts and notation. Remember thatthe complex number i is defined so that i2 = 1. Now every complex number can be writtenuniquely as z = x + iy where x and y are real numbers. One can also express z in polarco-ordinates as z = rei where r =

px2 + y2 is the norm or modulus of z and is the


39/46

COMPLEX ANALYSIS EXAMPLES 3

argument or phase of z. The expression of z in polar co-ordinates is not unique! Indeed1 = e2i = e4i = e6i = e8i = . This comes from the fact that the argument of a complexnumber is not a well-defined quantity. We will discuss this more in Section 4. However, onecan always define arg(x + iy) = arctan(y/x) when x and y are both positive.

To go the other way, when one is given a complex number in polar co-ordinates, one canwrite it as x + iy by setting x = r cos() and y = r sin(). This is a result of the famous

Euler (pronounced Oiler) formulaeix = cos(x) + i sin(x).

The complex conjugate of x + iy is x iy. The complex conjugate of rei is rei. Ingeneral, when taking the complex conjugate of an expression (no matter how complicated),one simply changes every instance of an i to a i. To multiply two complex numbers, onewrites

(x + iy)(u + iv) = xu yv + i(yu + xv) ,

reit

Reis

= rRei(t+s).

To divide complex numbers, one has to make the denominator real by multiplying the topand bottom by the conjugate of the denominator. We get

u + ivx + iy

= ux + vyx2 + y2

+ ivx uyx2 + y2

.

3. Analyticity

An analytic function can be characterized in many dierent ways. One way is to say thata function f is analytic in a region if every point w 2 is the center of an open disk of(possibly infinite) radius R > 0 completely contained in so that for every z in this diskone has

f(z) =1Xn=0

an(z w)n.

The main idea is that insisting that a function of a complex variable be analytic is actu-ally a strong condition and so we can make some very strong conclusions. Here are someconsequences of analyticity: if f is analytic in a region then...

(1) f is infinitely dierentiable;(2) if is a smooth curve whose interior is inside thenI

f(z)dz = 0;

(3) ifC is a circle whose interior is in then

f(n)(z) =n!

2iIC

f()

( z)n+1

d

for z inside C, where f(n) denotes the nth derivative of f;(4) ifD is a closed disk inside then

supz2D

|f(z)| = supz2@D

|f(z)|.

This property is called the Maximum Modulus Principle;(5) f() := {w : w = f(z) for some z2 } is an open set;


40/46


(6) if the closed disk centered at z of radius R is contained inside then

f(z) =

Z20

f(z+ Rei)d

2.

This is called the Mean Value Property for analytic functions.(7) As a consequence of (6), we know that u(x, y) = Re[f(x+iy)] and v(x, y) = Im[f(x+

iy)] are both harmonic functions, i.e.

@2u

@x2+

@2u

@y2= 0

and similarly for v.

Example. An example of a seemingly nice function that is not analytic is the functionf(z) = z. If we think of this as a function of two real variables x and y and think of fas f(x, y) = (x,y) then the function f is continuous and in fact infinitely dierentiablein both coordinates and has smooth gradient. However, f is not an analytic function of acomplex variable.

Example. Consider the function f(z) = (1

z)1. This function is analytic in the entire

complex plane except at the single point 1. We can therefore set = C \ {1} so that 0 isthe center of an open disk of radius 1 that is completely contained inside . In that disk,we write

1

1 z =1Xn=0

zn.

It is important to note that this series does not converge when z is not inside the disk ofradius 1 centered at 0.

Let us examine how to find the series representation for f(z) around a point other than0; for example around the point 3. In this case, we have

1

1 z =1

1 (z+ 3) + 3 =1

4 (z+ 3) =1

4

1

1 (z+ 3)/4 =1

4

1

Xn=0

z+ 34

n

,

which is a Taylor Series that converges for all z in an open disk centered at 3 and of radius4.

4. Complex Logarithm

The logarithm of a positive number x can be easily defined by

ln(x) =

Zx1

1

tdt.(4.1)

This is a simple definition that requires no knowledge of the exponential function or the

number e. However, for complex numbers we can define a logarithm, but it will not be aseasy. Consider the following line of reasoning: Let zbe a complex number. We want to definelog(z) to be a complex number w so that ew = z. However, such a choice of w is not unique.For example, one can set log(1) = 0 or log(1) = 2i. Furthermore, we want our logarithm tobe an inverse to the exponential function. It makes sense to want to define log(ez) = z for allcomplex numbers z. However, in this case we would define log(e2i) = 2i and log(e0) = 0,but you will notice that we just defined log(1) in two dierent ways. Obviously we havesome diculties to overcome.


41/46


To get around this, we make what is called a branch cut. A branch cut is a curve that hasone end at 0 and never intersects itself and tends to infinity. This curve helps us define thelogarithm by telling us where we will introduce the discontinuity for the logarithm function.We then define

log(z) = ln(|z|) + i arg(z)

where the ln(|z|) term is to be interpreted as in equation (4.1). The argument functionobviously has a discontinuity though we have much flexibility as to where we introduce thisdiscontinuity. The set of points at which we let the arg function be discontinuous is whatwe call our branch cut. A common choice of branch cut is along the positive real axis. Inthis case arg(z) will always lie between 0 and 2. Another common choice is the negativereal axis, where the arg function takes values between and .

Figure 3. A graph of themulti-valued functionarg(z).

Figure 4. The graph ofthe single-valued functionwith a discontinuity alongthe branch cut.

In order to make the arg function continuous in the whole plane, we would need to usesome very advanced complex analysis that is the beyond the scope of these notes. Figure 3though shows what is happening. As we travel (counterclockwise) around 0, the arg function

increases. In order to make it continuous, we would need to let it keep increasing as we movearound and around the origin. This would lead to the arg function being multi-valued,which causes problems. To get around this, we take a section of this spiral on which thearg function is single-valued. The price we pay is that it is no longer a continuous function.Figure 4 shows such a section where the branch cut is a straight line (i.e. the set of pointsalong which the graph is discontinuous is a straight line), but a branch cut can in generalbe any curve that does not intersect itself, starts at 0, and goes out to infinity.

Another possible choice of branch cut is shown in Figure 5.


42/46


-150 -100 -50 50 1 00 150

-100

-50

50

100

150

Figure 5. A possible branch cut for the logarithm.

With the choice of branch cut from Figure 5, we would write

log(1) = 0,

log(10i) = i

2,

log(10) = ln(10) + i,

log(50i) = ln(50) + i 72

,

log(100) = ln(100) + 6i,

log(100i) = ln(100) + i13

2,

and so on. You can see that as we work our way around the spiral, the argument functionis increasing, but if we cross the spiral we see a discontinuity in the argument function. Forexample, on the segment of the positive real axis that includes 1 and has endpoints lyingon the spiral, the argument function takes the value 0. On the segment of the positive realaxis that includes 25 and has endpoints lying on the spiral, the argument function takes thevalue 2i. On the segment of the positive real axis that includes 50 and has endpoints lyingon the spiral, the argument function takes the value 4i. Therefore, if we define to bethe open set that is the complex plane C with the pictured spiral removed, then log(z) is ananalytic function of z on .

Now we can define fractional powers of a variable z by using the logarithm.

Example. It is obvious that 12 = (

1)2 = 1 so how do we define

p1? In general, we define

pz = elog(z)/2. Therefore, we need to choose a branch cut for the logarithm to define thesquare root function. If we choose the branch cut along the negative real axis, then fromour discussion above it becomes clear that

p1 = 1. With this same branch cut, one haspi = ei/4. However, if we choose our branch cut to be along the positive real axis, thenpi = e3i/4.

As a final word of warning, because of this branch cut phenomenon, when dealing withthe complex logarithm, IT IS NO LONGER TRUE THAT log(zw) = log(z) + log(w).


43/46


5. Contour Integrals

We want to use complex analysis as a tool for evaluating dicult integrals. In order todo this we need to first review how to calculate a contour integral. A contour is a closedcurve that does not intersect itself (also called a simple closed curve). A contour divides thecomplex plane into two regions; one of them is bounded and the other is unbounded. Wecall the bounded region the interior of the contour. Here are some important points to keep

in mind:(1) Iff is analytic in an open set containing a contour and its interior thenI

f(z)dz = 0.

(2) If1 and 2 are two contours with 1 contained in the interior of 2 (with perhapssome overlap with 2) and f is analytic in the region between 1 and 2 thenI

1

f(z)dz =

I2

f(z)dz.

(3) If is the contour but traversed in the opposite direction thenI

f(z)dz = I

f(z)dz.

If no orientation is specified on a contour, it is assumed that the contour is given the counter-clockwise orientation. This means that if you were standing on the contour and walking inthe direction of the orientation then the interior of the contour would always be on your lefthand side.

Example. Let us calculateR

1z

dz where is the circle of radius 1 centered at 0. We write

z = ei where 2 [0, 2]. We calculate dz = ieid. We can therefore write our integral asZ

2

0

1ei

ieid = iZ

2

0

d = 2i.

Of course this does not depend on our parametrization, but lets just check to make sure. Ifwe instead parametrize the contour by z = e2i for 2 [0, 1] then dz = 2ie2id and soour integral becomes Z1

0

1

e2i2ie2id = 2i

Z10

d = 2i.

If we wanted to calculate the same integral around the same contour but with the oppositeorientation (i.e. clockwise) then (3) above tells us that the answer would be 2i.

Sometimes we are presented with a contour integral that is too hard to calculate, so weneed to estimate it. One useful estimate is sometimes called the ML-bound. The ML-boundallows us to put an upper bound on the magnitude of a contour integral in terms of thefunction being integrated and the length of the contour. More precisely, suppose we areasked to integrate

R

f(z)dz where f is some function that is analytic in a neighborhood ofthe contour . Let

L = length() , M = maxz2

|f(z)|.


44/46


Then Z

f(z)dz

ML.Applying this to the above example, we see that L is the length of the circumference of acircle of radius 1 so L = 2. The function 1/z has absolute value 1 whenever z is a point inthe unit circle, so in this case M = 1. Therefore, the ML-bound implies

Z{|z|=1}

1z

dz 2,

which the example calculation showed is indeed the case.

6. Further Topics

Ultimately we study complex analysis as a computational tool. The goal is to be able tocalculate things like

R1

1

dx1+x4

. One can also use complex analysis to find roots of analyticfunctions. Here are some key words and theorems that you might keep an eye out for as youlearn more complex analysis:

(1) The Residue Theorem: The key to calculating hard definite integrals.

(2) Rouches Theorem: Finds zeros of functions close to roots of other functions.

(3) The Argument Principle: Useful for counting zeros and poles.

(4) The Riemann Mapping Theorem: Useful for mapping arbitrary simply connectedregions to a disk.

(5) Cauchy Estimates: Restricts how fast an analytic function can grow as |z| !1.(6) Paley-Wiener Theorem: Relates analytic functions to their Fourier Transform.

(7) Schwartz Reflection Principle: One very common method of analytic continuation.

(8) Runges Theorem: Describes when an analytic function can be uniformly approxi-mated by polynomials.

(9) Montels Theorem: Describes when a sequence of holomorphic functions has a uni-formly convergent subsequence.


45/46


7. Practice Problems

7.1. Analyticity. A complex analytic function can be expressed in many dierent ways.However, some formulas are only valid in certain regions of the plane. Find the radius ofconvergence of the following Taylor series:

(1) P1

n=0 2nzn

(2)P1

n=0 nzn

(3)P1

n=0zn

n!

(4)P1

n=0 2n! zn!

n!

7.2. Complex Logarithm. The logarithm is the most common example of a function thatcan be locally defined as analytic, but requires a branch cut to be defined locally. Further-more, we need the logarithm to define fractional powers of a variable z. Express the following

as complex numbers, where we assume the logarithm is defined with branch cut along thenegative real axis:

(1) log(1)

(2) log(e2i)

(3) log(2i)

(4)p

i

7.3. Contour Integrals. Contour integrals arise frequently when complex analytic func-tions are involved. Calculate the following integrals where is the curve indicated:

(1)H

z2dz where is the circle with center 0 and radius 1 traversed counterclockwise.

(2)H

1zdz where is the circle with center 0 and radius 1 traversed counterclockwise.

(3)H

1z2+1dz where is the circle with center 0 and radius 2 traversed counterclockwise.

(4)H

1z2+1dz where is the circle with center 0 and radius

12 traversed counterclockwise.

7.4. Definite Integrals (Optional). Our primary motivation for studying complex anal-ysis is because of its utility as a computational tool. In particular, one can use the ResidueTheorem to calculate some otherwise very dicult integrals of common functions. After afull course in complex analysis, you should be able to calculate the following definite inte-grals. If you cant, you should just remember that they can be calculated using tools fromcomplex analysis, so that if you ever need to calculate such an integral, just consult yournearest complex analysis textbook or expert.


46/46


(1)R1

1

dx1+x4

(2)R1

1

e2ixt

cosh(x)dx (i.e. calculate the Fourier Transform of 1/ cosh(x))

References[1] L. Ahlfors, Complex Analysis, Third Edition, McGraw-Hill, 1979.[2] M. Homan and J. Marsden, Basic Complex Analysis, Third Edition, W. H. Freeman, 1998.[3] S. Lang, Complex Analysis, Fourth Edition, Springer Graduate Texts in Mathematics, 1998.[4] E. B. Sa and A. Snider, Fundamentals of Complex Analysis with Applications to Engineering, Science,

and Mathematics; Third Edition, Prentice-Hall, 2003.[5] R. Shakarchi and E. M. Stein, Complex Analysis, Princeton University Press, Princeton, NJ, 2003.

Documents

Bootcamp Math Workbook v1_0