Math Boot Campmath.bu.edu/people/mrmorse/bootcamp/10_MultivariateCalculus.pdf · Math Boot Camp 15: Multivariate Calculus 15.1: Functions of Several Variables 15.1: Functions of Several

Math Boot Camp

Math Boot CampLecture 10: Multivariate Calculus

Matthew Morse

8/31/2018

Math Boot Camp

15: Multivariate Calculus

15.1: Functions of Several Variables


In our discussion of calculus, we have limited our results tofunctions of a single variable. We have talked about derivativesand integrals of functions written like f (x).

We have briefly mentioned multivariable functions, and we wouldlike to extend calculus concepts to these multivariable functions.Before we do this, we will further develop the concepts andnotation of multivariable functions.

Math Boot Camp




We have discussed multivariable functions in the context of usingmultiple predictor variables to estimate a response variable. Forexample, if we want to use demographic information of acommunity to estimate the proportion of eligible voters whoactually voted in the last election, our predictor variables might be

x : per capita income

y : total population

and we look for a function that allows us to estimate theproportion of voters based on these predictors:

f (x , y) : proportion of voters

Math Boot Camp




Functions of several variables can be linear functions, for example

f (x , y) = 8x + 15y .

They can also be nonlinear functions.

Nonlinear functions

f (x , y) = xy

f (x1, x2) = x21 + 3x1 − 2x22 + 7

f (x , y , z) = ex+2y (3x + ln z)

Math Boot Camp



15.1: Functions of Vectors

In cases where we only have a few variables, it may make sense towrite them all out explicitly in the name of the function, likef (x , y) or f (x1, x2, x3). As the number of variables increases, itmakes more sense to write the input variables as a vector,

x = (x1, x2, . . . , xn).

Then we can write the function as a function of that vector, f (x).

Math Boot Camp



15.1: Vector valued functions

To further complicate things, we may have several responsevariables which we estimate based on the same set of predictorvariables. In addition to the proportion of the population whichvoted in the last election, we may also be interested in theproportion of voters for a particular candidate. We can write thisas two functions:

f1(x , y) : proportion of the population which voted

f2(x , y) : proportion of voters for a particular candidate

Just as we can consider the predictor variables as a vector, we canconsider the functions as a vector, f(x , y). This vector has twocomponents. The first component is the function f1(x , y), and thesecond component is the function f2(x , y).

Math Boot Camp



15.1: Matrix multiplication as a vector function

We can also write this as a vector function of a vector, f(x).

Recall that we wrote a system of linear equations as the matrixequation

Ax = b.

We can think of the vector b as a function of the vector x, sob = f(x), where f(x) = Ax.

Math Boot Camp


15.2: Calculus in Several Dimensions

15.2.1: Marginal change

For a function of one variable, f (x), the derivative ddx f (x) tells us

the rate of change of the function. Equivalently, it tells us theslope of the tangent line to the function. For functions of two ormore variables, we must be more precise when we talk about therate of change.

As an example, consider the function f (x , y) = x2 + y2. Whenconsidering how the value of the function changes, we need to askwhether x is changing, or y is changing, or both.

We start by considering the change when only one variable changesand all other variables are held fixed. If we start at a specific point,say (x , y) = (3, 4), we can ask about the change in the value ofthe function when x is allowed to change but y is held at 4. Thisis often called the marginal change of the function.

Math Boot Camp



15.2.1: Partial Derivatives

When we compute the instantaneous marginal rate of change of afunction (with respect to a particular variable), we call this thepartial derivative and write

∂

∂xf (x , y) or ∂x f (x , y)

The ∂ symbol is called the partial differential symbol, just like d isthe differential symbol for functions of one variable.

Since f (x , y) is a function of two variables, we must specify whichvariable we are allowing to change. ∂

∂x f (x , y) is the partialderivative with respect to x , so we allow x to change and hold yfixed. Similarly, ∂

∂y f (x , y) is the partial derivative with respect toy .

Math Boot Camp



15.2.1: Computing Partial Derivatives

To compute a partial derivative, treat all other variables asconstants, and compute the derivative following the normal rules ofdifferentiation.

Examples of Partial Derivatives

∂

∂x(x2 + y2) = 2x

∂

∂y(x2 + y2) = 2y

∂

∂xxy = y

∂

∂yxey = xey

Math Boot Camp



15.2.2: Gradients

Notice that a function of several variables has as many partialderivatives as it has variables. The function f (x1, x2, x3) has threepartial derivatives,

∂

∂x1f (x1, x2, x3) ,

∂

∂x2f (x1, x2, x3) , and

∂

∂x3f (x1, x2, x3).

Just as it is natural to collect several variables as a vector,x = (x1, x2, x3), it is natural to collect the partial derivatives of afunction into a vector, called the gradient. We write

∇f (x) =

(∂

∂x1f (x),

∂

∂x2f (x),

∂

∂x3f (x)

).

The symbol ∇ is called “nabla” or “del”, and it means thegradient, or the vector of partial derivatives.

Math Boot Camp



15.2.2: The Chain Rule in Several Dimensions

It is often useful to apply the chain rule in several dimensions.Suppose that we have a function of two variables, f (x , y), whereeach of those variables is in turn the function of another variable,t. The partial derivatives of f tell us how the value of the functionchanges as each variable changes, but we may be interested in howthe value of the function changes as both variables changetogether as t changes. We can write the derivative of f withrespect to t using the chain rule.

d

dtf (x , y) =

(∂

∂xf (x , y)

)dx

dt+

(∂

∂yf (x , y)

)dy

dt

Math Boot Camp



15.2.2: Example of Chain Rule

In our example where x is the per capita income, y is the totalpopulation, and f (x , y) is proportion of voters for a city, we maybe interested in the change in voting behavior over time, where thechange in voting is driven by the change in income and change inpopulation over time. Then

∂

∂xf (x , y) : change in voters due to change in income

∂

∂yf (x , y) : change in voters due to change in population

dx

dt: change in income over time

dy

dt: change in population over time

Math Boot Camp



15.2.2: Example of Chain Rule (Cont.)

Putting these together,

d

dtf (x , y) =

(∂

∂xf (x , y)

)dx

dt+

(∂

∂yf (x , y)

)dy

dt

is the change in voters over time.

Math Boot Camp


15.3: Derivatives of Multidimensional Functions

15.2.3: Derivatives of Multidimensional Functions

When we consider vector-valued functions, we can computederivatives of each function in the vector. If we have

f(x) = (f1(x1, x2), f2(x1, x2))

we can compute the partial derivative of either f1 or f2, and we cancompute either with respect to x1, or x2.The four partial derivatives are written

∂

∂x1f1(x1, x2)

∂

∂x2f1(x1, x2)

∂

∂x1f2(x1, x2)

∂

∂x2f2(x1, x2)

Math Boot Camp


15.3: Derivatives of Multidimensional Functions

15.2.3: The Jacobian matrix

Just as we collected the partial derivatives of a real-valuedfunctions as a vector, the gradient, we can collect the partialderivatives of a vector-valued function as a matrix. This is calledthe Jacobian matrix and written

J =

[∂∂x1

f1∂∂x2

f1∂∂x1

f2∂∂x2

f2

]

In a bit of confusing terminology, the determinant of the Jacobianmatrix is called the Jacobian. The matrix J is the the Jacobianmatrix. The determinant |J| is the Jacobian.

Math Boot Camp


15.2.4: Second-Order Derivatives


Just like we can compute the second-order derivative of a functionof one variable, we can compute the second-order partialderivatives of a function of several variables. For a function of twovariables, f (x , y), the partial derivatives

∂

∂xf (x , y) and

∂

∂yf (x , y)

are themselves functions of x and y , so we can compute the partialderivative of either partial derivative with respect to either variable,resulting in four second-order derivatives, written,

∂2

∂x2f (x , y)

∂2

∂y∂xf (x , y)

∂2

∂x∂yf (x , y)

∂2

∂y2f (x , y)

Math Boot Camp



15.2.4: Example of Second-Order Derivatives

Suppose f (x) = 3x2 + 2xy + 5y2. Then

∂

∂xf = 6x + 2y

∂2

∂x2f = 6

∂2

∂y∂xf = 2

∂

∂yf = 2x + 10y

∂2

∂x∂yf = 2

∂2

∂y2f = 10

Math Boot Camp



15.2.4: Mixed Second-Order Derivatives

A second-order partial derivative like ∂2

∂y∂x f (x , y) or ∂2

∂x∂y f (x , y) iscalled a mixed second-order derivative.

FACT: The value of a mixed second-order partial derivative doesnot depend on the order in which the partial derivatives arecomputed. That is,

∂2

∂y∂xf (x , y) =

∂2

∂x∂yf (x , y)

This equality holds for all “reasonable” functions.

Math Boot Camp



15.2.4: The Hessian Matrix

Once again, we can collect all of the second-order partialderivatives into a matrix, which we call the Hessian. For a functionof two variables, f (x , y), the Hessian H is given by

H =

[∂2

∂x2f ∂2

∂x∂y f∂2

∂y∂x f∂2

∂y2 f

]

Math Boot Camp


15.3: Concavity and Convexity

15.3: Concavity and Convexity

For a function of one variable, the second derivative, f ′′(x) tells uswhether the function is convex (curving up) or concave (curvingdown).

For functions of more than one variable, the situation is morecomplicated. A function which looks like a bowl is convex. Afunction which looks like a hill is concave. On the other hand, afunction which looks like a saddle is neither concave nor convex.

Just as we use the second derivative to determine if a function ofone variable is concave or convex, we can use the Hessian todetermine if a function of several variables is concave or convex.This requires looking at the eigenvalues of the matrix, which is amatrix property which we have not discussed.

Documents

Math Boot Campmath.bu.edu/people/mrmorse/bootcamp/10_MultivariateCalculus.pdf · Math Boot Camp 15: Multivariate Calculus 15.1: Functions of Several Variables 15.1: Functions of Several