Chapter 2 Mathematics of Optimization

• Many economic concepts can be expressed as functions (eg., cost, utility, profit),

– To be optimized (maximized or minimized), – Subject to or not subject to constraints (eg.,

income or resource constraints). • Optimizing a function of one variable

(unconstrained):

Maximization (one variable and no constraint)

• To maximize π=f(q), look at changes in π as q increases.

• As long as Δ π > 0, π increases.

• Can express Δ’s as a ratio ; could be + or -.

• Limit of ratio as Δq 0 is derivative of π=f(q).

• For q<q*, derivative >0. • For q>q*, derivative <0. • At q*, derivative = 0.

ΔqΔΠ

∆q>0 q* ∆q>0 q

∆Π<0

∆Π=0 Π

∆Π>0

Π=f(q)

Π =f(q) is concave.

Minimization (one variable and no constraint)

• To minimize AC=f(q), look at ∆AC as q

increases. • As long as ∆AC<0, AC

is decreasing. The change ratio is Limit of ratio as ∆q 0 is

derivative of AC=f(q). For q<q**, derivative < 0. For q>q**, derivative > 0. At q= q** , derivative = 0.

qAC∆∆

0dq

dAC< 0

dqdAC

>

q** q

AC

AC = f(q)

AC =f(q) is convex.

• First derivative is the slope of a line tangent to the function at a point.

• Derivative is denoted as:

• To maximize or minimize f(q), find q value where: • Derivative will be a constant if the original function

is linear; thus, the function has no Max or Min. • But if the function is nonlinear, its derivative will be

a function of q itself (the derivative varies as q varies).

• For derivative at a specific value of q, say q1, denote At q*,

f(q)fordqdfor

dqdACor

dqdΠ ′=′

0dqdf

=

.dqd

1qq=Π 0.

dqd

*qq =Π

=

q

Π

0

Slope = dπ/dq at the point of tangency.

• To find the q where Π is maximized or AC is minimized, find the derivative of f(q), and set it equal to 0 and solve for q to get q*.

• The “First Order Condition (FOC)” for a maximum or a minimum of an unconstrained function of one variable is:

FOC is Solve the FOC for q to

get q* (the optimal q). The FOC is a necessary condition for a maximum

(or minimum) but not a sufficient condition.

.0dqdf

=

Second Order Condition (SOC)

.0 is FOC The =dxdyy

x 0dxdy

=

Convex function

0 0

dxdy

= x

y Concave function

0

y

x

Inflection point

0dxdy

=

Concave then convex

Convex then concave

0

dxdy

x 0

FOC is necessary but not sufficient

0 x Increases through 0

dxdy

dxdy Decreases

through 0

x 0

Take the derivative (second derivative) of the derivative (first derivative). 0*x

dxyd2

2

> 2

2

dxyd

0 + 0 - Min

Max

0*xdx

yd2

2

<2

2

dxyd

x

2

2

dxyd Inflection

0 x x

x* x*

x* x*

x*

x*

x* x*

x*

0x*dx

yd2

2

=

Examples: y

x

MAX

MIN

0dxdy

=

0dxdy

> 0dxdy

<

0dx

yd2

2

>0dx

yd2

2

<

0dxdy

> 0dxdy

>

0dx

yd2

2

< 0dx

yd2

2

>

0dx

yd2

2

=

0dxdy

=

0dx

yd2

2

=0

dxyd2

2

=

Local maximum and local minimum

An inflection point is where the second derivative equals zero when evaluated at the q* identified by the FOC.

When the first derivative equals zero, you have a minimum, maximum, or an inflection point. Not all inflections have

SOC if have a maximum at q*

if have a minimum at q*

if have an inflection at q*

Max Min

FOC

SOC Solve FOC for q* and evaluate at q*.

,0f *q >′′

,0f *q =′′

For an unconstrained function of one variable

0f =′ 0f =′0f *q <′′

.0f =′

,0f *q <′′

0f *q >′′f ′′

Rules for Finding Derivatives 1.The derivative of a constant is 0. If b is a constant in y = b, then y = a + bx, the derivative of a with respect to x is 0. 2. The Power Rule – The derivative of a variable raise to a power is

.0dxdb

dxdy

==

,baxdx

dax 1bb

−=

.bxdxdx 1b

b−=,1

dxdx

=

,aaxdxdax 11 == −

Parameters Variable

• Examples: For y = a+bx2 y = a+bx+cx2+ex3

bx2dxdy

= 2ex3cx2bdxdy

++=Anything raised to the 0 power is 1!

b2dx

yd2

2

= ex6c2dx

yd2

2

+=

Review of the FOC and SOC for an unconstrained function of one variable:

Find x* from the FOC then look at SOC by substituting x* into

to see if second derivative evaluated at x* is >, <, or = 0.

( ),0dxdy

=2

2

dxyd

FOC 2bx=0 ⇒x*=0

b2f *x =′′

• The derivative of the natural log of x is the inverse of X, The actual rule is because If ,

x25

dxdy ⋅

=

.x1

dxxlnd

dxdy

==

xbc

dx)dbln(xc

=

xc

dxdclnxandclnx)ln(xc ==

),5ln(x3y 2+=xbc

dxdbclnxandbclnx)bln(xc ==

May not always have the parentheses.

The exponential function rule - For y = the derivative of the exponent (bx) times the original function times the natural log of the constant (a).

Can give constants different labels than above. If then

lna,balnaadx

dbxdx

da bxbxbx

==

,acy x= .clnacdxdy x=

alnadxda x

x

=

xxxx

e)1(eelnedxde

===,be)1(beelnbedx

de bxbxbxbx

===

,abx

( )bxa=

dxdy

e is the base of natural logarithms = 2.71828, so that lne = 1.

The Sum of Two Functions Rule –

The derivative of the sum of two functions of x is the sum of derivatives of the two functions:

Example: If y = ax + bx2, then a +2bx.

[ ] ).x(g)x(fdx

)x(g)x(fd ′+′=+

=dxdy

The Product Rule – The derivative of the product of two functions of x is the first times the derivative of the second plus the second times the derivative of the first.

Example: y = (2x-1)(2-9X2)

[ ] )x(f)x(g)x(g)x(fdx

)x(g)x(fd ′+′=⋅

2

22

2

x54x184)x184()x18x36(

)2)(x92()X18)(1X2(dxdy

−+=

−++−=

−+−−=

The Quotient Rule - The derivative of the quotient of two functions of x is the derivative of the top times the bottom minus the top times the derivative of the bottom all divided by the bottom squared. Ex 1: If Ex 2: If Or

[ ]2)x(g)x(g)x(f)x(g)x(f

dx)x(g)x(fd

′−′=

,2x

xy 2

3

+= ( )

4x4xx6x

4x4x)x2(x2xx3

dxdy

24

24

24

322

+++

=++

−+=

,x1y 4=

( ) 558

3

8

34

x4x

4x

x4x

)x4(1x0dxdy −−=

−=

−=

−=

5x4dxdy −−=,x

x1y 4

4−==

The Chain Rule - Allows differentiation of complex functions in terms of elementary functions. If and so that

then )x(fy = )z(gx =

)),z(g(fy =

dzdg

dxdf

dzdg

)z(dgdf

dzdx

dxdy

dzdy

⋅=⋅=⋅=

Examples:

1. 3123

12 xyso,z2zx,)z2z(y =+=+=

+=

+==

−2z2

dzdx)z2z(

31

dxdy

dzdy 3

22

( )3

22

322

)z2z(3

)1z(2)2z2(z2z31

+

+=++=

−

Examples:

3. )2x4xln(y 2 +−=

2x4x4x2)4x2(

2x4x1

dxdy

22 +−−

=−⋅+−

=

2. ,)1x3(y 2+= 6x183)1x3(2dxdy

+=+=w w

w

dxdw

dwdy

Functions of More than One Variable “Partial” derivatives – May want to know how y

changes as one of several x variables changes, ceteris paribus (as in demand curve).

Several derivatives can be taken; one with respect to

each Xi holding the other variables constant. Denoted as:

When taking this derivative, all other xi’s are treated as constants. Fortunately, the rules for derivatives of functions of one variable apply to partial derivatives also. We simply treat the other x’s as constants.

( )n321 x,...,x,x,xfy =

ixi

ffxy

i==

∂∂

Second Order Partial Derivatives are analogous to the early one-variable case. Denoted as:

ijji

2

fxxy

=∂∂

∂

There are four combinations in the two variable case, fij, fii, fji, and fjj; but fij and fji are equal (Young’s Theorem). The order of differentiation does not matter.

Partial derivatives reflect the units of measurement for y and x. If y = quantity of apples demanded per year in 100 pound units and x is the price of apples per pound, ∂y/∂x measures the change in the quantity of apples demanded in 100 pound units per year for a dollar change in the apple price, ceteris paribus. If demand were measured in pounds per year, the partial derivative would be 100 time larger than when demand is measured in 100 pounds per year.

“cross second partial” i≠j

“own second partial” i=j

Example 1: 22 I36P201000Q +−=

,P40PQ

−=∂∂ ,0

IPQ2

=∂∂

∂ 40PP

Q2

−=∂∂

∂

,I72IQ=

∂∂

,0PI

Q2

=∂∂

∂ 72II

Q2

=∂∂

∂

Example 2: IPI36P201000Q 22 ++−=

,IP40PQ

+−=∂∂ ,1

IPQ2

=∂∂

∂40

PPQ2

−=∂∂

∂

,PI72IQ

+=∂∂

,1PI

Q2

=∂∂

∂ 72II

Q2

=∂∂

∂

Total Differential The total differential shows what happens to y

as both x1 and x2 change simultaneously. The total differential is To find a maximum or minimum for y (where dy

= 0 as x1 and x2 change), the FOC is that all because then dy = 0! That is, all the first partial derivatives must be 0. This happens at the “top of the Hill” in

three dimensions or the “bottom of the hole”.

22

11

dxxfdx

xfdy

∂∂

+∂∂

=

0xf

i=∂

∂

x2

x1

f2

f1

Example 1: To find max. or min. for

we must find where both first partials equal zero (FOCs) and solve the FOCs simultaneously for and .

FOC 1:

FOC 2:

Example 2:

5,4xx2xxy 22

212

1 ++−+−=

02x2xy

11

=+−=∂∂

04x2xy

22

=+−=∂∂

2x2 1 =

4x2 2 = 2x*2 =

These FOCs are not sufficient; could be a maximum, a minimum, or a saddle point. All we know is that it is a flat spot! Need SOCs (later).

1x*1 =

*1x *

2x

,xx54xx2xxy Optimize 2122

212

1 −++−+−=FOC 1: FOC 2:

22xx0x22xxy

12211

+−=⇒=−+−=∂∂

0x42)-2(-2x0x42xxy

11122

=−++⇒=−+−=∂∂

2xand0x *2

*1 ==⇒ Plug these optimal values into y to get y* = 9.

Plug these optimal values into y to get y* = 10.

Implicit Functions • Usually express functions explicitly as y = f(x1, x2 ,…, xn) where y is a function of xi. • However, some functions will be expressed

implicitly as 0 = f(x1, x2 ,…, xn, y) where there is no dependent variable.

• Can find:

Remember the negative sign.

yi ff

dxdy i−=

iff

dydx yi −=and

i

j

ff

dxdx

j

i −=or

Example

044y2xyxy)f(x, 22 =++−+=

2x2fx −= 4y2fy +=

2yx1

2y1x

4y22x2

dxdy

+−

=+−

−=+−

−=

x12y

22x42y

dydx

−+

=−+

−= Remember the sign!

Examples of implicit functions are production possibility curves, indifference curves, budget constraints.

Envelope Theorem

• Deals with how the optimal value of the function (y*) changes as the value of one of the parameters changes (What are parameters?)

• Theorem states that one can calculate by holding independent variables at their optimal values in the function and finding .

ay∂∂ ∗

dady

)x,...,x,f(xy *n

*2

*1=∗

Example: bxaxxy 2 ++−=

First, find the FOC: ,0bax2dxdy

=++−=

2bax* +

=Second, solve FOC for x: Third, substitute x* into y to get:

4)ba(

2bab

2baa

2bay

22 +=

+

+

+

+

+−=∗

Forth, partially differentiate with respect to a:

(Partial derivative because b is constant)

∗y2

baay +

=∂∂ ∗

Thus, depends only on values of a and b, but no variables!

Assume b=2 then:

=∂∂

=

=∂∂

=

=∂∂

=

=+

=∂∂

1ay,0a

2ay,2a

3ay,4a

22a

ay

*

*

*

*

2ba

ay +

=∂∂ ∗

ay∂∂ ∗

Example cont’d From previous slide

In many-variable case, solve all FOC simultaneously for xi*(a) and substitute into y to get y* = f(x1

*, x2*, …, xn

*), then .ay

dady *

∂∂

=

Constrained Optimization

• Because we are looking at scarce resources, we usually must find an optimum value subject to one or more constraints (limits). These may prevent us from achieving the maximum (or minimum) of the objective function.

Lagrangian Multiplier Method Mathematical “voodoo” to short-cut optimization subject to a constraint. The following is an important format for setting up a constrained optimization problem.

Max (Min): )x,...,x,x,x(fy n321=s.t. )x,...,x,x,x(g0 n321=

Implicit form!

Where λ is the Lagrangian multiplier.

Constraint example: x1 + 2x2 + x3 = 3

Set up )x,...,x,x,x(g)x,...,x,x,x(fL n321n321 λ+=

)x,...,x,x,x(gxx2x30 n321321 =−−−=Value of constraint

Objective function to be optimized

Constraint

Find FOC for each xi and λ:

==λ∂∂

=λ+=∂∂

=λ+=∂∂

=λ+=∂∂

=λ+=∂∂

0)x,...,x,x(gL

0gfxL

0gfxL

0gfxL

0gfxL

n21

nnn

333

222

111

FOC

Lagrangian Multiplier Method (cont)

Solve these n + 1 FOCs simultaneously to find values for and

These are optimum values of the xis given the constraint.

Substituting the into y will give

These conditions are necessary, but not sufficient for an optimum. Look at Second Order Conditions (SOC) later.

,x,...,x,x,x *n

*3

*2

*1 .*λ

*ix .y*

)x,...,λ2g2(x)x,...,λ1g1(x)x,...,f(xL n1n1n1 ++=

You can use the Lagrangian method for any number of xi (i=1…n)and for any number of constraints. Just add another λj (j=1…m) for each constraint, gj (j=1…m).

Interpretation of λ

Solve the first FOC for λ to get:

1

11111 g

fλ,fλg0,λgf−

=−==+

Do it for all n FOC to get:

n

n

2

2

1

1

gf...

gf

gf

−==

−=

−=λ

Because all ratios equal λ, all ratios are equal. i

i

gfλ−

=

• At optimum (Max or Min of y), the negative of the ratios of the partial derivatives (objective/constraint) with respect to each xi are equal and equal to λ.

• The fi shows the marginal contribution (benefit) of xi to y (the objective function), while the –gi shows the marginal cost of xi in the constraint. That is, –gi shows how much the other xis must be reduced when xi is increased slightly, and therefore, how much y is foregone if the constraint is to hold.

• Because λ = the ratio of marginal benefit to marginal cost for all xi, all these ratios are equal. This condition is necessary for a maximum (or minimum). That is, the Margin Benefit/Marginal Cost ratios must be equal for all xi and equal to λ. If one xi ratio were larger than the others, then more xi should be used and less of other xis should be used until the ratios are equal again.

• λ* indicates how y * would be affected by relaxing the constraint slightly. λ* is the shadow price for a unit of the resource represented by the constraint (the value of an additional unit of the resource). If λ* = 0, the constraint is redundant and the optimum is unconstrained. An additional unit of the resource would be worth nothing in regard to increasing y.

• If λ* is large, relaxing the constraint would be very beneficial because the benefit/cost ratio is high.

• Relaxing the constraint refers to increasing (decreasing) the value of the constraint in a maximization (minimization) problem!

Duality

• Any constrained Max (Min) problem (called the primal problem) is associated with a constrained Min (Max) problem (called the dual problem). – Primal (Dual) may be: Max Utility s.t.

Income; or Min Cost s.t. Output level. – Dual (Primal) would be: Min Expenditures

s.t. Utility level; or Max Output s.t. Cost level.

Always two ways to look at any constrained optimization problem.

Example: Primal x5xy3yxz)y,x(f 22 ++−==Min:

s.t. 0y2x =−2y)xλ(05x3xyyxL 22 +−+++−=

05y3x2xL

=λ−++=∂∂

FOCs 02x3y2yL

=λ++−=∂∂

0y2xL=+−=

λ∂∂

1.

2.

3.

Primal example (cont.) Simultaneous Solution

0104y7x02λ02y3x02λ106y4x

=++

=++−=−++1. Multiply FOC 1 by 2 and add it to FOC 2.

2. Solve FOC 3 for x to get

3. Substitute x into above result to get:

This gives:

4. Substitute into x=2y from FOC 3 to get:

5. Substitute and into FOC 1 to get:

6. Substitute and into z to get:

7. Optimal solution is:

.y2x =

.010y4)y2(7 =++

.95y* −=

.910)95(2x* −=−=*y*x *y .910* =λ*x *y .81225z* −=

,910x* −= ,95y* −= ,81225z* −=

If you could relax the constraint by one unit (decrease it by one unit from 0 to -1), shows that would decrease by 10/9 of a unit. *λ *z

.910λ* =

Example: Dual

y2xw −=Max:

s.t.: 81225x5xy3yx 22 −=++−

5x)3xyyx81225(λ2yxL 22D −−+−−+−=

The constraint in implicit form is: 0 = –225/81 – f(x,y).

From the constraint of the primal.

From the objective function of the primal assuming optimality.

Dual example (cont.)

05x3xyyx81225

λL

0x3λy2λ2yL

05λ3λx2λ1xL

22D

DD

DDD

=−−+−−=∂∂

=−+−=∂∂

=−−−=∂∂ y

Solve these FOCs simultaneously to get:

,910x* −= ,95y* −= ,109λD* =

If we could relax (increase) the constraint by 1 unit, the optimal value of the objective function would increase by 9/10 of a unit. Notice that values for x* and y* are common to the primal and dual, while the value of λD* for the dual is the reciprocal of λ* for the primal.

FOCs

0w* =

)w( *

Envelope Theorem in Constrained Max.

• Theorem can be used as discussed earlier to examine how a function’s optimum value (y*) changes as a parameter in the problem changes. In the case at hand, differentiate optimal Lagrangian expression with respect to the parameter:

Max: )a;x,x(fy 21= st: )a;x,x(g0 21=

)a;x,x(g)a;x,x(fL 2121 λ+=Solve FOC for x1*, x2* and substitute into L to get L*, which is a function of parameters only.

Envelope Theorem …(cont.)

aL

dady **

∂∂

=Only difference is that y* is a constrained optimum.

If a is the value of the constraint, then

is the shadow price of the resource in the constraint.

See the section in the text on Inequality Constraints (pp. 45-47) for the Lagranian methods for solving problems with inequality constraints.

Warnings:

Not all optimization problems can be solved with calculus. Functions must be continuous at optimum. If not, use Linear Programming methods.

aL

dady *

**

∂∂

=λ=

Further discussion of FOC and SOC

We have already talked about FOC and SOC for functions of one independent variable and no constraints: y = f(x).

FOC

SOC Max

0}concave*xf0f

<′′=′

FOC

SOC

Min0}convex*xf

0f

>′′=′

y

x

concave

y

x

convex

X*

X*

y*

y*

Suppose

==

0f0f

2

1FOC

)x,x(fy 21=

Indicates a flat spot—

max, min, saddle point.

Two independent variables, no constraint.

Logic suggests that the own second partials evaluated at would indicate which (max or Min), but….

>

>

<

<

0f

0f

0f

0f

*2

*1

*2

*1

*2

*1

*2

*1

x,x22

x,x11

x,x22

x,x11

Max

Min

Assumes only x1 or x2 is changing!

*2

*1 xandx

SOC

However, what if both x1 and x2 are changing? Must consider a third SOC condition (partial derivatives are evaluated at X1* and X2* for SOCs):

0fff 2122211 >−

This condition must hold when evaluated at the optimal values of x1 and x2. This condition is one of three sufficient conditions (SOCs) when f is a function of two variables (unconstrained).

FOC for maximum or minimum

==

0f0f

2

1 max,min, saddle point

>−

<<

0fff

0f,0f2

122211

2211Concave function

>−

>>

0fff

0f,0f2

122211

2211 Convex function

SOC for a maximum:

Summary of FOC and SOC for functions of two independent variables and no constraint.

SOC for a minimum:

)x,f(xy 21=

• f1 and f2 locate a flat spot.

• f11 and f22 distinguish between a minimum and a maximum.

• Think of a saddle in case of a minimum. The flat spot on the saddle is not the lowest point on the saddle.

• f11f22 - f122 > 0 confirms that it is not a saddle

point.

Constrained Optimization

0xL

0xL

2

1

=∂∂

=∂∂

FOC

0λL

0xL

1

n

=∂∂

=∂∂

Solve the FOC simultaneously to get the optimal solution and plug the xi* into the appropriate expression above (SOC) to see if it is satisfied (or just determine the signs of the partial derivatives and determine if the expression has the appropriate sign).

The SOC for constrained maximization of a function of two variables, subject to a linear constraint (or its dual) is:

. xandat x evaluated ,0ffff2fff *2

*1

21222112

2211

<+−

If a constrained function has this property, it is strictly quasi-concave. In economic theory, we usually use this problem (e.g., Max. a utility function, s.t. a linear budget constraint (primal), or Min. linear expenditures, s.t. a fixed level of the utility function (dual). …

0λL

m

=∂∂

…

The SOC for constrained minimization of a function of two variables, subject to a linear constraint (or its dual) is:

. xand at x evaluated ,0ffff2fff *2

*1

21222112

2211

>+−If a constrained function has this property, it is strictly quasi-convex.