Concavity Convexity CW

7/30/2019 Concavity Convexity CW

1/27

Concavity-Convexity

From Chiang-Wainwright

Chapter 11


2/27

Second-Order Conditions

in Relation to Concavity and Convexity

Second-order conditions: can be stated in terms of the principal minors of the Hessian determinant or

the characteristic roots of the Hessian matrix

Concerned with the question of whether a stationary

point is the peak of a hill or the bottom of a valley. In the single-choice-variable case, with z = f(x), the hill

(valley) configuration is manifest in an inverse (U-shaped)curve.

For the two-variable functionz = f(x,y), the hill (valley)

configuration takes the form of a dome-shaped (bowl-shaped) surface

When three or more choice variables are present, thehills and valleys are no longer graphable, but we may thinkof "hills" and "valleys" on hypersurfaces.


3/27

Maximum Minimum

Saddle Point Inflection Point


4/27

Strict vs. non strict convexity / concavity

A function that gives rise to a hill (valley) over the entire domainis said to be a concave (convex) function.

Domain is entire Rn, where nis the number of choice variables.Therefore, concavity and convexity are global concepts.

We need to distinguish between concavity and convexity on theone hand, and strictconcavity and strictconvexity, on the otherhand. Non-strict: may contain flat portions (line on a curve, plane on a

surface.

Strict: no such line or plane segments

A strictly concave (strictly convex) function must be concave(convex), but the converse is not true.


5/27

Absolute vs. Relative Extremum

An extremum of a concave function must be a peakamaximum (as against minimum). Moreover, that maximummust be an absolute maximum (as against relative maximum),since the hill covers the entire domain.

However, that absolute maximum may not be unique,because multiple maxima may occur if the hill contains a flathorizontal top.

For strict concavity - the peak consists of a single point andthe absolute maximum be unique. A unique absolute

maximum is also referred to as a strongabsolute maximum.

The extremum of a strictly convexfunction must be a uniqueabsolute minimum.


6/27

Absolute vs. Relative Extremum

The properties of concavity and convexity are taken to be global inscope.

If they are valid only for a portion of the curve or surface (only on asubset Sof the domain), then the associated maximum and minimum

are relative (or local) to that subset of the domain, since we cannot becertain of the situation outside of subset S.

The sign definiteness ofd2z (or of the Hessian matrixH), the leadingprincipal minors of the Hessian determinant are evaluated only at thestationary point.

If we limit the verification of the hill or valley configuration to a smallneighborhood of the stationary point, we could only refer to relativemaxima and minima.


7/27

Sign Definiteness of d2z

ifd2z is everywhere negative (positive)semidefinite, the functionz = f(x1, x2,...,xn) mustbe concave (convex), and ifd2z is everywhere

negative (positive) definite, the function/must bestrictly concave (strictly convex).

For a twice continuously differentiable functionz

= f(x1, x2,...,xn), we concentrate exclusively onconcavity and maximum;


8/27


9/27

Absolute and Relative Maximum

(refer to diagram)

The first-order condition is necessary for z* to be a relative maximum,and the relative-maximum status of z* is, in turn, necessary for z* to bean absolute maximum, and so on.

If z* is a unique absolute maximum it is sufficient for z* to be arelative maximum.

The three ovals at the top have to do with the first- and second-orderconditions at the stationary point z*. Hence they relate only to a relativemaximum.

The diamonds and triangles in the lower part, on the other hand,describe global properties that enable us to draw conclusions about anabsolute maximum.

The stronger property of everywhere negative definiteness ofd2z issufficient, but not necessary, for the strict concavity offbecause strictconcavity offis compatible with a zero value ofd2z at a stationarypoint.


10/27

Checking Concavity and Convexity

Geometric Definition

The functionz = f(x1,x2) isconcaveiff, for any pair ofdistinct pointsMandNon its grapha surfacelinesegmentMNlies eitheron orbelowthe surface. Thefunction is strictly concave iff line segmentMN liesentirely below the surface, except atMandN.

The functionz = f(x1,x2) isconvexiff, for any pair ofdistinct pointsMandNon its grapha surfacelinesegmentMNlies eitheron or abovethe surface. The

function is strictly convex iff line segmentMN liesentirely above the surface, except atMandN


11/27

The functionz = f(x1,x

2) isconcaveiff, for any pair of distinct

pointsMandNon its graph -- a surface -- line segmentMNlieseitheron orbelowthe surface. The function is strictly concaveiff line segmentMN lies entirely below the surface, except atMandN.


12/27

Checking Concavity and Convexity

Algebraic Definition

Let u = (u1,u2)and v = (v1,v2)be any two distinct orderedpairs (2-vectors) in the domain ofz = f(x1,x2). Then the zvalues (height of surface) corresponding to these will be

f(u) = f(u1,u2)andf(v) = f(v1,v2),respectively.

Each point on the said line segment is a "weightedaverage" ofu and v. Thus we can denote this line segmentbyu+(1-)vwhere has a range of values0 < < 1.

The line segmentMNrepresents the set of all weightedaverages off(u) andf(v) andcan be expressed by f(u)+(1-)f(v),with varying from 0 to 1.


13/27

Algebraic definition

Along the arc MN, the values of the function fevaluated at various points on line segment uv, itcan be written simply asf[u + (1- )v].

Algebraic definition:

height of line segment height of arc

concaveA function is iff, for any pair of distinct points and

convex

in the domain of , and for 0 1,

( ) (1 ) ( ) (1 )

f u v

f

f u f v f u v


14/27

Algebraic definition

Note that, in order to exclude the two end points M andN from the height comparison, we restrict to the openinterval (0, 1) only.

For strict concavity and convexity change the weakinequalities and to the strict inequalities < and >,respectively.

The algebraic definition can be applied to a function of

any number of variables. The vectors u and v can beinterpreted as n-vectors instead of 2-vectors.


15/27

Theorems on Concavity and Convexity

Theorem I (linear function). Iff(x)is a linear function, then it isa concave function as well as a convex function, but not strictlyso.

Theorem II (negative of a function). Iff(x)is a concavefunction, then -f(x)is a convex function, and vice versa. Similarly,

iff(x)is a strictly concave function, then -f(x)is a strictly convexfunction, and vice versa.

Theorem III (sum of functions). Iff(x)and g(x)are bothconcave (convex) functions, then f(x) + g(x)is also a concave(convex) function. Iff(x)and g(x)are both concave (convex) and,

in addition, either one or both of them are strictly concave (strictlyconvex), then f(x) + g(x)is strictly concave (strictly convex).


16/27

Example:2 2

1 2

1 2 1 2

2 2

1 2 1 2

2 2

1 2 1 2

1

Check for concavity or convexity.

Let ( , ) and ( , ) be any two distinct points in the domain.Then we have

( ) ( , )

( ) ( , )

and [ (1 ) ] (

z x x

u u u v v v

f u f u u u u

f v f v v v v

f u v f u

1 2

1 2 2

value of value of

22

1 1 2 2

1 ) , (1 )

= (1 ) ] [ (1 )

Substituting these into (11.20), subtracting the right-side expression

from the left-side one, and collec

x x

v u v

u v u v

2 2 2 21 2 1 2 1 1 2 22 2

1 1 2 2

2 2

1 2

ting terms, we find their difference to be

(1 ) (1 ) 2 (1 )( )

(1 )[( ) ( ) ] 0

Therefore is strictly convex.

u u v v u v u v

u v u v

z x x


17/27

Differentiable Functions

If the function is differentiable, however, concavity andconvexity can also be defined in terms of its firstderivatives.

In the one-variable case, the definition is:

concaveA differentiable function ( ) is iff,

convex

for any given point u and any other point v in the domain,

( ) ( ) ' ( )

f x

f v f u f u v u


18/27

Comments:

Concavity and convexity will be strict, if the weakinequalities are replaced by the strictinequalities

< and >, respectively.

Interpreted geometrically, this definition depicts aconcave (convex) curve as one that lies on or below(above) all its tangent lines.

To qualify as a strictly concave (strictly convex) curve, onthe other hand, the curve must lie strictly below (above)all the tangent lines, except at the points of tangency.


19/27

Let A be any given point on the curve with height f(u) and withtangent line AB. Let xincrease from the value u. Then a strictlyconcave curve (as drawn) must, in order to form a hill, curl

progressively away from the tangent line AB, so that point C, withheight f(v), has to lie below point B.

In this case, the slope of line segment ACis less than that oftangent AB.

Slope of line segment (slope of )

( ) ( )'( )

Note: ( ) ( ) '( )( )

DCAC AB

AC

f v f uf u

v u

f v f u f u v u


20/27

Differentiable Functions

For two or more variables, the definition becomes:

1

1 1

1

concaveA differentiable function ( ) ( , ) is iff,

convex

for any given point ( , , ) and any other point ( , , ) in the domain,

( ) ( ) ( )( )

where ( ) i

n

n n

n

j j j

j

j

j

f x f x x

u u u v v v

f v f u f u v u

ff u

x

1s evaluated at ( , , ).nu u u

This definition requires the graph of a concave (convex) functionf(x) to lie on or below(above) all its tangent planes or hyperplanes.

For strictconcavity and convexity, the weak inequalities should be changed to strict

inequalities


21/27

Twice Differentiable Functions

1

2

1

2

concaveA twice differentiable function ( , , ) is iff,

convex

negativeis everywhere

positive

Consider a function ( ) ( , , ) If second order partial

derivatives exist, is defined:

.

n

n

z f x x

d z

f x f x x

d z

2

semidefinite.

concaveThe said function is if (but not only ifstri )

convex

negativeis everywhere definite

posit

c

v

l

i e

t y

.d z


22/27

Examples:

4 3 *

2 2

2 2

1 2 1 1 2 2

11 12

11

21 22

2

1. 4 0

''( ) 12 0 everywhere negative semidefinite

2. 2 , 2

2 02 and 4 0

0 2

Thus everywhere negative definite (satisfies the sufficient condition o

dz

z x x xdx

d z f x x

z x x f x f x

f ff

f f

d z

f

strict convexity.


23/27

Convex functions vs. convex sets

Convex sets and convex functions are distinct

concepts

important not to confuse them.

Geometric characterization of a convex set. Let Sbe a set of points in a 2-space or 3-space. If, for any

two points in set S, the line segment connecting these twopoints lies entirely in S, then Sis said to be a convex set.

a straight line satisfies this definition and constitutes a

convex set.

a single point is also considered as a convex set (by

convention).


24/27

b


25/27

Convex functions vs. convex sets

Convex functions and convex sets are notunrelated. Convex function needs a convex set for the domain.

(11.20) requires that, for any two points uand vin thedomain, all the convex combinations ofuand vspecifically, u+ (1 -)v, 0


26/27

( ) [ ( ) convex]S x f x k f x ( ) [ ( ) concave]S x f x k f x

The set S consists of all the x values associated with the segment of f(x)

lying on or below the broken horizontal line. Domain is a convex set.

Even a concave function g(x) can generate an associated convex set, given

some constant k.

Valid if we interpret x as a vector (for more than two variables).


27/27

Documents

Concavity Convexity CW