Tangentially continuous directional derivatives in nonsmooth analysis

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 61, No, l, APRIL 1989

Tangentially Continuous Directional Derivatives in Nonsmooth Analysis 1'2

R. C O R R E A 3 A N D A. J O F R E 4

Communicated by R. A. Tapia

Abstract. In this paper, we introduce a new class of nonsmooth functions in terms of a continuity property of the usual directional derivative. Under this approach, we study the subregular and the semismooth functions. Finally, we give conditions for a marginal function to be subregular and semismooth.

Key Words. Nonsmooth optimization, subregular functions, semismooth functions, marginal functions.

I . Introduction

In Section 2, we present some nonsmooth analysis concepts: directional derivative, generalized directional derivative, and the associated not ions of subdifferential and general ized gradient. A detailed explanat ion o f the subject can be found in the classical book on convex analysis (Ref. I) and in the recent, excellent b o o k on n o n s m o o t h analysis (Ref. 2).

In Section 3, we in t roduce a new class of locally Lipschitzian n o n s m o o t h functions, defined on a general topological linear space in terms of a continui ty proper ty o f the usual directional derivative. We prove that, for these functions, called direct ionally D-regular , the generalized gradient can be written as the closed convex hull o f the Gateaux derivative limits as the variable takes on values on the dense subset D of the space where the Gateaux derivative exists. We call a funct ion whose generalized gradient can be so written D-representable . The class o f directionally D-regula r

This research was supported in part by the Fondo Nacional de Ciencias, Santiago, Chile. 2 The authors thank A. Auslender and L. Thibault for discussions on the subject. 3 Profesor, Departamento de Matem~iticas y Cieneias de la Computaci6n, Universidad de

Chile, Santiago, Chile. 4 Profesor Instructor, Departamento de Matem~iticas, Universidad de La Serena, La Serena,

Chile.

t 0022-3239/89/0400-0001506.00/0 © 1989 Plenum Publishing Corporation

2 JOTA: VOL. 61, NO. 1, APRIL 1989

functions turns out to be a nonconvex balanced cone containing almost every class of useful nonsmooth locally Lipschitzian functions. Theorem 3.2 shows that the sum of two directionally D-regular functions, albeit not necessarily D-regular, is D-representable. Proposition 3.1 gives an important property of these functions: the difference of two directionally D-regular functions whose Gateaux derivatives coincide over D is a constant function.

In Section 4, we show (Theorem 4.2) that a subregular function [these functions were introduced by Clarke (Ref. 3) under the name regular, and were subsequently called subdifferentially regular by Rockafellar (Ref. 4)] is always directionally D-regular for every dense set D where the Gateaux derivative exists. In order to prove this, we generalize to a non-finite- dimensional space a result by Rockafellar (Ref. 5, Theorem 2), characteriz- ing subregularity in terms of a continuity property of the function's directional derivative. In formulating this characterization in Theorem 4.1, we assume the function to be directionally differentiable at every point of the space, a hypothesis that can be removed by replacing the directional derivative of the function by its lower Dini derivative (see Corollary 6.1). This result of Theorem 4.1 can be directly arrived at through a generalization of formula (12) in Lemma 4.1 by substituting for the usual directional derivative therein the lower Dini derivative. In such case, the integral mean-value theorem used in the demonstration can be replaced by a Dini mean-value theorem; see, for instance, the proof of Theorem 6.1. Since formulas (12), (13) and their generalizations make up a rather useful characterization of a locally Lipschitzian function's generalized directional derivative, it is curious to find that they have not been used in the literature. In Theorem 4.3, we show that the sum of a subregular function and a directionally D-regular one is D-representable, which implies, in particular, that the difference of two subregular functions is D-representable. Apart from DC (difference of two convex) functions, the literature has not yet included studies of functions that are the difference of two subregular ones. Finally, in Proposition 4.1, we show that the difference of two D-representable functions is a constant function when one of them is subregular and its generalized gradient contains that of the other. This result generalizes one by Rockafellar (Ref. 5, Corollary 3).

In Section 5, we introduce semismooth functions, defined by Mifflin for the finite-dimensional case (Ref. 6), and characterize them through a continuity property of the directional derivative. Such characterization allows obtaining in a simpler way most familiar properties of semismooth functions as well as deducing new ones. In this respect, just as we did for the case of subregular functions, we note that, in Theorem 5.1, the existence of the directional derivative is too strong a condition since the conclusion is equally valid if in formula (16) the ordinary directional derivative is

JOTA: VOL. 61, NO. i, APRIL 1989 3

replaced by the lower Dini derivative (see Corollary 6.3). It is shown that semismooth functions are directionally D-regular for every dense subset where the Gateaux derivative exists, that continuous convex functions are semismooth, that the composition of two semismooth functions is still semismooth, and finally that the difference of two semismooth functions such that the generalized gradient of one contains that of the other is a constant function.

In Section 6, sufficient conditions are given for the supremum of a family of subregular functions to be subregular and also for the supremum of a family of semismooth functions to be semismooth. Theorems 6.1 and 6.2 generalize significantly Clarke's (Ref. 3, Theorem 2.1) and Mifflin's (Ref. 6, Theorem 2) results. The notion of semicontinuity of a multifunction M : X ~ U, introduced in Definition 6.3 and used in the theorems mentioned above, corresponds to the l.s.c of M at (x, u) in the sense of Penot (Ref. 7).

2. Preliminary Definitions

The class of nonsmooth functions defined on a locally convex topological vector space (1.c.t.v.s.) X and taking on values on the real line R that nowadays draws the greatest attention consists of the so..called locally Lipschitzian functions, which we now define.

Definition 2.1. A function f : X ~ ~ is said to be locally Lipschitzian if, for all x ~ X, there exist a neighborhood V of x and a continuous seminorm p over X, such that

[f(y)-f(z)[~p(y-z), for all y, z c V. (1)

When f is a continuous convex (concave) function on X (afortiori locally Lipschitzian), the directional derivative

f ' (x; d ) = lim t-~[f(x + td)- f (x)] (2) t - > 0 +

plays a fundamental role in the so-called convex analysis, mainly developed by Moreau (Ref. 8) and Rockafellar (Ref. 1) in the 60's, and which may be considered at the present time as the first chapter of the nonsmooth analysis.

The fundamental property of this directional derivative, besides its existence, is that, as a function of the direction d, is finite, positively homogeneous, convex (i.e., sublinear) when f is convex, and concave (i,e., suplinear) when f is concave.

From these properties of f '(x; • ), we can define the subdifferential of a continuous convex func t ion f at a point x ~ X as the nonempty w*-compact


convex set

Of(x) = {x* c X*/f ' (x; d) >- (x*, d), for all d ~ X}, (3)

where X* is the topological dual of X and ( , ) represents the canonical bilinear form of the duality on X* x X. Analogously, w h e n f is a continuous concave function, the supditterential of f at x is

Of(x) = {x* ~ X*/f ' (x; d) <- (x*, d), for all d ~ X}.

It can be proved, in both cases, that f will be Gateaux-differentiable at x if and only if the set ~f(x) reduces to just one point, which corresponds to the Gateaux derivative o f f at x.

Outside the class of convex and concave functions, the directional derivative does not in general satisfy the aforesaid sublinearity or suplinearity property, and the results of convex analysis are not valid. The situation there is worse, because f '(x; d) might not even exist for a locally Lipschitzian function.

In 1973, Clarke (Ref. 9) introduced a new directional derivative which allows generalizing the supdifferential and subdifferential notions to a wider class of functions, namely, the locally Lipschitzian functions.

Definition 2.2. Let f : X ~ R be locally Lipschitzian. The upper and lower generalized directional derivatives of f at x ~ X in the direction d are defined respectively by

f ° ( x ; d ) = lim sup+ t-l[f(y+ td)-f(y)], (4) y---~X,t~O

fo(x; d) = lim inf t-l[f(y+ td)-f(y)]. (5) y ~ x , t ~ O +

It can be shown that f ° ( x ; .) is sublinear, that fo(x; ") is suplinear, andf'(x; d) coincides wi th f ° (x ; d) orfo(x; d) w h e n f is convex or concave, respectively. These generalized directional derivatives verify f ° ( x ; d ) = - fo(x; - d ) .

By means of this generalized directional derivative, Clarke extends the notions of subdifferential and supdifferential to the class of locally Lipschit- zian functions.

Definition 2.3. For a locally Lipschitzian function f : X ~ ~, the generalized gradient o f f at x ~ X is defined as

Of(x) = {x* ~ X*/f°(x; d) >- (x*, d), for all d ~ X}

= {x* c X*/fo(x; d) <- (x*, d), for all d c X}. (6)

It can be shown that of(x) is a nonempty w*-compact convex set in X*.

JOTA: VOL. 61, NO. 1, APRIL 1989 5

It is important to observe that, unlike the subdifferential and supdifferential notions, the generalized gradient does not privilege convexity or concavity. The characterization (7), given in the next section, clearly shows this idea.

If the generalized gradient of a locally Lipschitzian function at a point x~ X reduces to just one element, then the function will be Gateaux- differentiable at x. Unfortunately, the converse is not true, except when f is continuously Gateaux differentiable at x. In general, we know that the Gateaux derivative of f at x belongs to the genealized gradient of f at x.

It is important to point out that this generalized gradient notion is not interesting for every locally Lipschitzian function. We mention in this respect that Clarke has built a Lipschitzian function on R whose generalized gradient at all points coincides with [ -1 , 1].

3. D-Representable and Directionaily D-Regular Functions

The starting point of (nonconvex) nonsmooth analysis was the definition given by Clarke (Ref. 9) of the generalized gradient for a locally Lipschitzian real-valued function defined on R ". By using a result by Rademacher, which states that a real-valued function f which is locally Lipschitzian on R" is Gateaux differentiable Lebesgue almost-everywhere (a.e) in R ", Clarke defined the generalized gradient o f f at a point x e N" by

of(x) = co{lim V f (y ) / y ~ G}, (7) y ~ X

where co{A} denotes the convex hull of the set ,4, Vf(y) denotes the Gateaux derivative o f f at y, and G is the set (whose complement is Lebesgue-nult) where f is Gateaux differentiable. Clarke proved that (7) and (6) coincide.

The purpose of this section is to study under what conditions a locally Lipschitzian function f : X ~ R which is differentiable on a dense part D of X still satisfies formula (7) when G is replaced by D. Besides making it possible to extend equality (7) to more general spaces, the possibility of replacing G by D .in (7) corresponds to an important regularity property of the function f. The example that we mentioned at the end of the previous section clearly shows that there are Lipschitzian functions lacking this property.

A direct generalization of (7), when f is defined on an infinite- dimensional space, previously requires a generalization of Rademacher's theorem. In this sense, Christensen (Ref. 10) proved that, if X is a separable Banach space and f is a locally Lipschitzian function defined on X, then f is Gateaux-differentiable excepting on a Haar-null set (in ~ , the Haar-null


sets and Lebesgue-null sets coincide). Other results along this line have been obtained by Aronszajn (Ref. 11) and Mignot (Ref. 12). By taking (6) as the definition of the generalized gradient, and using Christensen's result, Thibault (Ref. 13) obtained the characterization

Of(x) = ~-6{ w* - l i m Vf(y)/y E G}, (8) y ~ X

that is, the w*-closed convex hull of the w*-limits of Gateaux derivatives Vf(y) when y converges to x, with y in the set where f is Gateaux differentiable.

Definition 3.1. A locally Lipschitzian function f defined on a 1.c.t.v.s. X, and Gateaux-differentiable at any point of a dense subset D in X (adh(D) = X) , is said to be D-representable at x if

Of(x) = ~-6{ w* - lira Vf(x)ly ~ O}. (9) y--~X

We say that f is D-representable if it is so at every x c X. It is evident that, if f is D-representable, so is Af for A ~ R, but it is

not clear whether the sum of two D-representable functions is D-representable.

The next theorem, due to Chung (Ref. 14, Theorem 1), gives a useful global characterization of the D-representation property.

Theorem 3.1. A locally Lipschitzian function f : X ~ R which is Gateaux differentiable at any point of a dense subset D is D-representable if and only if, for all x, d c X, the function ~ b ( t ) = f ( x + td) satisfies

~b'(t) -< lim sup (Vf(y), d), (10) y ~ x + t d

y ~ D

for Lebesgue a.e. t 6 R+, where the derivative

~b'(t) = l i m A - 1 ( 4 ~ ( t + A ) - 4~ ( t ) ) A ~ 0

exists.

Remark 3.1. Since the above statement of the theorem differs from Chung's, we give the demonstration in an appendix at the end of this paper.

Remark 3.2. According to the hypothesis of Gateaux differentiability of a locally Lipschitzian func t ion f on a dense subset in a 1.c.t.v.s. X, Lebourg (Ref. 15) proved that, if X is a separable Baire space (for example, a separable Banach space) and i f f admits generically a directional derivative

JOTA: VOL. 61, NO. t, APRIL 1989 7

f '(x; d) for every d ~ X (this will occur for supregular, subregular, and semismooth functions), then f is generically Gateaux differentiable (the term "generically" here means that the property holds on a subset G~ dense in X). Another recent result in this sense has been given by Zivkov (Ref. t6); it says that, if X is an H-space (in particular, a reflexive Banach space) and if for all x~ X the directional derivative f '(x; d) exists for all d in a dense subset of X, then f is generically Gateaux differentiable on 32.

By using a quite weak continuity notion for the directional derivative, we will introduce a class of functions for which (10) and afortiori (9) hold. It will be shown in the following sections that several well-known classes of nonsmooth functions are included in this one.

Definition 3.2. A locally Lipschitzian function f : X -~ N that is Gateaux differentiable at any point of a dense subset D in X will be said to be directionally D-regular if, for all (x, d) such that &'(0) exists, with 0 defined as in Theorem 3.1, the following equality holds:

~b'(0)= lim (Vf(x+~h),d). (11) t ~ O +

h o d x + t h e D

It is evident that (11) is equivalent to the continuity of (t, h)-~ f ' (x+ th; d) at (0, d), as a function defined on

/ ) = {(t, h) ~ ~+ x X[x + thc D w {x}}.

It is evident that, i f f is directionally D-regular, so is h f for 3. ~ N. The following example shows that the sum of two directionally D-regular functions is not necessarily directionally D-regular.

Example 3.1. Let f l : N ~ N be a continuous function such that: f~(x) = O, if x ~ 0; f l (x ) = ], if x -> ½; f l (x ) ~ [x - x 2, x], if x ~ [0, ½]; f~(1/n; 1) = 1 and f~(1/n; -1)---0, for all integers n > 2; is continuously differentiable at any point of D = g~\[{1/nln > 2}w {0}] with a derivative between 0 and 1. Let f2 : ~ -~ N be the function defined byf2(x) = x if x < 0, and f2(x) = 0, if x -> 0. It is clear that fl and f2 are directionally D-regular, but f =f~ +f2 is not. In fact, f is differentiable at 0 with a noncontinuous derivative there.

Though the sum of two directionally D-regular functions may not be directionally D-regular (see Example 3.1), we have the following result.

Theorem 3.2. The sum of two directionally D-regular functions is D-representable.

ProoL Let f l and f2 be two directionally D-regular functions from X to N. For x, d ~ X fixed, we know by Rademacher's theorem that, for

8 JOTA: VOL. 61, NO. l, APRIL 1989

Lebesgue a.e. t~R+, the derivatives ~ ( t ) and 44 (0 exist, where 4~1(t)= fa(x + td) and 4~2(t) =fz(x + td). Then, if we note that f = f ~ +f2 and ~b(t) = f ( x + td), we obtain for a.e. t e N+ the equalities

4,'(t) = ~ ( t ) + ~ ( t )

= tim+ ( V f l ( X + t d + r h ) , d ) (r,h)-~(O ,d) x+td+rh~D

+ lira+ (Vfz(x + td + rh), d) (r,h)~(O ,d) x+td+rhcD

= lim+ ( V f ( x + t d + r h ) , d ) (r,h )~(O ,d) x+td+rh~D

lim sup (Vf(y), d), y ~ x + t d

y e D

which completes the proof. []

The function f given in Example 3.1 is not directionally D-regular, but it is the sum of two directionally D-regular functions, so it is D-representable.

Proposition 3.1. Given two directionally D-regular functions f l and f2 from X to R such that Vf~(x) = Vf2(x) for all x ~ D, the function f l - f 2 turns out to be a constant function.

Proof. Since - f2 is directionally D-regular, from Theorem 3.2 one has that f~ - f2 is D-representable; then, for all x e X,

o(fl - f2 ) (x ) = E-6{ w* - lim V(fl - f2 ) (Y) /Y e D} = {0}. y'+x

By using Lebourg's mean-value theorem, we can say that, for all x, z e X, there exists u = x + t(z - x), with t e [0, 1] and u* E O(fl - f2) (u) , such that

(f , - f2 ) (x ) - (fl - f2 ) (z ) = (u*, x - z) = 0,

which lets us conclude the proof. []

Remark 3.3. An important class of nonsmooth functions consists of the almost differentiable functions (Ref. 17), that is to say, those whose Gateaux derivative is continuous relative to its domain. If X = ~N it is easily shown that for an almost differentiable function, (10) holds and thus such a function is D-representable for any dense subset D in the domain of Vf It does not appear easy to produce an example showing these functions not to be directionally D-regular.

JOTA: VOL. 61, NO. 1, APRIL i989 9

4. Subregular Functions

A very important class of locally Lipschitzian functions in nonsmooth analysis is that of the subregular functions, called regular by some authors and subdifferentially regular by others.

Definition 4.1. A locally Lipschitzian function f : X ~ ~ is said to be subregular at x p r o v i d e d f ' ( x ; d) exists a n d f ° ( x ; d) = f ' ( x ; d) for all d ~ X. A function f is subregular if it is so at every x s X.

The following theorem gives a characterization of subregular functions in terms of a continuity property o f f ' ( - , d).

Theorem 4.1. A locally Lipschitzian function f:X-->ff~, such that f '(x; d) exists for all x, d ~ X, is subregular at x if and only if the function f ' ( . , d) is u.s.c, at x for all d ~ X.

Proof. It is a direct consequence of formula (12) in the following lemma. D

Lemma 4.1. I f f : X-~ ~ is a locally Lipschitzian function such that f '(x; d) exists for all x, d e X, then

f ° ( x ; d ) = l i r a s u p f ' ( y ; d) (12) y ~ x

= lim sup - f ' (y; -d) , (t3) y--~X

for all x, d ~ X.

Proof. For any e > 0, there exist T > 0 and two neighborhoods V, W of x such that W+ td C V, for all w ~ W and t ~ [0, T], and

f '(z; d)-<l im s u p f ' ( y ; d ) + e , for all z~ V. y ~ x

Then, for w c W and t ~ ]0, T[, we have

fo f ( w + t d ) - f ( w ) = f ' (w+sd; d) ds

t[lim supf'(y; d) + El;

dividing by t and taking the upper limit as w ~ x and t ~ 0 +, we obtain

f ° ( x ; d) <-lim supf'(y; d)+~. y ~ x


Since E is arbitrary, and since the converse inequality is trivial, formula (12) follows.

Analogously, we can obtain the equality

f0(Y; d) = lim i n f f ' ( y ; d), y ~ x

which is equivalent to (13). []

From the above theorem, it follows readily that the class of subregular functions is a convex cone, that a continuous differentiable function is subregutar, and that a continuous convex function is subregular; in fact, for a continuous convex function f,

f ' (x; d) = inf{(f(x + td) -f(x))t-llt > 0}

(infimum of continuous functions). It is evident that the difference of two subregular functions is not necessarily subregular (a concave function is not in general subregular).

Theorem 4.2. I f f : X o ~ (or - f ) is a subregular function, then it is directionally D-regular, for any dense D in X where f is Gateaux differentiable.

Proof. In view of Theorem 4.1 it suffices to prove the 1.s.c. at (0, d) of the function (t, h) ~ 19 ~ f ' ( x + th; d), for all (x, d) such that 4d(0) exists; that is, - f ' (x ; -d )= f ' ( x ; d).

By using the u.s.c, o f f ' ( . , - d ) , we derive

lim inf f ' ( x+ th; d) = - lira sup f ' ( x+ th; -d ) ( t,h )~(O,d) (t,h )-~(O,.d)

( t , h ) ~ D ( t , h ) ~ D

----- - lira sup f ' ( y , - d ) y o x

>- - f ' (x; -d ) = f ' ( x ; d),

that is, the claimed lower semicontinuity. Since the class of directionally D-regular functions is balanced, the

result is also valid for - f []

This theorem shows that a subregular function is D-representable (for any D as in Definition 3.1), as is the sum of two subregular functions. As for the difference of two subregular functions, Example 3.1 shows that it is not necessarily directionally D-regular [ t ake f = f l - ( - f z ) ] , but from Theorems 3.2 and 4.2 we can conclude that it is D-representable. The following theorem improves the above conclusion.

JOTA: VOL. 61, NO. 1, APRIL 1989 !1

Theorem 4.3. The sum of a subregular function which is Gateaux differentiable at any point of a dense subset D in X and a D-representable one is D-representable.

Proof. The key to this property is the equality

4 / ( t ) = lim (Vf(y),d), (14) y ~ x + t d

y ~ D

for all x, d e X and all t ~ R+ such that &'(t) exists, with dp (t) =f(x + td), where f : X-~ R is a subregular function which is Gateaux differentiable at any point of D. In order to prove (14), we recall that the subregularity of f implies that

049(t) = {th'(t)} = {(x*, d)]x* ~ of(x + td)}.

On the other hand, if (y~) is a net in D converging to x+ td, the w*-upper semicontinuity of the multifunction Of ensures that every w*-accumulation point of (Vf(y , ) ) belongs to Of(x + td) and then the bounded net (Vf(y~), d) converges to 4;(t) . Since (y~) is an arbitrary net in D, equality (14) follows. Now, we can easily prove the result.

Let f l : X -~ ~ as above, and let f2: X ~ ~ be a D-representable function. Since for all (x, d) the derivatives ~b'l(t) and qS~(t) exist for a.e. teN+, we know by Theorem 3.1 that, for a.e. t such that both derivatives exist,

~b~(t) -< lim sup (Vfz(y), d); (15) y ~ x + t d

y ~ D

and from (14) and (15), we obtain, for all (x, d),

(61 + 4~2)'(t) -< lim sup (V(fl +f2)(Y), d), y ~ x + t d

y c D

for a.e. t e R~_ such that (~b~ + 4~2)'(t) exists. Thus, f ! +f2 is D-representable by Theorem 3.1. []

We complete this section with the following consequence of the above theorem.

Proposition 4.1. If f l and f2 are two D-representable functions such that ( i ) f l is subregular and (ii) Of2(x)COf~(x), for all x~ D, then f l - f , , is a constant function.

Proof. The subregularity of f~ implies that Of~(y)= {Vfl(y)}, for all y ~ D, and the inclusion (ii) implies in turn that Vf2(y)=Vfl(y) , for all y ~/9. From Theorem 4.3, we conclude the demonstration as in Proposition 3.1. []

12 JOTA: VOL. 61, NO. I, APRIL 1989

5. Semismooth Functions

Another important class of locally Lipschitzian functions in nonsmooth analysis is that of the semismooth functions.

Definition 5.1. A locally Lipschitzian function f : X ~ R will be called semismooth at x provided that, for all d ~ X, for every net (t~, ha) in •+ x X converging to (0, d), and for x* ~ Of(x+ t,h,,), the equality

lim (x*, d) = f ' ( x ; d) c t

holds. A function f is semismooth if it is so at every x ~ X. The following theorem gives a characterization ofsemismooth functions

in terms of a continuity property o f f ' ( . ; d).

Theorem 5.1. A locally Lipschitzian function f : X ~ R such that f ' (x; d) exists for all x, d ~ X, is semismooth at x if and only if the function

(t, h) ~ R+ x X ~ f ' ( x + th; d) (16)

is continuous at (0, d) for all d E X.

Proof. I f f is semismooth at x, for every net (t~, h~) in R+ x X converging to (0, d), we have

f ' ( x + t,h~; d) e (Of(x+ t,~h~), d),

and so we conclude that f ' ( x+ t~h~; d) converges to f ' (x; d); that is, the function in (16) is continuous at (0, d).

Conversely, let us suppose that the function in (16) is continuous at (0, d). Let (t~, h~) be a net converging to (0, d) and x*cOf(x+ t~h~); thus,

f°(x+t~h~; d)~(x*, d), for all ~; (17)

and, for any e > O, there exist an open neighborhood V of d and 6 > 0 such that

f ' ( x+th ' ;d)<_f ' (x ;d)+e/2 , for all h '~ V a n d t~]O, 3[. (18)

On the other hand, from formula (12), for every t c R+ and h ~ V, there exists h '~ V such that

f ' ( x + th'; d ) + e/2 >-f°(x + th; d). (19)

From (18), (19), and (17), we conclude that, for any e > 0, there exists C~o such that

f ' (x; d)+~>-(x *, d), for all a >-ao. (20)


Analogously, by using formula (13) we can see that, for any E>0 , there exists al such that

(x*, d) >-if(x; d) - ~, for all a >- a l . (21)

Then, from (20) and (21), we derive that, for any e > 0, there exists a2 (with a 2 ~ if0 and a2>-- Ogl) satisfying

I ( x * , d ) - f ' ( x ; d ) ] < - e , for all a ->a2 .

Remark 5.1. The use of Lemma 4.1 in the proof of Theorem 5.1 was suggested by Thibault, A direct demonstration of this theorem was given by the authors in Ref. 18, but it assumed f to be D-representable.

From the above theorem, it follows readily that the class of semismooth functions is a vector space, containing the subspace of the continuous differentiable functions.

Several properties of semismooth functions can be easily derived from Theorem 5.1. In what follows we give some.

Proposition 5.1. If f : X ~ ~ is a semismooth function which is Gateaux differentiabte at any point of a dense subset D in X, then f is directionally D-regular and a fortiori D-representable.

Proposition 5.2. I f f : X - ~ is a continuous convex function, then it is semismooth.

Proof. From the subregularity o f f and Theorem 3.2, one has the u.s.c. of the function in (16). There remains to be proven the 1.s.c., that is, for any E > 0 there exists a neighborhood V of 0 in X and 3 > 0 such that

f ' ( x ; d ) - ~ < - f ' ( x + t ( d + k ) ; d ) , for all k~ V a n d t o [ 0 , 3].

Since f is locally Lipschitzian, we have

i f (x , d) = lira t - l [ f ( x + t(d + k)) - f ( x ) ] ; t -~O + k ~ O

then, for any e > 0, there exist a neighborhood V' of 0 and 3 ' > 0 such that

f ' ( x , d) - e/2 < _ t - l [ f ( x + t(d + k)) - f ( x ) ]

= ( - t ) -~[f (x + td + tk - t(d + k)) - f ( x + td + tk)]

<--f'(x + td + tk; d + k) (by convexity)

<--if(x+ td + tk; d )+ f ' ( x + td + tk; k), (22)

for all k e V' and t ~ [0, 3'].


On the other hand, if p is the continuous seminorm corresponding to the Lipschitzian property o f f in a neighborhood W of x, we have

If'(z; k)l<-p(k), for all z~ W.

Then, there exist a neighborhood V of 0 with VC V' and 0 < ~5 < 8' such that

[f'(x+td+tk;k)[<-p(k)<-~/2, for all k~ V and t~[0 , 8].

From (22), we conclude the inequality

f ' ( x ; d ) - e < - f ' ( x + t ( d + k ) ; d ) , for atl k~ V and to [0 , ~]. E?

Remark 5.2. The demonstration of Proposition 5.2 was given by Thibault. A direct proof has been given by the authors in Ref. 18.

From the above proposition, it follows readily that the class of DC functions (difference of two convex functions) is a subspace of the class of semismooth functions.

Proposition 5.3. Let g ~ , . . . , g , be n functions from X to R and l :R n-*R, all of them semismooth. Then, the function f = 1 o g, with g = ( g l , . . . , gn), is semismooth.

ProoL For any (4 h )~ I~+xX, we have the formulas

f ' ( x + th; d ) = l'(g(x + th); g'(x + th; d))

= l'(g(x)+ t[g'(x; h)+ O(t)/t]; g'(x+ th; d)).

On the other hand, from the fact that l'(y + tk; • ) is Lipschitzian uniformly on (t, k), we can easily obtain the equality

lim t'(y+ tk; k ' )= lim l'(y+ tk; d), t - ~ O + t ~ O +

k~k '~d k--'.d

when the right-hand side limit exists. Then, from the semismoothness of h, g and Theorem 5.1, we find that

lim+ f ' ( x+th; d)= I'(g(x); g'(x; d ) ) = i f ( x ; d). (t,h)~(O ,d)

Theorem 5.1 allows us to conclude the proof. []

Proposition 5.4. Given a semismooth function f ~ : X ~ E whose generalized gradient aft is single-valued at any point of a dense subset D in X, and afortiori Gateaux differentiable at any point o f / 9 , if f2 :X ~ • is another function such that df2(x)COfl(x) for all x ~ X , then f l - f ~ is a constant function.

JOTA: VOL. 6t, NO. t, APRIL 1989 15

Proof. The inclusion Of2(x) COf~(x) and the fact thatf~ is semismooth imply the semismoothness of f2. Propositions 5.1 and 3.1 permit us to conclude the proof. [3

6. Marginal Functions

An important representation of the locally Lipschitzian functions corresponds to the so-called marginal functions.

Definition 6.1. Let U be a topological space, and let g : X x U-~ R be a function locally Lipschitzian on X uniformly in U. The function

f ( x ) =sup g(x,u) (23) u ~ U

is said to be the supmarginal function associated to g. In order for f to be locally Lipschitzian, we will assume that, for all x ~ X, the solution set M(x) = {u ~ U/ f (x ) = g(x, u)} is nonempty.

The study of supmarginal functions consists of obtaining properties for the function f by using properties of the function g. An important class of supmarginal functions are the so-called lower-C k functions (Refs. 5, 19).

In what follows, we will study the properties that must be satisfied by g for f to be subregular or semismooth. We will also study a directional derivative and generalized gradient characterization for the function f in terms of the directional derivative and generalized gradient of g ( . , u), which we will denote by g'~(x, u'; d) and Oxg(x, u), respectively.

We will begin by studying the directional derivative of a supmarginal function f. The most general result that we know in this sense, and which is given in Lemma 6.1, uses the following definitions.

Definition 6.2. The lower Dini derivative of a function f : X -~ N at a point x a X in the direction d is defined by

D f(x; d) = lira inf t - t [ f (x + td) - f ( x ) ] . t-+O +

(24)

Definition 6.3. A multifunction A : X ~ U is said to be semicontinuous (s.c.) at x e X, if for every net (x~) in X converging to x, there exist u ~ M(x) and a net (u~) with u~ E M(x~) such that u is an accumulation point of (us); i.e., there exists a subnet of (us) converging to u. A multifunction M is s.c. if it is so at every x c 32.


Lemma 6.1. L e t f : X - ~ be a supmarginal function. Let x, d ~ X be such that the multifunction t ~ R+ ~ M ( x + td) is s.c. at 0 and the function (t, u) ~ ~+ x U ~ D_~g(x + td, u; d) is u.s.c, at any point of {0} x M(x). Then, the directional derivative f ' (x; d) exists and can be characterized by

f ' (x ; d) = sup{g'(x, u; d)/u ~ M(x)}. (25)

Proof. See Ref. 20, Corollary 2.1. []

Remark 6.1. I f f is a supmarginal function, then M is s.c. at x ~ X if the function g(x;. ) is u.s.c, on U and there exists a relatively compact selection u(y) ~ M(y) for y in some neighborhood of x (in particular, if U is a compact topological space). On account of the hypothesis of semicontinuity of the Dini derivative of g, in practice it is in general sufficient to verify the u.s.c, for the function tcR+~D_xg(x+td, u; d) at 0, for each u~M(x ) . In this sense, it can be shown that the function t ~ R ÷ ~ D_~g(x+ td, u; d) is u.s.c. (resp., continuous) at 0, if and only if the function t c R+~ g(x+ td, u) is subregular (resp., semismooth) at 0; see Corollary 6.1 (resp., Corollary 6.3).

We now give a sufficient condition for a supmarginal function to be subregular, and we characterize its directional derivative and generalized gradient.

Theorem 6.1. Let f : X ~ R be a supmarginal function. If the multifunction M is s.c. at a point x ~ X and the function (y, u) ~ X × U-~ D~g(y, u; d) is u.s.c, at any point of{x} × M(x) , t hen f i s subregular at x and its directional derivative can be characterized by the formula

f ' ( x ; d) = sup{g'(x, u; d)/u ~ M(x)}. (26)

Proof. Formula (26) follows immediately from Lemma 6.1. Let (x~) and ( t , ) be such that x~-~x, t ~ O +, and

f ° (x ; d) = lira tSl[f(x~ + t,d) - f ( x ~ ) ] .

Since M is s.c. at x, there exists an accumulation point v in M(x) of a net (v~) with v~ E M(x , + t~d); and, from the definition of f and the Dini mean-value theorem (Ref. 21, Proposition 2), there exists ?~ c [0, t , ] such that

t~,~[f(x~ + t,~d) - f ( x , ) ] ~< t-~[g(x~ + tad, v,~) -g(x~, v~)]

<_ D_~g(x~ + 7~d, v~; d);


and taking the upper limit on ~, we deduce from the u.s.c, of _Dxg(', • ; d) and formula (26) the inequality

f ° ( x ; d)<- g'(x, v; d)<- f '(x; d),

which is precisely the subregularity of f at x.

From the above theorem, we can obtain an interesting generalization of Theorem 4.1.

Corollary 6.1. A locally Lipschitzian function f : X ~ ~ is subregular at x if and only if the function D f ( . , d) is u.s.c, at x for all d E X.

Proof. I f _Df(.; d) is u.s.c, at x for all d ~ X, from the theorem we conclude immediately the subregularity o f f at x. Conversely, i f f is subregular at x, we have D_f(x; d) = f ° ( x ; d), but D f(y; d) <fO(y; d) for any y c 3(, and f0( "; d) is u.s.c.; this implies the u.s.c, of _Df( .; d) at x. []

Corollary 6.2. I f f : X ~ ~ is a supmarginal function with the properties as in Theorem 6.1, then the generalized gradient of f a t x can be characterized by

Of(x) = cd{Oxg(X, u)/u ~ M(x)}. (27)

Proof. From Corollary 6.!,

g°(x, v; d) = g'x(x, v; d), for all v ~ M(x).

Formula (27) is then obtained directly from Theorem 6.1 since equality (26) can be written as

sup{(x*, d)lx* c Of(x)}

=sup{(x*,d)/x*£ U Oxg(x,v)} v~M(x)

= sup{(x*, d)/x* ~ -c-d{Oxg(X, v)/v ~ M(x)}}. []

Remark 6.2. From the above corollary, we see that the u.s.c, hypothesis of Dxg( •, • ; d) in Theorem 6.1 is a kind of subregularity of g at x uniformly in u. In fact, Corollary 6.1 says that, for u fixed, the function _Dxg(., u; d) is u.s.c, at x for all d 6 X if and only if g ( . , u) is subregular at x.

We now give a sufficient condition for a supmarginal function to be semismooth and we characterize its directional derivative.


Theorem 6.2. Let f : X ~ ~ be a supmarginal function. If for a fixed x e X and any d e X the multifunction (t, h) ~ g~+ × X ~ M ( x + th) is s.c. at (0, d) and the function (t, h, u ) e R + x X x U~D_xg(x+th , u; d) is continuous at any point of {0} × {d} x M ( x ) , then f is semismooth at x and its directional derivative can be characterized by

f ' ( x ; d) = sup{g'(x, u; d)lu c M(x)}. (28)

Proof. Formula (28) follows immediately from Lemma 6.1. Let (h,) , ( t ,) and (x*) be such that h~-~d, t~-+O + and x*eOf(x+t~h~) . First, we shall prove the inequality

lira sup ix*, d)<- f ' ( x ; d). (29) o~

Since ix*, d)<-f°(x+t~h~; d), for any neighborhood V, of ha and e > 0 there exist h" E V~ and ha c ]0, t 2] such that

(x*~, d) <- a 21[f(x + t~h2 + hod) - f ( x + t~h~)] + e;

and, since h ~ d , we can choose V~ such that h ' ~ d and then h ' + ( h ~ / t , ) d ~ d. Let (x~) be a subnet of (x*) such that

lim sup(x*, d) = lim(x~, d).

On the other hand, from the s.c. of M at x there must be a point u ~ M ( x ) and a subnet (u~) of (ut3), with ut~ ~ M(x+t13h;+h~d) , converging to u. T h e n ,

(x~, d) -< h ~l[g(x + t~h ~ + hrd , ur) - g(x + tvh ~, uv)] + e;

and, from a Dini mean-value theorem (Ref. 21), there exists £ ~ ]0, hv[ such that

ix*, d) <- D xg(x + tvh~ + £vd, uv; d) + e.

By using the continuity property assumed on the Dini derivative of g, and taking the limit over y in the above inequality, we obtain

lim sup(x*, d)<-g'~(x, u; d )+e .

Since e is arbitrary and u ~ M ( x ) , from formula (28) we deduce the desired inequality (29).

Now, we shall prove the inequality

f ' ( x ; d) <- lim infix*, d). (30) o t

Since ix*, d )>- fo(x+t , ho; d), we can obtain (as in the first part of this demonstration) the inequality

g'(x , u; d)<-lim infix*, d),

JOTA: VOL. 61, NO. !, APRIL 1989 19

where u ~ M ( x ) is an accumulation point of a net (u~) with u ~ M (x + t'~ h '~ ), t'~ --~ 0 +, and h'~ -> d. Let (u~) b e a subnet of ( u~ ) converging to u; we shall prove now that f ' (x; d)<-g'(x, u; d). If not so, there exist uo~ M(x) and e > 0 such that

g'(x, uo; d)>-g'~(x, u; d)+e,

and from the continuity hypothesis of the lower Dini derivative of g, jointly with the Lipschitzianity of D~g(x + th, u; • ) uniformly on (t, h, u), we see that there exists 130 such that

D xg(x+ th'~, Uo; h i ) -> Dxg(x + th'~, us; h'~) + e/2, _ ! for all/3 - /30 and t ~ [0, t~]; and, integrating from t = 0 to t - t~, we have

g(x + t'oh'~, Uo)-g(x, Uo)>- g(x + t'~h'~, us)-g(x , us)+ t~e/2,

that is,

g(x + t'~h'¢, Uo)+ g(x, u¢)>-f(x + t~h'c)+ f (x ) + t ; e /2 ,

which is a contradiction. Then, the semismoothness of f at x follows immediately from (29)

and (30). []

From the above theorem we can obtain an interesting generalization of Theorem 5.1:

Corollary 6.3. A locally Lipschitzian function f : X --> ~ is semismooth at x if and only if the function (t, h) e ~+ × X -> Df(x + th; d) is continuous at (0, d) for all d ~ X.

Remark 6.3. From the above corollary, we see that the continuity hypothesis of the lower Dini derivative of g in Theorem 6.2. is a kind of semismoothness of g at x uniformly in u. In fact, Corollary 6.3 says that, for u fixed, the function (t, h ) e ~ + x X--> D xg(x+ th; d) is continuous at (0, d) for all d c X if and only if g ( . , u) is semismooth at x.

Corollary 6.4. A lower Cl-function is semismooth.

7. Appendix

Proof of Theorem 3.1. First, let us observe that the D.representability of f at x is equivalent to

f ° ( x ; d) = tim sup(Vf(y), d), for all d ~ X. (31) y ~ x

y ~ D

Inequality (10) is easily obtained from equality (3t).


Let us prove the converse. For any e > 0, there exists some neighborhood V of x such that, for all z e V,

lim sup(Vf(y), d) -< lim sup(Vf(y), d) + e, (32) y ~ z y"~x

where all the upper limits are taken with y e/9. Let U be a neighborhood of x and 6 > 0 such that w + td e V, for all

we U and r e [ 0 , 6 ] . By writing (gw, d ( t ) = f ( w + t d ) one has, for all we U and h e ]0, 6[,

fo ~ ' f ( w + A d ) - f ( w ) = d~,,d(t) dt

<- [~ lira sup(Vf(y), d) dt [from (10)] 30 y .~w+td

-- [lira sup(Vf(y), d)+ e] dt [from (32)] 0 y-~x

= ,~ [lim sup(Vf(y), d ) + e]. y-~-x

Dividing by A and taking the upper limit when w ~ x and A ~ 0 ÷, we obtain, since E is arbitrary,

f ° ( x ; d) -< lim sup(Vf(y), d). y ~ x

The converse inequality is direct from the fact that f ° (y , d) -> (Vf(y), d) and f o ( . ; d) is u.s.c.s at x. []

References

1. ROCKAFELLAR, R. T., Convex Analysis, Princeton University Press, Princeton, New Jersey, 1970.

2. CLARKE, F. H , Optimization and Nonsmooth Analysis, Wiley-Interscience, New York, New York, 1983.

3. CLARKE, F. n., Generalized Gradients and Applications, Transactions of the American Mathematical Society, Vol. 205, pp. 247-262, 1975.

4. ROCKAFELLAR, R. T., Directionally Lipschitzian Functions and Subdifferential Calculus, Proceedings of the London Mathematical Society, Vol. 39, pp. 331-355, 1979.

5. ROCKAFELLAR, R. T., Favorable Classes of Lipschitz Continuous Functions in Subgradient Optimization, Nondifferentiable Optimization, Edited by E. Nurminski, Pergamon Press, New York, New York, 1982.

6. MIFFLIN, R., Semismooth and Semiconvex Functions in Optimization, SIAM Journal on Control and Optimization, Vol. 15, pp. 959-972, 1977.


7. PENOT, J. P., On Favorable Classes of Mappings in Nonlinear Analysis and Optimization, Preprint, Universit6 de Pau, Pau, France, 1986.

8. MOREAU, J. J., Fonctionnelles Convexes, Seminaire sur les Equations aux D6riv6es Partielles, College de France, Paris, France, 1986.

9. CLARKE, F.H., NecessaD, Conditions for Nonsmooth Problems in Optimat Controt and the Calculus of Variations, PhD Thesis, University of Washington, Seattle, Washington, 1973.

10. CHRISTENSEN, J. P. R., Topology and BoreI Structures, North-Holland, Amster- dam, Holland, 1974.

11. ARONSZAJN, N., Differentiability of Lipschitzian Functions, Studia Mathematica, VoI. 57, pp. 147-190, 1976.

12. MIGNOT, F., Controle clans les Inequations Variationnelles Ellipliques, Journal of Functional Analysis, Vol. 22, pp. 130-185, 1976.

13. THIBAULT, L., On Generalized Differentials and Subdifferentials of Lipschitz Vector-Valued Functions, Nonlinear Analysis, Vol. 6, pp. 1037-1053, 1982.

14. CHUNG, S. S., Remarques sur le Gradient Generalis~, Journal de Mathematiques Pures et Appliqu6es, Vol. 61, pp. 301-310, 1982.

15. LEBOURG, G., Generic Differentiability of Lipschitzian Functions, Transactions of the American Mathematical Society, Vol. 256, pp. 125-144, 1979.

16. ZIVKOV, N., Generic Gateaux Differentiability of Locally Lipschitzian Functions, Constructive Function Theory '81, Sofia, Bulgaria, pp. 590-594, 1983.

t7. SHOR, N. Z., On One Class of Almost-Differentiable Functions and a Method for Minimizing Functions of This Class, Kibernetika, Volo 4, pp. 65-70, 1972.

18. CORREA, R., and JOFRE, A., Some Properties of Semismooth and Regular Functions in Nonsmooth Analysis, Proceedings of the I FIP Working Conference, Edited by L. Contesse, R. Correa, and A. Weintraub, Springer-Verlag, Berlin, Germany, 1987.

19. SPINGARN, J. E., Submonotone SubdifferentiaIs of Lipschitz Functions, Transac- tions of the American Mathematical Society, Vol. 264, pp. 77-89, 1981.

20. CORREA, R., and SEEGER, A., Directional Derivative of a Minimax Function, Nonlinear Analysis, Vol. 9, pp. 13-22, 1985.

21. H I RIART-URRUTY, J. B., Note on the Mean. Value Theorem for Convex Functions. Boltettino della Unione Matematica Italiana, Vol. 17B, pp. 765-775, 1980.

Documents

Tangentially continuous directional derivatives in nonsmooth analysis