Journal of Convex Analysis Volume 7 (2000), No. 2, 243–269
Elements of Quasiconvex Subdifferential Calculus
Jean-Paul Penot Universit´ede Pau, Facult´edes Sciences, D´epartement de Math´ematiques, 64000 Pau, France. e-mail: [email protected]
Constantin Z˘alinescu∗ Universit´e“Al. I. CuzaÔ Ia¸si,Facult´ede Math´ematiques, Bd. Copou Nr. 11, 6600 Ia¸si,Romania. e-mail: [email protected]
Received July 7, 1998 Revised manuscript received April 24, 1999
A number of rules for the calculus of subdifferentials of generalized convex functions are displayed. The subdifferentials we use are among the most significant for this class of functions, in particular for quasiconvex functions: we treat the Greenberg-Pierskalla’s subdifferential and its relatives and the Plastria’s lower subdifferential. We also deal with a recently introduced subdifferential constructed with the help of a generalized derivative. We emphasize the case of the sublevel-convolution, an operation analogous to the infimal convolution, which has proved to be of importance in the field of quasiconvex functions. We provide examples delineating the limits of the rules we provide.
Keywords: harmonic sum, incident derivative, lower subdifferential, normal cone, quasiconvex function, subdifferential set, sublevel-convolution
1991 Mathematics Subject Classification: 26B25, 26A51, 26A27, 90C26
1. Introduction It is the purpose of the present paper to give some calculus rules for known subdifferentials which are used for generalized convex functions. We feel that without such rules, a number of problems involving generalized convex functions can hardly be dealt with. Some rules already exist (see [10], [5] for instance); but we endeavour to make a systematic study. In fact we concentrate on the most usual subdifferentials: the subdifferentials of the Greenberg-Pierskalla type and the lower subdifferential of Plastria. We also deal with a notion introduced in [5] which has not received much attention yet, although it seems to be well adapted to the case of quasiconvex functions. The operations we consider are the most useful ones: composition with a linear map into the source, composition with an increasing function at the range, suprema, performance functions. We also deal with an operation which has attracted some attention during the last few years, the sublevel-convolution h := f3g of the two functions f, g given by
h(w) := inf f(w x) g(x): x X ∈ − ∨ This operation, which is the analogue of the infimal convolution of convex analysis (the
∗The contribution of this author was done during his stay at Universit´ede Pau, spring 1998.
ISSN 0944-6532 / $ 2.50 c Heldermann Verlag 244 J.-P. Penot, C. Z˘alinescu/ Elements of quasiconvex subdifferential calculus
sum being replaced by the supremum operation ) is of fundamental importance for quasiconvex analysis inasmuch as the usual sum does∨ not preserve quasiconvexity whereas supremum does (see [1], [11], [12], [13], [14], [L-V]...). Moreover, one can check that the strict sublevel sets of h are given by
[h < r] = [f < r] + [g < r];
whereas the sublevel sets satisfy
[h r] = [f r] + [g r] ≤ ≤ ≤ whenever the infimum is attained in the formula defining h (then one says that the sublevel-convolution is exact). The interest of such an operation for regularization pur- poses is shown elsewhere (see [11], [9]). Our study relies on techniques from convex analysis which are described in our paper [8]. Section 2 is devoted to the case of subdifferentials of Greenberg-Pierskalla type. Such subdifferentials are easy to deal with, but they differ drastically from the usual Fenchel subdifferential as they are cones. The lower subdifferential of Plastria which is studied in section 3 also suffers from such a difference, but to a less extent, as it is a shady subset of the dual (i.e. is stable under homotheties of rate greater than 1), hence is unbounded or empty. The last section is devoted to a subdifferential which is a kind of compromise between the Plastria subdifferential and the incident subdifferential. For a Lipschitzian quasiconvex function, it has always nonempty values. A number of subdifferentials which have been proposed for the study of generalized quasi- convex functions have not been considered here for the sake of brevity (see [5] for a recent survey). However, we hope the present study will enable the reader to get a more precise idea of the possibilities and the limitations of what is called “quasiconvex analysisÔ in [6].
2. Subdifferentials of normal type
Hereafter, f is a function defined on a Hausdorff locally convex space X with dual X∗, ; ; f x with values in R or R := R and we suppose that ( 0) is finite. As ∪ {∞} ∪{−∞ ∞} in [8] we denote by R+ (resp. P) the set of nonnegative (resp. positive) real numbers. We devote this section to some rules for the calculus of subdifferentials of Greenberg- Pierskalla type. These subdifferentials are among the simplest concepts. They are akin to the normal cones to the corresponding sublevel sets [f f(x )] : ≤ 0 @νf(x ) := N([f f(x )]; x ); 0 ≤ 0 0 where the normal cone to a subset C of X at x X is given by 0 ∈
N(C; x ) := x∗ X∗ x∗; x x 0 x C : 0 { ∈ | h − 0i ≤ ∀ ∈ } We also consider the case of the star subdifferential introduced in [5], or rather a slight modification of it (it differs from the definition in [5] by 0 when x0 is not a minimizer). ~ { } It is as follows: x∗ @ f(x ) iff x∗; x x 0 for each x [f < f(x )]. In other terms 0 ∈ 0 h 0 − 0i ≤ ∈ 0 ~ @ f(x0) := N([f < f(x0)]; x0): J.-P. Penot, C. Z˘alinescu/ Elements of quasiconvex subdifferential calculus 245
Both definitions are variants of the classical Greenberg-Pierskalla’s subdifferential, given x @ f x x ; x x < x f < f x by 0∗ ∗ ( 0) iff 0∗ 0 0 for each [ ( 0)]. These subdifferentials are cones and∈ thus theyh are large− i and quite different∈ from the usual Fenchel subdifferential. Obviously ~ ν ~ @∗f(x ) @ f(x );@ f(x ) @ f(x ): 0 ⊂ 0 0 ⊂ 0 ~ Observe that x0 is a minimizer of f iff 0 @∗f(x0) iff @∗f(x0) = X∗, iff @ f(x0) = X∗. In ν ∈ ~ contrast, one always has 0 @ f(x0), 0 @ f(x0). Thus, although these subdifferentials are similar, they have distinct∈ features. Let∈ us give a criterion for their coincidence up to zero. Here f is said to be radially u.s.c. (upper semicontinuous) at some point x if for any y X the function t f(x + ty) is u.s.c. at 0. ∈ 7→ ~ Lemma 2.1. If @∗f(x0) is nonempty, then @ f(x0) = cl(@∗f(x0)): If f is radially u.s.c. ~ at each point of [f < f(x0)] then @ f(x0) = @∗f(x0) 0 . If there is no local minimizer 1 ~ ν ∪ { } of f in f − (f(x0)) then @ f(x0) = @ f(x0):
Proof. The first assertion is easy. For the second assertion, it suffices to observe that a linear form which is nonnegative on an absorbant subset is 0: 1 If there is no local minimizer of f in f − (f(x0)) then the sublevel set [f f(x0)] is ~≤ contained in the closure of the strict level set [f < f(x )]. Thus, any x∗ @ f(x ) being 0 0 ∈ 0 bounded above by x∗; x on [f < f(x )] is also bounded above by x∗; x on [f f(x )]: h 0 0i 0 h 0 0i ≤ 0 x∗ N([f f(x )]; x ): 0 ∈ ≤ 0 0 We observe that each subdifferential @? among the three we consider is homotone, i.e. ? ? given f, g such that f g and f(x0) = g(x0) one has @ f(x0) @ g(x0). The first assertion of the next lemma≤ follows from this observation. ⊂
Lemma 2.2. Let (fi)i I be an arbitrary family of functions finite at x0 and let f := ∈ ? ~ ν infi I fi. Suppose I(x0) := i I fi(x0) = f(x0) is nonempty. Then, for @ = @∗;@ ;@ ∈ one has { ∈ | } @?f(x ) @?f (x ): 0 ⊂ i 0 i I(x ) ∈\0 ~ If I(x0) = I, then equality holds for @∗;@ ; if moreover I(x) is nonempty for each x X (in particular if I is finite) then equality holds for @ν. ∈
Proof. For the second assertion, one observes that if I(x ) = I and if x [f < f(x )] 0 ∈ 0 (resp. x [f f(x0)] and if I(x) is nonempty), then, for some i I one has x [fi < f (x )] (resp.∈ x≤ [f f (x )]). ∈ ∈ i 0 ∈ i ≤ i 0 A similar result holds for suprema. Here we denote by co A (resp. coA) the convex hull (resp. the closed convex hull) of A.
Proposition 2.3. Let (fi)i I be an arbitrary family of functions finite at x0 and let f f I x∈ i I f x f x @? := supi I i. Suppose ( 0) := i( 0) = ( 0) is nonempty. Then, for = ~ ν ∈ { ∈ | } @∗;@ ;@ one has
@?f(x ) co @?f (x ) : (2.1) 0 ⊃ i 0 i I(x ) ∈[0 246 J.-P. Penot, C. Z˘alinescu/ Elements of quasiconvex subdifferential calculus
ν Equality holds for @ if I is finite and equal to I(x0), if Ci := [fi fi(x0)] is convex for i I x k I x C ≤ C X each ( 0) and if either for some ( 0) one has k ( i=k int i) = , or is a ∈ ∈ ∩ 6 6 ∅ C XI C Banach space, each i is closed and = R+ ∆ i I i ,T where ∆ is the diagonal of XI : − ∈ Q ¡
When I(x0) has two elements j; k only, the qualification condition of the preceding state- ment can be rewritten in the simpler (and more familiar) form X C C : = R+( j k) − ? Proof. Again the first assertion is a consequence of homotonicity. When @ f(x0) is closed, one can replace co by co in (2.1). The second assertion is a consequence of Lemma 2.1 and f f x f f x of the calculus of normal cones ([15] for instance) since [ ( 0)] = i I [ i i( 0)] ≤ ∈ ≤ when I = I(x0) is finite. T Proposition 2.4. Let f : X R , g : Y R and M : X Y R , M x; y f x g y → z ∪{∞}x ; y X→ Y∪{∞} r f ×x →g y∪{∞} ( ) = ( ) ( ). Consider 0 := ( 0 0) such that 0 := ( 0) = ( 0) R. ? ∨ ~ ν ∈ × ∈ Then for @ = @∗;@ ;@ one has
@?M(x ; y ) @?f(x ) @?g(y ); 0 0 ⊃ 0 × 0 ? ν with equality for @ = @ : When x0 and y0 are not local minimizers of f and g, respectively, equality holds for @? = @~; and
~ ~ @∗M(x ; y ) = @∗f(x ) @ g(y ) @ f(x ) @∗g(y ) : 0 0 0 × 0 ∪ 0 × 0 ¡ ¡ Proof. The first assertions are easy consequences of the preceding result. Let z∗ := ~ (x∗; y∗) @ M(z ). Thus ∈ 0
x x ; x∗ + y y ; y∗ 0 x [f < f(x )]; y [g < g(y )]: (2.2) h − 0 i h − 0 i ≤ ∀ ∈ 0 ∀ ∈ 0
Since x0 and y0 are not local minimizers of f and g; respectively, we get that
x x ; x∗ 0 x [f < f(x )]; y y ; y∗ 0 y [g < g(y )]; h − 0 i ≤ ∀ ∈ 0 h − 0 i ≤ ∀ ∈ 0 ~ ~ i.e. x∗ @ f(x0) and y∗ @ g(y0). Now let us suppose that z∗ @∗M(z0): If x∗ ~ ∈ ∈ ∈ ∈ @ f(x ) @∗f(x ) there exists u [f < f(x )] such that u x ; x∗ = 0; then, as 0 \ 0 ∈ 0 h − 0 i u x ; x∗ + y y ; y∗ < 0 for each y [g < g(y )]; we get that y∗ @∗g(y ): h − 0 i h − 0 i ∈ 0 ∈ 0 The following result about performance functions is simple and useful. Proposition 2.5. Let W and X be locally convex spaces and let F : W X R; × → p(w) := inf F (w; x);S(w) := x X F (w; x) = p(w) : x X ∈ { ∈ | }
? ~ Then for @ = @∗;@ , for any w0 W such that p(w0) is finite, and for any x0 S(w0) w @?p w w ; @∈?F w ; x S w ∈w W one has 0∗ ( 0) iff ( 0∗ 0) ( 0 0). When ( ) is non-empty for each , the same result∈ holds for @ν: ∈ ∈
Recall that if F is (convex) quasiconvex then p is (convex) quasiconvex. J.-P. Penot, C. Z˘alinescu/ Elements of quasiconvex subdifferential calculus 247
Proof. The result is essentially a consequence of the fact that w [p < p(w0)] iff there exists x X such that (w; x) [F < F (w ; x )] and of the relation∈ ∈ ∈ 0 0 (w∗; 0); (w w ; x x ) = w∗; w w : h 0 − 0 − 0 i h 0 − 0i The last assertion is obvious.
We also observe that when (w∗; 0) @∗F (w ; x ) then x S(w ): 0 ∈ 0 0 0 ∈ 0 In the sequel, given another locally convex space Y , we denote by L(X;Y ) the class of continuous linear maps from X into Y and for A L(X;Y ), At stands for the transpose of A. ∈ Proposition 2.6. Let f = g A, where A L(X;Y ), and g : Y R is finite at ? ◦ ∈ ~ ν → y0 = Ax0. Then, if @ is one of the subdifferentials @∗;@ ;@ , one has
At @?g(y ) @?f(x ): 0 ⊂ 0 If A is onto then ¡ t 1 ? ? (A )− @ f(x0) = @ g(Ax0): A R At X Y If is onto and ( ) is w∗-closed (in particular¡ if and are Banach spaces) one t ν ν t ~ has A (@ g(y0)) = @ f(x0); if moreover y0 is not a minimizer of g, then A (@ g(y0)) = ~ t @ f(x0) and A (@∗g(y0)) = @∗f(x0):
~ ~ Proof. Let us consider the case of @ . Given y∗ @ g(y ), for any x [f < f(x )] we ∈ 0 ∈ 0 have y := A(x) [g < g(y0)] hence y∗ A; x x0 = y∗; y y0 0 and x∗ := y∗ A ~ ∈ h ◦ − i h − i ≤ ◦ ∈ @ f(x0): t 1 ~ Suppose now that A is onto and y∗ (A )− (@ f(x0)). Let us show that y∗ belongs to ~ t ~∈ @ g(Ax0). Indeed, x∗ := A y∗ @ f(x0) and for each y Y such that g(y) < g(Ax0) there exists x X with y = Ax∈. So, ∈ ∈ t y Ax ; y∗ = Ax Ax ; y∗ = x x ;A y∗ 0: h − 0 i h − 0 i − 0 ≤ ~ Therefore y∗ @ g(Ax ). The other cases are similar.ª « ∈ 0 t ν 1 Suppose A is onto and R(A ) is w∗-closed; let x∗ @ f(x ). Given x A− (0) and ∈ 0 ∈ " 1; 1 we have f(x0 + "x) = f(x0) hence x∗; "x 0, and so x∗; x = 0. Thus ∈ {− } t h i ≤ h i x∗ N(A)⊥ = A (Y ∗) and there exists a continuous linear form y∗ on Y such that ∈ t 1 ν ν x∗ = y∗ A. It follows that y∗ (A )− (@ f(x )) = @ g(y ): ◦ ∈ 0 0 ~ When x0 is not a minimizer of f and x∗ @ f(x0), taking x X such that f(x) < f(x0) 1 ∈ ∈ one obtains that for any u A− (0) one has f(x + u) = g(Ax) = f(x) < f(x ) hence ∈ 0 x∗; x + u x0 0. It follows that x∗; u = 0 and x∗ = y∗ A for some y∗ Y ∗. As h − i ≤ ~ h i ◦ ∈ above one gets that y∗ @ g(y ) (and y∗ @∗g(y ) when x∗ @∗f(x )). ∈ 0 ∈ 0 ∈ 0 Another important chain rule is the following one. Proposition 2.7. Let g : X R and let ’ : R R be a nondecreasing ’ → ∪{∞}x X →’ t ∪{∞} t g x : function. Set ( ) = and consider 0 such that ( 0) R, where 0 := ( 0) Let f = ’ g. Then∞ ∞ ∈ ∈ ◦ ~ ~ ν ν @∗g(x ) @∗f(x );@ g(x ) @ f(x ) and @ g(x ) @ f(x ): 0 ⊂ 0 0 ⊂ 0 0 ⊃ 0 248 J.-P. Penot, C. Z˘alinescu/ Elements of quasiconvex subdifferential calculus
If g(x0) is not a local minimizer of ’ the first two inclusions are equalities while if g(x0) ν ν is not a local maximizer of ’ one has @ g(x0) = @ f(x0). If there is no local minimizer 1 ν ν ~ ~ of f in f − (f(x0)) then @ f(x0) = @ g(x0) = @ g(x0) = @ f(x0):
Proof. The inclusions are immediate since [f < f(x0)] [g < g(x0)] and [f f(x0)] [g g(x )]. When g(x ) is not a local minimizer (resp.⊂ maximizer) of ’ one≤ has [f⊃ < ≤ 0 0 f(x0)] = [g < g(x0)] (resp. [f f(x0)] = [g g(x0)]). The last assertion is a consequence ≤ ν ≤ ~ ν ~ of the other ones, and of the relations @ g(x0) @ g(x0), @ f(x0) = @ f(x0) when f satisfies the last assumption of Lemma 2:1. ⊂
The previous results can be used to compute the subdifferential of the sublevel-convolution h := f3g. The following rule, which mimics the classical rule for the infimal convolution, is a simple consequence of the formulas about sublevel sets. x ; y X x y z f x g y h z Proposition 2.8. Given 0 0 with 0 + 0 = 0, ( 0) = ( 0) = ( 0) R one has for h := f3g ∈ ∈
@~f(x ) @~g(y ) @~h(z ); 0 ∩ 0 ⊂ 0 ~ ~ @ f(x ) @∗g(y ) @∗f(x ) @ g(y ) @∗h(z ); 0 ∩ 0 ∪ 0 ∩ 0 ⊂ 0 @νf(x ) @νg(y ) @νh(z ); ¡ 0 ∩ 0¡ ⊃ 0 with equality in this last relation when for each z X there exist x; y X such that ∈ ∈ x + y = z, f(x) g(y) = h(z) i.e. the sublevel-convolution is exact. If x0 and y0 are not local minimizers∨ of f and g, respectively, equality holds in the first and the second relations.
h < h z f < f x g < g y z @~f x @~g y Proof. Since [ ( 0)] = [ ( 0)] + [ ( 0)], for any 0∗ ( 0) ( 0) and any x [f < f(x )], y [g < g(y )] we have ∈ ∩ ∈ 0 ∈ 0
z∗; x + y z = z∗; x x + z∗; y y 0; h 0 − 0i h 0 − 0i h 0 − 0i ≤ ~ ~ ~ hence z∗ @ h(z ). Similarly, when z∗ @ f(x ) @∗g(y ) or z∗ @∗f(x ) @ g(y ) 0 ∈ 0 0 ∈ 0 ∩ 0 0 ∈ 0 ∩ 0 one gets z∗ @∗h(x ): 0 ∈ 0 x y f g z Conversely, when 0 and 0 are not local minimizers of and , respectively, given 0∗ ~ ∈ @ h(z0), for any x [f < f(x0)], taking a sequence (yn) in [g < g(y0)] with limit y0 we observe that ∈ z ; x x z ; x y x y ; 0∗ 0 = lim 0∗ + n 0 0 0 h − i n h − − i ≤ ~ ~ hence z∗ @ f(x ). Similarly, z∗ @ g(y ): An argument similar to the one in the last 0 ∈ 0 0 ∈ 0 part of the proof of Proposition 2.4 yields the case z∗ @∗h(z ): 0 ∈ 0 The assertion about @ν is a consequence of the inclusion [f f(x )] + [g g(y )] [h ≤ 0 ≤ 0 ⊂ ≤ h(z0)] which is an equality when the sublevel convolution is exact.
3. Subdifferential calculus for lower subdifferentials We devote the present section to the calculus with the lower subdifferential of Plastria < [10]. It is defined by x∗ @ f(x ) iff 0 ∈ 0
x∗; x x f(x) f(x ) x [f < f(x )]: h 0 − 0i ≤ − 0 ∀ ∈ 0 J.-P. Penot, C. Z˘alinescu/ Elements of quasiconvex subdifferential calculus 249
We leave to the reader most of the task of writing the necessary modifications for the infradifferential of Guti´errez[2] given by x∗ @≤f(x ) iff 0 ∈ 0
x∗; x x f(x) f(x ) x [f f(x )]: h 0 − 0i ≤ − 0 ∀ ∈ ≤ 0 Obviously, one has
< ν < @≤f(x ) @ f(x );@≤f(x ) @ f(x );@ f(x ) @∗f(x ): 0 ⊂ 0 0 ⊂ 0 0 ⊂ 0
Again, we start with the observation that these subdifferentials are homotone. Now we consider the performance function p associated to a perturbation F : W X R by × → p(w) = infx X F (w; x): ∈ Lemma 3.1. p w F w ; x : w @
Proof. (see also [5]). (i) Let F (w; x) < F (w0; x0); it follows that p(w) < p(w0), whence
w w ; w∗ + x x ; 0 p(w) p(w ) F (w; x) F (w ; x ); h − 0 i h − 0 i ≤ − 0 ≤ − 0 0 < which shows that (w∗; 0) @ F (w ; x ). ∈ 0 0 (ii) Suppose that p(w ) < F (w ; x ). Then there exists x X such that F (w ; x) < 0 0 0 ∈ 0 F (w0; x0), whence
0 = w w ; w∗ + x x ; 0 F (w ; x) F (w ; x ) < 0; h 0 − 0 i h − 0 i ≤ 0 − 0 0 a contradiction. Therefore p(w0) = F (w0; x0). Let now w W be such that p(w) < p(w ). Then there exists (x ) X such that (F (w; x )) p∈(w); we may suppose that 0 n ⊂ n → F (w; xn) < F (w0; x0) = p(w0) for every n. Hence w w ; w x x ; F w; x F w ; x n : 0 ∗ + n 0 0 ( n) ( 0 0) N h − i h − i ≤ − ∀ ∈ < Taking the limit we get w w ; w∗ p(w) p(w ). Therefore w∗ @ p(w ). h − 0 i ≤ − 0 ∈ 0 Let us turn to composition with a continuous linear map. g Y A L X;Y x X g Ax Proposition 3.2. Let : R, ( ) and 0 be such that ( 0) R. Then, for f := g A → ∈ ∈ ∈ ◦ @ t Moreover, if R(A ) is w∗-closed (in particular if X and Y are Banach spaces) and Ax0 is not a minimizer of g, then < t < @ f(x0) = A (@ g(Ax0)) : 250 J.-P. Penot, C. Z˘alinescu/ Elements of quasiconvex subdifferential calculus Proof. The first assertion is contained in [10] Th. 3.5. Let us recall its simple proof. Let < y∗ @ g(Ax ) and f(x) < f(x ). Then g(Ax) < g(Ax ), and so ∈ 0 0 0 t x x ;A y∗ = Ax Ax ; y∗ g(Ax) g(Ax ) = f(x) f(x ): − 0 h − 0 i ≤ − 0 − 0 t < Therefore Aª y∗ @ f(x«): ∈ 0 t 1 < t < Suppose now that A is onto and y∗ (A )− (@ f(x0)). Then x∗ := A y∗ @ f(x0). Let y Y be such that g(y) < g(Ax ). There∈ exists x X such that y = Ax.∈ So, ∈ 0 ∈ t y Ax ; y∗ = Ax Ax ; y∗ = x x ;A y∗ f(x) f(x ) = g(y) g(Ax ): h − 0 i h − 0 i − 0 ≤ − 0 − 0 < Therefore y∗ @ g(Ax ): ª « ∈ 0 t Suppose now that R(A ) is w∗-closed and Ax0 is not a minimizer for g. It follows that t < R(A ) = (ker A)⊥. Let x∗ @ f(x0) @∗f(x0). By the proof of Proposition 2.6 x∗ t t ∈ ⊂ < ∈ R(A ). Thus x∗ = A y∗ for some y∗ Y ∗. By (3.1) we have that y∗ @ g(Ax0), and so t < ∈ ∈ x∗ A (@ g(Ax )). ∈ 0 < Remark 3.3. The same conclusions hold for @ replaced by @≤ in the preceding state- ment, even without asking that Ax0 is not a minimizer of f in the last part, with the same proof. The following result is a special case of what precedes, taking for A a projection. It can also be proved directly. Corollary 3.4. Let f : X R and let f : X Y R be defined by f(x; y) = f(x). If x ; y X Y f x → x × → f ( 0 0) with ( 0) R and if 0 is not a minimizer of , then ∈ × ∈ e e @ Proof. (i) Let us denote ’ g by f. Suppose first that t0 is a minimizer of ’. Then x0 is a minimizer of f, and so relation◦ (3.2) is obvious. < < Suppose now that t is not a minimizer of ’, and let t∗ @ ’(t ) and x∗ @ g(x ). 0 ∈ 0 ∈ 0 Since t is not a minimizer and ’ is nondecreasing, t∗ > 0. Consider x X such that 0 ∈ f(x) < f(x0). Then g(x) < g(x0), whence t∗(g(x) g(x )) ’(g(x)) ’(g(x )); x x ; x∗ g(x) g(x ): − 0 ≤ − 0 h − 0 i ≤ − 0 J.-P. Penot, C. Z˘alinescu/ Elements of quasiconvex subdifferential calculus 251 Multiplying the second relation by t∗ > 0, we obtain that x x0; t∗x∗ f(x) f(x0). < h − i ≤ − Therefore t∗x∗ @ f(x ): ∈ 0 < (ii) Let now u∗ @ f(x ). As t is not a local minimizer of ’, we have that [f < f(x )] = ∈ 0 0 0 [g < g(x0)]. Thus x x ; u∗ f(x) f(x ) < 0 x [ g < g(x )[: (3.4) h − 0 i ≤ − 0 ∀ ∈ 0 (a) Let t 0. Taking x [g < 0] and s > 0, we have 2x +sx [g < g(x )] = [ f < f(x ] 0 ≤ ∈ 0 ∈ 0 0 hence x0 + sx; u∗ 0. Therefore x0; u∗ = lims 0 x0 + sx; u∗ 0. Let t < t0. From → (3.4) weh obtain thati ≤ h i h i ≤ x; u∗ x ; u∗ + ’(t) ’(t ) x [g t]: h i ≤ h 0 i − 0 ∀ ∈ ≤ As inf g < t < 0, applying [8] Prop. 5.2 with its notation, we get sup w; u∗ w [g t] = t‹ (u∗) x ; u∗ + ’(t) ’(t ) < 0: (3.5) {h i | ∈ ≤ } ∂g(0) ≤ h 0 i − 0 ‹ u > = @g @g w Thus := ∂g(0)( ∗) 0. Since 0 (0) and (0) is ∗-closed, is finite. Using again 1 ∈ [8] Prop. 5.2 we get w∗ := − u∗ @g(0). By (3.5) we obtain that ∈ t x ; w∗ + ’(t) ’(t ) < x ; w∗ t < t : ≤ h 0 i − 0 h 0 i ∀ 0 Dividing by , taking the limit as t t0 and using the inclusion w∗ @g(0), we get → < ∈ < t0 = x0; w∗ = g(x0) = t0, whence w∗ @g(x0) @ g(x0) and @ ’(t0). The conclusionh followsi in this case. ∈ ⊂ ∈ (b) Suppose now that t > 0. Thus, for each t ]0; t [, using again [8] Prop. 5.2, we have 0 ∈ 0 0 sup w; u∗ w [g t] = t« (u∗) x ; u∗ + ’(t) ’(t ) < : ≤ {h i | ∈ ≤ } ∂g(0) ≤ h 0 i − 0 ∞ u @g u @g +@g g x ; u Therefore, ∗ R+ (0). If ∗ 0 (0) = 0 (0) = (dom )−, then 0 ∗ 0 ’∈1 t ’ t < ∈ · « uh > i ≤ and so 0 ( 2 0) ( 0) 0, a contradiction. Therefore := ∂g(0)( ∗) 0 and 1 ≤ − w∗ := − u∗ @g(0). Thus ∈ t x ; w∗ + ’(t) ’(t ) < x ; w∗ t ]0; t [: ≤ h 0 i − 0 h 0 i ∀ ∈ 0 < As above, it follows that x ; w∗ = g(x ) = t , whence w∗ @g(x ) @ g(x ) and h 0 i 0 0 ∈ 0 ⊂ 0 (t t0) ’(t) ’(t0), for each t ]0; t0[. Taking x = 0 in (3.4), we see that the preceding− ≤ inequality− is also valid for t =∈ 0, while for t < 0 the inequality is obvious. Thus @<’(t ) again. ∈ 0 Example 3.6. Let ’ : R R and g : R R be defined by ’(t) := t3, g(x) = max(x; 1). Then one has @→<’(0) = but @<→(’ g)(0) = [1; [. This example shows the necessity− of restrictive assumptions∅ such as sublinearity◦ of g:∞ Let us observe that for any function g and any x0 X such that t0 := g(x0) = inf g @<’ t ∈ X ∈ R, relation (3.3) holds if and only if ( 0) = . When dim = 1, relation (3.3) 6 ∅ holds whenever t0 = 0 and g is sublinear. One may ask whether the assumptions of the statement (ii) are crucial.6 252 J.-P. Penot, C. Z˘alinescu/ Elements of quasiconvex subdifferential calculus Example 3.7. Let ’; ’; ’; g : R R and g : R2 R be defined by: ’(t) = max(t; 0), ’(t) := min(1; 2’(t)), g(r; s) = r and→ → e e 2t if t 0, 1 x if x 0, ’(t) = ≤ g(x) = 2 ≤ (t if t > 0, (x if x > 0. e e x ; t g x ’ @<’ t @