Journal of Mathematical Analysis and Applications 237, 30᎐42Ž. 1999 Article ID jmaa.1999.6437, available online at http:rrwww.idealibrary.com on On the Subdifferentials of Quasiconvex and Pseudoconvex Functions and Cyclic Monotonicity* ² ³ View metadata, citation and similarAris papers Daniilidis at core.ac.ukand Nicolas Hadjisavvas brought to you by CORE provided by Elsevier - Publisher Connector Department of Mathematics, Uni¨ersity of the Aegean, 83200 Karlo¨assi, Samos, Greece Submitted by Arrigo Cellina Received January 7, 1998 The notions of cyclic quasimonotonicity and cyclic pseudomonotonicity are introduced. A classical result of convex analysis concerning the cyclic monotonicity of theŽ. Fenchel᎐Moreau subdifferential of a convex function is extended to corresponding results for the Clarke᎐Rockafellar subdifferential of quasiconvex and pseudoconvex functions. The notion of proper quasimonotonicity is also introduced. It is shown that this new notion retains the characteristic property of quasimonotonicityŽ i.e., a lower semicontinuous function is quasiconvex if and only if its Clarke᎐Rockafellar subdifferential is properly quasimonotone. , while it is also related to the KKM property of multivalued maps; this makes it more useful in applications to varia- tional inequalities. ᮊ 1999 Academic Press 1. INTRODUCTION Let X be a Banach space and f: X ª R jqÄ4ϱ a lower semicontinu- ousŽ. lsc function. According to a relatively recent result of Correa, Joffre, and ThibaultŽ seewx 7 for reflexive and wx 8 for arbitrary Banach spaces. , the function f is convex if and only if its Clarke᎐Rockafellar subdifferential Ѩ f is monotone. In the same line of research, much work has been done to characterize the generalized convexity of lsc functions by a corresponding generalized monotonicity of the subdifferential. Thus Lucwx 15 and, inde- * Work supported by a grant of the Greek Ministry of Industry and Technology. ² E-mail: [email protected] ³ E-mail: [email protected] 30 0022-247Xr99 $30.00 Copyright ᮊ 1999 by Academic Press All rights of reproduction in any form reserved. SUBDIFFERENTIALS AND QUASICONVEXITY 31 pendently, Aussel, Corvellec and Lassondewx 2 , showed that f is quasicon- vex if and only if Ѩ f is quasimonotone. Similarly, Penot and Quangwx 16 showed that if the function f is also radially continuous, then f is pseudoconvex if and only if Ѩ f is pseudomonotoneŽ in the sense of Karamardian and Schaiblewx 14 , as generalized for multivalued operators by Yaowx 20. In Section 2, we review these results, together with some notation and definitions, and show that in most cases the radial continuity assumption is not necessary. However, since the Clarke᎐Rockafellar subdifferential of a convex func- tion coincides with the classical Fenchel᎐Moreau subdifferentialwx 19 , it is not only monotone, but also cyclically monotonewx 17 . In Section 3 of this work, we define analogous notions of cyclic quasimonotonicity and cyclic pseudomonotonicity and show that the subdifferential of quasimonotone and pseudomonotone functions have these properties, respectively. Cyclic generalized monotonicity is not just a stronger property than the corre- sponding generalized monotonicity, but it expresses a behavior of a specific kind. In particular, an operator can even be strongly monotone without being cyclically quasimonotone. CyclicŽ. generalized monotonicity describes the behavior of an operator around a ``cycle'' consisting of a finite number of points. In Section 4 we consider instead the convex hull of such a cycle. We show that the definitions of monotone and pseudomonotone operators can be equiva- lently stated in terms of this convex hull. This is not so for quasimonotone operators; this leads to the introduction of the new notion of a properly quasimonotone operator. We show that this new notion, while retaining the important characteristics of quasimonotonicityŽ in particular, f is quasiconvex if and only if Ѩ f is properly quasimonotone. is often easier to handle; in particular, it is closely related to the KKM property of multival- ued maps. We show this by an application to Variational Inequalities. In addition, quasimonotonicity and proper quasimonotonicity are identical on one-dimensional spaces, which is probably the reason why the latter escaped attention. 2. RELATIONS BETWEEN GENERALIZED CONVEXITY AND GENERALIZED MONOTONICITY We denote by X* the dual of X and by Ž.x*, x the value of x* g X*at x g X. For x, y g X we set wxx, y s Ätx q Ž.1 y ty:0F t F 14 and define Ž x, yxw, x, y.Ž., and x, y analogously. Given a lsc function f: X ª R j Ä4qϱ with domain domŽ.f [ Ä x g X : fxŽ.- qϱ4 / л,the 32 DANIILIDIS AND HADJISAVVAS ᎐ g Clarke Rockafellar generalized derivative of f at x0 domŽ.f in the direction d g X is given byŽ seewx 19 . fxŽ.Ž.q tdЈ y fx ­ s fxŽ.0 , d sup lim sup inf ,Ž. 2.1 ␧) ª dЈgBdŽ. t 0 x f x0 ␧ to0 55 where Bd␧Ž.s ÄdЈ g X : dЈ y d - ␧4, t o 0 indicates the fact that t ) 0 ª ª ª ª and t 0, and x f x000means that both x x and fxŽ.fx Ž .. ᎐ The Clarke Rockafellar subdifferential of f at x0 is defined by Ѩ s g F ­ ᭙ g fxŽ.00Ä4x* X : Žx*, d .f Žx , d ., d X .2.2Ž. We recall that a function f is called quasiconvex, if for any x, y g X and z g wxx, y we have fzŽ.F maxÄ4fx Ž., fy Ž..2.3Ž. A lsc function f is called pseudoconvexwx 16 , if for every x, y g X, the following implication holds: ᭚ x* g Ѩ fxŽ.Ž: x*, y y x .G 0 « fxŽ.F fy Ž..2.4 Ž . It is knownwx 16 that a lsc pseudoconvex function which is also radially continuousŽ. i.e., its restriction to line segments is continuous , is quasicon- vex. Both quasiconvexity and pseudoconvexity of functions are often used in Optimization and other areas of applied mathematics when a convexity assumption would be too restrictivewx 5 . Let T: X ª 2X * be a multivalued operator with domain DTŽ.s Ä x g X : TxŽ./ л4. The operator T is called g Ž.i cyclically monotone, if for every x12, x ,..., xn X and every U g U g U g x1122TxŽ., x TxŽ.,..., xnnTxŽ.we have n U y F Ý Ž.xii, x q1 xi 02.5 Ž. is1 [ Ž.where xnq11x . Ž.ii monotone, if for any x, y g X, x* g TxŽ., and y* g TyŽ.we have Ž.y* y x*, y y x G 0. Ž2.6. SUBDIFFERENTIALS AND QUASICONVEXITY 33 Žiii . pseudomonotone, if for any x, y g X, x* g Tx Ž., and y* g Ty Ž. the following implication holds: Ž.x*, y y x G 0 « Ž.y*, y y x G 02.7 Ž. or equivalently, Ž.x*, y y x ) 0 « Ž.y*, y y x ) 0. Ž.2.8 Ž.iv quasimonotone, if for any x, y g X, x* g TxŽ., and y* g TyŽ. the following implication holds: Ž.x*, y y x ) 0 « Ž.y*, y y x G 0. Ž.2.9 The above properties were listed from the strongest to the weakest. We recall the hitherto known results connecting generalized convexity with generalized monotonicity: THEOREM 2.1. Let f: X ª R jqÄ4ϱ be a lower semicontinuous func- tion. Then Ž.i f is con¨ex if and only if Ѩ f is monotone wx8. In this case Ѩ f is also cyclically monotoneŽ see for instance wx17. Ž.ii f is quasicon¨ex if and only if Ѩ f is quasimonotone Ž see wx2 or w15 x. Ž.iii Let f be also radially continuous. Then f is pseudocon¨ex if and only if Ѩ f is pseudomonotoneŽ see wx4 or w16 x. We now show that pseudoconvexity of a function f implies quasiconvex- ity of f and pseudomonotonicity of Ѩ f, even without the radial continuity assumption: PROPOSITION 2.2. Let f: X ª R j Ä4ϱ be a lsc, pseudocon¨ex function with con¨ex domain. Then Ž.i f is quasicon¨ex Ž.ii Ѩ f is pseudomonotone. g g Proof. Ž.i Suppose that for some x12, x domŽ.f and some y ) s Ž.x12, x we have fy Ž.maxÄ fxŽ.Ž.1, fx 24Ä. Set m max fxŽ.Ž.1, fx 24. Since f is lower semicontinuous, there exists some ␧ ) 0 such that wx fyŽ.Ј ) m, for all yЈ g By␧ Ž.. From Ž 2.4 . it follows Ž see also 4. that the sets of local and global minimizers of the function f coincide; hence the point y cannot be a local minimizer, so there exists w g By␧Ž.such that fwŽ.- fy Ž.. Applying Zagrodny's Mean Value Theoremwx 21, Theorem 4.3 wxg w ª to the segment w, y , we obtain u w, y., a sequence un u, and 34 DANIILIDIS AND HADJISAVVAS U g Ѩ U y ) g unnfuŽ., such that Žu nn, y u . 0. Since y coÄ4 x12, x it follows U y ) g that Žuni, x u n. 0, for some i Ä41, 2 . Using relationŽ. 2.4 we get G G G m fxŽ.infu Ž .and, since f is lower semicontinuous, m fuŽ.. This clearly contradicts the fact that u g By␧Ž.. Ž.ii Let x* g Ѩ fxŽ.be such that Ž.x*, y y x ) 0. By partŽ. i , f is quasiconvex, so applying Theorem 2.1Ž. ii we conclude that Ѩ f is quasi- monotone. Hence Ž.y*, y y x G 0, for all y* g Ѩ fy Ž.