JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 97, No. I, pp. 29-45, APRIL 1998
Subdifferential Properties of Quasiconvex and Pseudoconvex Functions: Unified Approach1
D. AUSSEL2
Communicated by M. Avriel
Abstract. In this paper, we are mainly concerned with the characteriza- tion of quasiconvex or pseudoconvex nondifferentiable functions and the relationship between those two concepts. In particular, we charac- terize the quasiconvexity and pseudoconvexity of a function by mixed properties combining properties of the function and properties of its subdifferential. We also prove that a lower semicontinuous and radially continuous function is pseudoconvex if it is quasiconvex and satisfies the following optimality condition: 0sdf(x) =f has a global minimum at x. The results are proved using the abstract subdifferential introduced in Ref. 1, a concept which allows one to recover almost all the subdiffer- entials used in nonsmooth analysis.
Key Words. Nonsmooth analysis, abstract subdifferential, quasicon- vexity, pseudoconvexity, mixed property.
1. Preliminaries
Throughout this paper, we use the concept of abstract subdifferential introduced in Aussel et al. (Ref. 1). This definition allows one to recover almost all classical notions of nonsmooth analysis. The aim of such an approach is to show that a lot of results concerning the subdifferential prop- erties of quasiconvex and pseudoconvex functions can be proved in a unified way, and hence for a large class of subdifferentials. We are interested here in two aspects of convex analysis: the charac- terization of quasiconvex and pseudoconvex lower semicontinuous functions and the relationship between these two concepts.
1The author is indebted to Prof. M. Lassonde for many helpful discussions leading to a signifi- cant improvement in the presentation of the results. 2Assistant Professor, Mathematiques Appliquees, Universite de Perpignan, Perpignan, France. 29 0022-3239/98/0400-0029$15.00/0 © 1998 Plenum Publishing Corporation 30 JOTA: VOL. 97, NO. 1, APRIL 1998
The paper is organized as follows. Section 2 is devoted to the study of quasiconvex functions. We establish a characterization based on a mixed property combining properties of the function and properties of its subdiff- erential. The case of quasiconcave and quasiaffine functions, for which anal- ogous characterizations are obtained, is treated in Section 3. In Section 4, we generalize to the nonsmooth setting a result of Crouzeix and Ferland (Ref. 2) which describes the precise relation between quasiconvexity and pseudoconvexity. Moreover, this allows one to prove in an elegant way two characterizations of pseudoconvexity. Finally, we point out in Section 5 that the continuity assumption can be replaced in some results by the radially nonconstant hypothesis, already used by Diewert in 1981 (Ref. 3). In all the sequel, X denotes a Banach space, X* its topological dual, and < •, • > the duality pairing. Most functionsf: X-+R u { + 00} considered in this paper will be (at least) lower semicontinuous (l.s.c. in short), and dom f will denote the domain off
For a point-to-set map A: X-+X*, we let
Throughout this paper, d stands for the following abstract notion of subdifferential.
Definition 1.1. See Ref. 1. We call subdifferential, denoted by d, any operator which associates a subset Sf(x) of X* to any lower semicontinuous f: X-**R u {+00} and any xeX, and satisfies the following properties: (P1) df(x) = {x*eX*\ This abstract subdifferential allows one to recover, by a unique defini- tion, a large class of subdifferentials. Among this class, let us mention some examples (for precise definitions, see Ref. 1 and references therein): the Clarke-Rockafellar subdifferential dCRf; the lower and upper Dini subdifferentials <3D- f and <3D+ f; JOTA: VOL. 97, NO. 1, APRIL 1998 31 the lower Hadamard subdifferential dH- f, also called contingent subdifferential; the Frechet subdifferential dF f; the Lipschitz smooth subdifferential dLS f, also called proximal subdiff- erential d" f whenever X is a Hilbert space. The verification that these subdifferentials satisfy properties (P1) to (P3) is straightforward (see Ref. 4) and does not require any assumption on the Banach space. The use of the word d-differentiable is justified by the follow- ing observation: a function is <3°~-differentiable [resp. dH--differentiable, dF-differentiable] at x if and only if it is Gateaux [resp. Hadamard, Frechet] differentiable at x. In all the sequel, we will focus our attention on the class of subdifferen- tials which satisfy properties (P1) to (P3) and one of the following inclusions: This assumption is not needed in all the proofs and in any case is not really restrictive, since the Clarke-Rockafellar and the upper Dini subdifferentials are the biggest (in the sense of inclusion) among the classical subdifferentials. In particular, one has Let us recall the definition of the Clarke-Rockafellar subdifferential and the definition of the upper Dini subdifferential which will be helpful for the sequel, (one can take «=f(u) whenever f is l.s.c.); 32 JOTA: VOL. 97, NO. 1, APRIL 1998 Definition 1.2. See Ref. 1.3 A norm || • || on X is said to be d-smooth if all the (real-valued, convex, continuous) functions of the following form are 3-differentiable: (i) df[a,b](x) •= min,6[a,fc] ||x-c||2, where [a, b] is a closed segment in X, (ii) A2(x):=I.nnn\\x-vn\\2, where 2nJuM= 1, ^«>0, and (i>«) converges in X. We say that a Banach space admits a d-smooth renorm if it admits an equivalent norm which is 3-smooth. Let us give some interesting examples of such d-smooth norms: (a) a norm is ^"-smooth iff it is dfl~-differentiable on ^f\{0}, that is, according to a previous remark, iff it is Gateaux-differentiable on X\{0] (elementary proof); (b) any norm is dc/J-smooth since the functions rff0ifc] and A2 are locally Lipschitz. Concerning (a), we note that the same equivalence, with respect to Hadamard or Frechet differentiability, is true for d — dH- or d = dF. So, any separable [resp. reflexive] Banach space admits a <3D--smooth [resp. dF- smooth] renorm; see, e.g., Ref. 6. Finally, we need the following result which is a special case of the mean- value inequality stated in Ref. 1. Proposition 1.1. Let X be a Banach space with a 3-smooth renorm, and let f:X-»R u {+00} be a l.s.c. function. For any aedom f and beX such that f(a) 2. Two Characterizations of Quasiconvexity This section is devoted to the study of quasiconvex functions. More precisely, we extract from the proof of the main result of Aussel et al. (Ref. 7, Theorem 1) a mixed property (Qs) combining properties of the function 3Since this paper was submitted, J. N. Corvellec, M. Lassonde, and the author have shown that (i) can be deduced from (ii); see Proposition 2.3 in Ref. 5. JOTA: VOL. 97, NO. 1, APRIL 1998 33 and properties of the subdifferential. Using the same arguments as in Ref. 7, Theorem 1, we show that this mixed property characterizes the quasi- convexity of l.s.c. functions. This characterization will play a central role in the sequel. Let us recall that a function is quasiconvex if, for any x, yeX and any ze[x,y], one has It is well known that, in the differentiable case, quasiconvex functions satisfy In the nondifferentiable case, this property becomes Our first result asserts that the following slightly stronger mixed property actually characterizes the quasiconvexity of l.s.c. functions: Theorem 2.1. Let X be a Banach space with a 5-smooth renorm, and let f: X ->R u { + 00} be a l.s.c. function. Then, the following assertions are equivalent: (i) f is quasiconvex; Proof. (i) =» (ii) Case dfcdCRf. Let x,yedom df and x*edf(x) such that Hence, there exists e > 0 such that, for every ne N*, one can find xneBe/n(x) [and then y — xneBe(y — x)] and tne ]0, 1/n[ which satisfy Since f is quasiconvex, the previous inequality implies that, for every te [0, 1 ], we have and hence, by lower semicontinuity of f, (i) => (ii) Case df c BD+f. The proof is very simple. Indeed, if x, y and x*edD+f(x) satisfy fD+(x,y-x)>0, then f(z)>f(x) for some 34 JOTA: VOL. 97, NO. 1, APRIL 1998 ze ]x, y[, and by quasiconvexity of f, (ii) => (i) We essentially reproduce the first part of the proof of Ref. 7, Theorem 1. Let x, yedom f and According to Proposition 1.1, there exist sequences (xn) and (x*) such that xn->xe[x, z[, x*edf(xn), and Obviously, every l.s.c. quasiconvex function also satisfies property (0. But the converse is false, in general, for l.s.c. functions. Consider for example the function f defined by It satisfies property (Q) for d = dCR but is not quasiconvex. In previous works, Hassouni (Ref. 8, locally Lipschitz function) and Penot and Quang (Ref. 9, continuous function) characterize quasiconvexity using property (0. But property (Qs) is proved to be the good expression in the more general setting of l.s.c. functions. In the following result, we show that, for a continuous function or radially continuous function (that is, continuous on each segment), both properties are equivalent. Proposition 2.1. Let X be a Banach space with a d-smooth renorm. Every l.s.c. radially continuous function which satisfies (0 is quasiconvex. Proof. Let x, yeX, and let ze]x, y[ be such that f(z)>max[f(x), f(y)]. Applying Proposition 1.1 to the points x and z, we obtain two sequences (an) and (a*), with an-+ae[x, z[, a*edf(an), and The functionfbeing radially continuous, there exists According to Proposition 1.1 applied to point a = a + t(z-a)e ]a, z[ and point y, and using property (0, there exists a'e[a, z[ such thatf(a') The case d = dCR is given in Penot and Quang (Ref. 9). A point-to-set map A: X-+X* is quasimonotone if, for any x, yeX, the following holds: The equivalence between the quasiconvexity of a function and the quasimonotonicity of its subdifferential has been studied in Aussel et al. (Refs. 7, 1) for dedCR and in Luc (Ref. 10) for d=dCR. The aim of the next two results is to show that the implication f quasiconvex => df quasimonotone can be stated without any regularity assumption on f and that the reverse implication can be seen as a consequence of Theorem 2.1. Proposition 2.2. Let X be a Banach space. Then, the Clarke-Rockafel- lar subdifferential and the upper Dini subdifferential of any quasiconvex functionf: X-*R u {+00} are quasimonotone. Proof. The case of the Clarke-Rockafellar subdifferential consists of a slight refinement of the second part of the proof of Ref. 7, Theorem 1. We include the proof for completeness. Suppose that f is quasiconvex, and let x, yedom dCR f and x*edCR f(x) be such that These inequalities, according to the quasiconvexity assumption, yield Moreover, from the choice of y and e', the direction UB - v is an element of Bf(x-y). Thus, summing up the previous steps, for any e>0, there exists y>0 such that, for any veBr(y), any /Je5y(f(>0)> f(v) which clearly implies the desired inequality f1(y, x-y)<,0. For the case of the upper Dini subdifferential, as an immediate conse- quence of the quasiconvexity of the function f, we have or equivalently that Consequently, if x*edD+f(x) is such that Theorem 2.2. Let X be a Banach space with a <3-smooth renorm, and let f: .X-Ru {+00} be a l.s.c function. Then, f is quasiconvex iff df is quasimonotone. Proof. Since the abstract subdifferential df is assumed to be included in dCR for SD+ f, the "only if" part follows from Proposition 2.2. To prove the "if" part, we follow the lines of the second part of the proof of Ref. 1, Theorem 5.4. Let us assume that df is quasimonotone. According to Theorem 2.1, we just have to show that the l.s.c. functionf satisfies property (Qs). Let xedom df, jedom f(x^y), and ze[x,y[ such that f(z)>f(y). Applying Proposition 1.1 to y and z, we get a sequence (yn) converging to ye ]z,y] and a sequence (y*) such that y*edf(yn) and By quasimonotonicity of df, we have for all n and all x*edf(x). Then, Thus, the function f satisfies property (Qs), and the proof is complete. D 3. Quasiconcave and Quasiaffine Functions In this section, we consider the case of quasiconcave and quasiaffine functions for which analogous mixed characterizations are proved using appropriate mixed properties. JOTA: VOL. 97, NO. 1, APRIL 1998 37 A function f is said to be quasiconcave if -f is quasiconvex and it is said to be quasiaffine if f and -f are quasiconvex. Let us consider the following mixed property The characterization of the quasiconcavity of a function f by the property (27) cannot be deduced in general from Theorem 2.1. Indeed, considering the function (—f) in place of a functionfin Theorem 2.1 yields a charac- terization of the quasiconcavity of f in terms of —3(—f), which in general is different from df. The function f(x) = ^/\x\ is a standard counterexample for dCR, since dCR f(0) = R and dcr(-f)(0) = 0; [however, the subdifferen- tial equality dCR f=-dCR(-f) is true if f is assumed to be locally Lipschitz (see, e.g., Ref. 11). Proposition 3.1. Let X be a Banach space with a 5-smooth renorm, and let f: X->R u {+00} be a continuous function. Then,f is quasiconcave if and only if, for any x, y of X, the function f satisfies Proof. It follows the same line as the proof of Theorem 2.1. Let us assume that f satisfies property (Q-) and that x, yeX and ze[x, y[ are such that f(z) Let t1 and t2 be some positive numbers, with 0 and define two sequences (xn) and (zn) by For n large enough, we have For any n, the two points xn and zn = kxn + (1 - A )y [with A defined by z = Ax + (l -A)y] are on the segment line ]xn + tn(xn—y),y[. Using the quasicon- cavity off we have and hence, sincefis upper semicontinuous,f(z) >f(>>). Ifdf<=dD+f, then we havefD+(x, x—y)>0, and hence for every «, there exists tne ]0, \fn[ verifyingf(x + tn(x-y)) >f(x). But f is quasiconcave and xe]x + tn(x-y),y[, for all «. Consequently f(z)>f(y) andfsatisfies prop- erty (fiT). D Let us mention that one can easily find counterexamples which show that, for the "if" part of the proposition, the continuity hypothesis cannot be replaced by a lower or upper semicontinuity hypothesis. Consider for example the indicator function of the set {0, 1} or the indicator function of the set R\{0}. As an immediate consequence of Theorem 2.1 and Proposition 3.1, we obtain the following characterization of quasiamne functions. Corollary 3.1. Let X be a Banach space with a 3-smooth renorm, and letf: Ar-+Ru{+oo}bea continuous function. Then, the following assertions are equivalent: (i) fis quasiaffine; Indeed, the conjunction of the mixed properties (Qs) and (Qs ) is equivalent to 3x*edf(x): <**, rf> > 0=> fXid: t*-*f(x + td) is nondecreasing on K, which is exactly assertion (ii). Another equivalent way to describe assertion (ii) of the corollary is Using the corresponding expression in the differentiable case, Martos (Ref. 12) characterized quasiaffine functions in finite-dimensional spaces. 4. Pseudoconvexity The original definition of pseudoconvexity by Mangasarian in the differentiable setting was extended by many authors in the following way JOTA: VOL. 97, NO. 1, APRIL 1998 39 (see for instance Ref. 9): A functionfis said to be pseudoconvex if, for any x, yeX, one has 3x*edf(x): is pseudoconvex, with e.g. d = dCR, but not quasiconvex. Extending a result of Crouzeix and Ferland (Ref. 2) stated for differen- tiable functions, the following theorem gives the precise relation between pseudoconvexity and quasiconvexity for l.s.c. radially continuous functions. Theorem 4.1. Let X be a Banach space with a ^-smooth renorm, and letf: X->Rv {+00} be a l.s.c. and radially continuous function. Then, the following assertions are equivalent: (i) fis pseudoconvex; (ii) fis quasiconvex and (Qedf(x) => fhas a global minimum at x). Proof, (i) =*• (ii). From the definition of pseudoconvexity, it is obvi- ous that, if Oedf(x), then So x is a global minimum of the functionf. On the other hand, the function fis l.s.c., radially continuous and satisfies property (Q) of Section 2, since every pseudoconvex function satisfies property (Q). Then, from Proposition 2.1,f is quasiconvex. (ii) => (i). Let xedomdf, yeX, and x*edf(x) such that For every neN, the point yn satisfies Using Theorem 2.1, we obtain that, for every n,f(yn)>f(x) and by radial continuity off,f(y)>f(x). D In finite dimensions, for d = dD+ and for an upper semicontinuous func- tion, the implication (ii) => (i) is given in Komlosi (Ref. 13). Now using the relation between quasiconvexity and pseudoconvexity (Theorem 4.1) and the characterization of quasiconvexity by the quasimono- tonicity of its subdifferential (Theorem 2.2), it is possible to give two charac- terizations of l.s.c. radially continuous pseudoconvex functions. Let us recall that a point-to-set map A: X->X* is said to be pseudomonotone if, for any x, yeX, one has Theorem 4.2. Let X be a Banach space with a d-smooth renorm, and letf: A"-*Ru {+°o} be a l.s.c. and radially continuous function. Then, the following assertions are equivalent: Proof, (i) => (ii). Let jcedom df, yeX, and x*edf(x) such that <**,y-x}>0. From the definition of pseudoconvexity, it follows that f(x) We claim that 0 is not an element of df(y). Indeed, since f](x,y-x)>0, there exist e>0, x'eBc(x), and re]0, 1[ such that JOTA: VOL. 97, NO. 1, APRIL 1998 41 From Theorem 4.1, the function f is quasiconvex and then we have f(x') The idea of the proof of (i) => (iii) for dCR is due to Penot and Quang (Ref. 9). 5. Radially Nonconstant Functions Following Komlosi (Ref. 14), we say that a function f is radially non- constant (r.n.c. for short) if one cannot find any line segment on which f is constant; i.e., The purpose of this section is to give a different light to some of the preceding results by proving them under the r.n.c. assumption, instead of the continuity (or radial continuity) assumption. 42 JOTA: VOL. 97, NO. 1, APRIL 1998 Proposition 5.1. Let X be a Banach space with a 3-smooth renorm, and letf: X-^Ru { + 00} be a l.s.c. and radially nonconstant function. Iff is pseudoconvex, then df is pseudomonotone. Let us remark that Proposition 5.1 does not allow one to recover the implication (i) => (iii) of Theorem 4.2, since there exists continuous (and convex) functions which are not radially nonconstant. Proof. Assume, for a contradiction, that f is pseudoconvex and that df is not pseudomonotone. So, there exist x, yedomdf, x*edf(x), and y*edf(y) such that Since f is pseudoconvex and l.s.c., it follows that f(y)>f(b). But be]x, z[<=V, and thenf(6)>A>f(j). Consequently, we havef(a)>f(z). Using the same arguments one can prove that this is impossible too, thus supplying the desired contradiction. D As an immediate consequence of Proposition 5.1, we obtain the follow- ing relation between pseudoconvexity and quasiconvexity. Corollary 5.1. Let X be a Banach space with a d-smooth renorm. Every l.s.c., radially nonconstant, and pseudoconvex function is quasiconvex. Proof. Iffis a l.s.c., r.n.c., and pseudoconvex function, then according to Proposition 5.1, its subdifferential is pseudomonotone and hence quasimonotone. By Theorem 2.2, the function is then quasiconvex. D JOTA: VOL. 97, NO. 1, APRIL 1998 43 We say that a functionf:X-»R u{+oo} is: (a) strictly quasiconvex iff, for any x, yeX and any ze ]x, y[, one has (b) strictly pseudoconvex iff, for any x, yeX, the following holds: Obviously every strictly quasiconvex [resp. strictly pseudoconvex] func- tion is quasiconvex [resp. pseudoconvex]. Diewert (Ref. 3) observed that, conversely, every quasiconvex r.n.c. function is strictly quasiconvex. A consequence of the next result is that the same holds true for pseudo- convex functions. Proposition 5.2. Let X be a Banach space with a 3-smooth renorm, and letf: X-*R u {+00} be a l.s.c. function. Then, the following assertions are equivalent: (i) fis pseudoconvex and radially nonconstant; (ii) f is strictly quasiconvex and strictly pseudoconvex. Proof, (i) => (ii). According to Corollary 5.1, the radially noncon- stant function f is quasiconvex and hence strictly quasiconvex. To prove thatfis strictly pseudoconvex, let us assume, for a contradiction, that there exist xedom df, yedomf, and x*edf(x) such that <;c*,_y-x>^0 and f(x)^.f(y). Consequently, for any ze]x,y], we have 6. Final Remarks (i) Throughout this paper, two different assumptions are made. The first one is the inclusion of the subdifferential operator in the Clarke-Rock- afellar or upper Dini subdifferential [Assumption (H)]. The other one, is the existence of a d-smooth renorm. A swift reading is sufficient to mention that these hypotheses are not necessary in all the proofs. They are, in a sense, complementary. Indeed, in most equivalence proofs, one hypothesis allows to show an implication and the other is used to prove the converse. 44 JOTA: VOL. 97, NO. 1, APRIL 1998 To be more precise, a direct or indirect use of Proposition 1.1 requires the existence of a 3-smooth renorm, whereas Assumption (H) is necessary for proofs based on the definition of dD+ or BCR. (ii) Another way to characterize generalized convexity and to describe the relationships between different kinds of generalized convexity is to use criteria and properties based on the notion of generalized derivative instead of the subdifferential. In recent works [Komlosi (Refs. 14, 15), Penot (Ref. 16), etc.], some abstract approach with generalized derivatives are developed. Since some subdifferentials covered by our abstract definition are not defined by a generalized derivative (the Frechet and proximal subdifferentials, for example), these two approaches have to be considered as complementary points of view. References 1. AUSSEL, D., CORVELLEC, J. N., and LASSONDE, M., Mean-Value Property and Subdifferential Criteria for Lower Semicontinuous Functions, Transactions of the American Mathematical Society, Vol. 347, pp. 4147-4161, 1995. 2. CROUZEIX, J. P., and FERLAND, J. A., Criteria for Quasiconvexity and Pseudo- convexity: Relationships and Comparisons, Mathematical Programming, Vol. 23, pp. 193-205, 1982. 3. DIEWERT, W. E., Alternative Characterizations of Six Kinds of Quasiconcavity in the Nondifferentiable Case with Applications to Nonsmooth Programming, Gen- eralized Concavity in Optimization and Economics, Edited by S. Schaible and W. T. Ziemba, Academic Press, New York, New York, pp. 51-93, 1981. 4. AUSSEL, D., Theoreme de la Valeur Moyenne et Convexite Generalisee en Analyse Non Reguliere, PhD Thesis, Universite Blaise Pascal, Clermont II, France, 1994. 5. AUSSEL, D., CORVELLEC, J. N., and LASSONDE, M., Nonsmooth Constrained Optimization and Multidirectional Mean- Value Inequalities, Research Report 96/ 04, Universite des Antilles et de la Guyane, 1996. 6. DEVILLE, R., GODEFROY, G., and ZIZLER, V., Smoothness and Renormings in Banach Spaces, Longman Scientific and Technical, London, England, 1993. 7. AUSSEL, D., CORVELLEC, J. N., and LASSONDE, M., Subdifferential Charac- terization of Quasiconvexity and Convexity, Journal of Convex Analysis, Vol. 1, pp. 195-201, 1994. 8. HASSOUNI, A., Sous-Differentiels des Fonctions Quasiconvexes, PhD Thesis, Universite Paul Sabatier, Toulouse, France, 1983. 9. PENOT, J. P., and QUANG, H. P., On Generalized Convexity of Functions and Generalized Monotonicity of Set- Valued Maps, Journal of Optimization Theory and Applications, Vol. 92, pp. 343-356, 1997. 10. Luc, D. T., Characterizations of Quasiconvex Functions, Bulletin of the Austral- ian Mathematical Society, Vol. 48, pp. 393-405, 1993. JOTA: VOL. 97, NO. 1, APRIL 1998 45 11. ROCKAFELLAR, R. T., Generalized Directional Derivatives and Subgradients of Nonconvex Functions, Canadian Journal of Mathematics, Vol. 32, pp. 257-280, 1980. 12. MARTOS, B., Nonlinear Programming: Theory and Methods, North-Holland, Amsterdam, Holland, 1975. 13. KOMLOSI, S., Some Properties of Nondifferentiable Pseudoconvex Functions, Mathematical Programming, Vol. 26, pp. 232-237, 1983. 14. KOMLOSI, S., Generalized Monotonicity and Generalized Convexity, Journal of Optimization Theory and Applications, Vol. 84, pp. 361-376, 1995. 15. KOMLOSI, S., Monotonicity and Quasimonotonicity in Nonsmooth Analysis, Recent Advances in Nonsmooth Optimization, Edited by D. Z. Du, L. Qi, and R. S. Womersley, World Scientific Publishers, Singapore, pp. 101-124, 1994. 16. PENOT, J. P., Generalized Convexity in the Light of Nonsmooth Analysis, Recent Developments in Optimization, Lecture Notes in Economics and Mathematical Systems, Springer, Berlin, Germany, Vol. 429, pp. 269-290, 1995.