Wetnam Joumal of Mathematics 26:'2(199D 95-110 V Ie trn a nn ] oiLrrrrrnLarlL o,lf MtA\If ]HtlENt7\T ]t C SS o Springer-Verlag1998 Survey GeneralizedConvexity and SomeApplications to Vector Optimization* Dinh The Luc Institute of Mathematics, P.O. Box 631, Bo Ho 10000, Hanoi, Vietnam Received January 25, 1998 Abstract. In this paper we present an overview of recent developmentson the characterizations of convex and generalized convex functions via the nonsmooth analysis approach. Generalized convex vector functions are also consideredtogether with their applicationsto vector optimization. 1. Introduction Convex sets and convex functions have been studied for some time by Hdlder [19], Jensen[20], Minkowski [39] and many others. Due to the works of Fenchel, Moreau, Rockafellar in the 1960sand 1970s,convex analysis became one of the most beautiful and most developed branches of mathematics. It has a wide range of applications including optimization, operations resea.rch,economics, engineering, etc. However, severalpractical models involve functions which arenot exactly convex,but sharecertain properties of convex functions. These functions are a modification or generalizationof convex functions. The first generalization is probably due to de Finetti (1949) who introduced the notion of quasiconvexity.Other types of generalized convex functions were later developedin the works of Thy [45], Hanson [7], Mangasarian[37], Ponstein [41], Karamardianl2ll, Ortega-Rheinboldt [40], Avriel-Diewert-schaible-Zang[2]. This paper is an overview of recent developments on the study of convex and generalizedconvex functions. Our emphasisis on characterizationsofthese functions by the nonsmoothanalysis approach, namely, by subdifferential and directional derivatives. The second part of this overview is to study vector functions, mainly convex vector functions. We shall seethat presently,the study of generalizedconvex vector functions is quite far from being satisfactory.There still remain open problems in relation to the * Invited lecture delivered at the XXI AMASES Congress,Rome, 10-13 September,1997. 96 Dinh Thel;uc definitions ofgeneralized convex vector functions, their structureand characterizations. Nevertheless,many resultshave alreadybeen obtainedin the applicationsof generalized convex vector functions to multi-objective optimization. Becauseof the surveycharacter of the paper,proofs are not provided except for those of unpublishedresults. 2. Part A: Generalized Convex Functions 2.1. Most knportant GeneralizedConvex Functions Let A be a subsetof a finite-dimensional spaceR". It is said to be convex if it.contains the whole interval linking any two points of A, that is, ),a I (l - ),)b e A whenever a, b e A and,l. e (0, 1). In this section, / is assumedto be a real function defined on a nonempty convex set A c Rn. 2.1.l. ConvexFunctions Definition l. Thefunctionf issaidtobeconvexif,foreveryx,y € Aand), e (o, l), one has f ().x+ (1- r)y) < Lf (x)+ (l - L)f (y). (t) If inequality (l) holds strictly for every x * y, we say that f is strictly convexon A. Convex functions are a direct generalizationof affine functions because,for the latter, (1) becomes an equality. convex functions possessmany interesting properties. The readeris referred to Rockafellar l42l for a complete study of thesefunctions. Let us recall some properties that will be neededin understandinggeneralized convex functions. Rememberthat the epigraph of / is the set epif ::{(x,a) e Rn x R:.x € A, f (x) <u} and the level set at a e R is the set lev(f;a) :- {x e A: f(x) < a}. Proposition 2. Thefunction f is convex if and only if one of thefollowing conditions holds: (i) epi/ is convex; (ii) lev(/ i€; a)isconvexforalla e Randall{ € R"wherethesumfunctionf -t€ is definedby(f + 6)(x) : f(x) + (€,x). It is worth noting that according to Proposition 2(ii), convex functions have convex level sets; however, the converseis not true. A generalized convexity will be defined to meet this converse.The next result of Fenchel [13] and Mangasarian I38l deals with differentiable functions. GeneralizedCowexity to VectorOptimization 97 Proposition 3. AssumeA is open and f is dffirentiable on A. Then f is convex(resp. strictly cowex) if andonly if,foreachx, ! € A, x # y, onehas ftf(x)-vf0),r-y)>0 (resp.(y/(x) - V,f0), r - y) > 0). For nondifferentiable functions, one uses the subdifferential to characteize convexitv. We recall that the subdifferential of a function f at x e A is defined by lf(x) :: {E e R" : f(y) - f(x) > (E, y -x) forally e A}. For convex functions, the subdifferential is nonvoid at every relative interior point of A. Proposition 4. Assume af @) is nonemptyfor x e A. Then f is convex (resp. strictly convex)ifandonlyifforeachx,y € A, x I y andg e 0f (x), n eAf 0), onehas (€-ry,r-y)>0, (resp.(€-n,x-y) >0). The above result is due to Rockafellar [42]. rt has been recently extended by correa-Jofr6-Thibaudt and by Luc (see [29]) for the case where / is defined on a Banach spaceand 0/ is defined in a more generalform, including Rockafellar-Clarke's subdifferential, Michel-Penot's subdifferential, etc. Now, we turn to a subclassof strictly convex functions which is frequently used in optimization and economics. Definition 5. f is said to be strongly cotvex if there existsot > 0 such that f Q,x(I -I)y) <Lf (x) +(1 - L)f (y) -1.(1 - ),)o4lx-yll2 foreveryx,y e A,,1,e (0,1). Equivalently, / is strongly convex if there is cv > 0 such that the difference function f (x) - allxll2 is convex. Using Proposition 4, we obtain the following characteization of strong convexity. Proposition 6. Assume0f (x) is nonemptyfor x e A. Then f is strongly convexif and only if thereexists a > 0 suchthat, for eachx, y e A, x I y and { e Of (x), n e Of (y), one has G - q, x - y) - allx -)ll2 > 0. 2.1.2. QuasiconvexFunctions Definition 7. f is saidtobequasicowexif,foreveryx,y € A, x I y and), € (0, 1), one has - f ().x + (1 r)y) < max{f (x), f (y)}. (2) If (2) holds strictly, then f is said to be strictly quasiconvex.If (2) hotds strictly for those x, y with f (x) * f (y), then f is said to be semistrictly quasiconvex. 98 Dinh The Luc It is evident that strict quasiconvexityimplies semistnct Qua-si6sa1-gxrq'andquasicon- vexity. Semistrict quasiconvexityimplies quasiconrerin'prorrded rhe funcuon is lower semicontinuous. Remember that the level surface of f at cy € R is the set surt-rr: at '.: 11 1 I '. f(x): q\. Proposition 8. Wehave thefollowing: (i) / ts quasiconvexif and only if lev (f , u) is com'ex_for all cr = R'. (11) Assumef is continuousand A is strictly com'er. Then .f is semisrricrk quasiconvex ifandonlyiflev(f;u)isconvexandeitherler-(/:al:surt'r.f:ratorsurft_f:a)ls in the boundaryoflev(f; a); (111)Assume f is continuous.Then f is strictly quasicomexi.f and onh if levt/:cv) ls strictly colrvexand surf(f ; a) consistsof exteme poinrs o.fletr ,f : q t onh. Proof of (iii). It is alreadyknown [2]that lev(/; a) is strictlv conve\ ii -f is continuous, strictly quasiconvex.If surf(/; a) containsa non-extremepoint of levr-f : a r. then there arex,y, z elev(f;cv)suchthata: f(z) >_mar{-fr.rr../rrr}.: € (.r._r').On the by strict quasiconvexity, < mar{-fr.rt..f,)'}. other hand, "f(r) a contradiction. Conversely,if lev(/; a) is strictly convex, cy € R. then it follou's rn panicular that / is quasiconvex.Now, if it is not strictly quasiconvex.,*'e can find .r. -r'= -{ and : e (x, y) such that f (z) : max{/(x),,f(y)}, say, equal to .f(.rt. Then br quasiconvexity, f(z): f(r): f(x)forallx e [2,"r].Thismeansthatsurfr_f:_fr.trrdoesnotonly consistofextremepointsof lev(f; f (x)). Theproof iscomplete. r Next is a characterizationof quasiconvexity in the differentiable case. Proposition 9.fll Assumef is dffirentiable on an open com ei set ,1,C Rn. Then f is (x) < (t) -l') < quasiconvexifandonlyif,foreveryx,y € A, "f f impliesT-f()')(.r 0. 2.1.3. PseudoconvexFunctions We use Ortega-Rheinbold's definitions where no differentiabi-liq'assumptionis made. Definition 10. / ls said to be pseudoconvexif, for eve^' x. y e A and /(.r) < /(y), there exists B > O such that f().x* (1-r)y) =f(y)-x(l-L)p (3) for all.r, e (0, 1). f issaidtobestrictlypseudoconvexif,foreveryx,y,eA. .x f t and f (x) = f (y), there exists B as above. Observethat convexity implies pseudoconvexity,which implies semistrict quasicon- vexity, and strict convexity implies strict pseudoconvexity. Proposition ll. For dffirentiable functions, the above definition is equivalent to the classical one of Mangasarian [37]: f ispseudocowexif,forx,ye A, f (x) < f (y)impliesVf 0)(x -y) < O,and f isstrictpseudocorNexif,forx,ye A, x +y, f (x) S f $)impliesVf 0)@-y) < 0. Generalized.Convexity to VectorOptimization 99 For pseudoconvexfunctions, no known criteria exist using epigraph or level sets. 2.2. Extrema of GeneralizedConvex Functions For convex functions, it is known [13] that any local minimum is global. This property is very important in optimization since most existing theoretical and computational methods allow us to find local minima only. The concQptof quasiconvexityis developed under the influence of this property, namely, we have the following proposition. Proposition 12. Let f be a real function on A c R". Then (i) If f is quasicorwex,then every strict local minimum is strictly global minimum; (11) If f is continuous, then f is semistrictly quasicowex if and only if every local minimum is global minimum; (iii) If f is strictly quasiconvex,then it cannot have a local maximum and it may have a uniqueminimum. Actually, we can characterize quasiconvexity using extrema properties of the function on line segments.