Optimization of Convex Functions with Fenchel Biconjugation and Duality

International Journal of Advanced Technology and Engineering Exploration, Vol 5(42) ISSN (Print): 2394-5443 ISSN (Online): 2394-7454 Research Article http://dx.doi.org/10.19101/IJATEE.2018.542013 Optimization of convex functions with fenchel biconjugation and duality

Vinod Kumar Bhardwaj* Department of Mathematics, G.L.A. University, Mathura, India

Abstract Analysis of conjugation operations to induce a bijection between proper closed convex functions and to discuss the problems of boundedness of closed convex proper functions using continuity of conjugates. Present study shows a great contribution of biconjugation of convex functions in optimization. Fenchel biconjugation describes the relation of duality in optimization. Problems of finite dimensional Lagrangian convex duality theory and problems on duality gap are comparative of primal and dual solutions in convex optimization.

Keywords Convex functions, Fenchel biconjugation, Lagrangian duality, Optimization.

1.Introduction A duality gap can be easily defined by a problem The assumption of a closed function is a dissimilar to where the primal value is strictly greater than the dual previous definition by Rockafellar et al. [1, 2], value [10]. We investigate conditions ensuring that although they coincide for proper functions but our there is no duality gap and the result highlights the assumption easily extends to infinite dimensions [3]. proof of the Lagrangian necessary conditions which Von Neumann's minimax theorem had both the sets in fact describes the existence of a dual optimal C and D simplices [4]. Brouwer proved this by fixed solution [11, 12]. To prove that there is no duality point theorem. Fisher introduced information gap we consider following two different approaches: function useful in signal reconstruction [5]. Interior first uses the Slater condition to force attainment in point methods depend on the inequality in the dual problem and second (dual) approach uses Logarithmic homogeneity [6]. No one can consider compactness to force attainment in the primal biconjugate unless it itself has a closed convex problem [2]. The problem is said to be normal when epigraph, since any conjugate function must have a the value function v is lower semi continuous at 0 [1] closed convex epigraph. Nonempty compact convex and the problem is said to be stable if sets in E and everywhere-finite sub linear functions v(0)  ( orv (0)   ) [13]. On the gap have Fenchel conjugacy between them [7]. functions of prevariational inequalities Yang [11] and on the dual gap function for variational inequalities Finite dimensional Lagrangian convex duality Zhang et al. [12] are some good reference work on theorem provides comparative study on problems on duality gap functions. Chong et.al derived Sufficient duality gap and zero duality gap and analysis of dual and necessary conditions for the stable Fenchel and primal solutions in convex optimization. Elegant duality and for the total Fenchel duality , also proved feature of theory is the duality between a convex some sufficient and necessary conditions for the function and its Fenchel conjugate. The study of strong Fenchel duality and the strong converse optimization is significantly depends on its control to Fenchel duality using the properties of the epigraph describe duality theory for convex programming of the conjugated functions [14]. Jeyakumar et al. problems. The perfectly well-defined dual problems developed a robust theorem of the alternative for without any assumptions on the functions and g show parameterized convex inequality systems using the `weak duality inequality' [8]. This is great feature conjugate analysis and derives a new robust of finite-dimensional convex duality theory and its characteristic cone constraint qualification necessary development in infinite dimensions [9]. and sufficient for strong duality between the robust equivalent and its Lagrangian dual [15]. Wang et.al establishes some total and strong Fenchel dualities for convex optimization problems accompanying data *Author for correspondence uncertainty within the framework of robust 83

Bhardwaj optimization in locally convex Hausdorff vector is lower semi continuous at every point in E: this is in spaces [16]. Fajardo used generalized convex fact corresponding to h being closed, the result holds conjugation theory instead in place of Fenchel if and only if h has closed level sets. Any two conjugation and construct an alternative dual functions h and g satisfying hg (in which case problem, using the perturbational approach, for a  general optimization one defined on a separated we call h a minorant of g) must satisfy hg , locally convex topological space [17]. and hence hg    .

In the paper following points are focused:  Biconjugates of proper convex functions are 3.Fenchel biconjugation developed to discuss the optimality conditions. Theorem 3.1 (Fenchel biconjugation) The  Relationship between convex function and its properties below are equivalent, for any function biconjugate and the contribution of these in : optimization is described. (a) h is closed and convex;  Lagrangian Duality theorem is used to describe the (b) h = ; duality gap and zero duality gap between convex (c) for all points x in E, function and its conjugate. h( x ) sup ( x ) anaffine min orant of h (1)   2.Preliminaries and definitions Hence the conjugacy operation induces a bijection We have seen that for many important convex between proper closed convex functions. functions , the biconjugate hE:,   Proof. We can assume h is proper. Since conjugate  functions are always closed and convex we know h agrees identically with h. The bipolar cone property (b) implies property (a). Also, any affine  minorant  of h satisfies     , theorem shows K  K for any closed convex  hh  cone K. In the paper we isolate exactly the and hence property (c) implies (b). It remains to show (a) implies (c). circumstances when h = . We can easily check Fix a point x in E. Assume first that is a minorant of h (that is, hh point wise). Our explicit plan in the paper is to find x cl() domh , and fix any real r < h ( ). conditions on a point x in E guaranteeing Since h is closed, the set x h( x )> r is open, so h()() x h x . This becomes the key relationship for the study of duality in optimization. there is an open convex neighborhood U of with As we see in the paper, the conditions we need are h(x) > r on U. Now note that the set both geometric and topological. This is neither domh contU is nonempty, so we can apply particularly astonishing nor rigid. It is very to the Fenchel theorem to deduce that some element  consider a function with its biconjugate without of E satisfies having a closed convex epigraph, since any conjugate  rinf() h x UU () x   h ()    ()  (2) function must have a closed convex epigraph. On the x   other side, this restriction is not mainly strong, as the Now define an affine function previous study shows that convex functions (),()        r . Inequality (3.2) shows fundamentally have strong continuity properties. U that minorizes h, and by definition we know  Consider the function hE:,   is ()xr . Since r was arbitrary, (c) follows at the closed if its epigraph is a closed set. Which indicates point x = . that h is lower semi continuous at a point x in E if Suppose on the other hand does not lie in cl (dom rr liminfh ( x )( lim inf h ( x )) h ( x ) h). By the Basic separation theorem there is a real b x r  s and a nonzero element a of E satisfying for any sequence xxr  . A function is lower semi continuous if it ax, >b a,x , for all points x in dom h.

International Journal of Advanced Technology and Engineering Exploration, Vol 5(42)

The argument in the preceding paragraph shows there Strict convexity is also easy to recognize via is an affine minorant  of h. But now the affine conjugacy, using the following results. function ()(,) k a   b is a minorant of h Theorem 3.4 (Strict-smooth duality) A proper for all k 1,2,...... Evaluating these functions at closed convex function on E is essentially strictly convex if and only if its conjugate is essentially x = proves property (c) at . The final remark x smooth. follows easily.  We can immediately deduce that a closed convex What can we say about h when the function function hE:,  equals its hE:,   is not necessarily closed? To biconjugate if and only if it is proper or identically answer this question we introduce the idea of the  or  . Restricting the conjugacy bijection to closure of h, denoted cl h, defined by finite sub linear functions gives the following result. epi()() cl h cl epih (3) It is easy to verify that cl h is then well-defined. The Corollary 3.2 (Support functions) Fenchel definition immediately implies cl h is the largest conjugacy induces a bijection between everywhere- closed function minorizing h. Clearly if h is convex, finite sub linear functions and nonempty compact so is cl h. convex sets in E: Proposition 3.5 (Lower semi continuity and (a) If the set CE is compact, convex and closure) A convex function  is lower semi continuous at a nonempty then the support function C is everywhere finite and sub linear. point x where it is finite if and only if (x) = (cl (b) If the function is sub linear then h: E R )(x). In this case is proper.  h  C , where the set We can now answer the question we posed at the beginning of the section. C  E,() d  h d for all d  E is nonempty, compact and convex. Theorem 3.6 Suppose the function is convex. Proof. See in previous section. Conjugacy offers a convenient way to recognize when a convex function (a) If is somewhere finite then = cl h. has bounded level sets. (b) For any point x where h is finite, h(x) = hx() if and only if h is lower semi continuous at x. Theorem 3.3 (Moreau-Rockafellar) A closed convex proper function on E has bounded level sets if and only if its conjugate is continuous at 0. Proof. Observe first that since is closed and minorizes h, we know h cl h h . If is Proof. By Proposition, a convex function somewhere finite then (and hence cl h) is never hE:,   has bounded level sets if and  , by applying Proposition 3.6 (Lower semi only if it satisfies the growth condition fx() liminf >0 continuity and closure) to . On the other hand, if x  x h is finite and lower semi continuous at x then Proposition 3.6 shows cl h(x) is finite, and applying Since f is closed we can check that this is the proposition again to cl h shows once more that cl h is never . In either case, the Fenchel equivalent to the existence of a minorant of the form biconjugation theorem implies   kf ()  , for some constants  > 0 and k. cl h() cl h   h    cl h , so  Taking conjugates, this is in turn equivalent to f cl h h. being bounded above near 0, and the result then Part (a) is now immediate, while part (b) follows by follows by Theorem (Local boundedness). using Proposition 3.6 once more.

Bhardwaj

4.Lagrangian duality Proposition 4.1 (Dual optimal value) Consider a convex program like that of studied in (a) The primal optimal value p is v(0) . section-3.2 (b) The conjugate of the value function satisfies

 inffx ( )  ( ),if  0  (4) v ()   subjectto g( x ) 0  , otherwise   xE (c) The dual optimal value d is v(0) . Here, the function f and the components Proof. Part (a) is just the definition of p. Part (b) g, g ,...., g : E   ,  are convex, 12 m follows from the identities m and satisfy  dom f1 domgi . As before, the Lagrangian function v () sup Tmb  v ( b ) b  R  m LER:,    is defined by sup Tb  f ( x ) g ( x )  z  b , x  dom f , b  R m , z  R m L(;)()() x f xT g x .    Notice that the Lagrangian encapsulates all the sup Tm (g ( x )  z )  f ( x ) x  dom f , z  R information of the primal problem (4.3.1): clearly   

 inff ( x ) T g ( x ) x  dom f  sup  T z z  R m  f( x ), if xis feasible      supLx ( ; )   , otherwise Rm  ( ),if  0    so if we denote the optimal value of (4.3.1) by  , otherwise p , , we could rewrite the problem in the following form: Finally, we observe d  sup()()()   inf      inf vv      (5) mm p inf sup L ( x ; ) Rm RR xE m  R This makes it rather natural to consider an associated So part (c) follows. Notice the above result does not problem: use convexity. (6) d sup inf L ( x ; ) xE The reason for our interest in the relationship Rm  between a convex function and its biconjugate should Where is called the dual value. d  ,  now be clear, in light of parts (a) and (c) above. Thus the dual problem consists of maximizing over vectors in m the dual function  R Corollary 4.2 (Zero duality gap) Suppose the value of the primal problem (4.1) is finite. Then the primal ( ) inf Lx(;) . x and dual values are equal if and only if the value This dual problem is perfectly well-defined without function v is lower semi continuous at 0. In this case any assumptions on the functions and g. It is an the set of optimal dual solutions is v(0) . easy exercise to show the `weak duality inequality’

. Notice is concave. pd  Proof. By the previous result, there is no duality gap exactly when the value function satisfies It can happen that the primal value p is strictly larger vv(0)  (0) , so Theorem 3.8 proves the first than the dual value d. In this case we say there is a duality gap. In this section we investigate conditions assertion. ensuring there is no duality gap. As the chief tool in our analysis is the primal value function By part (b) of the previous result, dual optimal solutions  are characterized by the property vR:,m   , defined by 0v (  ) , or equivalently, vb( ) inf f()() x g x b . (7)   vv( )    (0)  0 . But we know

Below we summarize the relationships between these , so this property is equivalent to various ideas and pieces of notation. the condition  v(0) . 86

International Journal of Advanced Technology and Engineering Exploration, Vol 5(42)

The above result gave a new on the proof of the By the compact level set assumption, the sequence ( Lagrangian necessary conditions: this actually ) has a subsequence converging to some point x , demonstrates the existence of a dual optimal solution. and since all the functions are closed, we know To prove that there is no duality gap we consider f() x s and g() x b . We deduce following two different approaches: first uses the Slater condition to force attainment in the dual v() b s , so (b; s) lies in epi v as we required. problem and second (dual) approach uses When vb() is finite, the same argument with compactness to force attainment in the primal problem. replaced by (b , v ( b )) for each r shows the infimum is attained. Theorem 4.3 (Dual attainment) If the Slater If the functions are convex condition holds for the primal problem (4.1) then the f, g12 , g ,...., gm primal and dual values are equal, and the dual value then we know v is convex. If d is  then, then is attained if finite. again from the inequality , there is nothing to pd Proof. If p is  there is nothing to prove, since we prove. If dv(  (0)) is finite then Theorem3.8 know . If on the other hand p is finite then, as pd shows v  cl v , and the above argument shows cl in the proof of the Lagrangian necessary conditions, = . Hence  , and the the Slater condition forces v(0)  .Hence v is p v(0) v (0)  d result follows. finite and lower semi continuous at 0 and the result follows by Corollary 4.3.6 (Zero duality gap). An indirect way of stating the Slater condition is that 5.Conclusion Any proper convex function h with an affine there is a point xˆ in E for which the set minorant has its biconjugate h somewhere finite. m is compact for all real . R L(;) xˆ   (In fact, because E is finite-dimensional, is The second approach uses a `dual' condition to ensure somewhere finite if and only if h is proper) Notice the value function is closed. that if either the objective function f or any one of the constraint functions g, g ,...., g has Theorem 4.4 (Primal attainment) Suppose that the 12 m functions compact level sets then the compact level set condition in the primal attainment results hold. are f, g12 , g ,...., gm : E   ,  Relationship between convex function and its closed, and that for some real ˆ  0 and some biconjugate results a great contribution in optimization. Lagrangian Duality theorem describe vector ˆ in m , the function ˆ ˆT has  R 0 fg  duality gap and zero duality gap between convex compact level sets. Then the value function v defined function and its biconjugate. by equation (4.3.4) is closed, and the infimum in this equation is attained when finite. Consequently, if the Acknowledgment None. functions f, g12 , g ,...., gm are in addition convex and the dual value for the problem (4.1) is not , then the primal and dual values, p and d, are Conflicts of interest equal, and the primal value is attained when finite. The author has no conflicts of interest to declare.

r References Proof. If the points (,)bsr lie in epi v for [1] Rockafellar RT. Convex analysis. Princeton r 1,2,...... , and approach the point (b; s), University Press Princeton. New Jersey. 1997. [2] Rockafellar RT, Wets RJ. Variational analysis. r then for each integer r there is a point x in E Springer Science & Business Media; 2009. [3] Ekeland I, Temam R. Convex analysis and variational satisfying r 1 and rr f() x sr  r gb()x  problems. SIAM; 1999. Hence we deduce [4] Dem’yanov VF, Malozemov VN. Introduction to ()ˆf  ˆTTT g()() xrr  ˆ s r1   ˆ b   ˆ s   ˆ b . minimax. Courier Corporation; 1990. 0 0r 0 [5] Borwein JM, Lewis AS, Noll D. Maximum entropy spectral analysis using first order information. part I:

Bhardwaj

fisher information and convex duality. Mathematics of [14] Li C, Fang D, López G, López MA. Stable and total Operations Research. 1996; 21:442-68. fenchel duality for convex optimization problems in [6] Nesterov Y, Nemirovskii A. Interior-point polynomial locally convex spaces. SIAM Journal on Optimization. algorithms in convex programming. SIAM; 1994. 2009; 20(2):1032-51. [7] Phelps RR. Convex functions, monotone operators and [15] Jeyakumar V, Li GY. Strong duality in robust convex differentiability. Lecture Notes in Mathematics. 1989; programming: complete characterizations. SIAM 1364:425-30. Journal on Optimization. 2010; 20(6):3384-407. [8] Peressini AL, Sullivan FE, Uhl JJ. The mathematics of [16] Wang M, Fang D, Chen Z. Strong and total Fenchel nonlinear programming. New York: Springer-Verlag; dualities for robust convex optimization problems. 1988. Journal of Inequalities and Applications. 2015; [9] Holmes RB. Geometric functional analysis and its 2015:70. applications. Springer Science & Business Media; [17] Fajardo MD, Vidal J. Stable strong Fenchel and 1975. Lagrange duality for evenly convex optimization [10] Wright SJ. Primal-dual interior-point methods. SIAM; problems. Optimization. 2016; 65(9):1675-91. 1997. [11] Yang XQ. On the gap functions of prevariational Vinod Kumar Bhardwaj received his inequalities. Journal of Optimization Theory and Ph.D. degree in Mathematics from the Applications. 2003; 116(2):437-52. Dr. B. R. Ambedkar University, Agra [12] Zhang J, Wan C, Xiu N. The dual gap function for (India). He is currently Assistant variational inequalities. Applied Mathematics and Professor at the Department of Optimization. 2003; 48(2):129-48. Mathematics of GLA University, [13] Bertsekas DP. Nonlinear programming. Belmont: Mathura (India). His research interests Athena Scientific; 1999. are optimization problems. Email: [email protected]