DOCSLIB.ORG

Convex Optimization 3

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Convex Optimization 3 Convex Optimization 3. Convex Functions Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2018 SJTU Ying Cui 1 / 42 Outline Basic properties and examples Operations that preserve convexity The conjugate function Quasiconvex functions Log-concave and log-convex functions Convexity with respect to generalized inequalities SJTU Ying Cui 2 / 42 Definition ◮ convex: f : Rn R is convex if domf is a convex set and if → f (θx + (1 θ)y) θf (x) + (1 θ)f (y) − ≤ − for all x, y domf , and θ with 0 θ 1 ∈ ≤ ≤ ◮ geometric interpretation: line segment between (x, f (x)) and (y, f (y)) (i.e., chord from x to y) lies above graph of f (y, f(y)) (x, f(x)) Figure 3.1 Graph of a convex function. The chord (i.e. , line segment) be- tween any two points on the graph lies above the graph. ◮ concave: f is concave if f is convex − SJTU Ying Cui 3 / 42 Definition ◮ strictly convex: f : Rn R is strictly convex if domf is a → convex set and if f (θx + (1 θ)y) <θf (x) + (1 θ)f (y) − − for all x, y domf , x = y, and θ with 0 <θ< 1 ∈ 6 ◮ strictly concave: f is strictly concave if f is strictly convex − ◮ affine functions are both convex and concave ◮ any function that is convex and concave is affine SJTU Ying Cui 4 / 42 Examples on R convex: ◮ affine: ax+b on R, for any a, b R ∈ ◮ exponential: eax on R, for any a R ∈ ◮ powers: xα on R , for α 1 or α 0 ++ ≥ ≤ ◮ powers of absolute value: x p on R, for p 1 ◮ | | ≥ negative entropy: x log x on R++ concave: ◮ affine: ax+b on R, for any a, b R ∈ ◮ α powers: x on R++, for 0 α 1 ◮ ≤ ≤ logarithm: log x on R++ SJTU Ying Cui 5 / 42 n m n Examples on R and R × Examples on Rn: ◮ affine function f (x)=aT x+b is both convex and concave ◮ every norm is convex ◮ due to triangle inequality and homogeneity ◮ n p 1/p n lp-norms: x p=( xi ) for p 1 ( x = xi , || || i=1 | | ≥ || ||1 i=1 | | x ∞=maxk xk ) ◮ || || | | P P max function f (x) = max x , , xn is convex { 1 · · · } ◮ log-sum-exp f (x) = log(ex1 + + exn ) is convex · · · ◮ a differentiable approximation of the max function: x1 xn x1 xn log(e + +e ) log n max x , , xn log(e + +e ) ··· − ≤ { 1 ··· }≤ ··· m n Examples on R × : ◮ T m n affine function f (X )= tr(A X )+ b = i=1 j=1 Aij Xij + b is both convex and concave P P ◮ spectral (maximum singular value) norm f (X )= X = σ (X ) = (λ (X T X ))1/2 on is convex k k2 max max SJTU Ying Cui 6 / 42 Restriction of a convex function to a line ◮ a function f : Rn R is convex iff it is convex when restricted to any line→ that intersects its domain, i.e., ◮ g(t)= f (x + tv) is convex on t x + tv domf for all x domf and all v Rn { | ∈ } ∈ ∈ ◮ check convexity of a function of multiple variables by restricting it to a line and checking convexity of a function of one variable ◮ example: f : Sn R with f (X ) = logdet X , domf = Sn → ++ Consider an arbitrary line X = Z + tV Sn with Z, V Sn. ∈ ++ ∈ w. l. o. g., assume t = 0 is in the interval, i.e., Z Sn . ∈ ++ 1/2 1/2 1/2 1/2 g(t) = log det(Z + tV ) = log det(Z (I + tZ − VZ − )Z ) 1/2 1/2 =logdet Z + log det(I + tZ − VZ − ) n 1/2 1/2 =logdet Z + log(1 + tλi ) λi : eigenvalues of Z − VZ − i X=1 g is concave in t. Thus, f is concave. SJTU Ying Cui 7 / 42 Extended-value extension ◮ extended-value extension f˜ of a convex function f is f (x), x domf f˜(x)= ∈ , x / domf (∞ ∈ ◮ f˜ is defined on all Rn, and takes values in R ◮ recover domain of f from f˜ as domf = x f˜(∪x {∞}) < { | ∞} ◮ extension can simplify notation, as no need to explicitly describe the domain, or add the qualifier ‘for all x domf ’ ∈ ◮ basic defining inequality for convexity can be expressed as: for 0 <θ< 1, f˜(θx + (1 θ)y) θf˜(x) + (1 θ)f˜(y) for any x − ≤ − and y ◮ the inequality always holds for θ =0, 1 ◮ no need to mention the two conditions: domf is convex (can be shown by contradiction) and x, y domf (x, y Rn is used instead, which can be omitted) ∈ ∈ SJTU Ying Cui 8 / 42 First-order conditions Suppose f is differentiable, i.e., domf is open and gradient f (x)= ∂f (x) , , ∂f (x) exists at any x domf ∇ ∂x1 · · · ∂xn ∈ ◮ f is convex iff domf is convex and f (y) f (x)+ f (x)T (y x) for all x, y domf ≥ ∇ − ∈ ◮ first-order Taylor approx. of a convex function is a global underestimator of it; if first-order Taylor approx. of a function is always a global underestimator of it, then it is convex ◮ local information about a convex function (value and derivative at a point) implies global information (a global underestimator) ◮ if f is convex and f (x) = 0, then x is a global minimizer of f ∇ ◮ f is strictly convex iff domf is convex and f (y) > f (x)+ f (x)T (y x) for all x, y domf and x = y ∇ − ∈ 6 f(y) f(x) + ∇f(x)T (y − x) (x, f(x)) Figure 3.2 If f is convex and differentiable, then f(x)+∇f(x)T (y−x) ≤ f(y) for all x, y ∈ dom f. SJTU Ying Cui 9 / 42 Second-order conditions Suppose f is twice differentiable, i.e., domf is open and Hessian 2 n 2 ∂2f (x) f (x) S exists at any x domf , where f (x)ij = , ∇ ∈ ∈ ∇ ∂xi ∂xj i, j = 1, , n · · · ◮ f is convex iff domf is convex and 2f (x) 0 for all ∇ x domf ∈ ◮ for a function on R, this reduces to domf is an interval and f ′′(x) 0 for all x in the interval ◮ 2f (x≥) 0 means the graph of f has positive (upward) curvature∇ at x ◮ if domf is convex and 2f (x) 0 for all x domf , then f is strictly convex ∇ ≻ ∈ ◮ the converse is not true, e.g., f (x)= x 4 is strictly convex but f ′′(0) = 0 SJTU Ying Cui 10 / 42 Second-order conditions Examples ◮ quadratic function: f (x) = (1/2)xT Px + qT x + r (P Sn) ∈ f (x)= Px + q, 2f (x)= P ∇ ∇ convex iff P Sn ∈ + ◮ least-squares objective: f (x)= Ax b 2 = xT AT Ax 2xT AT b + bT b k − k2 − f (x) = 2AT (Ax b), 2f (x) = 2AT A ∇ − ∇ convex for all A Rm n (as AT A 0 for all A Rm n) ∈ × ∈ × ◮ quadratic-over-linear function: f (x, y)= x2/y 2 y y T 2f (x, y)= 0 ∇ y 3 x x − − convex for all x R and y R (as zzT 0 for all z Rn) ∈ ∈ ++ ∈ SJTU Ying Cui 11 / 42 Second-order conditions Examples ◮ n log-sum-exp: f (x) =log k=1 exp xk is convex ◮ proof: P 2 1 1 T f (x)= diag(z) zz (zk = exp xk ) ∇ 1T z − (1T z)2 to show 2f (x) 0, we must verify that v T 2f (x)v 0 for all v: ∇ ∇ ≥ 2 2 ( zk v )( zk ) ( vk zk ) v T 2f (x)v = k k k − k 0 ∇ ( z )2 ≥ P P k k P 2 2 P since ( k vk zk ) ( k zk vk )( k zk ) (from Cauchy-Schwarz T ≤T T 2 inequality (a a)(b b) (a b) by treating ai = vi √zi and P P≥ P bi = √zi ) ◮ n 1/n n geometric mean: f (x) = ( k=1 xk ) on R++ is concave (similar proof as for log-sum-exp) Q SJTU Ying Cui 12 / 42 Sublevel set and superlevel set Sublevel set ◮ α-sublevel set of f : Rn R: x domf f (x) α → { ∈ | ≤ } ◮ sublevel sets of a convex function are convex ◮ the converse is false (e.g., f (x)= exp x is not convex (indeed, strictly concave) but all its− sublevel sets are convex) Superlevel set ◮ α-superlevel set of f : Rn R: x domf f (x) α → { ∈ | ≥ } ◮ superlevel sets of a concave function are convex To establish convexity of a set, express it as a sublevel set of a convex function, or as the superlevel set of a concave function. SJTU Ying Cui 13 / 42 Epigraph and hypergraph ◮ graph of f : Rn R: → (x, f (x)) x domf Rn+1 { | ∈ }⊆ ◮ epigraph of f : Rn R: → epi f = (x, t) Rn+1 x domf , f (x) t Rn+1 { ∈ | ∈ ≤ }⊆ ◮ f is convex iff epi f is a convex set ◮ hypograph of f : Rn R: → hypo f = (x, t) Rn+1 x domf , f (x) t Rn+1 { ∈ | ∈ ≥ }⊆ ◮ f is concave iff hypo f is a convex set epi f f Figure 3.5 Epigraph of a function f, shown shaded. The lower boundary, shown darker, is the graph of f. SJTU Ying Cui 14 / 42 Jensen’s inequality and extensions ◮ basic inequality: if f is convex, x, y domf and 0 θ 1, ∈ ≤ ≤ then f (θx + (1 θ)y) θf (x) + (1 θ)f (y) − ≤ − ◮ extension to convex combinations of more than two points: if f is convex, x , , xk domf , and θ , ,θk 0 with 1 · · · ∈ 1 · · · ≥ θ + + θk = 1, then 1 · · · f (θ x + + θk xk ) θ f (x )+ + θk f (xk ) 1 1 · · · ≤ 1 1 · · · ◮ extensions to infinite sums and integrals (if p(x) 0 on ≥ S domf , p(x)dx = 1, then ⊆ S f p(x)xdx f (x)p(x)dx, provided the integrals exist) S R S ◮ ≤ extensionR to expected R values: if f is convex and X is a random variable such that X domf w.p.
Load more

Recommended publications
  • CORE View Metadata, Citation and Similar Papers at Core.Ac.Uk
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Bulgarian Digital Mathematics Library at IMI-BAS Serdica Math. J. 27 (2001), 203-218 FIRST ORDER CHARACTERIZATIONS OF PSEUDOCONVEX FUNCTIONS Vsevolod Ivanov Ivanov Communicated by A. L. Dontchev Abstract. First order characterizations of pseudoconvex functions are investigated in terms of generalized directional derivatives. A connection with the invexity is analysed. Well-known first order characterizations of the solution sets of pseudolinear programs are generalized to the case of pseudoconvex programs. The concepts of pseudoconvexity and invexity do not depend on a single definition of the generalized directional derivative. 1. Introduction. Three characterizations of pseudoconvex functions are considered in this paper. The first is new. It is well-known that each pseudo- convex function is invex. Then the following question arises: what is the type of 2000 Mathematics Subject Classification: 26B25, 90C26, 26E15. Key words: Generalized convexity, nonsmooth function, generalized directional derivative, pseudoconvex function, quasiconvex function, invex function, nonsmooth optimization, solution sets, pseudomonotone generalized directional derivative. 204 Vsevolod Ivanov Ivanov the function η from the definition of invexity, when the invex function is pseudo- convex. This question is considered in Section 3, and a first order necessary and sufficient condition for pseudoconvexity of a function is given there. It is shown that the class of strongly pseudoconvex functions, considered by Weir [25], coin- cides with pseudoconvex ones. The main result of Section 3 is applied to characterize the solution set of a nonlinear programming problem in Section 4. The base results of Jeyakumar and Yang in the paper [13] are generalized there to the case, when the function is pseudoconvex.
    [Show full text]
  • Robustly Quasiconvex Function
    Robustly quasiconvex function Bui Thi Hoa Centre for Informatics and Applied Optimization, Federation University Australia May 21, 2018 Bui Thi Hoa (CIAO, FedUni) Robustly quasiconvex function May 21, 2018 1 / 15 Convex functions 1 All lower level sets are convex. 2 Each local minimum is a global minimum. 3 Each stationary point is a global minimizer. Bui Thi Hoa (CIAO, FedUni) Robustly quasiconvex function May 21, 2018 2 / 15 Definition f is called explicitly quasiconvex if it is quasiconvex and for all λ 2 (0; 1) f (λx + (1 − λ)y) < maxff (x); f (y)g; with f (x) 6= f (y): Example 1 f1 : R ! R; f1(x) = 0; x 6= 0; f1(0) = 1. 2 f2 : R ! R; f2(x) = 1; x 6= 0; f2(0) = 0. 3 Convex functions are quasiconvex, and explicitly quasiconvex. 4 f (x) = x3 are quasiconvex, and explicitly quasiconvex, but not convex. Generalised Convexity Definition A function f : X ! R, with a convex domf , is called quasiconvex if for all x; y 2 domf , and λ 2 [0; 1] we have f (λx + (1 − λ)y) ≤ maxff (x); f (y)g: Bui Thi Hoa (CIAO, FedUni) Robustly quasiconvex function May 21, 2018 3 / 15 Example 1 f1 : R ! R; f1(x) = 0; x 6= 0; f1(0) = 1. 2 f2 : R ! R; f2(x) = 1; x 6= 0; f2(0) = 0. 3 Convex functions are quasiconvex, and explicitly quasiconvex. 4 f (x) = x3 are quasiconvex, and explicitly quasiconvex, but not convex. Generalised Convexity Definition A function f : X ! R, with a convex domf , is called quasiconvex if for all x; y 2 domf , and λ 2 [0; 1] we have f (λx + (1 − λ)y) ≤ maxff (x); f (y)g: Definition f is called explicitly quasiconvex if it is quasiconvex and
    [Show full text]
  • Convexity I: Sets and Functions
    Convexity I: Sets and Functions Ryan Tibshirani Convex Optimization 10-725/36-725 See supplements for reviews of basic real analysis • basic multivariate calculus • basic linear algebra • Last time: why convexity? Why convexity? Simply put: because we can broadly understand and solve convex optimization problems Nonconvex problems are mostly treated on a case by case basis Reminder: a convex optimization problem is of ● the form ● ● min f(x) x2D ● ● subject to gi(x) 0; i = 1; : : : m ≤ hj(x) = 0; j = 1; : : : r ● ● where f and gi, i = 1; : : : m are all convex, and ● hj, j = 1; : : : r are affine. Special property: any ● local minimizer is a global minimizer ● 2 Outline Today: Convex sets • Examples • Key properties • Operations preserving convexity • Same for convex functions • 3 Convex sets n Convex set: C R such that ⊆ x; y C = tx + (1 t)y C for all 0 t 1 2 ) − 2 ≤ ≤ In words, line segment joining any two elements lies entirely in set 24 2 Convex sets Figure 2.2 Some simple convexn and nonconvex sets. Left. The hexagon, Convex combinationwhich includesof x1; its : :boundary : xk (shownR : darker), any linear is convex. combinationMiddle. The kidney shaped set is not convex, since2 the line segment between the twopointsin the set shown as dots is not contained in the set. Right. The square contains some boundaryθ1x points1 + but::: not+ others,θkxk and is not convex. k with θi 0, i = 1; : : : k, and θi = 1. Convex hull of a set C, ≥ i=1 conv(C), is all convex combinations of elements. Always convex P 4 Figure 2.3 The convex hulls of two sets in R2.
    [Show full text]
  • Local Maximum Points of Explicitly Quasiconvex Functions
    Local maximum points of explicitly quasiconvex functions Item Type Article Authors Bagdasar, Ovidiu; Popovici, Nicolae Citation Bagdasar, O. & Popovici, (2015) 'Local maximum points of explicitly quasiconvex functions' Optimization Letters, 9: 769. doi:10.1007/s11590-014-0781-3 DOI 10.1007/s11590-014-0781-3 Publisher Springer Journal Optimization Letters Rights Archived with thanks to Optimization Letters Download date 25/09/2021 20:26:30 Link to Item http://hdl.handle.net/10545/620886 Optimization Letters manuscript No. (will be inserted by the editor) Local maximum points of explicitly quasiconvex functions Ovidiu Bagdasar · Nicolae Popovici Received: date / Accepted: date Abstract This work concerns (generalized) convex real-valued functions defined on a nonempty convex subset of a real topological linear space. Its aim is twofold. The first concerns explicitly quasiconvex functions. As a counterpart of some known results, it is shown that any local maximum point of such a function is actually a global minimum point whenever it belongs to the intrinsic core of the function's domain. Secondly, we establish a new characterization of strictly convex normed spaces by applying this property for a particular class of convex functions. Keywords Local maximum point · Relative algebraic interior · Convex function · Explicitly quasiconvex function · Strictly convex space · Least squares problem 1 Introduction Optimization problems involving explicitly quasiconvex objective functions, i.e., real-valued functions which are both quasiconvex and semistrictly quasiconvex, have been intensively studied in the literature, beginning with the pioneering works by Martos [6] and Karamardian [5]. These functions are of special interest since they preserve several fundamental properties of convex functions.
    [Show full text]
  • A Class of Functions That Are Quasiconvex but Not Polyconvex
    University of Tennessee, Knoxville TRACE: Tennessee Research and Creative Exchange Masters Theses Graduate School 12-2003 A Class of Functions That Are Quasiconvex But Not Polyconvex Catherine S. Remus University of Tennessee - Knoxville Follow this and additional works at: https://trace.tennessee.edu/utk_gradthes Part of the Mathematics Commons Recommended Citation Remus, Catherine S., "A Class of Functions That Are Quasiconvex But Not Polyconvex. " Master's Thesis, University of Tennessee, 2003. https://trace.tennessee.edu/utk_gradthes/2219 This Thesis is brought to you for free and open access by the Graduate School at TRACE: Tennessee Research and Creative Exchange. It has been accepted for inclusion in Masters Theses by an authorized administrator of TRACE: Tennessee Research and Creative Exchange. For more information, please contact [email protected]. To the Graduate Council: I am submitting herewith a thesis written by Catherine S. Remus entitled "A Class of Functions That Are Quasiconvex But Not Polyconvex." I have examined the final electronic copy of this thesis for form and content and recommend that it be accepted in partial fulfillment of the requirements for the degree of Master of Science, with a major in Mathematics. Henry C. Simpson, Major Professor We have read this thesis and recommend its acceptance: Charles Collins, G. Samuel Jordan Accepted for the Council: Carolyn R. Hodges Vice Provost and Dean of the Graduate School (Original signatures are on file with official studentecor r ds.) To the Graduate Council: I am submitting herewith a thesis written by Catherine S. Remus entitled “A Class of Functions That Are Quasiconvex But Not Polyconvex”.
    [Show full text]
  • Convex Functions
    Convex Functions Prof. Daniel P. Palomar ELEC5470/IEDA6100A - Convex Optimization The Hong Kong University of Science and Technology (HKUST) Fall 2020-21 Outline of Lecture • Definition convex function • Examples on R, Rn, and Rn×n • Restriction of a convex function to a line • First- and second-order conditions • Operations that preserve convexity • Quasi-convexity, log-convexity, and convexity w.r.t. generalized inequalities (Acknowledgement to Stephen Boyd for material for this lecture.) Daniel P. Palomar 1 Definition of Convex Function • A function f : Rn −! R is said to be convex if the domain, dom f, is convex and for any x; y 2 dom f and 0 ≤ θ ≤ 1, f (θx + (1 − θ) y) ≤ θf (x) + (1 − θ) f (y) : • f is strictly convex if the inequality is strict for 0 < θ < 1. • f is concave if −f is convex. Daniel P. Palomar 2 Examples on R Convex functions: • affine: ax + b on R • powers of absolute value: jxjp on R, for p ≥ 1 (e.g., jxj) p 2 • powers: x on R++, for p ≥ 1 or p ≤ 0 (e.g., x ) • exponential: eax on R • negative entropy: x log x on R++ Concave functions: • affine: ax + b on R p • powers: x on R++, for 0 ≤ p ≤ 1 • logarithm: log x on R++ Daniel P. Palomar 3 Examples on Rn • Affine functions f (x) = aT x + b are convex and concave on Rn. n • Norms kxk are convex on R (e.g., kxk1, kxk1, kxk2). • Quadratic functions f (x) = xT P x + 2qT x + r are convex Rn if and only if P 0.
    [Show full text]
  • ABOUT STATIONARITY and REGULARITY in VARIATIONAL ANALYSIS 1. Introduction the Paper Investigates Extremality, Stationarity and R
    1 ABOUT STATIONARITY AND REGULARITY IN VARIATIONAL ANALYSIS 2 ALEXANDER Y. KRUGER To Boris Mordukhovich on his 60th birthday Abstract. Stationarity and regularity concepts for the three typical for variational analysis classes of objects { real-valued functions, collections of sets, and multifunctions { are investi- gated. An attempt is maid to present a classification scheme for such concepts and to show that properties introduced for objects from different classes can be treated in a similar way. Furthermore, in many cases the corresponding properties appear to be in a sense equivalent. The properties are defined in terms of certain constants which in the case of regularity proper- ties provide also some quantitative characterizations of these properties. The relations between different constants and properties are discussed. An important feature of the new variational techniques is that they can handle nonsmooth 3 functions, sets and multifunctions equally well Borwein and Zhu [8] 4 1. Introduction 5 The paper investigates extremality, stationarity and regularity properties of real-valued func- 6 tions, collections of sets, and multifunctions attempting at developing a unifying scheme for defining 7 and using such properties. 8 Under different names this type of properties have been explored for centuries. A classical 9 example of a stationarity condition is given by the Fermat theorem on local minima and max- 10 ima of differentiable functions. In a sense, any necessary optimality (extremality) conditions de- 11 fine/characterize certain stationarity (singularity/irregularity) properties. The separation theorem 12 also characterizes a kind of extremal (stationary) behavior of convex sets. 13 Surjectivity of a linear continuous mapping in the Banach open mapping theorem (and its 14 extension to nonlinear mappings known as Lyusternik-Graves theorem) is an example of a regularity 15 condition.
    [Show full text]
  • Characterisations of Quasiconvex Functions
    BULL. AUSTRAL. MATH. SOC. 26B25, 49J52 VOL. 48 (1993) [393-406] CHARACTERISATIONS OF QUASICONVEX FUNCTIONS DINH THE LUC In this paper we introduce the concept of quasimonotone maps and prove that a lower semicontinuous function on an infinite dimensional space is quasiconvex if and only if its generalised subdifferential or its directional derivative is quasimonotone. 1. INTRODUCTION As far as we know quasiconvex functions were first mentioned by de Finetti in his work "Sulle Straficazioni Convesse" in 1949, [6]. Since then much effort has been focused on the study of this class of functions for they have much in common with convex functions and they are used in several areas of science including mathematics, operations research, economics et cetera (see [15]). One of the most important properties of convex functions is that their level sets are convex. This property is also a fundamental geometric characterisation of quasiconvex functions which sometimes is treated as their definition. However, the most attractive characterisations of quasiconvex functions are those which involve gradients (a detailed account of the current state of research on the topic can be found in [l]). As to generalised derivatives of quasiconvex functions, very few results exist (see [1,7]). In [7], a study of quasiconvex functions is presented via Clarke's subdifferential, but the authors restrict themselves to the case of Lipschitz functions on a finite dimensional space only. The aim of our paper is to characterise lower semicontinuous quasiconvex functions in terms of generalised (Clarke-Rockafellar) subdifferentials and directional derivatives in infinite dimensional spaces. Namely, we shall first introduce a new concept of quasi- monotone maps; and then we prove the equivalence between the quasiconvexity of a function and the quasimonotonicity of its generalised subdifferential and its directional derivative.
    [Show full text]
  • Sum of Squares and Polynomial Convexity
    CONFIDENTIAL. Limited circulation. For review only. Sum of Squares and Polynomial Convexity Amir Ali Ahmadi and Pablo A. Parrilo Abstract— The notion of sos-convexity has recently been semidefiniteness of the Hessian matrix is replaced with proposed as a tractable sufficient condition for convexity of the existence of an appropriately defined sum of squares polynomials based on sum of squares decomposition. A multi- decomposition. As we will briefly review in this paper, by variate polynomial p(x) = p(x1; : : : ; xn) is said to be sos-convex if its Hessian H(x) can be factored as H(x) = M T (x) M (x) drawing some appealing connections between real algebra with a possibly nonsquare polynomial matrix M(x). It turns and numerical optimization, the latter problem can be re- out that one can reduce the problem of deciding sos-convexity duced to the feasibility of a semidefinite program. of a polynomial to the feasibility of a semidefinite program, Despite the relative recency of the concept of sos- which can be checked efficiently. Motivated by this computa- convexity, it has already appeared in a number of theo- tional tractability, it has been speculated whether every convex polynomial must necessarily be sos-convex. In this paper, we retical and practical settings. In [6], Helton and Nie use answer this question in the negative by presenting an explicit sos-convexity to give sufficient conditions for semidefinite example of a trivariate homogeneous polynomial of degree eight representability of semialgebraic sets. In [7], Lasserre uses that is convex but not sos-convex. sos-convexity to extend Jensen’s inequality in convex anal- ysis to linear functionals that are not necessarily probability I.
    [Show full text]
  • Subgradients
    Subgradients Ryan Tibshirani Convex Optimization 10-725/36-725 Last time: gradient descent Consider the problem min f(x) x n for f convex and differentiable, dom(f) = R . Gradient descent: (0) n choose initial x 2 R , repeat (k) (k−1) (k−1) x = x − tk · rf(x ); k = 1; 2; 3;::: Step sizes tk chosen to be fixed and small, or by backtracking line search If rf Lipschitz, gradient descent has convergence rate O(1/) Downsides: • Requires f differentiable next lecture • Can be slow to converge two lectures from now 2 Outline Today: crucial mathematical underpinnings! • Subgradients • Examples • Subgradient rules • Optimality characterizations 3 Subgradients Remember that for convex and differentiable f, f(y) ≥ f(x) + rf(x)T (y − x) for all x; y I.e., linear approximation always underestimates f n A subgradient of a convex function f at x is any g 2 R such that f(y) ≥ f(x) + gT (y − x) for all y • Always exists • If f differentiable at x, then g = rf(x) uniquely • Actually, same definition works for nonconvex f (however, subgradients need not exist) 4 Examples of subgradients Consider f : R ! R, f(x) = jxj 2.0 1.5 1.0 f(x) 0.5 0.0 −0.5 −2 −1 0 1 2 x • For x 6= 0, unique subgradient g = sign(x) • For x = 0, subgradient g is any element of [−1; 1] 5 n Consider f : R ! R, f(x) = kxk2 f(x) x2 x1 • For x 6= 0, unique subgradient g = x=kxk2 • For x = 0, subgradient g is any element of fz : kzk2 ≤ 1g 6 n Consider f : R ! R, f(x) = kxk1 f(x) x2 x1 • For xi 6= 0, unique ith component gi = sign(xi) • For xi = 0, ith component gi is any element of [−1; 1] 7 n
    [Show full text]
  • NP-Hardness of Deciding Convexity of Quartic Polynomials and Related Problems
    NP-hardness of Deciding Convexity of Quartic Polynomials and Related Problems Amir Ali Ahmadi, Alex Olshevsky, Pablo A. Parrilo, and John N. Tsitsiklis ∗y Abstract We show that unless P=NP, there exists no polynomial time (or even pseudo-polynomial time) algorithm that can decide whether a multivariate polynomial of degree four (or higher even degree) is globally convex. This solves a problem that has been open since 1992 when N. Z. Shor asked for the complexity of deciding convexity for quartic polynomials. We also prove that deciding strict convexity, strong convexity, quasiconvexity, and pseudoconvexity of polynomials of even degree four or higher is strongly NP-hard. By contrast, we show that quasiconvexity and pseudoconvexity of odd degree polynomials can be decided in polynomial time. 1 Introduction The role of convexity in modern day mathematical programming has proven to be remarkably fundamental, to the point that tractability of an optimization problem is nowadays assessed, more often than not, by whether or not the problem benefits from some sort of underlying convexity. In the famous words of Rockafellar [39]: \In fact the great watershed in optimization isn't between linearity and nonlinearity, but convexity and nonconvexity." But how easy is it to distinguish between convexity and nonconvexity? Can we decide in an efficient manner if a given optimization problem is convex? A class of optimization problems that allow for a rigorous study of this question from a com- putational complexity viewpoint is the class of polynomial optimization problems. These are op- timization problems where the objective is given by a polynomial function and the feasible set is described by polynomial inequalities.
    [Show full text]
  • Monotonic Transformations: Cardinal Versus Ordinal Utility
    Natalia Lazzati Mathematics for Economics (Part I) Note 10: Quasiconcave and Pseudoconcave Functions Note 10 is based on Madden (1986, Ch. 13, 14) and Simon and Blume (1994, Ch. 21). Monotonic transformations: Cardinal Versus Ordinal Utility A utility function could be said to measure the level of satisfaction associated to each commodity bundle. Nowadays, no economist really believes that a real number can be assigned to each commodity bundle which expresses (in utils?) the consumer’slevel of satisfaction with this bundle. Economists believe that consumers have well-behaved preferences over bundles and that, given any two bundles, a consumer can indicate a preference of one over the other or the indi¤erence between the two. Although economists work with utility functions, they are concerned with the level sets of such functions, not with the number that the utility function assigns to any given level set. In consumer theory these level sets are called indi¤erence curves. A property of utility functions is called ordinal if it depends only on the shape and location of a consumer’sindi¤erence curves. It is alternatively called cardinal if it also depends on the actual amount of utility the utility function assigns to each indi¤erence set. In the modern approach, we say that two utility functions are equivalent if they have the same indi¤erence sets, although they may assign di¤erent numbers to each level set. For instance, let 2 2 u : R+ R where u (x) = x1x2 be a utility function, and let v : R+ R be the utility function ! ! v (x) = u (x) + 1: These two utility functions represent the same preferences and are therefore equivalent.
    [Show full text]