DOCSLIB.ORG

Arxiv:1902.10921V1 [Math.OC] 28 Feb 2019 Programming Nvriy Ejn,109,P .Ciaemi:Yxia@Buaa and E-Mail: Mathematics China of R

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Arxiv:1902.10921V1 [Math.OC] 28 Feb 2019 Programming Nvriy Ejn,109,P .Ciaemi:Yxia@Buaa and E-Mail: Mathematics China of R Noname manuscript No. (will be inserted by the editor) A survey of hidden convex optimization Yong Xia Received: date / Accepted: date Abstract Motivated by the fact that not all nonconvex optimization problems are difficult to solve, we survey in this paper three widely-used ways to reveal the hidden convex structure for different classes of nonconvex optimization problems. Finally, ten open problems are raised. Keywords convex programming quadratic programming quadratic matrix programming fractional programming· Lagrangian dual · semidefinite programming · · · Mathematics Subject Classification (2010) 90C20, 90C25, 90C26, 90C32 1 Introduction Nowadays, convex programming becomes very popular due to not only its various real applications but also the charming property that any local solution also remains global optimality. However, it does not mean that convex programming problems are easy to solve. First, it may be difficult to identify convexity. Actually, deciding whether a quartic polynomials is globally convex is NP-hard [1]. Second, evaluating a convex function is also not always easy. For example, the induced matrix norm Ax p A p = sup k k k k x x p =06 k k This research was supported by National Natural Science Foundation of China under grants arXiv:1902.10921v1 [math.OC] 28 Feb 2019 11822103, 11571029 and Beijing Natural Science Foundation Z180005. Y. Xia LMIB of the Ministry of Education; School of Mathematics and System Sciences, Beihang University, Beijing, 100191, P. R. China E-mail: [email protected] 2 Yong Xia is a convex function. But evaluating A p is NP-hard if p =1, 2, [35]. Third, convex programming problems mayk bek difficult to solve.6 It has been∞ shown in [20] that the general mixed-binary quadratic optimization problems can be equivalently reformulated as convex programming over the copositive cone Co := A Rn×n : A = AT , xT Ax 0, x 0, x Rn . { ∈ ≥ ∀ ≥ ∈ } Notice that checking whether A Co is NP-Complete [52]. Like a coin has two sides, there∈ are quite a few nonconvex optimization problems could be globally and efficiently solved in polynomial time. The rea- son behind this observation is that most of them belong to the hidden convex optimization, i.e., they admit equivalent convex programming reformulations. In the past two decades, hidden convex optimization problems were studied case by case in literature. In this survey, we summarize three class of ap- proaches to reveal the hidden convex structure, namely, nonlinear transforma- tion, Lagrangian dual and the primal tight convex relaxation. The remainder of this paper is organized as follows. Section 2 presents the most common used nonlinear transformations for different classes of problems. Section 3 shows how Lagrangian duality and its variations achieve zero gap. Section 4 summarizes some classes of nonconvex optimization problems ad- mitting tight primal relaxations. As a concluding remark, ten open problems are raised in Section 5. Rn Throughout the paper, let v( ) be the optimal value of problem ( ). + de- notes the nonnegative orthant in·Rn. For a matrix A, denote by A ·( )0 that A is positive (semi)definite. The trace of A is defined as the sum≻ of its diago- n nal elements, i.e., tr(A)= i=1 Aii. λmin(A) and λmax(A) denote the minimal and maximal eigenvalues of A, respectively. In is the identity matrix of order T P n. x = √x x stands for the ℓ2 norm of a vector x. diag(x1,..., xn) returns k k T n a diagonal matrix with diagonal elements x1,...,xn. e = (1, 1,..., 1) R . Denote by conv Ω the convex hull of Ω. E denotes the number of elements∈ in the set E. For{ a} real value x,[x] returns| | the largest integer less than or equal to x. 2 Nonlinear transformation In this section, we survey some nonlinear transformation approaches widely used in convexifying different classes of nonconvex optimization problems. 2.1 A univariate example Univariate examples are not always as simple as converting the concave func- tion √x to y( 0) by introducing the one-to-one mapping x = y2 with y 0. The approach≥ [77] for reducing the duality gap for box constrained noncon-≥ vex quadratic program requires solving the following nonconvex subproblem 2 (G) max φ(θ) := min a1θ,a2δ (θ) , (1) θ { } Hidden convex optimization 3 where a1, a2 are two positive scalars, 2 2 δ (θ) = min (xi yi) − i I J ∈X∪ s.t. x C, yi [ 1, 1], i I, ∈ ∈ − ∈ √1 θ ωiyi 1, i J, − ≤ ≤ ∈ C is a linear manifold and ωi 1, 1 is given. The objective function φ(θ) (1) is not concave and thus (G)∈ {− is a} nonconvex minimization. Introducing σ = √1 θ, one can reformulate (G) as the following convex program: − max σ √a2 (x y) s.t. · − √a1, √a1 σ ≤ · 1 yi 1, i I, − ≤ ≤ ∈ σ ωiyi 1, i J, ≤ ≤ ∈ x C, σ 0. ∈ ≥ 2.2 p-th power approach Consider the following constrained nonconvex optimization problems: (P) min f(x): gi(x) bi,i =1,...,m, x S , { ≤ ∈ } where f(x), gi(x) (i =1, 2,...,m) are strictly positive over S and S is closed, bounded, connected and of a full dimension. It is known [41] that if the per- turbation function w(y) := min f(x): gi(x) yi,i =1,...,m, x S x∈S { ≤ ∈ } is locally convex in a closed and non-degenerate neighbourhood of b, then there is no duality gap between (P ) and its Lagrangian dual in the sense that the inner minimization is local and the outer maximization is in the neighbourhood of the optimal Lagrangian multiplier. In order to achieve such a zero duality gap, Li [43] first introduced the p-th power transformation: p p p (Pp) min f (x): g (x) b ,i =1,...,m, x S . { i ≤ i ∈ } Under some additional assumptions, Li [43] showed that, for a sufficiently large p p, the perturbation function of (Pp) is a convex function of y for any y in a neighbourhood of b. For more applications of the p-th power formulation, we refer to [45,46] and references therein. More generalizations of the p-th power convexification approach are further studied in [44,75]. Recently, a novel shifted p-th power reformulation is introduced in [84]: p p (Pp,µ) min f(x) : (gi(x)+ µi) (bi + µi) ,i =1,...,m, x S . { ≤ ∈ } 4 Yong Xia Surprisingly, under the same assumptions, there is a parametric vector µ Rm such that p = 3 is sufficient to guarantee the convexity of the perturbation∈ p function of (Pp,µ) in terms of (y+µ) with y lying in a neighbourhood of b. Be- sides, the assumption on the strict positiveness of f(x) and gi(x) is redundant in this new reformulation. For more details, we refer to [84]. 2.3 Minimal-volumn ellipsoid cover Consider the geometric problem of finding n-dimensional ellipsoid of minimal volume covering a set of m given points ai, i =1,...,m: −1 (MVE) minQ,x un det(Q ) T s.t. (x ai) Q(x ai) 1,i =1, . , m, p− − ≤ n where un is the volume of the unit ball in R and the objective is the n- dimensional volume. 1 By first introducing M = Q 2 (which is well defined as Q 0) and z = Mx, and then taking a logarithmic transformation for the objective≻ function, we can reformulate the nonconvex optimization problem (MVE) as the following convex program [70]: minM,z un log det(M) − T s.t. (z Mai) (z Mai) 1,i =1, . , m, − − ≤ where the fact det(Q−1) = (det(M))−2 is used. 2.4 Multiplicative programming Consider the linear multiplicative programming [40]: T T (LMP) maxx∈C (d1 x + c1)(d2 x + c2) T s.t. d x + ci 0, i =1, 2, i ≥ where C is a convex set. (LMP) is NP-hard [51] to solve if replacing the max- imization with minimization. The objective function of (LMP) is nonconcave but quasiconcave [40]. The hidden convexity of (LMP) is viewed by the equiv- alent convex programming problem in the sense that both sharing the same optimal solution: T T maxx∈C log(d1 x + c1) + log(d2 x + c2) T s.t. d x + ci 0, i =1, 2. i ≥ Hidden convex optimization 5 Moreover, we notice that (LMP) has the following new second-order cone programming representation: maxx∈C z T s.t. d x + ci 0, i =1, 2, i ≥ s dT x + c , t dT x + c , ≤ 1 1 ≤ 2 2 s t − s + t. 2z ≤ 2.5 Geometric programming Geometric program was first introduced in the book [26]. It is an optimization problem of the form (GP) min f0(x) s.t. fi(x) 1, i =1, . , m, (2) ≤ gj(x)=1, j =1, . , p, (3) n aij where gj (j =1,...,p) are monomials, i.e. the form cj i=1 xi with cj > 0, and fi (i =0,...,m) are posynomials, i.e., the sum of several monomials. We also assume that the constraints (2)-(3) implicitly implyQ that all the variables are positive. Obviously, the general (GP) is a nonconvex optimization problem. One can verify that introducing logarithmic change of variables yi = log(xi) and log- arithmic transformations of fi(x) yields a convex programming reformulation of (GP): y min log f0(e ) (4) y s.t. log fi(e ) 0, i =1, . , m, (5) y ≤ log gj (e )=0, j =1, . , p, (6) y yi where (e )i = e . 2.6 Fractional programming Consider the linear fractional program cT x + α (LF) min f(x)= (7) dT x + β s.t.
Load more

Recommended publications
  • 4. Convex Optimization Problems
    Convex Optimization — Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form • convex optimization problems • quasiconvex optimization • linear optimization • quadratic optimization • geometric programming • generalized inequality constraints • semidefinite programming • vector optimization • 4–1 Optimization problem in standard form minimize f0(x) subject to f (x) 0, i =1,...,m i ≤ hi(x)=0, i =1,...,p x Rn is the optimization variable • ∈ f : Rn R is the objective or cost function • 0 → f : Rn R, i =1,...,m, are the inequality constraint functions • i → h : Rn R are the equality constraint functions • i → optimal value: p⋆ = inf f (x) f (x) 0, i =1,...,m, h (x)=0, i =1,...,p { 0 | i ≤ i } p⋆ = if problem is infeasible (no x satisfies the constraints) • ∞ p⋆ = if problem is unbounded below • −∞ Convex optimization problems 4–2 Optimal and locally optimal points x is feasible if x dom f and it satisfies the constraints ∈ 0 ⋆ a feasible x is optimal if f0(x)= p ; Xopt is the set of optimal points x is locally optimal if there is an R> 0 such that x is optimal for minimize (over z) f0(z) subject to fi(z) 0, i =1,...,m, hi(z)=0, i =1,...,p z x≤ R k − k2 ≤ examples (with n =1, m = p =0) f (x)=1/x, dom f = R : p⋆ =0, no optimal point • 0 0 ++ f (x)= log x, dom f = R : p⋆ = • 0 − 0 ++ −∞ f (x)= x log x, dom f = R : p⋆ = 1/e, x =1/e is optimal • 0 0 ++ − f (x)= x3 3x, p⋆ = , local optimum at x =1 • 0 − −∞ Convex optimization problems 4–3 Implicit constraints the standard form optimization problem has an implicit
    [Show full text]
  • Convex Optimisation Matlab Toolbox
    UNLOCBOX: USER’S GUIDE MATLAB CONVEX OPTIMIZATION TOOLBOX Lausanne - February 2014 PERRAUDIN Nathanaël, KALOFOLIAS Vassilis LTS2 - EPFL Abstract Nowadays the trend to solve optimization problems is to use specific algorithms rather than very gen- eral ones. The UNLocBoX provides a general framework allowing the user to design his own algorithms. To do so, the framework try to stay as close from the mathematical problem as possible. More precisely, the UNLocBoX is a Matlab toolbox designed to solve convex optimization problem of the form K min ∑ fn(x); x2C n=1 using proximal splitting techniques. It is mainly composed of solvers, proximal operators and demonstra- tion files allowing the user to quickly implement a problem. Contents 1 Introduction 1 2 Literature 2 3 Installation and initialization2 3.1 Dependencies.........................................2 3.2 GPU computation.......................................2 4 Structure of the UNLocBoX2 5 Problems of interest4 6 Solvers 4 6.1 Defining functions......................................5 6.2 Selecting a solver.......................................5 6.3 Optional parameters......................................6 6.4 Plug-ins: how to tune your algorithm.............................6 6.5 Returned arguments......................................8 7 Proximal operators9 7.1 Constraints..........................................9 8 Example 10 References 16 LTS2 - EPFL 1 Introduction During your research, you have most likely faced the situation where you need to solve a convex optimiza- tion problem. Maybe you were trying to find an optimal combination of parameters, you were transforming some process into something automatic or, even more simply, your algorithm was equivalent to solving convex problem. In this situation, you usually face a dilemma: you need a specific algorithm to solve your problem, but you do not want to spend hours writing it.
    [Show full text]
  • Conditional Gradient Sliding for Convex Optimization ∗
    CONDITIONAL GRADIENT SLIDING FOR CONVEX OPTIMIZATION ∗ GUANGHUI LAN y AND YI ZHOU z Abstract. In this paper, we present a new conditional gradient type method for convex optimization by calling a linear optimization (LO) oracle to minimize a series of linear functions over the feasible set. Different from the classic conditional gradient method, the conditional gradient sliding (CGS) algorithm developed herein can skip the computation of gradients from time to time, and as a result, can achieve the optimal complexity bounds in terms of not only the number of callsp to the LO oracle, but also the number of gradient evaluations. More specifically, we show that the CGS method requires O(1= ) and O(log(1/)) gradient evaluations, respectively, for solving smooth and strongly convex problems, while still maintaining the optimal O(1/) bound on the number of calls to LO oracle. We also develop variants of the CGS method which can achieve the optimal complexity bounds for solving stochastic optimization problems and an important class of saddle point optimization problems. To the best of our knowledge, this is the first time that these types of projection-free optimal first-order methods have been developed in the literature. Some preliminary numerical results have also been provided to demonstrate the advantages of the CGS method. Keywords: convex programming, complexity, conditional gradient method, Frank-Wolfe method, Nesterov's method AMS 2000 subject classification: 90C25, 90C06, 90C22, 49M37 1. Introduction. The conditional gradient (CndG) method, which was initially developed by Frank and Wolfe in 1956 [13] (see also [11, 12]), has been considered one of the earliest first-order methods for solving general convex programming (CP) problems.
    [Show full text]
  • Duality Gap Estimation Via a Refined Shapley--Folkman Lemma | SIAM
    SIAM J. OPTIM. \bigcircc 2020 Society for Industrial and Applied Mathematics Vol. 30, No. 2, pp. 1094{1118 DUALITY GAP ESTIMATION VIA A REFINED SHAPLEY{FOLKMAN LEMMA \ast YINGJIE BIy AND AO TANG z Abstract. Based on concepts like the kth convex hull and finer characterization of noncon- vexity of a function, we propose a refinement of the Shapley{Folkman lemma and derive anew estimate for the duality gap of nonconvex optimization problems with separable objective functions. We apply our result to the network utility maximization problem in networking and the dynamic spectrum management problem in communication as examples to demonstrate that the new bound can be qualitatively tighter than the existing ones. The idea is also applicable to cases with general nonconvex constraints. Key words. nonconvex optimization, duality gap, convex relaxation, network resource alloca- tion AMS subject classifications. 90C26, 90C46 DOI. 10.1137/18M1174805 1. Introduction. The Shapley{Folkman lemma (Theorem 1.1) was stated and used to establish the existence of approximate equilibria in economy with nonconvex preferences [13]. It roughly says that the sum of a large number of sets is close to convex and thus can be used to generalize results on convex objects to nonconvex ones. n m P Theorem 1.1. Let S1;S2;:::;Sn be subsets of R . For each z 2 conv i=1 Si = Pn conv S , there exist points zi 2 conv S such that z = Pn zi and zi 2 S except i=1 i i i=1 i for at most m values of i. Remark 1.2.
    [Show full text]
  • Subdifferentiability and the Duality
    Subdifferentiability and the Duality Gap Neil E. Gretsky ([email protected]) ∗ Department of Mathematics, University of California, Riverside Joseph M. Ostroy ([email protected]) y Department of Economics, University of California, Los Angeles William R. Zame ([email protected]) z Department of Economics, University of California, Los Angeles Abstract. We point out a connection between sensitivity analysis and the funda- mental theorem of linear programming by characterizing when a linear programming problem has no duality gap. The main result is that the value function is subd- ifferentiable at the primal constraint if and only if there exists an optimal dual solution and there is no duality gap. To illustrate the subtlety of the condition, we extend Kretschmer's gap example to construct (as the value function of a linear programming problem) a convex function which is subdifferentiable at a point but is not continuous there. We also apply the theorem to the continuum version of the assignment model. Keywords: duality gap, value function, subdifferentiability, assignment model AMS codes: 90C48,46N10 1. Introduction The purpose of this note is to point out a connection between sensi- tivity analysis and the fundamental theorem of linear programming. The subject has received considerable attention and the connection we find is remarkably simple. In fact, our observation in the context of convex programming follows as an application of conjugate duality [11, Theorem 16]. Nevertheless, it is useful to give a separate proof since the conclusion is more readily established and its import for linear programming is more clearly seen. The main result (Theorem 1) is that in a linear programming prob- lem there exists an optimal dual solution and there is no duality gap if and only if the value function is subdifferentiable at the primal constraint.
    [Show full text]
  • Arxiv:2011.09194V1 [Math.OC]
    Noname manuscript No. (will be inserted by the editor) Lagrangian duality for nonconvex optimization problems with abstract convex functions Ewa M. Bednarczuk · Monika Syga Received: date / Accepted: date Abstract We investigate Lagrangian duality for nonconvex optimization prob- lems. To this aim we use the Φ-convexity theory and minimax theorem for Φ-convex functions. We provide conditions for zero duality gap and strong duality. Among the classes of functions, to which our duality results can be applied, are prox-bounded functions, DC functions, weakly convex functions and paraconvex functions. Keywords Abstract convexity · Minimax theorem · Lagrangian duality · Nonconvex optimization · Zero duality gap · Weak duality · Strong duality · Prox-regular functions · Paraconvex and weakly convex functions 1 Introduction Lagrangian and conjugate dualities have far reaching consequences for solution methods and theory in convex optimization in finite and infinite dimensional spaces. For recent state-of the-art of the topic of convex conjugate duality we refer the reader to the monograph by Radu Bot¸[5]. There exist numerous attempts to construct pairs of dual problems in non- convex optimization e.g., for DC functions [19], [34], for composite functions [8], DC and composite functions [30], [31] and for prox-bounded functions [15]. In the present paper we investigate Lagrange duality for general optimiza- tion problems within the framework of abstract convexity, namely, within the theory of Φ-convexity. The class Φ-convex functions encompasses convex l.s.c. Ewa M. Bednarczuk Systems Research Institute, Polish Academy of Sciences, Newelska 6, 01–447 Warsaw Warsaw University of Technology, Faculty of Mathematics and Information Science, ul.
    [Show full text]
  • Lagrangian Duality and Perturbational Duality I ∗
    Lagrangian duality and perturbational duality I ∗ Erik J. Balder Our approach to the Karush-Kuhn-Tucker theorem in [OSC] was entirely based on subdifferential calculus (essentially, it was an outgrowth of the two subdifferential calculus rules contained in the Fenchel-Moreau and Dubovitskii-Milyutin theorems, i.e., Theorems 2.9 and 2.17 of [OSC]). On the other hand, Proposition B.4(v) in [OSC] gives an intimate connection between the subdifferential of a function and the Fenchel conjugate of that function. In the present set of lecture notes this connection forms the central analytical tool by which one can study the connections between an optimization problem and its so-called dual optimization problem (such connections are commonly known as duality relations). We shall first study duality for the convex optimization problem that figured in our Karush-Kuhn-Tucker results. In this simple form such duality is known as Lagrangian duality. Next, in section 2 this is followed by a far-reaching extension of duality to abstract optimization problems, which leads to duality-stability relationships. Then, in section 3 we specialize duality to optimization problems with cone-type constraints, which includes Fenchel duality for semidefinite programming problems. 1 Lagrangian duality An interesting and useful interpretation of the KKT theorem can be obtained in terms of the so-called duality principle (or relationships) for convex optimization. Recall our standard convex minimization problem as we had it in [OSC]: (P ) inf ff(x): g1(x) ≤ 0; ··· ; gm(x) ≤ 0; Ax − b = 0g x2S n and recall that we allow the functions f; g1; : : : ; gm on R to have values in (−∞; +1].
    [Show full text]
  • Additional Exercises for Convex Optimization
    Additional Exercises for Convex Optimization Stephen Boyd Lieven Vandenberghe January 4, 2020 This is a collection of additional exercises, meant to supplement those found in the book Convex Optimization, by Stephen Boyd and Lieven Vandenberghe. These exercises were used in several courses on convex optimization, EE364a (Stanford), EE236b (UCLA), or 6.975 (MIT), usually for homework, but sometimes as exam questions. Some of the exercises were originally written for the book, but were removed at some point. Many of them include a computational component using one of the software packages for convex optimization: CVX (Matlab), CVXPY (Python), or Convex.jl (Julia). We refer to these collectively as CVX*. (Some problems have not yet been updated for all three languages.) The files required for these exercises can be found at the book web site www.stanford.edu/~boyd/cvxbook/. You are free to use these exercises any way you like (for example in a course you teach), provided you acknowledge the source. In turn, we gratefully acknowledge the teaching assistants (and in some cases, students) who have helped us develop and debug these exercises. Pablo Parrilo helped develop some of the exercises that were originally used in MIT 6.975, Sanjay Lall developed some other problems when he taught EE364a, and the instructors of EE364a during summer quarters developed others. We’ll update this document as new exercises become available, so the exercise numbers and sections will occasionally change. We have categorized the exercises into sections that follow the book chapters, as well as various additional application areas. Some exercises fit into more than one section, or don’t fit well into any section, so we have just arbitrarily assigned these.
    [Show full text]
  • Efficient Convex Quadratic Optimization Solver for Embedded MPC Applications
    DEGREE PROJECT IN ELECTRICAL ENGINEERING, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2018 Efficient Convex Quadratic Optimization Solver for Embedded MPC Applications ALBERTO DÍAZ DORADO KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE Efficient Convex Quadratic Optimization Solver for Embedded MPC Applications ALBERTO DÍAZ DORADO Master in Electrical Engineering Date: October 2, 2018 Supervisor: Arda Aytekin , Martin Biel Examiner: Mikael Johansson Swedish title: Effektiv Konvex Kvadratisk Optimeringslösare för Inbäddade MPC-applikationer School of Electrical Engineering and Computer Science iii Abstract Model predictive control (MPC) is an advanced control technique that requires solving an optimization problem at each sampling instant. Sev- eral emerging applications require the use of short sampling times to cope with the fast dynamics of the underlying process. In many cases, these applications also need to be implemented on embedded hard- ware with limited resources. As a result, the use of model predictive controllers in these application domains remains challenging. This work deals with the implementation of an interior point algo- rithm for use in embedded MPC applications. We propose a modular software design that allows for high solver customization, while still producing compact and fast code. Our interior point method includes an efficient implementation of a novel approach to constraint softening, which has only been tested in high-level languages before. We show that a well conceived low-level implementation of integrated constraint softening adds no significant overhead to the solution time, and hence, constitutes an attractive alternative in embedded MPC solvers. iv Sammanfattning Modell prediktiv reglering (MPC) är en avancerad regler-teknik som involverar att lösa ett optimeringsproblem vid varje sampeltillfälle.
    [Show full text]
  • 1 Linear Programming
    ORF 523 Lecture 9 Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Any typos should be emailed to a a [email protected]. In this lecture, we see some of the most well-known classes of convex optimization problems and some of their applications. These include: • Linear Programming (LP) • (Convex) Quadratic Programming (QP) • (Convex) Quadratically Constrained Quadratic Programming (QCQP) • Second Order Cone Programming (SOCP) • Semidefinite Programming (SDP) 1 Linear Programming Definition 1. A linear program (LP) is the problem of optimizing a linear function over a polyhedron: min cT x T s.t. ai x ≤ bi; i = 1; : : : ; m; or written more compactly as min cT x s.t. Ax ≤ b; for some A 2 Rm×n; b 2 Rm: We'll be very brief on our discussion of LPs since this is the central topic of ORF 522. It suffices to say that LPs probably still take the top spot in terms of ubiquity of applications. Here are a few examples: • A variety of problems in production planning and scheduling 1 • Exact formulation of several important combinatorial optimization problems (e.g., min-cut, shortest path, bipartite matching) • Relaxations for all 0/1 combinatorial programs • Subroutines of branch-and-bound algorithms for integer programming • Relaxations for cardinality constrained (compressed sensing type) optimization prob- lems • Computing Nash equilibria in zero-sum games • ::: 2 Quadratic Programming Definition 2. A quadratic program (QP) is an optimization problem with a quadratic ob- jective and linear constraints min xT Qx + qT x + c x s.t. Ax ≤ b: Here, we have Q 2 Sn×n, q 2 Rn; c 2 R;A 2 Rm×n; b 2 Rm: The difficulty of this problem changes drastically depending on whether Q is positive semidef- inite (psd) or not.
    [Show full text]
  • Bounding the Duality Gap for Problems with Separable Objective
    Bounding the Duality Gap for Problems with Separable Objective Madeleine Udell and Stephen Boyd March 8, 2014 Abstract We consider the problem of minimizing a sum of non-convex func- tions over a compact domain, subject to linear inequality and equality constraints. We consider approximate solutions obtained by solving a convexified problem, in which each function in the objective is replaced by its convex envelope. We propose a randomized algorithm to solve the convexified problem which finds an -suboptimal solution to the original problem. With probability 1, is bounded by a term propor- tional to the number of constraints in the problem. The bound does not depend on the number of variables in the problem or the number of terms in the objective. In contrast to previous related work, our proof is constructive, self-contained, and gives a bound that is tight. 1 Problem and results The problem. We consider the optimization problem Pn minimize f(x) = i=1 fi(xi) subject to Ax ≤ b (P) Gx = h; N ni Pn with variable x = (x1; : : : ; xn) 2 R , where xi 2 R , with i=1 ni = N. m1×N There are m1 linear inequality constraints, so A 2 R , and m2 linear equality constraints, so G 2 Rm2×N . The optimal value of P is denoted p?. The objective function terms are lower semi-continuous on their domains: 1 ni fi : Si ! R, where Si ⊂ R is a compact set. We say that a point x is feasible (for P) if Ax ≤ b, Gx = h, and xi 2 Si, i = 1; : : : ; n.
    [Show full text]
  • Linear Programming and Convex Optimization
    Lecture 2 : Linear programming and convex optimization Rajat Mittal ? IIT Kanpur We talked about optimization problems and why they are important. We also looked at one of the class of problems called least square problems. Next we look at another class, 1 Linear Programming 1.1 Definition Linear programming is one of the well studied classes of optimization problem. We already discussed that a linear program is one which has linear objective and constraint functions. This implies that a standard linear program looks like P T min i cixi = c x T subject to ai xi ≤ bi 8i 2 f1; ··· ; mg n Here the vectors c; a1; ··· ; am 2 R and scalars bi 2 R are the problem parameters. 1.2 Examples { Max flow: Given a graph, start(s) and end node (t), capacities on every edge; find out the maximum flow possible through edges. s t Fig. 1. Max flow problem: there will be capacities for every edge in the problem statement The linear program looks like: P max fs;ug f(s; u) P P s.t. fu;vg f(u; v) = fv;ug f(v; u) 8v 6= s; t; ∼ 0 ≤ f(u; v) ≤ c(u; v) Note: There is another one which can be made using the flow through paths. ? Thanks to books from Boyd and Vandenberghe, Dantzig and Thapa, Papadimitriou and Steiglitz { Another example: This time, I will give the linear program and you will tell me what real world situation models it :). max 2x1 + 4x2 s.t. x1 + x2 ≤ 10; x1 ≤ 4 1.3 Solving linear programs You might have already had a course on linear optimization.
    [Show full text]