DOCSLIB.ORG

A Short Course on Semidefinite Programming

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Short Course on Semidefinite Programming A Short Course on Semidefinite Programming (in order of appearance) Henry Wolkowicz, Etienne de Klerk, Levent Tunc¸el, Franz Rendl [email protected] Department of Combinatorics and Optimization University of Waterloo A Short Course on Semidefinite Programming – p. 1 About these Notes: Semidefinite Programming, SDP, refers to optimization problems where the vector variable is a symmetric matrix which is required to be positive semidefinite. Though SDPs (under various names) have been studied as far back as the 1940s, the interest has grown tremendously during the last ten years. This is partly due to the many diverse applications in e.g. engineering, combinatorial optimization, and statistics. Part of the interest is due to the great advances in efficient solutions for these types of problems. These notes summarize the theory, algorithms, and applications for semidefinite programming. They were prepared for a minicourse given at the Workshop on Large Scale Nonlinear and Semidefinite Programming, held in memory of Jos Sturm, May 12, 2004, at the University of Waterloo. A Short Course on Semidefinite Programming – p. 2 Preface The purpose of these notes is three fold: first, they provide a comprehensive treatment of the area of Semidefinite Programming, a new and exciting area in Optimization; second, the notes illustrate the strength of convex analysis in Optimization; third, they emphasize the interaction between theory and algorithms and solutions of practical problems. A Short Course on Semidefinite Programming – p. 3 1 Introduction and Motivation 1.1 Outline • Basic Properties and Notation • Examples • Historical Notes A Short Course on Semidefinite Programming – p. 4 1.2 Basic Properties and Notation Basic linear Semidefinite Programming looks just like Linear Programming p∗ = max trace CX (hC,Xi) (PSDP) s.t. A(X) ∼= AX = b (linear) X 0, (X ∈P) (nonneg) C,X ∈ Sn Sn := space of n × n symmetric matrices A Short Course on Semidefinite Programming – p. 5 Space of Symmetric Matrices A ∈ Sn := space of n × n symmetric matrices A is positive semidefinite (positive definite) (A 0 (A ≻ 0)) if xT Ax ≥ 0(> 0), ∀x =6 0. denotes the Löwner partial order, [53] A B if B − A 0 (positive semidefinite) TFAE: 1. A 0 (A ≻ 0) 2. the eigenvalues λ(A) ≥ 0 (λ(A) > 0) A Short Course on Semidefinite Programming – p. 6 Linear transformation A; Adjoint: A : Sn → Rm, A∗ : Rm → Sn n A defined by associated set {Ai ∈ S , i = 1,...m} : ∗ m (AX)i = trace(AiX); A y = i=1 yiAi, where P hAX,yi = hX, A∗yi , ∀X ∈ Sn, ∀y ∈ Rm n P = S+ - cone of positive semidefinite matrices replaces n R+ - nonnegative orthant A Short Course on Semidefinite Programming – p. 7 SDP or LMI n n trace CX = hC,Xi = i=1 j=1 CijXij SDP is equivalent to (LinearP P Matrix Inequalities): ∗ n n p = max i=1 j=1 CijXij (PSDP) s.t. hA ,Xi = b , i = 1,...m P iP i X 0, (X ∈P) A Short Course on Semidefinite Programming – p. 8 1.2.1 Duality payoff function; player Y to player X (Lagrangian) L(X,y) := trace(CX) + yT (b −AX) (= hC,Xi + hy, b −AXi) Optimal (worst case) strategy for player X: p∗ = max min L(X,y) X0 y For each fixed X 0: y free yields hidden constraint b −AX = 0; recovers primal problem (PSDP). A Short Course on Semidefinite Programming – p. 9 Rewrite Lagrangian/payoff L(X,y) = trace(CX) + yT (b −AX) = bT y + trace (C −A∗y) X ∗ using adjoint operator, A y = i yiAi hA∗y,Xi = hy,PAXi , ∀X,y Then: p∗ = max min L(X,y) ≤ d∗ := min max L(X,y) X0 y y X0 For dual: for each fixed y, that X 0 yields hidden constraint g(y) := C −A∗y 0 A Short Course on Semidefinite Programming – p. 10 Hidden Constraint p∗ = max min L(X,y) ≤ d∗ := min max L(X,y) X0 y y X0 dual obtained from optimal strategy of competing player Y; use hidden constraint g(y) = C −A∗y 0 d∗ = min bT y (DSDP) s.t. A∗y C for the primal p∗ = max trace CX (PSDP) s.t. AX = b X 0 A Short Course on Semidefinite Programming – p. 11 1.2.2 Weak Duality - Optimality Proposition 1.2.2.1 (Weak Duality) If X feas. in (PSDP), y feas. in (DSDP), Z = A∗y − C 0 is slack variable, then trace CX − bT y = −trace XZ ≤ 0. Proof. (Direct - using: trace ZX = trace X1/2X1/2Z = trace X1/2Z1/2Z1/2X1/2 = kX1/2Z1/2k2 ≥ 0; Note: XZ = 0 iff trace XZ = 0) trace CX − bT y = trace(A∗y − Z)X − bT y = trace yT AX − trace ZX − bT y = yT (AX − b) − trace ZX = −trace ZX, A Short Course on Semidefinite Programming – p. 12 Characterization of optimality for the dual pair X 0,y (and slack Z 0) A∗y − Z = C dual feasibility AX = b primal feasibility ZX = 0 complementary slackness And ZX = µI perturbed Forms the basis for: interior point methods (central path Xµ,yµ, Zµ); (primal simplex method, dual simplex method) A Short Course on Semidefinite Programming – p. 13 Strong Duality primal value = hC,Xi = hA∗y − Z,Xi dual feasibility = hy, AXi − hZ,Xi adjoint = hy, bi − hZ,Xi primal feasibility = hy, bi complementary slackness = dual value A Short Course on Semidefinite Programming – p. 14 1.2.3 Preliminary Examples Example 1.2.3.1 Minimizing the Maximum Eigenvalue Arises in e.g. stability of differential equations • The mathematical problem: ⋄ given A(x) ∈ Sn depending linearly on vector x ∈ Rm ⋄ Find x to minimize the maximum eigenvalue of A(x) • SDP Model: ⋄ the largest eigenvalue λmax(A(x)) ≤ α iff λmax(A(x) − αI) ≤ 0 iff A(x) − αI 0. ⋄ The DSDP (in dual form) is: max −α s.t. A(x) − αI 0. A Short Course on Semidefinite Programming – p. 15 Pseudoconvex Optimization Example 1.2.3.2 Pseudoconvex (Nonlinear) Optimization T 2 d∗ = min (c x) (PCP) dT x s.t. Ax + b ≥ 0, (given Ax + b ≥ 0 ⇒ dT x > 0) Then (using 2 × 2 determinant), (PCP) is equivalent to d∗ = min t Diag (Ax + b) 0 0 T s.t. 0 t c x 0 0 cT x dT x A Short Course on Semidefinite Programming – p. 16 1.2.4 Further Properties, Definitions • For matrices P, Q compatible for multiplication, trace P Q = trace QP. • Every symmetric matrix S has an orthogonal decomposition S = QΛQT , where Λ is diagonal with the eigenvalues of S on the diagonal, and Q has orthonormal columns consisting of eigenvectors of S. • X 0 ⇒ P T XP 0 (congruence) A Short Course on Semidefinite Programming – p. 17 • For S symmetric, TFAE: 1. S 0 (≻ 0) 2. vT Sv ≥ 0, ∀v ∈ Rn (vT Sv > 0, ∀0 =6 v ∈ Rn) 3. λ(S) ≥ 0, nonneg. eigs (> 0, pos. eigs) 4. S = P T P for some P (for some nonsingular P ) 1 2 1 5. S =(S 2 ) , for some S 2 0 (≻ 0) A Short Course on Semidefinite Programming – p. 18 • SDP - Linear Matrix Inequality g(y) = C −A∗y 0 ∗ T adjoint operator A y = i yiAi, Ai = Ai • g(y) is a P-convex constraint, i.e. P g(λx+(1−λ)y) λg(x)+(1−λ)g(y), ∀0 ≤ λ ≤ 1, ∀x,y ∈ Rm equivalently, for every K 0, the real valued function fK(x) := trace Kg(x) is a convex function, i.e. m fK(λx+(1−λ)y) ≤ λfK(x)+(1−λ)fK(y), ∀0 ≤ λ ≤ 1, ∀x,y ∈ R A Short Course on Semidefinite Programming – p. 19 Geometry - Feasible Set • feasible set {x ∈ Rm : g(x) 0} is convex set • optimum point is on the boundary (g(x) is singular) • boundary of feasible set is not smooth (piecewise smooth, algebraic surface) A Short Course on Semidefinite Programming – p. 20 1.2.5 Historical Notes • arguably most active area in optimization (see HANDBOOK OF SEMIDEFINITE PROGRAMMING: Theory, Algorithms, and Applications, 2000, [94], for comprehensive results, history, references, ... and books [15, 95, 96]) • Lyapunov over 100 years ago on stability analysis of differential equations • Bohnenblust 1948 on geometry of the cone of SDPs, [10] • Yakubovitch in the 1960’s and Boyd and others on convex optimization in control in the 1980’s, e.g. solving Ricatti Equations (called LMIs), e.g. [13],[92, 91] A Short Course on Semidefinite Programming – p. 21 More: Historical Notes • matrix completion problems (another name for SDP) started early 1980’s, continues to be a very active area of research, [19],[28], and e.g.: [47, 46, 45, 44, 48, 40, 32, 20, 41]. (More recently,it is being used to solve large scale SDPs.) • combinatorial optimization applications 1980’s: Lovász theta function [52]; the strong approximation results for the max-cut problem by Goemans-Williamson, e.g. [27], survey papers: [25, 26],[78]. • linear complementarity problems can be extended to problems over the cone of semidefinite matrices, e.g. [22, 39, 43, 42, 59]. A Short Course on Semidefinite Programming – p. 22 More: Historical Notes • Complexity, Distance to Ill-Posedness, and Condition Numbers SDP is a convex program and it can be solved to any desired accuracy in polynomial time, see seminal work of Nesterov and Nemirovski e.g. [63, 64, 68, 66, 62, 65, 67]. Another measure of complexity is the distance to ill-posedness: e.g. work by Renegar [80, 84, 83, 82, 81]. • Cone Programming this is a generalization of SDP, also called generalized linear programming, in paper by Bellman and Fan 1963, [8]. Other books deal with problems over cones date back to 60s, e.g.
Load more

Recommended publications
  • An Algorithm Based on Semidefinite Programming for Finding Minimax
    Takustr. 7 Zuse Institute Berlin 14195 Berlin Germany BELMIRO P.M. DUARTE,GUILLAUME SAGNOL,WENG KEE WONG An algorithm based on Semidefinite Programming for finding minimax optimal designs ZIB Report 18-01 (December 2017) Zuse Institute Berlin Takustr. 7 14195 Berlin Germany Telephone: +49 30-84185-0 Telefax: +49 30-84185-125 E-mail: [email protected] URL: http://www.zib.de ZIB-Report (Print) ISSN 1438-0064 ZIB-Report (Internet) ISSN 2192-7782 An algorithm based on Semidefinite Programming for finding minimax optimal designs Belmiro P.M. Duarte a,b, Guillaume Sagnolc, Weng Kee Wongd aPolytechnic Institute of Coimbra, ISEC, Department of Chemical and Biological Engineering, Portugal. bCIEPQPF, Department of Chemical Engineering, University of Coimbra, Portugal. cTechnische Universität Berlin, Institut für Mathematik, Germany. dDepartment of Biostatistics, Fielding School of Public Health, UCLA, U.S.A. Abstract An algorithm based on a delayed constraint generation method for solving semi- infinite programs for constructing minimax optimal designs for nonlinear models is proposed. The outer optimization level of the minimax optimization problem is solved using a semidefinite programming based approach that requires the de- sign space be discretized. A nonlinear programming solver is then used to solve the inner program to determine the combination of the parameters that yields the worst-case value of the design criterion. The proposed algorithm is applied to find minimax optimal designs for the logistic model, the flexible 4-parameter Hill homoscedastic model and the general nth order consecutive reaction model, and shows that it (i) produces designs that compare well with minimax D−optimal de- signs obtained from semi-infinite programming method in the literature; (ii) can be applied to semidefinite representable optimality criteria, that include the com- mon A−; E−; G−; I− and D-optimality criteria; (iii) can tackle design problems with arbitrary linear constraints on the weights; and (iv) is fast and relatively easy to use.
    [Show full text]
  • A Lifted Linear Programming Branch-And-Bound Algorithm for Mixed Integer Conic Quadratic Programs Juan Pablo Vielma, Shabbir Ahmed, George L
    A Lifted Linear Programming Branch-and-Bound Algorithm for Mixed Integer Conic Quadratic Programs Juan Pablo Vielma, Shabbir Ahmed, George L. Nemhauser, H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, 765 Ferst Drive NW, Atlanta, GA 30332-0205, USA, {[email protected], [email protected], [email protected]} This paper develops a linear programming based branch-and-bound algorithm for mixed in- teger conic quadratic programs. The algorithm is based on a higher dimensional or lifted polyhedral relaxation of conic quadratic constraints introduced by Ben-Tal and Nemirovski. The algorithm is different from other linear programming based branch-and-bound algo- rithms for mixed integer nonlinear programs in that, it is not based on cuts from gradient inequalities and it sometimes branches on integer feasible solutions. The algorithm is tested on a series of portfolio optimization problems. It is shown that it significantly outperforms commercial and open source solvers based on both linear and nonlinear relaxations. Key words: nonlinear integer programming; branch and bound; portfolio optimization History: February 2007. 1. Introduction This paper deals with the development of an algorithm for the class of mixed integer non- linear programming (MINLP) problems known as mixed integer conic quadratic program- ming problems. This class of problems arises from adding integrality requirements to conic quadratic programming problems (Lobo et al., 1998), and is used to model several applica- tions from engineering and finance. Conic quadratic programming problems are also known as second order cone programming problems, and together with semidefinite and linear pro- gramming (LP) problems are special cases of the more general conic programming problems (Ben-Tal and Nemirovski, 2001a).
    [Show full text]
  • Linear Programming
    Lecture 18 Linear Programming 18.1 Overview In this lecture we describe a very general problem called linear programming that can be used to express a wide variety of different kinds of problems. We can use algorithms for linear program- ming to solve the max-flow problem, solve the min-cost max-flow problem, find minimax-optimal strategies in games, and many other things. We will primarily discuss the setting and how to code up various problems as linear programs At the end, we will briefly describe some of the algorithms for solving linear programming problems. Specific topics include: • The definition of linear programming and simple examples. • Using linear programming to solve max flow and min-cost max flow. • Using linear programming to solve for minimax-optimal strategies in games. • Algorithms for linear programming. 18.2 Introduction In the last two lectures we looked at: — Bipartite matching: given a bipartite graph, find the largest set of edges with no endpoints in common. — Network flow (more general than bipartite matching). — Min-Cost Max-flow (even more general than plain max flow). Today, we’ll look at something even more general that we can solve algorithmically: linear pro- gramming. (Except we won’t necessarily be able to get integer solutions, even when the specifi- cation of the problem is integral). Linear Programming is important because it is so expressive: many, many problems can be coded up as linear programs (LPs). This especially includes problems of allocating resources and business 95 18.3. DEFINITION OF LINEAR PROGRAMMING 96 supply-chain applications. In business schools and Operations Research departments there are entire courses devoted to linear programming.
    [Show full text]
  • Integer Linear Programs
    20 ________________________________________________________________________________________________ Integer Linear Programs Many linear programming problems require certain variables to have whole number, or integer, values. Such a requirement arises naturally when the variables represent enti- ties like packages or people that can not be fractionally divided — at least, not in a mean- ingful way for the situation being modeled. Integer variables also play a role in formulat- ing equation systems that model logical conditions, as we will show later in this chapter. In some situations, the optimization techniques described in previous chapters are suf- ficient to find an integer solution. An integer optimal solution is guaranteed for certain network linear programs, as explained in Section 15.5. Even where there is no guarantee, a linear programming solver may happen to find an integer optimal solution for the par- ticular instances of a model in which you are interested. This happened in the solution of the multicommodity transportation model (Figure 4-1) for the particular data that we specified (Figure 4-2). Even if you do not obtain an integer solution from the solver, chances are good that you’ll get a solution in which most of the variables lie at integer values. Specifically, many solvers are able to return an ‘‘extreme’’ solution in which the number of variables not lying at their bounds is at most the number of constraints. If the bounds are integral, all of the variables at their bounds will have integer values; and if the rest of the data is integral, many of the remaining variables may turn out to be integers, too.
    [Show full text]
  • A Framework for Solving Mixed-Integer Semidefinite Programs
    A FRAMEWORK FOR SOLVING MIXED-INTEGER SEMIDEFINITE PROGRAMS TRISTAN GALLY, MARC E. PFETSCH, AND STEFAN ULBRICH ABSTRACT. Mixed-integer semidefinite programs arise in many applications and several problem-specific solution approaches have been studied recently. In this paper, we investigate a generic branch-and-bound framework for solving such problems. We first show that strict duality of the semidefinite relaxations is inherited to the subproblems. Then solver components like dual fixing, branch- ing rules, and primal heuristics are presented. We show the applicability of an implementation of the proposed methods on three kinds of problems. The re- sults show the positive computational impact of the different solver components, depending on the semidefinite programming solver used. This demonstrates that practically relevant mixed-integer semidefinite programs can successfully be solved using a general purpose solver. 1. INTRODUCTION The solution of mixed-integer semidefinite programs (MISDPs), i.e., semidefinite programs (SDPs) with integrality constraints on some of the variables, received increasing attention in recent years. There are two main application areas of such problems: In the first area, they arise because they capture an essential structure of the problem. For instance, SDPs are important for expressing ellipsoidal uncer- tainty sets in robust optimization, see, e.g., Ben-Tal et al. [10]. In particular, SDPs appear in the context of optimizing robust truss structures, see, e.g., Ben-Tal and Nemirovski [11] as well as Bendsøe and Sigmund [12]. Allowing to change the topological structure then yields MISDPs; Section 9.1 will provide more details. Another application arises in the design of downlink beamforming, see, e.g., [56].
    [Show full text]
  • Abstracts Exact Separation of K-Projection Polytope Constraints Miguel F
    13th EUROPT Workshop on Advances in Continuous Optimization 8-10 July [email protected] http://www.maths.ed.ac.uk/hall/EUROPT15 Abstracts Exact Separation of k-Projection Polytope Constraints Miguel F. Anjos Cutting planes are often used as an efficient means to tighten continuous relaxations of mixed-integer optimization problems and are a vital component of branch-and-cut algorithms. A critical step of any cutting plane algorithm is to find valid inequalities, or cuts, that improve the current relaxation of the integer-constrained problem. The maximally violated valid inequality problem aims to find the most violated inequality from a given family of cuts. $k$-projection polytope constraints are a family of cuts that are based on an inner description of the cut polytope of size $k$ and are applied to $k\times k$ principal minors of a semidefinite optimization relaxation. We propose a bilevel second order cone optimization approach to solve the maximally violated $k$ projection polytope constraint problem. We reformulate the bilevel problem as a single-level mixed binary second order cone optimization problem that can be solved using off-the-shelf conic optimization software. Additionally we present two methods for improving the computational performance of our approach, namely lexicographical ordering and a reformulation with fewer binary variables. All of the proposed formulations are exact. Preliminary results show the computational time and number of branch-and-bound nodes required to solve small instances in the context of the max-cut problem. Global convergence of the Douglas{Rachford method for some nonconvex feasibility problems Francisco J.
    [Show full text]
  • A Survey of Numerical Methods for Nonlinear Semidefinite Programming
    Journal of the Operations Research Society of Japan ⃝c The Operations Research Society of Japan Vol. 58, No. 1, January 2015, pp. 24–60 A SURVEY OF NUMERICAL METHODS FOR NONLINEAR SEMIDEFINITE PROGRAMMING Hiroshi Yamashita Hiroshi Yabe NTT DATA Mathematical Systems Inc. Tokyo University of Science (Received September 16, 2014; Revised December 22, 2014) Abstract Nonlinear semidefinite programming (SDP) problems have received a lot of attentions because of large variety of applications. In this paper, we survey numerical methods for solving nonlinear SDP problems. Three kinds of typical numerical methods are described; augmented Lagrangian methods, sequential SDP methods and primal-dual interior point methods. We describe their typical algorithmic forms and discuss their global and local convergence properties which include rate of convergence. Keywords: Optimization, nonlinear semidefinite programming, augmented Lagrangian method, sequential SDP method, primal-dual interior point method 1. Introduction This paper is concerned with the nonlinear SDP problem: minimize f(x); x 2 Rn; (1.1) subject to g(x) = 0;X(x) º 0 where the functions f : Rn ! R, g : Rn ! Rm and X : Rn ! Sp are sufficiently smooth, p p and S denotes the set of pth-order real symmetric matrices. We also define S+ to denote the set of pth-order symmetric positive semidefinite matrices. By X(x) º 0 and X(x) Â 0, we mean that the matrix X(x) is positive semidefinite and positive definite, respectively. When f is a convex function, X is a concave function and g is affine, this problem is a convex optimization problem. Here the concavity of X(x) means that X(¸u + (1 ¡ ¸)v) ¡ ¸X(u) ¡ (1 ¡ ¸)X(v) º 0 holds for any u; v 2 Rn and any ¸ satisfying 0 · ¸ · 1.
    [Show full text]
  • Chapter 11: Basic Linear Programming Concepts
    CHAPTER 11: BASIC LINEAR PROGRAMMING CONCEPTS Linear programming is a mathematical technique for finding optimal solutions to problems that can be expressed using linear equations and inequalities. If a real-world problem can be represented accurately by the mathematical equations of a linear program, the method will find the best solution to the problem. Of course, few complex real-world problems can be expressed perfectly in terms of a set of linear functions. Nevertheless, linear programs can provide reasonably realistic representations of many real-world problems — especially if a little creativity is applied in the mathematical formulation of the problem. The subject of modeling was briefly discussed in the context of regulation. The regulation problems you learned to solve were very simple mathematical representations of reality. This chapter continues this trek down the modeling path. As we progress, the models will become more mathematical — and more complex. The real world is always more complex than a model. Thus, as we try to represent the real world more accurately, the models we build will inevitably become more complex. You should remember the maxim discussed earlier that a model should only be as complex as is necessary in order to represent the real world problem reasonably well. Therefore, the added complexity introduced by using linear programming should be accompanied by some significant gains in our ability to represent the problem and, hence, in the quality of the solutions that can be obtained. You should ask yourself, as you learn more about linear programming, what the benefits of the technique are and whether they outweigh the additional costs.
    [Show full text]
  • Introduction to Semidefinite Programming (SDP)
    Introduction to Semidefinite Programming (SDP) Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. 1 1 Introduction Semidefinite programming (SDP ) is the most exciting development in math- ematical programming in the 1990’s. SDP has applications in such diverse fields as traditional convex constrained optimization, control theory, and combinatorial optimization. Because SDP is solvable via interior point methods, most of these applications can usually be solved very efficiently in practice as well as in theory. 2 Review of Linear Programming Consider the linear programming problem in standard form: LP : minimize c · x s.t. ai · x = bi,i=1 ,...,m n x ∈+. x n c · x Here is a vector of variables, and we write “ ” for the inner-product n “ j=1 cj xj ”, etc. n n n Also, + := {x ∈ | x ≥ 0},and we call + the nonnegative orthant. n In fact, + is a closed convex cone, where K is called a closed a convex cone if K satisfies the following two conditions: • If x, w ∈ K, then αx + βw ∈ K for all nonnegative scalars α and β. • K is a closed set. 2 In words, LP is the following problem: “Minimize the linear function c·x, subject to the condition that x must solve m given equations ai · x = bi,i =1,...,m, and that x must lie in the closed n convex cone K = +.” We will write the standard linear programming dual problem as: m LD : maximize yibi i=1 m s.t. yiai + s = c i=1 n s ∈+. Given a feasible solution x of LP and a feasible solution (y, s)ofLD , the m m duality gap is simply c·x− i=1 yibi =(c−i=1 yiai) ·x = s·x ≥ 0, because x ≥ 0ands ≥ 0.
    [Show full text]
  • First-Order Methods for Semidefinite Programming
    FIRST-ORDER METHODS FOR SEMIDEFINITE PROGRAMMING Zaiwen Wen Advisor: Professor Donald Goldfarb Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY 2009 c 2009 Zaiwen Wen All Rights Reserved ABSTRACT First-order Methods For Semidefinite Programming Zaiwen Wen Semidefinite programming (SDP) problems are concerned with minimizing a linear function of a symmetric positive definite matrix subject to linear equality constraints. These convex problems are solvable in polynomial time by interior point methods. However, if the number of constraints m in an SDP is of order O(n2) when the unknown positive semidefinite matrix is n n, interior point methods become impractical both in terms of the × time (O(n6)) and the amount of memory (O(m2)) required at each iteration to form the m × m positive definite Schur complement matrix M and compute the search direction by finding the Cholesky factorization of M. This significantly limits the application of interior-point methods. In comparison, the computational cost of each iteration of first-order optimization approaches is much cheaper, particularly, if any sparsity in the SDP constraints or other special structure is exploited. This dissertation is devoted to the development, analysis and evaluation of two first-order approaches that are able to solve large SDP problems which have been challenging for interior point methods. In chapter 2, we present a row-by-row (RBR) method based on solving a sequence of problems obtained by restricting the n-dimensional positive semidefinite constraint on the matrix X. By fixing any (n 1)-dimensional principal submatrix of X and using its − (generalized) Schur complement, the positive semidefinite constraint is reduced to a sim- ple second-order cone constraint.
    [Show full text]
  • Chapter 7. Linear Programming and Reductions
    Chapter 7 Linear programming and reductions Many of the problems for which we want algorithms are optimization tasks: the shortest path, the cheapest spanning tree, the longest increasing subsequence, and so on. In such cases, we seek a solution that (1) satisfies certain constraints (for instance, the path must use edges of the graph and lead from s to t, the tree must touch all nodes, the subsequence must be increasing); and (2) is the best possible, with respect to some well-defined criterion, among all solutions that satisfy these constraints. Linear programming describes a broad class of optimization tasks in which both the con- straints and the optimization criterion are linear functions. It turns out an enormous number of problems can be expressed in this way. Given the vastness of its topic, this chapter is divided into several parts, which can be read separately subject to the following dependencies. Flows and matchings Introduction to linear programming Duality Games and reductions Simplex 7.1 An introduction to linear programming In a linear programming problem we are given a set of variables, and we want to assign real values to them so as to (1) satisfy a set of linear equations and/or linear inequalities involving these variables and (2) maximize or minimize a given linear objective function. 201 202 Algorithms Figure 7.1 (a) The feasible region for a linear program. (b) Contour lines of the objective function: x1 + 6x2 = c for different values of the profit c. x x (a) 2 (b) 2 400 400 Optimum point Profit = $1900 ¡ ¡ ¡ ¡ ¡ 300 ¢¡¢¡¢¡¢¡¢ 300 ¡ ¡ ¡ ¡ ¡ ¢¡¢¡¢¡¢¡¢ ¡ ¡ ¡ ¡ ¡ ¢¡¢¡¢¡¢¡¢ 200 200 c = 1500 ¡ ¡ ¡ ¡ ¡ ¢¡¢¡¢¡¢¡¢ ¡ ¡ ¡ ¡ ¡ ¢¡¢¡¢¡¢¡¢ c = 1200 ¡ ¡ ¡ ¡ ¡ 100 ¢¡¢¡¢¡¢¡¢ 100 ¡ ¡ ¡ ¡ ¡ ¢¡¢¡¢¡¢¡¢ c = 600 ¡ ¡ ¡ ¡ ¡ ¢¡¢¡¢¡¢¡¢ x1 x1 0 100 200 300 400 0 100 200 300 400 7.1.1 Example: profit maximization A boutique chocolatier has two products: its flagship assortment of triangular chocolates, called Pyramide, and the more decadent and deluxe Pyramide Nuit.
    [Show full text]
  • Linear Programming
    Stanford University | CS261: Optimization Handout 5 Luca Trevisan January 18, 2011 Lecture 5 In which we introduce linear programming. 1 Linear Programming A linear program is an optimization problem in which we have a collection of variables, which can take real values, and we want to find an assignment of values to the variables that satisfies a given collection of linear inequalities and that maximizes or minimizes a given linear function. (The term programming in linear programming, is not used as in computer program- ming, but as in, e.g., tv programming, to mean planning.) For example, the following is a linear program. maximize x1 + x2 subject to x + 2x ≤ 1 1 2 (1) 2x1 + x2 ≤ 1 x1 ≥ 0 x2 ≥ 0 The linear function that we want to optimize (x1 + x2 in the above example) is called the objective function.A feasible solution is an assignment of values to the variables that satisfies the inequalities. The value that the objective function gives 1 to an assignment is called the cost of the assignment. For example, x1 := 3 and 1 2 x2 := 3 is a feasible solution, of cost 3 . Note that if x1; x2 are values that satisfy the inequalities, then, by summing the first two inequalities, we see that 3x1 + 3x2 ≤ 2 that is, 1 2 x + x ≤ 1 2 3 2 1 1 and so no feasible solution has cost higher than 3 , so the solution x1 := 3 , x2 := 3 is optimal. As we will see in the next lecture, this trick of summing inequalities to verify the optimality of a solution is part of the very general theory of duality of linear programming.
    [Show full text]