Inner approximation of convex cones via primal-dual ellipsoidal norms by Miaolan Xie A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Master of Mathematics in Combinatorics and Optimization Waterloo, Ontario, Canada, 2016 c Miaolan Xie 2016 Author's Declaration I hereby declare that I am the sole author of this thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I understand that my thesis may be made electronically available to the public. ii Abstract We study ellipsoids from the point of view of approximating convex sets. Our focus is on finding largest volume ellipsoids with specified centers which are contained in certain convex cones. After reviewing the related literature and establishing some fundamental mathematical techniques that will be useful, we derive such maximum volume ellipsoids for second order cones and the cones of symmetric positive semidefinite matrices. Then we move to the more challenging problem of finding a largest pair (in the sense of geometric mean of their radii) of primal-dual ellipsoids (in the sense of dual norms) with specified centers that are contained in their respective primal-dual pair of convex cones. iii Acknowledgments First and foremost, I thank my supervisor Levent Tun¸celfor his continuous guidance and mentorship. His patience, dedication and passion inspired me more than just academically. The material in this thesis is based upon research supported in part by NSERC Discov- ery Grants, William Tutte Postgraduate Scholarship, Sinclair Graduate Scholarship, U.S. Office of Naval Research under award number: N00014-15-1-2171. I would like to express my gratitude for the financial support and to University of Waterloo. I thank Stephen Vavasis and Joseph Cheriyan for taking their time reading the thesis with their busy schedule and for their thoughtful comments. I thank Ritvik Ramkumar and Ewout van den Berg for the helpful discussion and pointing out the useful reference. I thank Mehdi Karimi, Nishad Kothari and all my friends in University of Waterloo and in the C&O Department for the genuine help and support throughout the journey. Last but not least, I thank my parents for the support, encouragement and love all these years. iv Table of Contents Author's Declaration ii Abstract iii Acknowledgments iv List of Figures vii 1 Introduction1 2 Convex sets, convex cones, self-concordant barrier functions and some optimality conditions 13 2.1 Fundamental concepts............................. 13 2.2 Convex optimization problem formulations and optimality conditions... 20 2.3 Optimizing a linear function over an ellipsoid................. 24 3 Ellipsoids approximating convex sets, L¨owner-Johnellipsoid and L¨owner's theorem 28 3.1 Motivation and L¨owner-Johntheorem..................... 28 3.2 Optimality conditions for an ellipsoid being L¨owner-Johnellipsoid..... 31 3.3 Some general observations on the maximum volume ellipsoids in convex cones 40 v 4 Maximum volume ellipsoids with specified centers contained in convex cones 43 4.1 Background................................... 43 4.2 Maximum volume ellipsoids in the nonnegative orthant........... 46 4.3 Maximum volume ellipsoids in the 2-by-2 positive semidefinite cone.... 49 4.3.1 The maximum volume ellipsoid over all self-adjoint positive definite operators................................ 51 4.4 Maximum volume ellipsoids in second order cones.............. 58 4.5 Maximum volume ellipsoids in positive semidefinite cones.......... 64 4.5.1 The maximum volume ellipsoids over the set of automorphism oper- ators................................... 64 4.5.2 The maximum volume ellipsoids over all self-adjoint positive definite operators by geometry......................... 65 4.5.3 The maximum volume ellipsoids over all self-adjoint positive definite operators by contact points and duality................ 71 4.6 Maximum volume ellipsoids in special homogeneous cones.......... 77 5 Largest primal-dual pairs of ellipsoids with specified centers in convex cones 81 5.1 Largest primal-dual pairs of ellipsoids in positive semidefinite cones.... 81 5.2 Largest primal-dual pairs of ellipsoids in self-scaled (symmetric) cones... 86 5.3 Largest primal-dual pairs of ellipsoids for homogeneous and hyperbolicity cones....................................... 88 5.4 Other generalizations of primal-dual pairs of ellipsoids............ 90 6 Conclusion and future research 94 References 96 vi List of Figures 1.1 epi(f) = f(x; µ): x; µ 2 R; µ ≥ x2g: ......................2 1.2 Illustration for dual norms in R2........................ 10 2.1 Using ellipsoid EC as outer approximation for a convex set C........ 25 2.2 Using ellipsoid EK centered atx ¯ as a local, inner approximation for cone K. 25 3.1 The minimum volume ellipsoid containing and the maximum volume ellip- soid contained in a equilateral simplex in R2.................. 30 3.2 Illustration of the affine transformation..................... 35 3.3 Illustration of the translation and affine transformation............ 36 3.4 Illustration of Proposition 3.3.2 pointwise on an ellipsoid........... 41 3.5 Illustration of Proposition 3.3.2......................... 41 4.1 Growing the radius given a fixed center and ellipsoidal shape until it hits n one of the boundaries of R+........................... 47 vii Chapter 1 Introduction In this introduction chapter, we will introduce some fundamental concepts, and describe the problems we studied with their motivation. After that, we will present the overall structure for the rest of the thesis. Some subsets of finite dimensional Euclidean spaces have the property that for every pair of points in the set, the line segment that joins the pair of points is entirely contained in the set. Such sets are called convex. A compact convex set with non-empty interior is called a convex body, and a convex hull of a set C is the smallest (with respect to set inclusion) convex set that contains C, denoted by conv(C). Let Rn denote the n-dimensional Euclidean space with the standard inner-product, and E denote a general Euclidean space of dimension n without a specific inner product. Let n n R+ denote the non-negative orthant of R , which is the set of vectors whose entries are all nonnegative. The Euclidean ball centered atx ¯ 2 Rn with radius r is denoted as: n 2 B(¯x; r) := fx 2 R : h(x − x¯);I(x − x¯)i ≤ r g: By the above definition on convexity, we have the empty set, Rn, the non-negative orthant, and any Euclidean ball are all convex sets. A closed half space is the set n fx 2 R : ha; xi ≤ bg; for some a 2 Rn and b 2 R. Notice that when a is the zero vector, ha; xi = 0, if b ≥ 0, the set is the whole space Rn, and if b < 0, the set is the empty set. Every closed half space is a closed convex set. 1 y epi(f) f(x) = x2 x Figure 1.1: epi(f) = f(x; µ): x; µ 2 R; µ ≥ x2g: The intersection of any collection (finite or infinite) of convex sets is always convex. Hence, the intersection of any collection of closed half spaces is again closed and convex. A polyhedron is the intersection of a finite set of closed half spaces, and a polytope is a polyhedron that is bounded. An equivalent definition for polytope is that it is the convex hull of finitely many points. This equivalence has connections to Minkowski's work from more than 100 years ago. A face of a polyhedron P is the intersection of P and some of its defining closed half spaces. We can represent a function using a geometric set, by defining the notion of epigraph. The epigraph of a function f : E ! [−∞; +1] is the set of points lying on or above its graph, i.e., epi(f) := f(x; µ): x 2 E; µ 2 R; µ ≥ f(x)g: Consider the function f : R ! R, f(x) = x2 as illustrated in Figure 1.1. The shaded area is the epigraph of f. A convex function is a function whose epigraph is a convex set. For instance, the epigraph of the function g : R ! R, g(x) := x3; x 2 R is not convex, so g is not a convex function. Convex optimization deals with the problem of minimizing convex functions over convex sets. Any convex optimization problem can be written as: min : f(x) x 2 F; 2 where f : E ! R is a convex function, and F ⊆ E is a convex set. This optimization problem is equivalent to the problem of the following form, with the introduction of a new variable t: min : t f(x) ≤ t x 2 F: The set of feasible solutions for the constraint f(x) ≤ t is precisely the set: x epi(f) = 2 ⊕ : f(x) ≤ t : t E R Since f is a convex function, by definition, its epigraph is a convex set. F is convex to begin with, so the intersection of F and epi(f) must again be a convex set. Hence, the feasible region of the new formulation is convex. With this new equivalent formulation, we see that any convex optimization problem can be reduced to a problem of minimizing a linear function over a convex feasible region. Thus, redefining E, F and x, we have the following general form for convex optimization problems: min : hc; xi x 2 F; where c 2 E∗ defines the linear objective function, and F ⊆ E is a convex set. A local minimum solution of a convex optimization problem is a solution that has the minimum objective value within a neighboring set of possible solutions, and a global min- imum solution is a solution that has the minimum objective value among all possible solutions. Convex optimization problems are in general \easier" to deal with than gen- eral optimization problems, since any local minimum solution is automatically a global minimum solution.