Convex Optimization Solutions Manual Stephen Boyd Lieven Vandenberghe January 4, 2006 Chapter 2 Convex sets Exercises Exercises Definition of convexity n 2.1 Let C R be a convex set, with x1; : : : ; xk C, and let θ1; : : : ; θk R satisfy θi 0, ⊆ 2 2 ≥ θ1 + + θk = 1. Show that θ1x1 + + θkxk C. (The definition of convexity is that this holds· · · for k = 2; you must show it· ·for· arbitrary2 k.) Hint. Use induction on k. Solution. This is readily shown by induction from the definition of convex set. We illus- trate the idea for k = 3, leaving the general case to the reader. Suppose that x1; x2; x3 C, 2 and θ1 + θ2 + θ3 = 1 with θ1; θ2; θ3 0. We will show that y = θ1x1 + θ2x2 + θ3x3 C. ≥ 2 At least one of the θi is not equal to one; without loss of generality we can assume that θ1 = 1. Then we can write 6 y = θ1x1 + (1 θ1)(µ2x2 + µ3x3) − where µ2 = θ2=(1 θ1) and µ2 = θ3=(1 θ1). Note that µ2; µ3 0 and − − ≥ θ2 + θ3 1 θ1 µ1 + µ2 = = − = 1: 1 θ1 1 θ1 − − Since C is convex and x2; x3 C, we conclude that µ2x2 + µ3x3 C. Since this point 2 2 and x1 are in C, y C. 2 2.2 Show that a set is convex if and only if its intersection with any line is convex. Show that a set is affine if and only if its intersection with any line is affine. Solution. We prove the first part. The intersection of two convex sets is convex. There- fore if S is a convex set, the intersection of S with a line is convex. Conversely, suppose the intersection of S with any line is convex. Take any two distinct points x1 and x2 S. The intersection of S with the line through x1 and x2 is convex. 2 Therefore convex combinations of x1 and x2 belong to the intersection, hence also to S. 2.3 Midpoint convexity. A set C is midpoint convex if whenever two points a; b are in C, the average or midpoint (a + b)=2 is in C. Obviously a convex set is midpoint convex. It can be proved that under mild conditions midpoint convexity implies convexity. As a simple case, prove that if C is closed and midpoint convex, then C is convex. Solution. We have to show that θx + (1 θ)y C for all θ [0; 1] and x; y C. Let − 2 2 2 θ(k) be the binary number of length k, i.e., a number of the form (k) 1 2 k θ = c12− + c22− + + ck2− · · · with ci 0; 1 , closest to θ. By midpoint convexity (applied k times, recursively), 2 f g θ(k)x + (1 θ(k))y C. Because C is closed, − 2 lim (θ(k)x + (1 θ(k))y) = θx + (1 θ)y C: k − − 2 !1 2.4 Show that the convex hull of a set S is the intersection of all convex sets that contain S. (The same method can be used to show that the conic, or affine, or linear hull of a set S is the intersection of all conic sets, or affine sets, or subspaces that contain S.) Solution. Let H be the convex hull of S and let be the intersection of all convex sets that contain S, i.e., D = D D convex; D S : D f j ⊇ g We will show that H = by showing that H and H. D \ ⊆ D D ⊆ First we show that H . Suppose x H, i.e., x is a convex combination of some ⊆ D 2 points x1; : : : ; xn S. Now let D be any convex set such that D S. Evidently, we have 2 ⊇ x1; : : : ; xn D. Since D is convex, and x is a convex combination of x1; : : : ; xn, it follows that x D2. We have shown that for any convex set D that contains S, we have x D. This means2 that x is in the intersection of all convex sets that contain S, i.e., x 2. 2 D Now let us show that H. Since H is convex (by definition) and contains S, we must have H = D for some DD⊆in the construction of , proving the claim. D 2 Convex sets Examples n T 2.5 What is the distance between two parallel hyperplanes x R a x = b1 and x n T f 2 j g f 2 R a x = b2 ? j g Solution. The distance between the two hyperplanes is b1 b2 = a 2. To see this, consider the construction in the figure below. j − j k k 2 x2 = (b2= a )a k k PSfrag replacements 2 a x1 = (b1= a )a T k k a x = b2 T a x = b1 The distance between the two hyperplanes is also the distance between the two points x1 and x2 where the hyperplane intersects the line through the origin and parallel to the normal vector a. These points are given by 2 2 x1 = (b1= a )a; x2 = (b2= a )a; k k2 k k2 and the distance is x1 x2 2 = b1 b2 = a 2: k − k j − j k k 2.6 When does one halfspace contain another? Give conditions under which x aT x b x a~T x ~b f j ≤ g ⊆ f j ≤ g (where a = 0, a~ = 0). Also find the conditions under which the two halfspaces are equal. 6 6 Solution. Let = x aT x b and ~ = x a~T x ~b . The conditions are: H f j ≤ g H f j ≤ g ~ if and only if there exists a λ > 0 such that a~ = λa and ~b λb. • H ⊆ H ≥ = ~ if and only if there exists a λ > 0 such that a~ = λa and ~b = λb. • H H Let us prove the first condition. The condition is clearly sufficient: if a~ = λa and ~b λb for some λ > 0, then ≥ aT x b = λaT x λb = a~T x ~b; ≤ ) ≤ ) ≤ i.e., ~. H ⊆ H To prove necessity, we distinguish three cases. First suppose a and a~ are not parallel. This means we can find a v with a~T v = 0 and aT v = 0. Let x^ be any point in the intersection 6 of and ~, i.e., aT x^ b and a~T x ~b. We have aT (x^ + tv) = aT x^ b for all t R. H H ≤ ≤ ≤ 2 However a~T (x^ + tv) = a~T x^ + ta~T v, and since a~T v = 0, we will have a~T (x^ + tv) > ~b for sufficiently large t > 0 or sufficiently small t < 0.6 In other words, if a and a~ are not parallel, we can find a point x^ + tv that is not in ~, i.e., ~. 2 H H H 6⊆ H Next suppose a and a~ are parallel, but point in opposite directions, i.e., a~ = λa for some λ < 0. Let x^ be any point in . Then x^ ta for all t 0. However for t large enough H − 2 H ≥ we will have a~T (x^ ta) = a~T x^ + tλ a 2 > ~b, so x^ ta ~. Again, this shows ~. − k k2 − 62 H H 6⊆ H Exercises Finally, we assume a~ = λa for some λ > 0 but ~b < λb. Consider any point x^ that satisfies aT x^ = b. Then a~T x^ = λaT x^ = λb > ~b, so x^ ~. 62 H The proof for the second part of the problem is similar. 2.7 Voronoi description of halfspace. Let a and b be distinct points in Rn. Show that the set of all points that are closer (in Euclidean norm) to a than b, i.e., x x a 2 x b 2 , f j k − k ≤ k − k g is a halfspace. Describe it explicitly as an inequality of the form cT x d. Draw a picture. ≤ Solution. Since a norm is always nonnegative, we have x a 2 x b 2 if and only if x a 2 x b 2, so k − k ≤ k − k k − k2 ≤ k − k2 x a 2 x b 2 (x a)T (x a) (x b)T (x b) k − k2 ≤ k − k2 () − − ≤ − − xT x 2aT x + aT a xT x 2bT x + bT b () − ≤ − 2(b a)T x bT b aT a: () − ≤ − Therefore, the set is indeed a halfspace. We can take c = 2(b a) and d = bT b aT a. This makes good geometric sense: the points that are equidistan−t to a and b are giv−en by a hyperplane whose normal is in the direction b a. − 2.8 Which of the following sets S are polyhedra? If possible, express S in the form S = x Ax b; F x = g . f j g n (a) S = y1a1 + y2a2 1 y1 1; 1 y2 1 , where a1; a2 R . f j − ≤ ≤ − ≤ ≤ g 2 n T n n 2 (b) S = x R x 0; 1 x = 1; xiai = b1; xia = b2 , where f 2 j i=1 i=1 i g a1; : : : ; an R and b1; b2 R. 2n T 2 P P (c) S = x R x 0; x y 1 for all y with y 2 = 1 . f 2 j ≤ k k g n T n (d) S = x R x 0; x y 1 for all y with yi = 1 . f 2 j ≤ i=1 j j g Solution. P (a) S is a polyhedron. It is the parallelogram with corners a1 + a2, a1 a2, a1 + a2, 2 − − a1 a2, as shown below for an example in R .