Convexity in Rn 1 N CONVEXITY in R Juan Pablo Xandri
Total Page:16
File Type:pdf, Size:1020Kb
Convexity in Rn 1 N CONVEXITY IN R Juan Pablo Xandri Up to this point we have studied metric spaces from a very general perspective. What we aim to do now is to study optimization problems on these spaces. Even when there are well de…ned optimization methods on metric spaces, the challenge that we will face now is that these are quite diverse. In this chapter we will focus on the analysis of concepts and properties that are speci…c to the case of Rn, and which will help us in the next chapter to analyze the main optimization methods in Rn. In this and following notes we will introduce the concepts of convexity and di¤erentiability for the particular case of normed spaces, as is the case of Rn. Even when these concepts can be generalized to spaces such as Rm; Rk , (R) or even spaces of bounded sequences, this will not be necessaryB for this course.L Convex Sets De…nition We say that a set is convex if whenever we consider any two elements of the set, then the line segment connecting these elements is also contained in this set. Graphically, if we have the points x; y A as in Figure 10, then the line segment that joins them (xy) is also contained in2 the set. Figure 10 In Figure 11 we present an example of a set that is not convex: by considering the points x and y in the set A, we see that there is an element z xy such that z = A. 2 2 Figure 11 In this section we will formalize the concept of convex sets and present some of their properties. De…nition 1 (Convex Combination) Given x; y Rn, we say that z Rn is a linear convex combination of x and y there2 exists [0; 1] such2 that () 2 z = x + (1 ) y: De…nition 2 (Convex Set) We say that a set C Rn is a convex set if for all pair of points x; y C and for all [0; 1], the point z x + (1 ) y C. Equivalently, a set2 is said to be convex2 if whenever x; y C, then also all2 their linear convex combinations are contained in C. 2 2 Convexity in Rn Hence, in order to prove that a set C is convex, we need to prove a "theorem": basically, we consider any two points x; y C and a number [0; 1] as given, and we need to prove that x + (1 ) 2C. We will now consider2 some simple examples of convex sets. 2 Examples of Convex Sets (1) Normed Ball: In this example we prove that for any norm N, the set n BN (a; r) = x R : N (x a) < r dN (x; a) < r f 2 () g is a convex set. Consider any x; y BN (a; r) and [0; 1]. We want to 2 2 show that the point z = x + (1 ) y is contained in BN (a; r). In order to do so, notice that N (z a) = N (x + (1 ) y a) = N ( (x a) + (1 )(y a)) N ( (x a)) + N ((1 )(y a)) = N (x a) + (1 ) N (y a) (1) (2) < r + (1 ) r = r; (3) where in (1) we use the subadditivity of normed spaces, in (2) the property that states that N ( x) = N (x), and in (3) the fact that x; y BN (a; r). j j 2 (2) Polyhedra: A polyhedron is a set characterized by a collection of linear equalities and P inequalities. Let A be a matrix m n, B a matrix p n, and a Rm and 2 b Rp two vectors. A polyhedron is characterized as follows: 2 n = x R : Ax a and Bx = b : P f 2 g We will prove that any polyhedron is a convex set. Once again, we prove that if x; y and [0; 1], then z = x + (1 ) y . Thus, we have 2 P 2 2 P to prove that Az a and Bz = b. Notice that Az = A (x + (1 ) y) = Ax + (1 ) Ay a + (1 ) a = a; Bz = B (x + (1 ) y) = Bx + (1 ) By = b + (1 ) b = b: Therefore, z . 2 P Convexity in Rn 3 Properties of Convex Sets In this section we will prove that convexity is preserved under certain operations over sets. These properties will aid us to de…ne some fundamental concepts involv- ing convex sets. Proposition 3 (Interesection of Convex Sets) Let be a collection of sets F in Rn such that every C is convex. Then, the set 2 F = C C C \2F is also convex. Proof. Consider x; y and [0; 1]. We want to prove that z = x + (1 ) y . Notice then2 C that 2 2 C x x C for all C and 2 C () 2 2 F y y C for all C . 2 C () 2 2 F Hence, x (1 ) y C for all C = x + (1 ) y , which is what we wanted to prove.2 2 2 F ) 2 C This proposition will enable us to de…ne the concept of convex hull of a set. De…nition 4 (Convex Hull) Let A Rn be any set. We say that C A is the convex hull of A (and we write C = co(A)) if it is the "smallest" convex set that contains A. By smallest we mean that if A B and B is convex, then co (A) B. n n Theorem 5 Given a set A R , de…ne A = C R : A C and C is convex . Then, F f g co (A) = C = ?: 6 C A \2F n Proof. The fact that C = ? is obvious given that R A. Since A C 6 2 F C A \2F for every C A = A C, and by the previous proposition we know that 2 F ) C A \2F C is a convex set. Hence, this set is a candidate to be the convex hull of A. C A Since\2F co (A) is the smallest convex set that contains A, it must also be the case that, since C is convex, we must have that co (A) C. On the other C A C A \2F \2F 4 Convexity in Rn hand, if there is an x C such that x = co (A) = x = C. Hence, we 2 2 ) 2 C A C A \2F \2F conclude that co (A) = C. C A \2F We will now prove a result that will be very useful when we want to show that certain sets are convex: the Cartesian product of convex sets is also a convex set. m k Proposition 6 (Product of Convex Sets) Let C1 R and C2 R be con- m k vex sets. Then, C = C1 C2 R R is a convex set. Proof. Consider x = (x1; x2) C1 C2 and y = (y1; y2) C1 C2, and pick a [0; 1]. We have that 2 2 2 x+(1 ) y = (x1; x2)+(1 )(y1; y2) = (x1 + (1 ) y1; x2 + (1 ) x2) C1 C2 2 given that C1 and C2 are convex sets. Separation of Convex Sets One of the fundamental results regarding convex sets is the so called Hyperplane Separation Theorem. Basically, this theorem tells us that if we take two convex sets C; D Rn such that C D = ?, then we can …nd a hyperplane that separates them. In order to understand\ this, consider the case of n = 2. This theorem tells us that we can trace a straight line between both sets which separates the plane n into two half-planes, S1 and S2 such that S1 S2 = R and C S1 and D S2. We can see this graphically in Figure 12. [ Figure 12 (Separation of Convex Sets) In Figure 13 we can also appreciate an example of how this theorem might not hold if one of the two sets is not convex. Figure 13 In order to be able to prove this theorem, we must formalize the concepts of "lines" and half-spaces. De…nition 7 (Hyperplanes) Given a vector a Rn and a constant b R, we 2 2 say that the set (a; b) Rn is a hyperplane if x (a; b) aT x = b. H 2 H () Convexity in Rn 5 + De…nition 8 (Half-spaces) Given a hyperplane (a; b), de…ne the sets S (a; b) ;S (a; b) H Rn as + n T S (a; b) = x R : a x b , and 2 n T S (a; b) = x R : a x b . 2 We say that S+ (a; b) is the positive half-space of and, accordingly, that H S (a; b) is the negative half-space of . H Now that we have de…ned the previous concepts, we can state the Hyperplane Separation Theorem. Theorem 9 (Hyperplane Separation Theorem) Let C; D Rn be two non- empty convex sets such that C D = ?. Then, there exist a Rn and b R such + \ 2 2 that C S (a; b) and D S (a; b). This theorem will be very relevant for many applications, for example, for the proof of the Second Welfare Theorem (if we have enough time, we will consider it later on as an example). Brouwer’sFixed-Point Theorem In this section we will once again study …xed-point problems. Brouwer’stheorem, which is very relevant for many applications, is easy to understand, but extremely di¢ cult to prove (it requires the knowledge of concepts of algebraic topology that are well beyond the scope of this course).