Lecture 2: Convex sets August 28, 2008 Lecture 2 Outline • Review basic topology in Rn • Open Set and Interior • Closed Set and Closure • Dual Cone • Convex set • Cones • Affine sets • Half-Spaces, Hyperplanes, Polyhedra • Ellipsoids and Norm Cones • Convex, Conical, and Affine Hulls • Simplex • Verifying Convexity Convex Optimization 1 Lecture 2 Topology Review n Let {xk} be a sequence of vectors in R n n Def. The sequence {xk} ⊆ R converges to a vector xˆ ∈ R when kxk − xˆk tends to 0 as k → ∞ • Notation: When {xk} converges to a vector xˆ, we write xk → xˆ n • The sequence {xk} converges xˆ ∈ R if and only if for each component i: the i-th components of xk converge to the i-th component of xˆ i i |xk − xˆ | tends to 0 as k → ∞ Convex Optimization 2 Lecture 2 Open Set and Interior Let X ⊆ Rn be a nonempty set Def. The set X is open if for every x ∈ X there is an open ball B(x, r) that entirely lies in the set X, i.e., for each x ∈ X there is r > 0 s.th. for all z with kz − xk < r, we have z ∈ X Def. A vector x0 is an interior point of the set X, if there is a ball B(x0, r) contained entirely in the set X Def. The interior of the set X is the set of all interior points of X, denoted by R (X) • How is R (X) related to X? 2 • Example X = {x ∈ R | x1 ≥ 0, x2 > 0} R 2 (X) = {x ∈ R | x1 > 0, x2 > 0} R n 0 (S) of a probability simplex S = {x ∈ R | x 0, e x = 1} Th. For a convex set X, the interior R (X) is also convex Convex Optimization 3 Lecture 2 Closed Set Def. The complement of a given set X ⊆ Rn is the set of all vectors that do not belong to X: the complement of X = {x ∈ Rn | x∈ / X} = Rn \ X Def. The set X is closed if its complement Rn \ X is open • Examples: Rn and ∅ (both are open and closed) 2 {x ∈ R | x1 ≥ 0, x2 > 0} is open or closed? hyperplane, half-space, simplex, polyhedral sets? • The intersection of any family of closed set is closed • The union of a finite family of closed set is closed • The sum of two closed sets is not necessarily closed • Example: C1 = {(x1, x2) | x1 = 0, x2 ∈ R} C2 = {(x1, x2) | x1x2 ≥ 1, x1 ≥ 0} C1 + C2 is not closed! • Fact: The sum of a compact set and a closed set is closed Convex Optimization 4 Lecture 2 Closure Let X ⊆ Rn be a nonempty set Def. A vector xˆ is a closure point of a set X if there exists a sequence {xk} ⊆ X such that xk → xˆ Closure points of X = {(−1)n/n | n = 1, 2,...}, Xˆ = {1 − x | x ∈ X}? • Notation: The set of closure points of X is denoted by cl(X) • What is relation between X and cl(X)? Th. A set is closed if and only if it contains its closure points, i.e., X is closed iff cl(X) ⊂ X Th. For a convex set, the closure cl(X) is convex Convex Optimization 5 Lecture 2 Boundary Let X ⊆ Rn be a nonempty set Def. The boundary of the set X is the set of closure points for both the set X and its complement Rn \ X, i.e., n bd(X) = cl(X) ∩ cl(R \ X) n • Example X = {x ∈ R | g1(x) ≤ 0, . , gm(x) ≤ 0}. Is X closed? What constitutes the boundary of X? Convex Optimization 6 Lecture 2 Dual Cone Let K be a nonempty cone in Rn Def. The dual cone of K is the set K∗ defined by K∗ = {z | z0x ≥ 0 for all x ∈ K} • The dual cone K∗ is a closed convex cone even when K is neither closed nor convex • Let S be a subspace. Then, S∗ = S⊥. • Let C be a closed convex cone. Then, (C∗)∗ = C. • For an arbitrary cone K, we have (K∗)∗ = cl(conv(K)). Convex Optimization 7 Lecture 2 Convex set • A line segment defined by vectors x and y is the set of points of the form αx + (1 − α)y for α ∈ [0, 1] • A set C ⊂ Rn is convex when, with any two vectors x and y that belong to the set C, the line segment connecting x and y also belongs to C Convex Optimization 8 Lecture 2 Examples Which of the following sets are convex? • The space Rn • A line through two given vectors x and y l(x, y) = {z | z = x + t(y − x), t ∈ R} • A ray defined by a vector x {z | z = λx, λ ≥ 0} • The positive orthant {x ∈ Rn | x 0} ( componentwise inequality) 2 • The set {x ∈ R | x1 > 0, x2 ≥ 0} 2 • The set {x ∈ R | x1x2 = 0} Convex Optimization 9 Lecture 2 Cone A set C ⊂ Rn is a cone when, with every vector x ∈ C, the ray {λx | λ ≥ 0} belongs to the set C • A cone may or may not be convex n 2 • Examples: {x ∈ R | x 0}{x ∈ R | x1x2 ≥ 0} For a two sets C and S, the sum C + S is defined by C + S = {z | z = x + y, x ∈ C, y ∈ S} (the order does nor matter) Convex Cone Lemma: A cone C is convex if and only if C + C ⊆ C Proof: Pick any x and y in C, and any α ∈ [0, 1]. Then, αx and (1 − α)y belong to C because... Using C + C ⊆ C, it follows that ... Reverse: Let C be convex cone, and pick any x, y ∈ C. Consider 1/2(x + y)... Convex Optimization 10 Lecture 2 Affine Set A set C ⊂ Rn is a affine when, with every two distinct vectors x, y ∈ C, the line {x + t(y − x) | t ∈ R} belongs to the set C • An affine set is always convex • A subspace is an affine set A set C is affine if and only if C is a translated subspace, i.e., C = S + x0 for some subspace S and some x0 ∈ C Dimension of an affine set C is the dimension of the subspace S Convex Optimization 11 Lecture 2 Hyperplanes and Half-spaces Hyperplane is a set of the form {x | a0x = b} for a nonzero vector a Half-space is a set of the form {x | a0x ≤ b} with a nonzero vector a The vector a is referred to as the normal vector • A hyperplane in Rn divides the space into two half-spaces {x | a0x ≤ b} and {x | a0x ≥ b} • Half-spaces are convex • Hyperplanes are convex and affine Convex Optimization 12 Lecture 2 Polyhedral Sets A polyhedral set is given by finitely many linear inequalities C = {x | Ax b} where A is an m × n matrix • Every polyhedral set is convex • Linear Problem minimize c0x subject to Bx ≤ b, Dx = d The constraint set {x | Bx ≤ b, Dx = d} is polyhedral. Convex Optimization 13 Lecture 2 Ellipsoids Let A be a square (n × n) matrix. • A is positive semidefinite when x0Ax ≥ 0 for all x ∈ Rn • A is positive definite when x0Ax > 0 for all x ∈ Rn, x 6= 0 An ellipsoid is a set of the form 0 −1 E = {x | (x − x0) P (x − x0) ≤ 1} where P is symmetric and positive definite • x0 is the center of the ellipsoid E 2 • A ball {x | kx − x0k ≤ r} is a special case of the ellipsoid (P = r I) • Ellipsoids are convex Convex Optimization 14 Lecture 2 Norm Cones A norm cone is the set of the form C = {(x, t) ∈ Rn × R | kxk ≤ t} • The norm k · k can be any norm in Rn • The norm cone for Euclidean norm is also known as ice-cream cone • Any norm cone is convex Convex Optimization 15 Lecture 2 Convex and Conical Hulls A convex combination of vectors x1, . , xm is a vector of the form Pm α1x1 + ... + αmxm αi ≥ 0 for all i and i=1 αi = 1 The convex hull of a set X is the set of all convex combinations of the vectors in X, denoted conv(X) A conical combination of vectors x1, . , xm is a vector of the form λ1x1 + ... + λmxm with λi ≥ 0 for all i The conical hull of a set X is the set of all conical combinations of the vectors in X, denoted by cone(X) Convex Optimization 16 Lecture 2 Affine Hull An affine combination of vectors x1, . , xm is a vector of the form Pm t1x1 + ... + tmxm with i=1 ti = 1, ti ∈ R for all i The affine hull of a set X is the set of all affine combinations of the vectors in X, denoted aff(X) The dimension of a set X is the dimension of the affine hull of X dim(X) = dim(aff(X)) Convex Optimization 17 Lecture 2 Simplex A simplex is a set given as a convex combination of a finite collection of vectors v0, v1, . , vm: C = conv{v0, v1 . , vm} The dimension of the simplex C is equal to the maximum number of linearly independent vectors among v1 − v0, . , vm − v0. Examples • Unit simplex {x ∈ Rn | x 0, e0x ≤ 1}, e = (1,..., 1), dim -? • Probability simplex {x ∈ Rn | x 0, e0x = 1}, dim -? Convex Optimization 18 Lecture 2 Practical Methods for Establishing Convexity of a Set Establish the convexity of a given set X • The set is one of the “recognizable” (simple) convex sets such as polyhedral, simplex, norm cone, etc • Prove the convexity by directly applying the definition For every x, y ∈ X and α ∈ (0, 1), show that αx + (1 − α)y is also in X • Show that the set is obtained from one of the simple convex sets through an operation that preserves convexity Convex Optimization 19 Lecture 2 Operations Preserving Convexity n n n m Let C ⊆ R , C1 ⊆ R , C2 ⊆ R , and K ⊆ R be convex sets.