Lecture 2: Convex Sets

Home , Affine hull, Cone (topology), Conical combination, Convex combination, Convex cone

Lecture 2: Convex sets

August 28, 2008 Lecture 2 Outline

• Review basic topology in Rn • Open Set and Interior • Closed Set and Closure • Dual Cone • Convex set • Cones • Aﬃne sets • Half-Spaces, Hyperplanes, Polyhedra • Ellipsoids and Norm Cones • Convex, Conical, and Aﬃne Hulls • Simplex • Verifying Convexity

Convex Optimization 1 Lecture 2 Topology Review

n Let {xk} be a sequence of vectors in R

n n Def. The sequence {xk} ⊆ R converges to a vector xˆ ∈ R when kxk − xˆk tends to 0 as k → ∞

• Notation: When {xk} converges to a vector xˆ, we write xk → xˆ

n • The sequence {xk} converges xˆ ∈ R if and only if for each component i: the i-th components of xk converge to the i-th component of xˆ i i |xk − xˆ | tends to 0 as k → ∞

Convex Optimization 2 Lecture 2 Open Set and Interior

Let X ⊆ Rn be a nonempty set Def. The set X is open if for every x ∈ X there is an open ball B(x, r) that entirely lies in the set X, i.e., for each x ∈ X there is r > 0 s.th. for all z with kz − xk < r, we have z ∈ X

Def. A vector x0 is an interior point of the set X, if there is a ball B(x0, r) contained entirely in the set X Def. The interior of the set X is the set of all interior points of X, denoted by R (X) • How is R (X) related to X? 2 • Example X = {x ∈ R | x1 ≥ 0, x2 > 0} R 2 (X) = {x ∈ R | x1 > 0, x2 > 0} R n 0 (S) of a probability simplex S = {x ∈ R | x 0, e x = 1} Th. For a convex set X, the interior R (X) is also convex

Convex Optimization 3 Lecture 2 Closed Set

Def. The complement of a given set X ⊆ Rn is the set of all vectors that do not belong to X: the complement of X = {x ∈ Rn | x∈ / X} = Rn \ X Def. The set X is closed if its complement Rn \ X is open • Examples: Rn and ∅ (both are open and closed) 2 {x ∈ R | x1 ≥ 0, x2 > 0} is open or closed? hyperplane, half-space, simplex, polyhedral sets? • The intersection of any family of closed set is closed • The union of a ﬁnite family of closed set is closed • The sum of two closed sets is not necessarily closed • Example: C1 = {(x1, x2) | x1 = 0, x2 ∈ R} C2 = {(x1, x2) | x1x2 ≥ 1, x1 ≥ 0} C1 + C2 is not closed! • Fact: The sum of a compact set and a closed set is closed

Convex Optimization 4 Lecture 2 Closure

Let X ⊆ Rn be a nonempty set Def. A vector xˆ is a closure point of a set X if there exists a sequence

{xk} ⊆ X such that xk → xˆ Closure points of X = {(−1)n/n | n = 1, 2,...}, Xˆ = {1 − x | x ∈ X}? • Notation: The set of closure points of X is denoted by cl(X)

• What is relation between X and cl(X)?

Th. A set is closed if and only if it contains its closure points, i.e., X is closed iﬀ cl(X) ⊂ X

Th. For a convex set, the closure cl(X) is convex

Convex Optimization 5 Lecture 2 Boundary

Let X ⊆ Rn be a nonempty set Def. The boundary of the set X is the set of closure points for both the set X and its complement Rn \ X, i.e.,

n bd(X) = cl(X) ∩ cl(R \ X)

n • Example X = {x ∈ R | g1(x) ≤ 0, . . . , gm(x) ≤ 0}. Is X closed? What constitutes the boundary of X?

Convex Optimization 6 Lecture 2 Dual Cone

Let K be a nonempty cone in Rn Def. The dual cone of K is the set K∗ deﬁned by

K∗ = {z | z0x ≥ 0 for all x ∈ K}

• The dual cone K∗ is a closed convex cone even when K is neither closed nor convex • Let S be a subspace. Then, S∗ = S⊥. • Let C be a closed convex cone. Then, (C∗)∗ = C. • For an arbitrary cone K, we have (K∗)∗ = cl(conv(K)).

Convex Optimization 7 Lecture 2 Convex set

• A line segment deﬁned by vectors x and y is the set of points of the form αx + (1 − α)y for α ∈ [0, 1]

• A set C ⊂ Rn is convex when, with any two vectors x and y that belong to the set C, the line segment connecting x and y also belongs to C

Convex Optimization 8 Lecture 2 Examples

Which of the following sets are convex? • The space Rn • A line through two given vectors x and y l(x, y) = {z | z = x + t(y − x), t ∈ R} • A ray deﬁned by a vector x {z | z = λx, λ ≥ 0}

• The positive orthant {x ∈ Rn | x 0} ( componentwise inequality) 2 • The set {x ∈ R | x1 > 0, x2 ≥ 0} 2 • The set {x ∈ R | x1x2 = 0}

Convex Optimization 9 Lecture 2 Cone

A set C ⊂ Rn is a cone when, with every vector x ∈ C, the ray {λx | λ ≥ 0} belongs to the set C

• A cone may or may not be convex n 2 • Examples: {x ∈ R | x 0}{x ∈ R | x1x2 ≥ 0} For a two sets C and S, the sum C + S is deﬁned by C + S = {z | z = x + y, x ∈ C, y ∈ S} (the order does nor matter) Convex Cone Lemma: A cone C is convex if and only if C + C ⊆ C Proof: Pick any x and y in C, and any α ∈ [0, 1]. Then, αx and (1 − α)y belong to C because... . Using C + C ⊆ C, it follows that ... Reverse: Let C be convex cone, and pick any x, y ∈ C. Consider 1/2(x + y)...

Convex Optimization 10 Lecture 2 Aﬃne Set

A set C ⊂ Rn is a affine when, with every two distinct vectors x, y ∈ C, the line {x + t(y − x) | t ∈ R} belongs to the set C • An affine set is always convex • A subspace is an affine set

A set C is aﬃne if and only if C is a translated subspace, i.e.,

C = S + x0 for some subspace S and some x0 ∈ C

Dimension of an aﬃne set C is the dimension of the subspace S

Convex Optimization 11 Lecture 2 Hyperplanes and Half-spaces

Hyperplane is a set of the form {x | a0x = b} for a nonzero vector a

Half-space is a set of the form {x | a0x ≤ b} with a nonzero vector a The vector a is referred to as the normal vector • A hyperplane in Rn divides the space into two half-spaces {x | a0x ≤ b} and {x | a0x ≥ b} • Half-spaces are convex • Hyperplanes are convex and aﬃne

Convex Optimization 12 Lecture 2 Polyhedral Sets A polyhedral set is given by ﬁnitely many linear inequalities C = {x | Ax b} where A is an m × n matrix

• Every polyhedral set is convex • Linear Problem minimize c0x subject to Bx ≤ b, Dx = d

The constraint set {x | Bx ≤ b, Dx = d} is polyhedral.

Convex Optimization 13 Lecture 2 Ellipsoids

Let A be a square (n × n) matrix. • A is positive semidefinite when x0Ax ≥ 0 for all x ∈ Rn • A is positive definite when x0Ax > 0 for all x ∈ Rn, x 6= 0 An ellipsoid is a set of the form 0 −1 E = {x | (x − x0) P (x − x0) ≤ 1} where P is symmetric and positive definite

• x0 is the center of the ellipsoid E 2 • A ball {x | kx − x0k ≤ r} is a special case of the ellipsoid (P = r I) • Ellipsoids are convex

Convex Optimization 14 Lecture 2 Norm Cones

A norm cone is the set of the form C = {(x, t) ∈ Rn × R | kxk ≤ t}

• The norm k · k can be any norm in Rn • The norm cone for Euclidean norm is also known as ice-cream cone • Any norm cone is convex

Convex Optimization 15 Lecture 2 Convex and Conical Hulls

A convex combination of vectors x1, . . . , xm is a vector of the form Pm α1x1 + ... + αmxm αi ≥ 0 for all i and i=1 αi = 1 The convex hull of a set X is the set of all convex combinations of the vectors in X, denoted conv(X)

A conical combination of vectors x1, . . . , xm is a vector of the form

λ1x1 + ... + λmxm with λi ≥ 0 for all i The conical hull of a set X is the set of all conical combinations of the vectors in X, denoted by cone(X)

Convex Optimization 16 Lecture 2 Aﬃne Hull

An affine combination of vectors x1, . . . , xm is a vector of the form Pm t1x1 + ... + tmxm with i=1 ti = 1, ti ∈ R for all i The affine hull of a set X is the set of all affine combinations of the vectors in X, denoted aff(X)

The dimension of a set X is the dimension of the aﬃne hull of X dim(X) = dim(aﬀ(X))

Convex Optimization 17 Lecture 2 Simplex A simplex is a set given as a convex combination of a ﬁnite collection of vectors v0, v1, . . . , vm: C = conv{v0, v1 . . . , vm} The dimension of the simplex C is equal to the maximum number of linearly independent vectors among v1 − v0, . . . , vm − v0. Examples • Unit simplex {x ∈ Rn | x 0, e0x ≤ 1}, e = (1,..., 1), dim -? • Probability simplex {x ∈ Rn | x 0, e0x = 1}, dim -?

Convex Optimization 18 Lecture 2 Practical Methods for Establishing Convexity of a Set

Establish the convexity of a given set X

• The set is one of the “recognizable” (simple) convex sets such as polyhedral, simplex, norm cone, etc

• Prove the convexity by directly applying the deﬁnition For every x, y ∈ X and α ∈ (0, 1), show that αx + (1 − α)y is also in X

• Show that the set is obtained from one of the simple convex sets through an operation that preserves convexity

Convex Optimization 19 Lecture 2 Operations Preserving Convexity

n n n m Let C ⊆ R , C1 ⊆ R , C2 ⊆ R , and K ⊆ R be convex sets. Then, the following sets are also convex:

• The intersection C1 ∩ C2 = {x | x ∈ C1 and x ∈ C2}

• The sum C1 + C2 of two convex sets • The translated set C + a • The scaled set tC = {tx | x ∈ C} for any t ∈ R

• The Cartesian product C1 × C2 = {(x1, x2) | x1 ∈ C1, x2 ∈ C2}

• The coordinate projection {x1 | (x1, x2) ∈ C for some x2} • The image AC under a linear transformation A : Rn 7→ Rm: AC = {y ∈ Rm | y = Ax for some x ∈ C} • The inverse image A−1K under a linear transformation A : Rn 7→ Rm: A−1K = {x ∈ Rn | Ax ∈ K}

Convex Optimization 20