Assignment 1: from the Definition of Convexity to Helley Theorem Exercise 1 Mark in the Following List the Sets Which Are Convex

Assignment 1: From the Deﬁnition of Convexity to Helley Theorem

Exercise 1 Mark in the following list the sets which are convex:

2 2 1. {x ∈ R : x1 + i x2 ≤ 1, i = 1, ..., 10}

2 2 2 2 2. {x ∈ R : x1 + 2ix1x2 + i x2 ≤ 1, i = 1, ..., 10} 2 2 2 2 3. {x ∈ R : x1 + ix1x2 + i x2 ≤ 1, i = 1, ..., 10} 2 2 2 4. {x ∈ R : x1 + 5x1x2 + 4x2 ≤ 1} 10 2 2 2 2 5. {x ∈ R : x1 +2x2 +3x3 +...+10x10 ≤ 2004x1 −2003x2 +2002x3 −...+1996x9 −1995x10} 2 6. {x ∈ R : exp{x1} ≤ x2}

2 7. {x ∈ R : exp{x1} ≥ x2} n n P 2 8. {x ∈ R : xi = 1} i=1 n n P 2 9. {x ∈ R : xi ≤ 1} i=1 n n P 2 10. {x ∈ R : xi ≥ 1} i=1 n 11. {x ∈ R : max xi ≤ 1} i=1,...,n

n 12. {x ∈ R : max xi ≥ 1} i=1,...,n

n 13. {x ∈ R : max xi = 1} i=1,...,n

n 14. {x ∈ R : min xi ≤ 1} i=1,...,n

n 15. {x ∈ R : min xi ≥ 1} i=1,...,n

n 16. {x ∈ R : min xi = 1} i=1,...,n

Exercise 2 Which ones of the following three statements are true?

1. The convex hull of a closed set in Rn is closed

2. The convex hull of a closed convex set in Rn is closed

3. The convex hull of a closed and bounded set in Rn is closed and bounded

For true statements, present proofs; for wrong, give counterexamples.

Hint: Recall that a bounded and closed subset of Rn is compact and that there exists Caratheodory Theorem.

The next two exercises are optional.

1 Exercise 3 A cake contains 300 g1) of raisins (you may think of every one of them as of a 3D ball of positive radius). John and Jill are about to divide the cake according to the following rules: • ﬁrst, Jill chooses a point a in the cake; • second, John makes a cut through a, that is, chooses a 2D plane Π passing through a and takes the part of the cake on one side of the plane (both Π and the side are up to John, with the only restriction that the plane should pass through a); all the rest goes to Jill.

1. Prove that it may happen that Jill cannot guarantee herself 76 g of the raisins

2. Prove that Jill always can choose a in a way which guarantees her at least 74 g of the raisins

3. Consider n-dimensional version of the problem, where the raisins are n-dimensional balls, the cake is a domain in Rn, and “a cut” taken by John is deﬁned as the part of the cake contained in the half-space {x ∈ Rn : eT (x − a) ≥ 0}, where e 6= 0 is the vector (“inner normal to the cutting hyperplane”) chosen by John. 300 Prove that for every > 0, Jill can guarantee to herself at least n+1 − g of raisins, but 300 in general cannot guarantee to herself n+1 + g.

Exercise 4 Prove the following Kirchberger’s Theorem: n Assume that X = {x1, ..., xk} and Y = {y1, ..., ym} are ﬁnite sets in R , with k + m ≥ n + 2, and all the points x1, ..., xk, y1, ..., ym are distinct. Assume that for any subset S ⊂ X ∪ Y comprised of n + 2 points the convex hulls of the sets X ∩ S and Y ∩ S do not intersect. Then the convex hulls of X and Y also do not intersect.

Hint: Assume, on contrary, that the convex hulls of X and Y intersect, so that

k m X X λixi = µjyj i=1 j=1 P P for certain nonnegative λi, i λi = 1, and certain nonnegative µj, j µj = 1, and look at the expression of this type with the minimum possible total number of nonzero coeﬃcients

λi, µj.

1)grams

2 Assignment 2: General Theorem on Alternative

General Theorem on Alternative has several particular cases (in fact equivalent to the GTA). The goal of the subsequent exercises is to become acquainted with these particular cases.

Exercise 5 Derive from GTA the following Gordan’s Theorem on Alternative Let A be an m×n matrix. One of the inequality systems

(I) Ax < 0, x ∈ Rn,

(II) AT y = 0, 0 6= y ≥ 0, y ∈ Rm, has a solution if and only if the other one has no solutions.

Exercise 6 Derive from GTA the following Inhomogeneous Farkas Lemma A linear inequality

aT x ≤ p (1) is a consequence of a solvable system of linear inequalities

Nx ≤ q (2) if and only if it is a ”linear consequence” of the system and the trivial inequality

0T x ≤ 1, i.e., if it can be obtained by taking weighted sum, with nonnegative coeﬃcients, of the inequalities from the system and this trivial inequality. Algebraically: (1) is a consequence of solvable system (2) if and only if

a = N T ν for some nonnegative vector ν such that

νT q ≤ p.

Exercise 7 Derive from GTA the following Motzkin’s Theorem on Alternative The system

Sx < 0, Nx ≤ 0 has no solutions if and only if the system

ST σ + N T ν = 0, σ ≥ 0, ν ≥ 0, σ 6= 0 has a solution.

Exercise 8 Which of the following systems of linear inequalities with 2 unknowns have, and which have no solutions (for the systems which are solvable, point out a solution; for the unsolv- able systems, explain why they are so):

3  x + y ≥ 2  • 2x − y ≥ 1  −5x + y ≥ −5  x + y ≥ 2  • 2x − y ≥ 1  −5x + y ≥ −4  x + y ≥ 2  • 2x − y ≥ 1  −5x + y ≥ −3.5

Exercise 9 Consider the linear inequality

x + y ≤ 2 and the system of linear inequalities

x ≤ 1 −x ≤ −100

Our inequality clearly is a consequence of the system – it is satisﬁed at every solution to it (simply because there are no solutions to the system at all). According to the Inhomogeneous Farkas Lemma, the inequality should be a linear consequence of the system and the trivial inequality 0 ≤ 1, i.e., there should exist nonnegative ν1, ν2 such that 1 1 −1 = ν + ν , ν − 1000ν ≤ 2, 1 1 0 2 0 1 2 which clearly is not the case. What is the reason for the observed “contradiction”?

4 Assignment 3: Separation and Extreme Points

Exercise 10 Mark in the below list the cases when the linear form f T x separates the sets S and T

• S = {0} ⊂ R, T = {0} ⊂ R, f T x = x

• S = {0} ⊂ R, T = [0, 1] ⊂ R, f T x = x

• S = {0} ⊂ R, T = [−1, 1] ⊂ R, f T x = x

3 3 q 2 2 T • S = {x ∈ R : x1 = x2 = x3}, T = {x ∈ R : x3 ≥ x1 + x2}, f x = x1 − x2

3 3 q 2 2 T • S = {x ∈ R : x1 = x2 = x3}, T = {x ∈ R : x3 ≥ x1 + x2}, f x = x3 − x2

3 3 2 T • S = {x ∈ R : −1 ≤ x1 ≤ 1}, T = {x ∈ R : x1 ≥ 4}, f x = x1 2 2 2 T • S = {x ∈ R : x2 ≥ x1, x1 ≥ 0}, T = {x ∈ R : x2 = 0}, f x = −x2

Exercise 11 Consider the set    x1 + x2 + ... + x2004 ≥ 1     x1 + 2x2 + 3x3... + 2004x2004 ≥ 10   2004 2 2 2 2  M = x ∈ R : x1 + 2 x2 + 3 x3... + 2004 x2004 ≥ 10    ......   2002 2002 2002 2002   x1 + 2 x2 + 3 x3 + ... + 2004 x2004 ≥ 10 

Is it possible to separate this set from the set {x1 = x2 = ... = x2004 ≤ 0}? If yes, what could be a separating plane?

Exercise 12 Let M be a convex set in Rn and x be an extreme point of M. Prove that if

m X x = λixi i=1 is a representation of x as a convex combination of points xi ∈ M with positive weights λi, then x = x1 = ... = xm.

Exercise 13 Let M be a closed convex set in Rn and x¯ be a point of M. Prove that if there exists a linear form aT x such that x¯ is the unique maximizer of the form on M, then x¯ is an extreme point of M.

Exercise 14 Find all extreme points of the set

2 {x ∈ R | −x1 + 2x2 ≤ 8, 2x1 + x2 ≤ 9, 3x1 − x2 ≤ 6, x1, x2 ≥ 0}.

Exercise 15 Mark with ”y” those of the below statements which are true:

• If M is a nonempty convex set in Rn which does not contain lines, then M has an extreme point

• If M is a convex set in Rn which has an extreme point, then M does not contain lines

5 • If M is a nonempty closed convex set in Rn which does not contain lines, then M has an extreme point

• If M is a closed convex set in Rn which has an extreme point, then M does not contain lines

• If M is a nonempty bounded convex set in Rn, then M is the convex hull of Ext(M)

• If M is a nonempty closed and bounded convex set in Rn, then M is the convex hull of Ext(M)

• If M is a nonempty closed convex set in Rn which is equal to the convex hull of Ext(M), then M is bounded

Exercise 16 A n × n matrix π is called double stochastic, if all its entries are nonnegative, and the sums of entries in every row and every column are equal to 1, as it is the case with the unit matrix or, more generally, with a permutation matrix – the one which has exactly one nonzero entry (equal to 1) in every column and every row, e.g.,

 0 1 0  π =  0 0 1  . 1 0 0

Double stochastic matrices of a given order n clearly form a nonempty bounded convex polyhedral set D in Rn×n. What are the extreme points of the set? The answer is given by the following

Theorem (Birkhoﬀ) The extreme points of the polytope D of n×n double stochastic matrices are exactly the permutation matrices of order n.

Try to prove the Theorem. Hint: Using the algebraic characterization of extreme points of a polyhedral set, verify that in an extreme point, there is a column with just one nonzero entry.

The Birkhoﬀ Theorem is the source of a number of important inequalities; some of these inequalities will be the subject of optional exercises in the coming hometasks.

6 Assignment 4: Convex Functions

Exercise 17 Mark by ”c” those of the following functions which are convex on the indicated domains: • f(x) ≡ 1 on R • f(x) = x on R • f(x) = |x| on R • f(x) = −|x| on R

• f(x) = −|x| on R+ = {x ≥ 0} • exp{x} on R • exp{x2} on R • exp{−x2} on R • exp{−x2} on {x | x ≥ 100}

Exercise 18 Prove that the following functions are convex on the indicated domains: x2 2 • y on {(x, y) ∈ R | y > 0} • ln(exp{x} + exp{y}) on the 2D plane.

Exercise 19 A function f deﬁned on a convex set Q is called log-convex on Q, if it takes real positive values on Q and the function ln f is convex on Q. Prove that • a log-convex on Q function is convex on Q • the sum (more generally, linear combination with positive coeﬃcients) of two log-convex functions on Q also is log-convex on the set. Hint: use the result of the previous Exercise + your knowledge on operations preserving convexity

1 n T Exercise 20 For n-dimensional vector x, let xb = (xb , ..., xb ) be the vector obtained from x T by rearranging the coordinates in the non-ascending order. E.g., with x = (2, 1, 3, 1) , xb = (3, 2, 1, 1)T . Let us fix k, 1 ≤ k ≤ n. k • Is the function xb (k-th largest entry in x) a convex function of x? 1 k • Is the function sk(x) = xb + ... + xb (the sum of k largest entries in x) convex? Exercise 21 Consider a Linear Programming program min{cT x : Ax ≤ b} x with m × n matrix A, and let x∗ be an optimal solution to the problem. It means that x∗ is a minimizer of differentiable convex function f(x) = cT x on convex set Q = {x | Ax ≤ b} and therefore, according to Proposition 2.5.1, ∇f(x∗) should belong to the normal cone of A at x∗ – this is the necessary and sufficient condition for optimality of x∗. What does this condition mean in terms of the data A, b, c?

7 Optional exercises:

Exercise 22 Let f(x) be a convex symmetric function of x ∈ Rn (symmetry means that f P remains unchanged when permuting the coordinates of the argument, as it is the case with xi, i or max xi). Prove that if π is a double stochastic n × n matrix (see Optional exercise from i Assignment 3), then f(πx) ≤ f(x) ∀x

Exercise 23 Let f(x) be a convex symmetric function on Rn. For a symmetric n × n matrix X, let λ1(X) ≥ λ2(X) ≥ ... ≥ λn(X) be the eigenvalues of X taken with their multiplicities and arranged in the nondecreasing order. Prove

1. For every orthogonal n × n matrix U and symmetric n × n matrix X,

f(diag(UXU T )) ≤ f(λ(X))

where diag(A) stands for the diagonal of square matrix;

2. The function F (X) = f(λ(X)) of symmetric n × n matrix X is convex.

Exercise 24 For a 10×10 symmetric matrix X, what is larger – the sum of two largest diagonal entries or the sum of two largest eigenvalues?

8 Assignment 5: Optimality Conditions in Convex Programming

Exercise 25 Find the minimizer of a linear function

f(x) = cT x on the set n n X p Vp = {x ∈ R | |xi| ≤ 1}; i=1 here p, 1 < p < ∞, is a parameter. What happens with the solution when the parameter becomes 0.5?

Exercise 26 Let a1, ..., an > 0, α, β > 0. Solve the optimization problem ( ) n a min X i : x > 0, X xβ ≤ 1 x xα i i=1 i i

Exercise 27 Consider the optimization problem n √ o max ξT x + τt + ln(t2 − xT x):(t, x) ∈ X = {t > xT x} x,t where ξ ∈ Rn, τ ∈ R are parameters. Is the problem convex2)? What is the domain in space of parameters where the problem is solvable? What is the optimal value? Is it convex in the parameters?

Exercise 28 Consider the optimization problem

max {ax + by + ln(ln y − x) + ln(y):(x, y) ∈ X = {y > exp{x}}} , x,y where a, b ∈ R are parameters. Is the problem convex? What is the domain in space of parameters where the problem is solvable? What is the optimal value? Is it convex in the parameters?

2)A maximization problem with objective f(·) and certain constraints and domain is called convex if the equivalent minimization problem with the objective (−f) and the original constraints and domain is convex.

9 Assignment 6: Optimality Conditions

Exercise 29 Consider the problem of minimizing the linear form

f(x) = x2 + 0.1x1 on the 2D plane over the triangle with the vertices (1, 0), (0, 1), (0, 1/2) (draw the picture!). 1) Verify that the problem has unique optimal solution x∗ = (1, 0) 2) Verify that the problem can be posed as the LP program

min {x2 + 0.1x1 : x1 + x2 ≤ 1, x1 + 2x2 ≥ 1, x1, x2 ≥ 0} . x Prove that in this formulation of the problem the KKT necessary optimality condition is satisﬁed at x∗. What are the active at x∗ constraints? What are the corresponding Lagrange multipliers? 3) Verify that the problem can be posed as a nonlinear program with inequality constraints

min {x2 + 0.1x1 : x1 ≥ 0, x2 ≥ 0, (x1 + x2 − 1)(x1 + 2x2 − 1) ≤ 0} . x

Is the KKT Optimality condition satisﬁed at x∗?

Exercise 30 Consider the following elementary problem:

n 2 o min f(x1, x2) = x − x2 : h(x) ≡ x2 = 0 x 1 with the evident unique optimal solution (0, 0). Is the KKT condition satisﬁed at this solution? Rewrite the problem equivalently as

n 2 2 o min f(x1, x2) = x − x2 : h(x) ≡ x = 0 . x 1 2 What about the KKT condition in this equivalent problem?

Exercise 31 Consider an inequality constrained optimization problem

min {f(x): gi(x) ≤ 0, i = 1, ..., m} . x

Assume that x∗ is locally optimal solution, f, gi are continuously diﬀerentiable in a neighbourhood of x∗ and the constraints gi are concave in this neighbourhood. Prove that x∗ is locally optimal solution to the linearized problem

n T T o min f(x∗) + (x − x∗) ∇f(x∗):(x − x∗) ∇gj(x∗) = 0, j ∈ J(x∗) = {j : gj(x∗) = 0} . x

Is x∗ a KKT point of the problem?

Exercise 32 Let a1, ..., an be positive reals, and let 0 < s < r be two reals. Find maximum and minimum of the function n X r ai|xi| i=1 on the surface n X s |xi| = 1. i=1

10 Optional exercises:

Exercise 33 Recall S-Lemma:

If A, B are two symmetric n × n matrices such that

x¯T Bx¯ > 0

for certain x¯, then the implication

xT Bx ≥ 0 ⇒ xT Ax ≥ 0

holds true iﬀ there exists λ ≥ 0 such that A − λB 0

(P 0 means that P is symmetric positive semideﬁnite: P = P T and xT P x ≥ 0 for all x ∈ Rn). The proof given in Lecture on optimality conditions (see Transparencies) was incomplete – it was taken for granted that certain optimization problem has an optimal solution. Fill the gap in the proof or ﬁnd an alternative proof.

Exercise 34 Let A = AT be a symmetric n × n matrix, and let B,C be m × n matrices, C 6= 0. Then all matrices A + BT ∆C + CT ∆T B corresponding to m × m matrices ∆ with k ∆ k≤ 1 3) are positive semidefinite iff there exists λ ≥ 0 such that the matrix " # A − λCT C −BT −B λIm is positive semidefinite. This fact has important applications in Control and Optimization under uncertainty.

3) here k ∆ k is the standard matrix norm: k ∆ k= max {k ∆u k2:k u k2≤ 1}. The statement remains true u rP 2 when replacing the matrix norm with the Frobenius norm k ∆ kF ro= ∆ij . i,j