Problem Set 7 Convex Optimization (Winter 2018)

Problem set 7 Convex Optimization (Winter 2018)

Due Friday, February 23rd at 5pm

• Upload both a pdf writeup of your work and a zip ﬁle containing any supplemental ﬁles to Canvas by the due date. Submissions without a non-zipped pdf writeup won’t be graded.

• Late homeworks will be penalized 25% per day and will not be accepted after the next Monday at 5pm unless you have made arrangements with us well before the due date. We strongly encourage you to typeset your solutions.

• You may collaborate with other students to complete this assignment, but all submitted work should be yours. When you go to write down your solutions, please do so on your own.

Description

In this assignment, you will experiment with the log-barrier method. Partiallly completed Python code is included with the assignment. Areas that you need to ﬁll in are marked with “TODO” comments. You should turn in an archive containing all of your source code, and a separate pdf document containing all plots and answers to underlined questions.

1 Log-Barrier Function

Most of the functions in the ﬁle “algorithms.py” are the same as in previous programming as- signments (but ﬁlled-in this time). The objective log barrier function in “algorithms.py” takes as parameters an objective function f0, a constraint function φ (which has the same calling convention as the objective function), a vector x, and a scaling parameter t, and returns the value, gradient and Hessian of the log-barrier objective:

f(x, t) = tf0(x) + φ(x)

1 1.1 Implementation

The function quadratic log barrier in “functions.py” is supposed to compute the value, gradient, and Hessian of: 1 X f(x, t) = t x>Qx + v>x − log (a x + b ) 2 i,: i i

This is the log-barrier objective function f(x, t) = tf0(x) + φ(x) for a quadratic objective f0, log-barrier function φ, and linear constraints Ax + b ≥ 0. Fill in this function.

1.2 Experiments

The function main quadratic in “functions.py” minimizes a quadratic objective subject to three linear constraints, with t = 3, (note: this is an extremely small value of t), and plots the iterates of gradient descent and Newtons method. What do you observe? Remember that you can zoom in on the plots! The function main quadratic2 in “functions.py” plots the number of iterations required by gradient 1 descent and Newton’s method for t between 8 and 2 (once more, these are extremely small values of t, but the performance of gradient descent should give you a clue as to why we didn’t use larger t). What do you observe?

2 Log Barrier Method

2.1 Implementation–Scalar Constraints

One of the most appealing properties of the log-barrier method is that, given oracle access to each constraint function fi and its ﬁrst and second derivatives, for constraints of the form fi(x) ≥ 0, we may calculate the value, gradient, and Hessian of the log-barrier function gi(x) = − log fi(x). The function objective scalar constraints in “algorithms.py” implements this. It takes as param- P P eters a list of constraint functions fi and a point x, and should return i gi(x), i ∇gi(x) and P 2 i ∇ gi(x). Each constraint function takes x as a parameter, and returns fi(x), ∇fi(x), and 2 ∇ fi(x). Fill in the function objective scalar constraints.

2 2.2 Implementation–Log Barrier

The function quadratic in “functions.py” implements a constraint function which returns the value, gradient, and Hessian of: 1 f (x) = x>Qx + v>x + c i 2 which will be used to implement constraints fi(x) ≥ 0 (in this case, we will need Q 0). The function log barrier in “algorithms.py” contains a mostly-complete implementation of the log-barrier method (Algorithm 11.1 in Boyd and Vandenberghe). This function takes (among other things) a list of constraint functions as a parameter, and constructs the log-barrier objective P f(x, t) = tf0(x) − i log fi(x) using objective log barrier and your ﬁlled-in version of objective scalar constraints. Fill in the function log barrier.

2.3 Experiments

The function main linear in “functions.py” optimizes a linear program:

min v>x x > s.t. ai x ≥ bi ∀i

The parameter µ controls how quickly the log-barrier scaling parameter t changes between inner iterations of Newton. The generated plot shows the total number of Newton iterations performed for a run of the log-barrier method for various values of µ (with t0 = 1). Before running the code, describe the dependence on µ which you expect to see here. Does the plot meet your expectations? What solution is found for this LP?