TMA521/MMA511 Large-Scale Optimization Lecture 1 Introduction: simple/difficult problems, matroid problems Ann-Brith Str¨omberg Professor of Mathematical Optimization 2019–11–04 1 / 33 TMA521/MMA511 Optimization, project course ◮ Examiner, lecturer & course responsible Ann-Brith Str¨omberg (room L2087, [email protected]) ◮ Teaching assistant Edvin Ablad˚ (Fraunhofer-Chalmers, [email protected]) ◮ Teacher Michael Patriksson ◮ Schedule ◮ 13–14 lectures ◮ 2 workshops (mandatory presence; optional dates) ◮ 2 student presentations/oppositions (mandatory presence; optional dates) ◮ Linux computers booked 2–4 hours per week for project work ◮ Course homepage in Canvas ◮ Two projects, including written reports ◮ Lagrangian relaxation for a VLSI design problem (Matlab) ◮ Column generation applied to a real production scheduling problem (AMPL/Cplex, Matlab) 2 / 33 Requirements for passing the course and examination 1. Each student must take active part in two scheduled workshops, and in the two course projects 2. The findings of your projects are to be presented in the form of two written reports 3. Each group shall present one of the projects at a seminar, and act as opponents at another group’s presentation of the other project 4. Each student must take active part in all of these activities 5. Join a project group in Canvas (before Friday, November 8) 6. All handing in of programs and reports in Canvas 7. Project groups consist of two students 8. To be able to reach a mark higher than “pass” (i.e., 3 or G) you can take an oral exam, based on the lecture material of the course and on the projects. 3 / 33 Some practicalities about the workshops and presentations ◮ Each of the two workshop sessions will be split into 3–4 occasions (see detailed course plan) ◮ Each student: sign up for one occasion/workshop (w46 & w49) ◮ Each of the two presentation/opposition sessions will be split into 3–4 occasions (see detailed course plan) ◮ Each project group: sign up for one presentation occasion (w49 or w51) and one opposition occasion (the other week, i.e., w51 or w49) ◮ Join a project group in Canvas (not later than Thursday this week) ◮ The classrooms and hours may be slightly adjusted ◮ Doodles/choodles for signing up will be posted in Canvas 4 / 33 Some more practicalities ◮ # students on the course ≈ 3 times more than ever We are happy but a little stressed, so please be patient ... ◮ Linux lab rooms are booked for project work (see TimeEdit). Teachers will be present if/when needed—we will try to adapt during the course 5 / 33 Course evaluation ◮ Questionnaires will be sent out to all students registered for the course after the exam week ◮ The examiner calls the course representatives to the introductory, (recommended) middle, and final meetings Student representatives (Chalmers): ◮ Carl Blomgren (MPENM) [email protected] ◮ Frida Eriksson (MPENM) [email protected] ◮ Yinfan Hu (MPSCM) [email protected] ◮ Viktor Ingemarsson (TKMAS) [email protected] ◮ Tai-ting Lee (UTBYTE) [email protected] ◮ Any volountary GU student? 6 / 33 Course literature ◮ The compendium “Lagrangean duals and subgradient methods for linear and mixed-binary linear optimization”, Str¨omberg (2019) ◮ The book chapter “A primer in column generation”, Desrosiers and Lubbecke¨ (2005) ◮ Articles and book chapters (hand-outs/links on the course homepage) ◮ Lecture notes (published in Canvas) 7 / 33 Aim ◮ Purpose of the course: provide an overview of the most important principles for the efficient solution of practical large-scale optimization problems, from theoretical foundations via modelling to method implementations ◮ Starting with a series of lectures, the course work is then gradually concentrated on project work, in which the knowledge gained is applied to efficiently solve some relevant large-scale optimization problems 8 / 33 Learning outcome After completion of this course, the student should ◮ be able to independently analyze and suggest modelling and method principles for a variety of practical large-scale optimization problems, and ◮ have suffient knowledge to utilize these principles successfully in practice through the use of optimization software tools. 9 / 33 Content ◮ Most large-scale optimization problems have inherent structures that can be exploited to solve them efficiently ◮ The course deals with some principles through which large-scale optimization problems can be attacked ◮ Techniques: decomposition–coordination. Based on convexity theory, polyhedral theory, integer optimization and duality theory. ◮ Three practical moments: ◮ An exercise/workshop on modelling and solution of a design problem (Wednesday or Thursday, November 13–14) ◮ Two project assignments (including the second workshop, on December 2), where large-scale optimization problems are solved using theory and techniques presented during the lectures 10 / 33 Content, cont’d. ◮ Problems: Complexity, unimodularity, convexity, minimal spanning trees, knapsack problems, location problems, generalized assignment, travelling salesperson problems, network design, set covering problems ◮ Principles: Decomposition & coordination, restriction, projection, variable fixing, neighbourhood, Lagrange relaxation, subproblem, coordinating master problem ◮ Methods: Cutting planes, Lagrangian heuristics, column generation, Dantzig-Wolfe decomposition, Benders decomposition, local search, modern tree search methods 11 / 33 General topic: Turn difficult problems into sequences of simpler ones using decomposition and coordination Prerequisites ◮ Linear Optimization (Linear Programming, LP), [Mixed] Integer Linear Optimization ([M]ILP), Non-Linear Optimization (NLP), Decomposition methods covered ◮ Lagrangian relaxation of Mixed Binary Linear Optimization (MBLP) ◮ Subgradient optimization (for Lagrangean duals of LP/MBLP) ◮ Heuristics and Greedy Algorithms (for MBLP and matroids) ◮ Branch & Bound (for MBLP) ◮ Dantzig–Wolfe decomposition (for LP) ◮ Column generation (for LP and MBLP) ◮ Benders decomposition (for MILP, NLP) 12 / 33 Important properties of simple problems ◮ What we here call simple problems can be solved in polynomial time with respect to the problem size (i.e., numbers of variables, constraints, ...) ◮ For simple problems, there exist polynomial algorithms preferably with a small largest exponent ◮ E.g., sorting an array of n elements can be done in time proportional to at most (i.e., worst case performance) ◮ n2 operations (bubble sort) ◮ n log n operations (heapsort, quicksort) 13 / 33 Examples of simple problems ◮ Network flow problems (see Wolsey): ◮ Shortest paths ◮ Maximum flows ◮ Minimum cost (single-commodity) network flows ◮ Transportation and assignment problems ◮ Maximum cardinality/weight matchings ◮ Problems over simple matroids (see Lawler) ◮ E.g., spanning trees and forests ◮ Linear optimization problems 14 / 33 Polynomial time solvable problems 15 / 33 Example: Shortest path Find the shortest path from node 1 to node 7 d4=9 i = 2 5 d9=2 d =2 1 d3=1 d8=4 7 1 3 d6=2 d5=3 6 d10=8 d2=2 4 d7=5 di = length of edge i Shortest path from node 1 to node 7 d4=9 2 5 d9=2 d =2 1 d3=1 d8=4 7 1 3 d6=2 d5=3 6 d10=8 d2=2 4 d7=5 Total length: 12 16 / 33 Example: Maximum flow Find the maximum flow from node 1 to node 7 A maximum flow from node 1 to node 7 k4=4 x4=3 2 5 2 5 k9=2 x9=2 k =6 x =6 1 k3=4 1 x3=3 k8=1 7 x8=1 7 1 3 1 3 k6=2 k5=3 6 k10=6 x6=0 x5=3 6 x10=5 k2=2 x2=1 4 k7=1 4 x7=1 ki = flow capacity of arc i (minimum arc capacity: 0) xi = optimal flow through arc i (≥ 0). Total flow: 7 A minimum cut separating nodes 1 and 7 k4−x4=1 2 5 k9−x9=0 k −x =0 1 1 k3−x3=1 k8−x8=0 7 1 3 k6−x6=2 k −x =0 5 5 6 k10−x10=1 k2−x2=1 4 k7−x7=0 ki − xi = residual flow capacity on arc i. Total cut capacity: k9 + k8 + k5 + k7 = 7 17 / 33 Matroids and the greedy algorithm (Lawler) ◮ Greedy algorithm ◮ Create a “complete solution” by iteratively choosing the best alternative ◮ Never regret a previous choice ◮ Which problems can be solved using such a simple method? ◮ Problems whose feasible sets can be described by matroids 18 / 33 Matroids and independent sets ◮ Given a finite set E and a family F of subsets of E: If I ∈ F and I′ ⊆I imply I′ ∈ F, then the elements of F are called independent ◮ A matroid M = (E, F) is a structure in which E is a finite set of elements and F is a family of subsets of E, such that 1. ∅∈F and all proper subsets of a set I in F are in F 2. If Ip and Ip+1 are sets in F with |Ip| = p and |Ip+1| = p + 1, then ∃ an element e ∈ Ip+1 \ Ip such that Ip ∪{e}∈F ◮ Let M = (E, F) be a matroid and A⊆E: If I and I′ are maximal independent subsets of A, then |I| = |I′| 19 / 33 Example I: Matric matroids ◮ E = a set of column vectors in Rn ◮ F = the set of linearly independent subsets of vectors in E. 10210 ◮ Let n = 3 and E = [e1,..., e5]= 01100 32001 ◮ We have: ◮ {e1, e2, e3}∈F and {e2, e3}∈F but ◮ {e1, e2, e3, e5} 6∈ F and {e1, e4, e5} 6∈ F 20 / 33 Example II: Graphic matroids ◮ E = {e1, e2, e3, e4, e5, e6, e7} = the set of edges in an undirected graph ◮ F = the set of all cycle-free subsets of edges in E e2 e2 e2 e1 e3 e5 E = {e ,..., e } 1 7 e7 e7 e4 e4 e2 e6 e1 e3 e 5 e2 e2 e7 e6 e4 e1 e3 e1 e5 e7 e7 e4 ◮ {e1, e2, e4, e7} ∈ F, {e2, e4, e7} ∈ F, {e2, e3, e5} 6∈ F, {e1, e2, e3, e7} ∈ F, {e1, e4, e5, e6, e7} 6∈ F, {e2} ∈ F 21 / 33 Matroids and the greedy algorithm applied to Example II ◮ Let w(e) be the cost of element e ∈E.