Linear and Integer Programming, Duality and Examples
Total Page:16
File Type:pdf, Size:1020Kb
LP algorithmis: History Primal-Dual Integer Linear programming Examples Linear and Integer Programming, duality and examples Algorithmics GEI-FIB UPC Spring 2018 A-GEI Linear and Integer Programming, duality and examples LP algorithmis: History Primal-Dual Integer Linear programming Examples LP People Kantorovich (1912-1986) Dantzig (1914-2005) von Neumman (1903-1957) Khachiyan (1952-2005) Karmarkar (1957) A-GEI Linear and Integer Programming, duality and examples LP algorithmis: History Primal-Dual Integer Linear programming Examples History 1939, Leonid Kantorovich (1939): for use during World War II to plan expenditures and returns to reduce costs to the army and increase losses to the enemy 1947 (secret until), George B. Dantzig: simplex method John von Neumann: the theory of duality as a linear optimization solution 1979, Leonid Khachiyan: LP was shown to be solvable in polynomial time 1984, Narendra Karmarkar: new interior-point method for solving LP problems First example (Dantzig's): Find the best assignment of 70 people to 70 job A-GEI Linear and Integer Programming, duality and examples LP algorithmis: History Primal-Dual Integer Linear programming Examples History: Simplex algorithm Klee and Minty 1972: The simplex algorithm takes exponential time in the worst case Spielman and Teng 2001: when the inputs to the simplex algorithm are slightly randomly perturbed, the expected running time of the simplex algorithm is polynomial for any inputs. Kelner and Spielman 2006: A polynomial time randomized simplex algorithm that works on any inputs. A-GEI Linear and Integer Programming, duality and examples The best lower bound, for any x? max bT y s.t. AT y = c y ≥ 0 ∗ T if x opt, y Ax is a general But as we are maximizing linear combination of equa- this is equivalent to tions, if we can select y so that max bT y T ∗ T ∗ y Ax = c x , s.t. AT y ≤ c DUAL cT x∗ ≥ y T b y ≥ 0 LP algorithmis: History Primal-Dual Integer Linear programming Examples Searching for a lower bound: The best one? LP (PRIMAL) min cT x s.t. Ax ≥ b x ≥ 0 A-GEI Linear and Integer Programming, duality and examples The best lower bound, for any x? max bT y s.t. AT y = c y ≥ 0 But as we are maximizing this is equivalent to max bT y s.t. AT y ≤ c DUAL y ≥ 0 LP algorithmis: History Primal-Dual Integer Linear programming Examples Searching for a lower bound: The best one? LP (PRIMAL) min cT x s.t. Ax ≥ b x ≥ 0 if x∗ opt, y T Ax is a general linear combination of equa- tions, if we can select y so that y T Ax∗ = cT x∗, cT x∗ ≥ y T b A-GEI Linear and Integer Programming, duality and examples max bT y s.t. AT y = c y ≥ 0 But as we are maximizing this is equivalent to max bT y s.t. AT y ≤ c DUAL y ≥ 0 LP algorithmis: History Primal-Dual Integer Linear programming Examples Searching for a lower bound: The best one? The best lower bound, for LP (PRIMAL) any x? min cT x s.t. Ax ≥ b x ≥ 0 if x∗ opt, y T Ax is a general linear combination of equa- tions, if we can select y so that y T Ax∗ = cT x∗, cT x∗ ≥ y T b A-GEI Linear and Integer Programming, duality and examples But as we are maximizing this is equivalent to max bT y s.t. AT y ≤ c DUAL y ≥ 0 LP algorithmis: History Primal-Dual Integer Linear programming Examples Searching for a lower bound: The best one? The best lower bound, for LP (PRIMAL) any x? T min c x max bT y s.t. Ax ≥ b s.t. AT y = c x ≥ 0 y ≥ 0 if x∗ opt, y T Ax is a general linear combination of equa- tions, if we can select y so that y T Ax∗ = cT x∗, cT x∗ ≥ y T b A-GEI Linear and Integer Programming, duality and examples max bT y s.t. AT y ≤ c DUAL y ≥ 0 LP algorithmis: History Primal-Dual Integer Linear programming Examples Searching for a lower bound: The best one? The best lower bound, for LP (PRIMAL) any x? T min c x max bT y s.t. Ax ≥ b s.t. AT y = c x ≥ 0 y ≥ 0 ∗ T if x opt, y Ax is a general But as we are maximizing linear combination of equa- this is equivalent to tions, if we can select y so that y T Ax∗ = cT x∗, cT x∗ ≥ y T b A-GEI Linear and Integer Programming, duality and examples LP algorithmis: History Primal-Dual Integer Linear programming Examples Searching for a lower bound: The best one? The best lower bound, for LP (PRIMAL) any x? T min c x max bT y s.t. Ax ≥ b s.t. AT y = c x ≥ 0 y ≥ 0 ∗ T if x opt, y Ax is a general But as we are maximizing linear combination of equa- this is equivalent to tions, if we can select y so that max bT y T ∗ T ∗ y Ax = c x , s.t. AT y ≤ c DUAL cT x∗ ≥ y T b y ≥ 0 A-GEI Linear and Integer Programming, duality and examples Theorem (Strong duality) If the primal has an optimal solution x∗ then the dual has an optimal solution y ∗ such that cT x∗ = bT y ∗ There are additional conditions for a pair (x; y) of primal-dual feasible solutions allowing improvements on the cost of the objective function. Those ideas lead to the family of primal-dual algorithms. Jack R. Edmonds gave the first (sophisticated) application of the primal-dual paradigm in his work on maximum weight matchings in arbitrary graphs (1965). LP algorithmis: History Primal-Dual Integer Linear programming Examples Strong duality theorem Working from the dual trying to get the best lower bound we come back to the primal. A-GEI Linear and Integer Programming, duality and examples LP algorithmis: History Primal-Dual Integer Linear programming Examples Strong duality theorem Working from the dual trying to get the best lower bound we come back to the primal. Theorem (Strong duality) If the primal has an optimal solution x∗ then the dual has an optimal solution y ∗ such that cT x∗ = bT y ∗ There are additional conditions for a pair (x; y) of primal-dual feasible solutions allowing improvements on the cost of the objective function. Those ideas lead to the family of primal-dual algorithms. Jack R. Edmonds gave the first (sophisticated) application of the primal-dual paradigm in his work on maximum weight matchings in arbitrary graphs (1965). A-GEI Linear and Integer Programming, duality and examples LP algorithmis: History Primal-Dual Integer Linear programming Examples Integer Linear Programming The ILP problem is defined: Given matrix A (n × m) together with vectors b and c, solve min cT x subject to Ax ≥ b x 2 Z Big difference between LP and ILP: LP has polynomial time algorithms but ILP is NP-hard. A-GEI Linear and Integer Programming, duality and examples There are a myriad of solvers packages: GLPK: https://www.gnu.org/software/glpk/ LP-SOLVE: http://www3.cs.stonybrook.edu/ CPLEX: http://ampl.com/products/solvers/solvers-we-sell/cplex/ GUROBI Optimizer: http://www.gurobi.com/products/gurobi-optimizer LP algorithmis: History Primal-Dual Integer Linear programming Examples Solvers for LP Due to the importance of LP and ILP as models to solve optimization problem, there is a very active research going on to design new algorithms and heuristics to improve the running time for solving LP (algorithms) IPL (heuristics or approximation algorithms). A-GEI Linear and Integer Programming, duality and examples LP algorithmis: History Primal-Dual Integer Linear programming Examples Solvers for LP Due to the importance of LP and ILP as models to solve optimization problem, there is a very active research going on to design new algorithms and heuristics to improve the running time for solving LP (algorithms) IPL (heuristics or approximation algorithms). There are a myriad of solvers packages: GLPK: https://www.gnu.org/software/glpk/ LP-SOLVE: http://www3.cs.stonybrook.edu/ CPLEX: http://ampl.com/products/solvers/solvers-we-sell/cplex/ GUROBI Optimizer: http://www.gurobi.com/products/gurobi-optimizer A-GEI Linear and Integer Programming, duality and examples The problem is NP-hard. Can we formulate VC as an ILP problem? Variables: x1 ::: xn, xi = 1 if i 2 S, 0 otherwise. Pn Objective function: i=1 xi . Restrictions: for every edge( i; j) 2 E, xi + xj ≥ 1 xi 2 f0; 1g The ILP formulation can be computed in polytime. Vertex Cover LP algorithmis: History Prob 91 Primal-Dual Prob 89 Integer Linear programming Prob 90 Examples Prob 83 ILP for vertex cover vc Given a graph G = (V ; E), we want to find a set S, with minimum number of vertices, so that every edge in G has at least one end point in S. A-GEI Linear and Integer Programming, duality and examples Can we formulate VC as an ILP problem? Variables: x1 ::: xn, xi = 1 if i 2 S, 0 otherwise. Pn Objective function: i=1 xi . Restrictions: for every edge( i; j) 2 E, xi + xj ≥ 1 xi 2 f0; 1g The ILP formulation can be computed in polytime. Vertex Cover LP algorithmis: History Prob 91 Primal-Dual Prob 89 Integer Linear programming Prob 90 Examples Prob 83 ILP for vertex cover vc Given a graph G = (V ; E), we want to find a set S, with minimum number of vertices, so that every edge in G has at least one end point in S.