1. LP solvers 2. LP solver in Maple 3. GLPK - GNU linear programming kit 4. LP model representations 5. Branch and bound method for mixed Integer linear programming
CP412 ADAII Introduction # 1 1. LP solvers LP algorithms have been implemented in many packages https://en.wikipedia.org/wiki/Linear_programming
• Proprietary packages – Dedicated packages: CPLEX, free for academic use, LINDO – General math tool: Maple, MATLAB, Mathematica, – Plugin: Excel solver – Libraries for specific LP apps, or embedded applications
• Open source packages – CLP -- a LP solver from COIN-OR – GLPK -- GNU Linear Programming Kit,
CP412 ADAII Introduction # 2 2. LP solver in Maple • Use Optimization package • Features – LP solver, integer programming by branch and bound, missed LP program – None-linear programming – Support matrix format • Small scale, testing, education Example:
with(Optimization): LPSolve( 13*A+23*B, {0 <= A, 0 <= B, 4*A+4*B <= 160, 5*A+15*B <= 480, 35*A+20*B <= 1190}) CP412 ADAII Introduction # 3 3. GLPK
• GLPK is available free of charge under GLU license
• Features – Simplex method, dual simplex method, mixed integer linear programming, implemented in C – Include Java wrapper, plus many third-party wrappers for other languages – Can be compiled as stand-alone solver, or used as embedded LP solver – Provide library and APIs for application development. – Support major LP input file format – Interfaces to database
CP412 ADAII Introduction # 4 GLPK in Action
• Download – https://sourceforge.net/projects/winglpk • Build glpk library • Build embedded solver • Build stand alone solver: glpsol.exe glpsol --help glpsol --lp cplextest.lp -o result.txt glpsol --lp beer.lp -o result.txt glpsol --mps beer.mps -o result.txt --max glpsol -m beer.mod -o result.txt Demo • GLPK/Solution information https://en.wikibooks.org/wiki/GLPK/Solution_information
CP412 ADAII Introduction # 5 4. LP model representations
• Common demanding: – Need language to represent LP problems so that computer program can read in LP program – Need standards for LP problem language for portability – Need simple, efficient, flexible file format
• Commonly used formats – LP -- a simple format proposed by CPLEX – MPS -- Mathematical Programming System – AMPL -- like GNU MathProg modeling language
CP412 ADAII Introduction # 6 LP format
LP format is a simple and basic language for LP expression. It is directly based on LP math expression. LP format example: Maximize obj: 13 A + 23 B Subject To c1: 5 A + 15 B <= 480 c2: 4 A + 4 B <= 160 c3: 35 A + 20 B <= 1190 #integer A B Bounds 0 <= A 0 <= B End CP412 ADAII Introduction # 7 MPS
• MPS (Mathematical Programming System) is a file format for presenting and archiving linear programming (LP) and mixed integer programming problems.
• The format was named after an early IBM LP product and has emerged as a de facto standard ASCII medium among most of the commercial LP solvers. Essentially all commercial LP solvers accept this format, and it is also accepted by the open-source COIN- OR system.
• With the acceptance of algebraic modeling languages MPS usage has declined. For example, according to the NEOS server statistics in January 2011 less than 1% of submissions were in MPS form compared to 59.4% of AMPL and 29.7% of GAMS submissions.
CP412 ADAII Introduction # 8 MPS features
• MPS is column-oriented (as opposed to entering the model as equations), and all model components (variables, rows, etc.) receive names. MPS is an old format, so it is set up for punch cards:
• Fields start in column 2, 5, 15, 25, 40 and 50. Sections of an MPS file are marked by so-called header cards, which are distinguished by their starting in column 1.
• It is typical to use upper-case throughout the file for historical reasons, many MPS-readers will accept mixed-case for anything except the header cards, and some allow mixed-case anywhere.
• The names for the individual entities (constraints or variables) are not important to the solver; one should pick meaningful names, or easy names for a post-processing code to read.
CP412 ADAII Introduction # 9 MPS example NAME BEER ROWS N COST L CORN L HOPS L MALS COLUMNS A COST 13 CORN 5 A HOPS 4 MALS 35 B COST 23 CORN 15 B HOPS 4 MALS 20 RHS RHS1 CORN 480 RHS1 HOPS 160 RHS1 MALS 1190 BOUNDS LO BND1 A 0 LO BND1 B 0 ENDATA CP412 ADAII Introduction # 10 AMPL -- A Mathematical Programming Language
• GNU MathProg is a modeling language intended for describing linear mathematical programming models.
• A MathProg application consists of two parts: model and data, they be in the same file or in separate file.\ – Model descriptions written in the GNU MathProg language consist of a set of statements – data blocks constructed by the user from the language.
• In a process called translation, a program called the model translator analyzes the model description and translates it into internal data structures, which may be then used either for generating mathematical programming problem instance or directly by a program called the solver to obtain numeric solution of the problem.
CP412 ADAII Introduction # 11 Advantage
One advantage of AMPL is the similarity of its syntax to the mathematical notation of optimization problems.
This allows for a very concise and readable definition of problems in the domain of optimization.
Many modern solvers available on the NEOS Server (formerly hosted at the Argonne National Laboratory, currently hosted at the University of Wisconsin, Madison) accept AMPL input.
According to the NEOS statistics AMPL is the most popular format for representing mathematical programming problems.
CP412 ADAII Introduction # 12 AMPL features • a mix of declarative and imperative programming styles. Formulating optimization models occurs via declarative language elements such as sets, scalar and multidimensional parameters, decision variables, objectives and constraints, which allow for concise description of most problems in the domain of mathematical optimization.
• Procedures and control flow statements are available in AMPL for the exchange of data with external data sources such as spreadsheets, databases, XML and text files
• data pre- and post-processing tasks around optimization models the construction of hybrid algorithms for problem types for which no direct efficient solvers are available.
• To support re-use and simplify construction of large-scale optimization problems, AMPL allows separation of model and data.
CP412 ADAII Introduction # 13 AMPL features • AMPL supports a wide range of problem types, among them: – Linear programming – Quadratic programming – Nonlinear programming – Mixed-integer programming – Mixed-integer quadratic programming with or without convex quadratic constraints – Mixed-integer nonlinear programming – Second-order cone programming – Global optimization – Semidefinite programming problems with bilinear matrix inequalities – Complementarity theory problems (MPECs) in discrete or continuous variables – Constraint programming
• AMPL invokes a solver in a separate process which has these advantages: – User can interrupt the solution process at any time – Solver errors do not affect the interpreter CP412 ADAII Introduction # 14 AMPL example set BEERS; set CROPS; var quantity_produced{j in BEERS}, >=0; param selling_price{j in BEERS}; param crops_available{i in CROPS}; param crops_needed{i in CROPS, j in BEERS}; maximize revenue: sum{j in BEERS} selling_price[j]*quantity_produced[j]; s.t. enough_crop{i in CROPS}: sum{j in BEERS} crops_needed[i,j]*quantity_produced[j] <= crops_available[i]; data; set BEERS := A B; set CROPS := corn hops malt; param selling_price := A 13 B 23; param crops_available := corn 480 hops 160 malt 1190; param crops_needed : A B := corn 5 15 hops 4 4 malt 35 20; end;
CP412 ADAII Introduction # 15 5. Branch-and-Bound Search for MILP Mixed Integer Linear Programming (MILP) are LP problems with integer constraints on all or some variables.
Example: max 7y1 + 4y2 + 19y3 s.t. y1 + y2 ≤ 2 y2 + y3 ≤ 1 y1, y2, y3 are in {0, 1}
General method: 1. enumerate all possible combination of discrete/integer variable values. 2. For each of the such combinations, the integer variables have fixed values, solve the corresponding partial (LP) problem. 3. Search the optimal over all above partial problems.
Introduction # 16 n i
Issue: the number of fixed combinations can be exponential, e.g x ,..., : 1,..., 10 , 1,..., ,| | 10 n ︶ ︷ ︸ ︸ ︷︵ 1 x x
The number of combinations fixed discretei variable values can be infinite. n This is the issue of general Discrete Optimization Problem: maximize f(x) : s.t. x is in a discrete set of vectors in Rn
Can we do better?
How can we reduce the number partial problems to search for.
Introduction # 17 Branch-and-Bound Search 1. Construct a sequence of partial problems related in a tree structure. Each node represent a partial problem, a child node is a partial problem derived by assigning a fixed value for a free integer variable (not assigned a fixed value yet). The root is the original problem. The leaves are partial problems with all integer variables have fixed value. The tree is called Branch-and-Bound tree 2. Start from root, use depth-first-search 3. At a node, try to solve the partial MILP problem at the node. If solved, then assign it to current solution, if it is better than the current solution or current solution is null. Go to parent node else if the node problem has an upper bound less than the current solution, go the parent node else go to next child
When a node is solved, no need to search sub-problems under the node If not solved, but if the node problem has an upper bound worse than the current solution, then no need to search the child nodes as all nodes are no better than the current node.
Introduction # 18 Branch-and-Bound tree
A partial solution has some discrete variables fixed, while other left free (denoted by #).
Example: y = (1,#, 0,#) is a partial solution with y1 = 1 and y3 = 0, while y2 and y4 are free; its completions are (1, 0, 0, 0), (1, 1, 0, 0), (1, 0, 0, 1), and (1, 1, 0, 1)
Introduction # 19 How to find upper bound of a partial problem
Try to solve its relaxation problem (such as treating integer variables as continuous variables). The optimal value of the relaxation problem provides an upper bound of the partial problem.
Constraint Relaxation: Optimization problem (R) is said to be a constraint relaxation of Optimization problem (P) if : 1. every feasible solution to (P) is also feasible in (R), and 2. (P) and (R) have the same objective function
Introduction # 20 Properties of constraint relaxation
Property 1: If the constraint relaxation problem (R) is not feasible, then original problem (P) is not feasible.
Property 2: If the constraint relaxation problem (R) has an optimal solution S, its objective value provides an upper bound for the original problem (P),
Property 3: If the constraint relaxation problem (R) has an optimal solution S, which is also a feasible solution of original problem (P), then S is an optimal solution of (P).
Introduction # 21 constraint relaxation for MILP
y 0, 1 0 y 1 i ︷ ︸
i
LP relaxations: LP relaxations of a MILP problem are formed by treating any discrete variables as continuous, while retaining all other constraints. Heuristics algorithm for MILP
• MILP Heuristics: 1. Solve the LP problem of LP relaxation of an MILP, derive an optimum solution X*. 2. Round up/down of integer variable of in X* to derive a feasible solution of MLIP X’*. 3. Output X’*, stop.
LP relaxations may produce optimal solutions that are easily “rounded” to good feasible solution for the corresponding MILP problem
CP412 ADAII Introduction # 23 Branch-and-Bound algorithm for MILP
Start from root node of B&B tree. Solve for the LP relaxation problem at a node, if 1. no feasible solution Action: Terminate by Infeasibility — The partial MILP problem is infeasible
2. all relaxed binary variables are integers at the optimum Action: terminate by completion — an optimum for the MILP problem is found. Update current solution, go up to parent node
3. some relaxed binary variables have fractional value at the optimum. If the optimal value is no better than the current solution, go to parent. Otherwise Action: Branch — choose one of the relaxed variables, create next child by fixing the next value of the variable, go to the child node CP412 ADAII Introduction # 24 Terminating Partial Solutions
1. The candidate problem has an Infeasible LP relaxation Action: Terminate by infeasibility — The candidate problem is itself infeasible
2. The candidate problem has a LP relaxation whose optimal value is no better than the current incumbent solution value Action: Terminate by value dominance — No feasible completion of the candidate model can improve on the incumbent
3. The candidate problem has a LP relaxation whose optimal solution with all relaxed binary variables equal to 0 or 1 Action 1: Terminate by Completion — This is an optimum for the candidate problem Action 2: Update incumbent (if applicable)
CP412 ADAII Introduction # 25 Terminating Branch-and-Bound Search
B&B search stops when every partial solution in the tree has been either branched or terminated
The final incumbent is a global optimum, if one exists
The model is infeasible, otherwise one might also decide the stop B&B search when sufficiently close to the optimum
CP412 ADAII Introduction # 26 Branch-and-Bound Heuristics
Heuristics for Branching variable selection:
Depth-first search selects an active partial solution with the most component fixed — i.e., one deepest in the search tree
Best-first search selects an active partial solution with best parent Bounds
Depth-forward best-back search selects a deepest active partial solution after branching a node, but one with best parent bound after a termination.
CP412 ADAII Introduction # 27 B&B example
Introduction # 28 CP412 ADAII Introduction # 29 No further solution better than round(21.65) = 21.
CP412 ADAII Introduction # 30 Branch-and-Bound in Parallel
Use parallel system or distributed system.
When branch, branching k children nodes, and create k processes, each solves partial problem at a node.
When update the incumbent, update the incumbent on all nodes
When terminate a node, terminate all its decedent nodes
CP412 ADAII Introduction # 31