sign in sign up
<<
Home , Branch and bound, Linear programming, Metaheuristic

Chapter 5.2 and 5.3 Sara Gestrelius Last time: Classification

Hybrid

Low-level High-level

Relay Teamwork Relay Teamwork LRH LTH HRH HTH 5.2 Combining metaheuristics with mathematical programming

Mathematical Programming Low-level replay Low-level teamwork High-level relay High-level teamwork Mathematical Programming

MP vs MH Enumerative Relaxation and decomposition methods Cutting Plane and Pricing algorithms Mathematical Programming Why?

Mathematical Programming: Exact but expensive in memory and time.

Metaheuristics: Not exact but manageable in memory and time.

(mainly if your problem is discrete and/or have nonlinear hard constraints/objective ) Mathematical Programming Enumerative algorithms : 1. Branch: Divide problem e.g. x=1 x=0 if x binary. 2. Bound: Keep track of upper bound (i.e. best solution found so far) and also lower bound for all nodes. 3. Prune: If lower bound > upper bound, don’t investigate that node further. Mathematical Programming Enumerative algorithms Branch and bound:

50

130 134 130 Mathematical Programming Enumerative algorithms Branch and bound:

50

130 134 130

143 130 Mathematical Programming Enumerative algorithms Branch and bound:

50

130 134 130

143 130 Mathematical Programming Enumerative algorithms Branch and bound:

50

130 134 130

143 130 140 143 Mathematical Programming Enumerative algorithms : Bellman’s Principle is a necessary condition: “The sub-policy of an optimal policy is in itself optimal with regard to the start and end states” Mathematical Programming Enumerative algorithms Dynamic Programming: 1. Divide the problem into stages N. For each stage there should be a number of states x. 2. Define the cost for the initial states in the initial stage. 3. Define a recursive relation for a state at stage k given states of previous stages. Mathematical Programming Enumerative algorithms

Dynamic Programming: Example: The . 1. Stages = items, states = how much weight capacity to use at this and all preceding stages, i.e. for this an all preceding items (0,1,2,3,4,5). 2. Cost of initial state, 3. Recursive relation: Item Weight Utility (u) (w) 1 2 65 2 3 80 3 1 30 Mathematical Programming Relaxation and decomposition methods

Linear programming relaxation: • Ignore the integrality of an IP. • Handy for finding lower bounds. Mathematical Programming Relaxation and decomposition methods

Lagrangian relaxation: • Move a constraint into the objective function (and weigh it with )

Mathematical Programming Relaxation and decomposition methods

Benders Decomposition: • A lot, but basically divide into master and sub- problem and use sub-problem to generate new constraints to master. Mathematical Programming Cutting plane and pricing algorithms

Cutting planes: Add constraints to relaxed IP to generate tighter IP relaxations.

http://mat.gsia.cmu.edu/orclass/integer/node14.html Mathematical Programming Cutting plane and pricing algorithms

Column generation (pricing): Master problem (without all variables) and sub-problem (that generates variables that are likely to be useful to the Master). Master

Duals New columns

Subproblem Low-level relay hybrid

S-metaheuristics in exact methods Exact methods in S-metaheuristics Low-level Relay Hybrid S-metaheuristics in exact methods

Branch and Bound: 1. Node selection strategy: Which node should we branch on next? How? Use problem knowledge and S-metaheuristics. 2. Upper bound generation: Use S-metaheuristics to make complete solution from partial at nodes.

Cutting planes and Pricing algorithms: 2. generation: Extract set of violated capacity constraints of the relaxed problem. 3. Generate new variables (columns): what it says. Low-level Relay Hybrid Exact methods in S-metaheuristics

1. Very large neighbourhoods: use mathematical programming to search them and find the best or improving solution in the (whole or subset) neighbourhood (B&B, DP, Network Flow algorithms, algorithms).

2. Lower Bounds: use in S-. May also use other information e.g. Lagrangian multipliers. Low-level Teamwork hybrid

P-metaheuristics in exact methods Exact methods in P-metaheuristics Low-level Teamwork Hybrid P-metaheuristics in exact methods

Branching • Which node to branch on? Which values to use in branching? Which node to solve next? • Genetic algorithms have been used for solving the node selection problem. Low-level Teamwork Hybrid P-metaheuristics in exact methods

Local Branching (sort of like local search in MIP). Local branching can be used for binary variables. Assumes s is a partial solution (of the binaries in the problem). Then let the optimization solve the problem for the k-opt neighbourhood of s, and if no better solution is found, solve it for the other branch (change k or more variables).

Low-level Teamwork Hybrid Exact methods in p-metaheuristics

Recombination, mutation: • Use an exact algorithm for combining parents into new children, mutation.

Exact decoding: • Use exact algorithms for generating a solution from the incomplete encoding used by the . Low-level Teamwork Hybrid Exact methods in p-metaheuristics

Exact search ingredients: • Lower bounds: Gilmore-Lawler lower bounds and dual variable values have been used in the construction phase of ant colony to solve the quadratic . Also, lower bounds have been used in EA for mutation/crossover, where solutions that exceed a given bound are deleted.

• Partial solutions: The partial solutions of Branch and Bound may be interesting initial solutions. High-level relay hybrid

Information provided by metaheuristics Information provided by exact methods High-level Relay Hybrid Information provided by metaheuristics

1. Good upper bound: Solve the problem using a and start from that heuristic. Helps you prune more.

2. Fixing variables: Partition variables into set X and Y, let metaheuristic fix variables in set X and solve for variables in Y using exact methods.

3. Domain reduction: Heuristically perform a domain reduction for the decision variables and solve exactly within the reduced domain. High-level Relay Hybrid Information provided by exact algorithms

1. Partial solutions: Complete with metaheuristic.

2. Problem reduction: Used -search to reduce scheduling problem size and then used with simplified objective function to fins .

3. Relaxed optimal solution and their duals: May be exploited by metaheuristics. High-level teamwork hybrid Parallel cooperation Specialist cooperation High-level Teamwork Hybrid Parallel Cooperation

Partial solution Branch and bound (new initial solution) S-metaheuristic algorithm Initial solution Non-explored nodes

Upper bound Best solution found Best solution (new upper bound)

E.g. combining Branch and Bound with . The SA algorithm send improved upper bounds to B&B, and any integer solution found by B&B is used as an alternative reheated solution. High-level teamwork hybrid Specialist cooperation

Branch and X Optimal solution S-metaheuristic for subproblems, lower bounds Subproblems, upper bound and so on Communication medium

Subpopulation of solutions Optimal solutions for relaxed LP solver problems and duals P-metaheuristic Partial solutions, and so on

E.g. local search used to generate new columns. 5.2 Combining metaheuristics with

Constraint Programming Hybrids Constraint Programming

CP vs MH Propagation and search Constraint programming Constraint programming vs. Metaheuristics

Constraint Programming: good at solving combinatorial optimization problems that are tightly constrained.

Metaheuristics: good at solving problems that are underconstrained. Constraint programming Search and propagate

Constraints: Constraint programs consists of a set of variables linked by a set of constraints, e.g. the all_different constraint. The problem is solved by iterating between:

1. Propagate: Remove all values from variable domains that can’t be in a feasible solution. There’s lot’s of code for this already.

2. Search: Tree search, partition problem into sub-problems by adding new constraints. You’ll probably need to write your own code for this. 1. Branch ordering? 2. Variable selection? Hybrids

Low-level relay hybrid Low-level teamwork hybrid High-leve relay hybrid High-level teamwork hybrid Hybrids Low-level relay hybrid

• Neighbourhoods with expensive testing of feasibility: CP is very good at feasibly! • Large neighbourhoods: Once again, CP is good at feasibility…? Hybrids Low-level teamwork hybrids Embedding metaheuristics into Constraint Programming: • Node improvement: Use metaheuristics to improve or repair the nodes of the search tree. • Discrepancy-based search algorithms: Generate near- greedy paths in a serch tree. • Branch-ordering: Use metaheuristic to decide which child node to investigate first. • Variable selection: Choose which branches to make. • Branching restriction: Filter the branches of the search tree. Hybrids Low-level teamwork hybrids Embedding constraint programming into metaheuristics: • Recombination, large neighbourhoods, lower bounds, partial solutions, decoders… Hybrids High-level relay hybrids

Information provided by metaheuristic: • Same as for when combining with MP.

Information provided by CP: • Same as for when combining with MP. Hybrids High-level teamwork hybrids

CP Best neighbour, optimal solutions for sub-problems, domain reductions, and so on

Communication Metaheuristic medium Sub-problems, upper bound, constraints and so on Optimal solutions for relaxed problems MP and duals, partial solutions, lower bounds and so on

Not that many. www.sics.se