Improved Branch and Bound Algorithms for Integer

IMPROVED BRANCH AND BOUND ALGORITHMS FOR

INTEGER PROGRAMMING

Brian Borchers

A Thesis Submitted to the Graduate

Faculty of Rensselaer Polytechnic Institute

in Partial Fulllment of the

Requirements for the Degree of

DOCTOR OF PHILOSOPHY

Ma jor Sub ject Mathematical Sciences

Approved by the

Examining Committee

Professor John E Mitchell Thesis Adviser

Professer Joseph G Ecker Member

Professor George Hab etler Member

Professor Carlton E Lemke Member

Professor John Mittenthal Member

Rensselaer Polytechnic Institute

Troy New York

June

For Graduation August

IMPROVED BRANCH AND BOUND ALGORITHMS FOR

INTEGER PROGRAMMING

Brian Borchers

An Abstract of a Thesis Submitted to the Graduate

Faculty of Rensselaer Polytechnic Institute

in Partial Fulllment of the

Requirements for the Degree of

DOCTOR OF PHILOSOPHY

Ma jor Sub ject Mathematical Sciences

The original of the complete thesis is on le

in the Rensselaer Polytechnic Institute Library

Approved by the

Examining Committee

Professor John E Mitchell Thesis Adviser

Professer Joseph G Ecker Member

Professor George Hab etler Member

Professor Carlton E Lemke Member

Professor John Mittenthal Member

Rensselaer Polytechnic Institute

Troy New York

June

For Graduation August

CONTENTS

LIST OF TABLES iv

LIST OF FIGURES vii

ACKNOWLEDGEMENT ix

ABSTRACT x

Intro duction and Historical Review

The Branch and Bound Metho d

Interior Point Metho ds for Linear Programming

Using the DualAne Metho d in a Branch and Bound Co de for Mixed

Integer Linear Programming Problems

The DualAne Metho d

The DualAne Metho d for Problems with Upp er Bounds

Starting and Stopping the DualAne Metho d

Detecting Fractional Variables

The Branch and Bound Algorithm

Branching Strategies

Computational Details

Sample Problems

Computational Results

Using the PrimalDual Metho d in a Branch and Bound Co de for Mixed

Integer Linear Programming Problems

The PrimalDual Metho d

The PrimalDual Metho d and Complementary Slackness

Detecting Fractional Variables

Warm starting the PrimalDual Metho d

Implementing the PrimalDual Metho d

The Exp erimental Branch and Bound Co de

More Sample Problems ii

Computational Results

Solving Mixed Integer Nonlinear Programming Problems

Solving the Subproblems

Heuristics for Detecting Fractional Solutions

Generating Lower Bounds

The Branch and Bound Algorithm

Computational Results

Discussion and Conclusions

LITERATURE CITED

APPENDICES

A Benchmarks for the PrimalDual Co de

B A Small Network Design Problem iii

LIST OF TABLES

Table Sample problem statistics

Table OSL results for problem set one

Table Branch and b ound results for problem set one

Table Fixed order branch and b ound results for problem set one

Table OSL results for problem set two

Table Branch and b ound results for problem set two

Table Fixed order branch and b ound results for problem set two

Table OSL results for problem set three

Table Branch and b ound results for problem set three

Table Fixed order branch and b ound results for problem set three

Table OSL results for problem set four

Table Branch and b ound results for problem set four

Table Fixed order branch and b ound results for problem set four

Table Problem set one without early branching

Table Problem set two without early branching

Table Problem set three without early branching

Table Problem set four without early branching

Table Problem set one without warm start

Table Problem set two without warm start

Table Problem set three without warm start

Table Problem set four without warm start

Table Sample problem statistics

Table Simplex iterations p er second in EKKSSLV and EKKMSLV iv

Table OSL results for problem set one

Table Branch and b ound results for problem set one

Table Problem set one without early branching

Table Problem set one without warm start

Table OSL results for problem set two

Table Branch and b ound results for problem set two

Table Problem set two without early branching

Table Problem set two without warm start

Table OSL results for problem set three

Table Branch and b ound results for problem set three

Table Problem set three without early branching

Table Problem set three without warm start

Table OSL results for problem set four

Table Branch and b ound results for problem set four

Table Problem set four without early branching

Table Problem set four without warm start

T able OSL results for problem set ve

Table Branch and b ound results for problem set ve

Table Problem set ve without early branching

Table Problem set ve without warm start

Table OSL results for problem sets eight through ten

Table Branch and b ound results for problem sets eight through ten

Table Problem set eight through ten without early branching

Table Problems eight through ten without warm start

Table OSL results for problem set six

Table Branch and b ound results for problem set six v

Table OSL results for problem set seven

Table Branch and b ound results for problem set seven

Table Branch and b ound results for p erturb ed problem set six

Table OSL results for p erturb ed problem set six

Table Characteristics of the sample problems

Table Computational results for the co de without heuristics

Table Computational results for the co de with heuristics

Table A OSL simplex solutions of the LP relaxations

Table A OSL primaldual solutions of the LP relaxations

Table A Exp erimental primaldual solutions of the LP relaxations

Table A Exp erimental dualane solutions of the LP relaxations

Table B Flows b etween cities

Table B Distances b etween cities vi

LIST OF FIGURES

Figure A branch and b ound example

Figure The branch and b ound algorithm

Figure The revised branch and b ound algorithm

Figure Lower b ounds for a sample problem

Figure Estimates of x for a sample problem

Figure The branch and b ound algorithm

Figure Subproblems solved relative to OSL

Figure CPU time relativetoOSL

Figure Subproblems solved relative to OSL

Figure Iterations p er subproblem

Figure CPU time relativetoOSL

Figure The central tra jectory

Figure The branch and b ound algorithm

Figure Subproblems solved relative to OSL

Figure Iterations p er subproblem

Figure CPU time p er subproblem relativetoOSL

Figure CPU times relativetoOSL

Figure The branch and b ound algorithm for MINLP

Figure A LP results for problem sets one through ten

Figure A Problem b efore minimum degree ordering

Figure A Problem after minimum degree ordering

Figure A The Cholesky factors for problem

Figure A Problem b efore minimum degree ordering vii

Figure A Problem after minimum degree ordering

Figure A The Cholesky factors for problem

Figure A Problem b efore ordering

Figure A Problem after minimum degree ordering

Figure A The Cholesky factors for problem viii

ACKNOWLEDGEMENT

Iwould liketotake this opp ortunitytothankmy advisor John Mitchell for in

tro ducing me to the whole eld of interior p oint metho ds and sup ervising the work

that wentinto this thesis Iwould also like to thank the memb ers of the thesis

committeeJoeEcker George Hab etler Carlton E Lemke and John Mittenthal

My ancee Suzanne Blo o dgo o d provided moral supp ort and pro ofread the text of

the thesis for sp elling errors Our cat Dustmop kept me company during long nights

sp ent at the terminal

This researchwas partially supp orted by ONR Grantnumb er NJ

ABSTRACT

This thesis is an investigation into improved branch and b ound algorithms for linear

and nonlinear mixed integer programming Heuristics are develop ed for determining

when the solution to a subproblem will involve fractional variables When this

situation is detected the algorithm branches early creating new subproblems with

the fractional variables xed at integer values In this wayweavoid the work of

solving the current subproblem to optimality

Branch and b ound co des for mixed integer linear programming problems have

traditionally used the simplex metho d to solve the linear programming subprob

lems that arise Early detection of fractional variables is dicult with the simplex

metho d b ecause a variable might b ecome basic and thus fractional at the last it

eration of the simplex metho d In contrast with an interior p oint metho d variables

converge steadily to their optimal values This makes it relatively easy to detect

fractional variables

To b e useful in a branch and b ound algorithm an interior p oint metho d must

generate lower b ounds which are used to fathom subproblems and primal solution

estimates which the heuristics use to detect a fractional solution Most interior p oint

metho ds satisfy these requirements but we concentrate on the dualane metho d

and the primaldual path following metho d We describ e exp erimental co des based

on b oth metho ds Computational results are provided for a numb er of randomly

generated problems as well as for problems taken from the literature These results

indicate that the early branching idea is very eective but more work is needed

h and b ound co des comp etitive with simplex based to make the exp erimental branc

branch and b ound co des

The early branching idea is also used in a branch and b ound co de for mixed

integer nonlinear programs This co de uses the sequential quadratic programming x

metho d to solve the NLP subproblems The SQP metho d generates estimates of

the optimal solution at each iteration which the early branching heuristics use

Computational results are provided for a numb er of sample problems taken from

the literature On some problems the co de with early branching uses less CPU

time than the co de without early branching xi

CHAPTER

Intro duction and Historical Review

Mixed integer programming problems are optimization problems in which some of

the decision variables are restricted to integer values while other decision variables

may b e continuous The integer variables typically enco de choices from a small set

of options available to the decision maker In many cases these choices are binary

yes or no decisions Such decisions can b e enco ded byinteger variables which

are restricted to the values zero and one The ob jective function and constraints

can b e linear or nonlinear and nonlinear ob jective functions and constraints maybe

convex or nonconvex We will restrict our attention to zeroone mixed integer linear

programs and zeroone mixed integer nonlinear programs with convex ob jective

functions and constraints

Many optimization problems can b e formulated as mixed integer linear pro

grams with zeroone variables including facility lo cation problems scheduling prob

lems set covering and partitioning problems matching problems and linear ordering

problems Salkin and Mathur provide many examples of zeroone integer linear pro

gramming mo dels in their b o ok Some applications of zeroone mixed integer

nonlinear programs with convex constraints and ob jective functions include network

design chemical pro cess synthesis pro duct marketing and

capital budgeting

Thereareanumb er of approaches to solving mixed integer linear programming

problems including branch and b ound algorithms implicit enumeration algorithms

cutting plane algorithms Benders decomp osition and group theoretic approaches

Of these metho ds the most p opular general purp ose metho d is clearly branch and

y commerciallyavailable co des including MPSXMIP OSL Sciconic b ound Man

and Lindo implement branch and b ound algorithms The other approaches have

b een somewhat successful in the researchenvironment but are much less widely

used in practice Much less work has b een done on algorithms for mixed inte

ger nonlinear programs Approaches to solving these problems include branch and

b ound algorithms generalized Benders decomp osition and algorithms based on

outer approximation

The Branch and Bound Metho d

We will concentrate on branch and b ound approaches to mixed integer linear

and nonlinear programs We restrict our attention to branch and b ound algorithms

for zeroone mixed integer programs but the branch and b ound metho d can also b e

applied to problems with integer variables that are not restricted to the values zero

and one The metho d was rst describ ed by Land and Doig in Dakin

Beale and Small Drieb eck Tomlin and others improved the basic algorithm in a

number of ways There are manysurvey articles and textb o oks that

cover branch and b ound algorithms for mixed integer linear programs including

Pap ers that discuss branch and b ound algorithms for mixed integer

nonlinear programs include

The branch and b ound algorithm for zeroone mixed integer programming

works byenumerating p ossible combinations of the zeroone variables in a branch

and b ound tree Eachnodeofthetreeisacontinuous optimization problem based

on the original problem but with some of the integer variables xed at zero or one

The remaining zeroone variables are allowed to takeonanyvalue in the range

between zero and one The ro ot of the tree is the original problem with all of the

integrality constraints relaxed

At each iteration the algorithm selects a leaf no de from the tree and solves

the corresp onding subproblem There are four p ossible results If the subproblem is

infeasible then any further restriction of the subproblem w ould also b e infeasible

so we can ignore all descendents of the current subproblem If the optimal value

of the current subproblem is larger than the ob jectivevalue of a known integer

solution then no restriction of the current subproblem will give a b etter integer

solution and we can ignore all descendents of the current subproblem This pro cess

of deleting the current subproblem and all of its p otential children from the tree is

called fathoming If the optimal solution to the current subproblem satises all of

the integrality constraints and has a lower ob jectivevalue than any known integer

solution then we record the ob jectivevalue of this integer solution Finally if none

of the ab ove conditions are satised one of the zeroone variables is fractional at

optimalityWe create two new subproblems with the variable alternately xed at

zero and one The new subproblems are added to the branch and b ound tree as the

children of the current subproblem We then select the next subproblem to solve

The algorithm stops when there are no more unfathomed subproblems in the branch

and b ound tree

Figure shows the branch and b ound tree for the following sample problem

min x x

sub ject to x x

x x

x x f g

The algorithm b egins by solving the LP relaxation of this problem The op

timal solution is x x with optimal ob jectivevalue z Since x

is fractional at optimality the algorithm creates two new subproblems with x

and x The algorithm next solves the subproblem with x xed at zero The

solution to this subproblem is x x with ob jectivevalue z Again

the algorithm creates two new subproblems with x xed at zero and one The

algorithm attempts to solve the subproblem with x xed at zero and discovers that

the subproblem is infeasible It then solves the subproblem with x xed at This

x x

z z

Infeasible x

Figure Abranch and b ound example

subproblem has an integer optimal solution with x x and optimal ob

jectivevalue z The algorithm then solves the subproblem with x xed at one

and since this is larger This subproblem has an optimal ob jectivevalue of z

than the subproblem can b e fathomed by b ound Thus the optimal solution is

x x with ob jectivevalue z

Aowchart of the basic branch and b ound algorithm is shown in Figure

Although most of the steps in this algorithm are quite simple some of the steps

require additional explanation

In step two the algorithm picks the next subproblem There are many options

here but most branch and b ound algorithms rely on two basic strategies In depth

rst search the algorithm picks one of the most recently created subproblems This

results in subproblems with a large numb er of xed variables and these subproblems

Initialize BB

Start

Tree

Tree Empty

PickaNew

Subproblem

Solvethe

Subproblem

Infeasible or Yes Delete

Output Soln

lb ub Subproblem

End

Yes

Branch Fractional

Var

Integer Yes

ubminubob j

Soln

Figure The branch and b ound algorithm

are more likely to haveaninteger solution A second strategy aimed at improving

the lower b ound on the optimal ob jectivevalue as quickly as p ossible is to pick the

subproblem that has the lowest estimated ob jectivevalue There are many metho ds

for estimating the optimal ob jectivevalue of a subproblem including the p enalties

prop osed by Drieb eckandTomlin and pseudo costs describ ed in

In step three any metho d for solving the continuous subproblems can b e used

For mixed integer linear programs the simplex metho d has b een very eective but

we will use interior p ointmethodstosolve the subproblems For mixed integer

nonlinear programs any metho d for solving convex nonlinear programming problems

can b e used Gupta and Ravindran used the Generalized Reduced Gradient GRG

metho d Our exp erimental co des use the Sequential Quadratic Programming

SQP metho d

In step six wemay nd more than one fractional variable The algorithm

must decide whichvariable to branchon One p ossibilityistopickthevariable

which is closest to an integer value By branching on this variable and then solving

the subproblem in whichthevariable has b een rounded to the nearest integer value

we increase the chances of nding an integer solution Another solution is to branch

on the most fractional variable The resulting subproblems are likely to havesig

nicantly larger optimal ob jectivevalues than the parent subproblem Pseudo costs

and p enalties can also b e used to select the branching variable

umeration techniques are often used within a branch and b ound Implicit en

algorithm to decrease the size of the branch and b ound tree In implicit enu

meration the algorithm uses logical relationships implied by the linear constraints

to x variables at zero or one instead of explicitly enumerating combinations of the

zeroone variables in the branch and b ound tree For example if we are given the

constraint x x and x has b een xed at one then x must b e zero in any

integer feasible solution Implicit enumeration techniques can also b e used to derive

lower b ounds for subproblems These techniques are used in IBMs Optimization

Subroutine Library OSL

Singhal Marsten and Morin havedevelop ed a branch and b ound co de called

ZOOM in which the branching variables are selected b efore solving anyofthe

subproblems This xed order branchandboundscheme will b e discussed

further in Chapter

In practice iterative algorithms are used to solve the subproblems If an iter

ative algorithm computes lower b ounds during the solution of a subproblem these

lower b ounds can sometimes b e used to fathom a subproblem by b ound without

solving the subproblem to optimalityFor example if the dual simplex metho d is

used to solve an LP subproblem and a dual solution is obtained that has a higher

ob jectivevalue than a known integer solution then the current subproblem can b e

fathomed by b ound This metho d is used bya numb er of existing branch and b ound

co des

We will develop branch and b ound algorithms that use heuristics to detect

zeroone variables which app ear to b e converging to fractional values If the algo

rithm detects such a fractional variable then it can branch early without solving

the current subproblem to optimalityAowchart for the revised branch and b ound

algorithm is shown in Figure After each iteration of the algorithm to solve the

subproblem wecheck to see if we can fathom the current subproblem by b ound or

if there is a zeroone variable that app ears to b e converging to a fractional value In

the latter case the algorithm branches without solving the subproblem to optimality

If these heuristics err in declaring a variable fractional when it actually has an

integer value at optimality then the algorithm will waste time on the additional sub

problems that are created However the algorithm will eventually nd an optimal

integer solution since all p ossible integer solutions will still b e enumerated

Initialize BB

Start

Tree

Tree Empty Pick a New

Subproblem

Do an Iteration

of the Solution

Algorithm

Infeasible or Delete Yes

Output Soln

lb ub Subproblem

End

Yes

Branch Fractional

Var

Integer Yes

ubminubob j

Soln

Figure The revised branch and b ound algorithm

There are two other p otential problems The algorithm might branch early on

a subproblem which is actually infeasible Any further restrictions of the subproblem

will also b e infeasible and the algorithm will eventually discover that each of these

subproblems is infeasible The algorithm might also branch early when the optimal

value of the current subproblem is large enough to fathom the subproblem by b ound

Again the algorithm would waste time by doing this but it would ultimately fathom

all of the resulting subproblems

In order to eectively apply the early branching idea we need a metho d for

solving the subproblems whichgives go o d estimates of the optimal values of the zero

one variables well b efore an optimal solution has b een found For linear problems

the commonly used simplex metho d is not a go o d choice since a zeroone variable

could b ecome basic and thus fractional at the very last iteration of the simplex

algorithm Fortunatelyinterior p oint metho ds do give go o d estimates of the optimal

solution relatively early in the solution pro cess

Interior Point Metho ds for Linear Programming

Recentinterest in interior p oint metho ds dates back to when Narendra

Karmarkar published a description of a new p olynomial time algorithm for linear

programming Karmarkars work was of great practical and theoretical im

p ortance Although Khachian had shown in that the ellipsoid metho d can b e

used to solve linear programming problems in p olynomial time the ellipsoid metho d

was not successful in practice Karmarkars pro jective algorithm also had the de

sirable prop erty of p olynomial running time but more imp ortantlyversions of the

algorithm were eective in practice

Since the publication of Karmarkars pap er the eld of interior p oint metho ds

t for linear programming has grown quickly A recent bibliographyoninterior p oin

metho ds lists nearly a thousand pap ers There are a numb er of go o d survey

pap ers including Computational studies that indicate interior p oint

metho ds can outp erform the simplex metho d on various kinds of problems include

We will use twointerior p oint metho ds for linear programming The rst of

these is the ane rescaling metho d which Dikin describ ed originally in the s

Unfortunately the algorithm was not develop ed into a practical metho d for

solving linear programs at that time This metho d was indep endently rediscovered

in the s as a variant of Karmarkars algorithm Weuseadualversion

of the ane rescaling metho d describ ed in The second metho d that we

will use is the primaldual path following metho d This metho d is based on the

sequential unconstrained minimization technique of Fiacco and McCormick

Weuseaversion of the algorithm develop ed by Lustig Marsten and Shanno in

One reason that interior p oint metho ds had previously not b een successful

was the lack of good software for solving the systems of linear equations that arise

in interior p oint metho ds The matrices of these systems of equations often have

far more zero entries than nonzero entries and software can takeadvantage of this

bysaving only the nonzero entries in the matrix Ecient co des for solving these

sparse systems were rst develop ed in the late s Our interior p oint co des make

use of routines for solving sparse systems of equations from the Yale Sparse Matrix

Package YSMP and IBMs extended scientic subroutine library ESSL

The b o oks by George and Liu and Du Erisman and Reid describ e techniques for

solving sparse systems of equations

Figure shows the sequence of dual ob jectivevalues generated by the dual

ane metho d in solving a sample problem Note that the lower b ounds given

by these dual ob jectivevalues are very go o d after only a few iterations while

iterations are needed to solve the problem to optimality Figure shows the

2000

1500

1000 Lower Bound

500

0 0 5 10 15 20 25 30

Iterations

Figure Lower b ounds for a sample problem

sequence of estimates of x for the same problem Its clear after ab out iterations

that x will have a fractional value at optimality In this example the dualane

metho d quickly pro duces useful lower b ounds and estimates of the optimal values

of the zeroone variables As we shall see in Chapters and these prop erties of

interior p oint metho ds are a very imp ortant factor in the p erformance of branch

and b ound co des based on interior p oint metho ds

Throughout the thesis wemake use of the following notation Upp er case

letters suchasA are used to denote matrices A refers to the jth column of the

matrix ALower case letters are used to representvectors and scalars We will use

DI AGx to denote the diagonal matrix whose diagonal elements are the elements

of the vector xFrequentlywe will asso ciate a vector suchas w with a diagonal

matrix suchasW DI AGw We sometimes use b old face typ e to distinguish

vectors from scalars We denote the jth elementofa vector v by v Sup erscripts j

1.4

1.2

0.8

0.6 Estimate of X(9)

0.4

0.2

0 0 5 10 15 20 25 30

Iterations

Figure Estimates of x for a sample problem

are used to denote the iteration count within algorithms Thus x would b e used

for the primal solution at the kth iteration of an algorithm We use jj x jj to denote

the norm of the vector xWe will also need the norm denoted by jj x jj

The remainder of this thesis is organized as follows In Chapter we describ e

a branch and b ound co de for mixed integer linear programs that uses the dual

ane metho d to solve its LP subproblems Computational results for four sets of

sample problems are presented In Chapter we describ e a branchandboundcode

for mixed integer linear programming problems that uses the primaldual interior

p oint metho d Computational results for the four sets of sample problems from

Chapter and from three new sets of sample problems are given In Chapter we

describ e a branch and b ound co de for convex mixed integer nonlinear programs and

present computational results from a numb er of sample problems Our conclusions

are presented in Chapter

CHAPTER

Using the DualAne Metho d in a Branch and Bound Co de for Mixed

Integer Linear Programming Problems

In this chapter we describ e two exp erimental branch and b ound co des that use

the dualane metho d to solve the LP subproblems These co des use the early

branching idea in an attempt to improve p erformance

The remainder of this chapter is divided into sections In sections and

we describ e the interior p oint metho d that we use to solve the subproblems We

develop pro cedures for warm starting the dual ane metho d and detecting fractional

variables in sections and The branch and b ound algorithm is describ ed in

section Two strategies for selecting subproblems to b e solved are describ ed in

section In section we describ e a numb er of computational details that have

aected the p erformance of the exp erimental co des In section we describ e four

sets of sample problems Our computational results are presented in section

The DualAne Metho d

In this section we present the dualane interior p oint metho d for solving

linear programs The metho d presented here is describ ed in

We b egin with a pair of primal and dual problems

min c x

sub ject to Ax b

and

max b y

sub ject to A y z c

We will assume that the constraint matrix A has full row rank Consequentlyin

any feasible solution y is determined by

y AA Ac Az

The dualane metho d b egins with an initial dual feasible solution y z

with z Ateach iteration the algorithm rescales the problem so that z moves

to ethevector of all ones In this rescaled problem the algorithm moves in a

direction that retains feasibility and improves the ob jective function This direction

is obtained by pro jecting the gradient of the ob jective function onto the null space

of the constraints The algorithm moves almost as far as p ossible in this direction

without losing nonnegativityof z This pro cess is rep eated until a convergence

criterion is met

k k

At the k th iteration of the algorithm wehave a dual solution y z Let

Z DI AGz Then let

z Z z

A AZ

and

c Z c

Now the scaled problem is

max b y

sub ject to A y z c

Herez e is the current solution and

y AA Ac Az

We can rewrite the scaled problem as

T T

max b AA Ac Az

T T

sub ject to A AA Ac Azz c

We can ignore the constant term in the ob jective function and rewrite the equality

constraints to get

T T

Az max b AA

T T T T

Ac sub ject to I A AA Az I A AA

Let

T T

dz A AA b

This is the gradient of the ob jective function of the scaled problem The direction

dz is already in the null space of the constraints since

T T T T

AA AA b I A AA

In the original unscaled problem this direction is equivalentto

T T

dz A AZ A b

In order to retain dual feasibilitywemust have dz A dy so

dy AZ A b

T T T T

Note that since b dy b AZ A b b dy is always p ositive Thus the algorithm

shows monotonic improvementateachstep

To determine the maximum p ossible step length wesolve the onevariable LP

max

sub ject to z dz d

k k

We can now compute z and y using

k k

z z dz

and

k k

y y dy

The factor of ensures that z will remain strictly p ositive In practice it is b etter

to compute

k k

y y dy

and

k T k

z c A y

This helps to ensure that the new solution will b e dual feasible

To nd primal solutions well solve the following least squares problem at each

iteration of the algorithm

min x z

i i

sub ject to Ax b

Any solution to this problem satises the primal constraints Ax bIfwe assume

nondegeneracy when z is dual optimal the complementary slackness conditions

ensure that we will nd exactly one solution to the least squares problem with

x z This will b e an optimal primal solution Unfortunatelywhen z is

i i

not optimal the solution to this least squares problem mayhave elements that are

less than zero Thus the solutions to this least square problem will not always b e

primal feasible

Let

x Z dz

We claim that x solves the least squares problem To see this rst multiply

A by x

k T

Ax AZ A dy

T T

AZ A AZ A b

Next compute the gradient of the ob jective function of the least squares problem

and pro ject it onto the null space of the constraints The gradientisZ xThe

pro jection onto the null space of the constraints is

T T k

p I A AA AZ x

T T

I A AA Adz

T T T

A AA AA dy I

Since the pro jection of the gradient is zero x Z dz is the optimal solution

to the least squares problem

An alternative pro cedure for nding primal solutions is describ ed in

It has the advantage of generating truly feasible primal solutions However this

metho d requires additional computations that increase the computational work p er

iteration Furthermore we sometimes found that the onevariable LP that o ccurs in

this metho d was unsolvable b ecause of numerical diculties So wehave continued

to use the simpler pro cedure for nding primal solutions In practice wehavehadno

trouble nding go o d primal solutions using the technique describ ed in this section

There have b een several pro ofs of convergence for primal and dual versions

of the ane scaling algorithm Unfortunately all of these

pro ofs are based on assumptions of nondegeneracy or restricted step lengths For

example the most recent pap er by Tsuchiya and Muramatsu shows that the

algorithm converges when the step length is limited to of the distance to the

b oundary of the feasible region This pap er also shows that the optimal primal and

dual solutions will b e p oints in the interior of the optimal faces This ensures that

the optimal solution will have strict complementarity The consequences of this will

b e discussed in section Although our algorithm do es not use a limited step

length the algorithm do es converge in practice

The DualAne Metho d for Problems with Upp er Bounds

The dualane metho d of the previous section has b een extended to problems

with upp er b ounds on the primal variables by Marsten Saltzman Shanno Pierce

and Ballintijn In this section we derive the dualane metho d for problems

with b ounded primal variables and discuss ways of obtaining primal solutions

We consider problems of the form

P min c x

sub ject to Ax b

x u

Again A is an m by n matrix c is an n by vector and b is a m by vector

For simplicitywe require that all the variables have explicit upp er b ounds This

restriction will b e relaxed in Chapter

The LP dual of this problem is

T T

D max b y u w

sub ject to A y w z c

w z

We will reformulate these problems so that they t into the framework of the

previous section Let

b c x y z

u s w w slack

and

I I

Our problems P and D are clearly equivalentto

P min c x

sub ject to Ax b

and the dual problem

D max b y

sub ject to A y z c

Note that the equality constraints in D imply that w w

slack

We can now apply the metho d of the previous section to problems P and

b can b e written as A D The equation dy AZ

AZ A AZ dy b

B C

Z A Z W dw u

Solving the ab ove system of equations we obtain

dw Z W Z A dy u

Substituting into the equations we get

T T

AZ A dy b AZ Z W Z A dy u

AZ Z Z W Z A dy b AZ Z W u

At this p oint we need two easy identities

Z Z W W Z W

and

Z Z Z W Z Z W

Let

D Z W

Then the equations for dy and dw b ecome

dy AD A b AW D u

and

dw W D A dy Z W D u

Finallywemust have A dy dw dz to maintain dual feasibilitysowe compute

dz dw A dy

Primal solutions can b e obtained by

x Z dz

In terms of x and sthisis

x Z dz

and

s W dw

However since wewant to ensure that x s u it is simpler to compute

s u x

The dualane metho d for problems with upp er b ounds on the primal vari

ables is summarized in the following algorithm

Algorithm

Given an initial dual feasible solution w y z with w z repeat the fol lowing

iteration until some convergence criterion has been met

Let W DI AGw and Z DI AGz

D Z W

dy AD A b AW D u

dw W D A dy Z W D u

dz dw A dy

x Z dz

Find the maximum step size that ensures z w Ifthestepsizeis

unbounded the primal problem is infeasible and the dual is unbounded

y y dy

w w dw

z c A y w

Starting and Stopping the DualAne Metho d

Because the primal problem has explicit upp er b ounds on all of its variables

we can use the following formula taken from to generate an initial dual solution

with w z Let

kck

y b

kA bk

y then let If c A

w z c A y

j j j j

Otherwise let

y c z w A

j j j

where is some p ositive constant The exp erimental co des use jj c jj n

Eachtimewe start work on a new subproblem in the branch and b ound al

gorithm we will need an initial dual feasible solution Since a child subproblem

is derived from its parent subproblem by xing a primal variable at zero or one

the last dual solution of the parent subproblem will automatically b e feasible for

the child subproblem However some elements of z and w in this solution maybe

very small The dualane metho d p erforms p o orly when it is started from a p oint

close to one of the b oundaries of the feasible set For this reason we center the

dual solution By simultaneously increasing w and z by equal amounts the dual

i i

solution is moved away from the b oundary of the set of dual solutions so that the

dualane metho d can pro ceed without diculty In our exp erimental co des we

tof w and z that is smaller than before add jj c jj n to each elemen

starting work on a new subproblem The choice of has consequences that will b e

discussed further in sections and

In step seven the dual problem is considered unbounded if dz and dw are

b oth nonnegative In practice this seldom happ ens Instead b ecause of numerical

diculties in the calculation of dz and dw some elements of dz and dw that should

b e p ositive but near zero are really slightly negative When this happ ens the

algorithm generates a sequence of dual solutions with steadily increasing ob jective

values Thus we stop the algorithm and declare the problem primal infeasible if no

optimal solution has b een found after iterations

Since the x iterates are not necessarily primal feasible they do not provide

upp er b ounds on the optimal ob jectivevalue Thus we cannot use the duality gap

to detect convergence In our dualane co de we use the convergence criterion

jv v j

n n

max jv j

where v is the dual ob jectivevalue at the nth iteration Our exp erimental co des

use the value This has worked well in practice although it provides no

absolute guarantee that the current dual solution is near optimal

Because the dual constraints involve w z it is p ossible to simultaneously

adjust w and z without losing dual feasibility By simultaneously decreasing w

i i i

and z we decrease u w and thus improve the dual ob jectivevalue Of course this

pro cess is limited by the constraints w andz This metho d is used in

the exp erimental co des to generate a go o d lower b ound after each iteration of the

dualane algorithm The lower b ound is given by

T T

w w lb maxlb b y u

wherew minw z

i i i

Detecting Fractional Variables

Tobranch earlywe need a heuristic for detecting zeroone variables which

will b e fractional at optimality This heuristic do es not need to b e accurate

but it should b e able to nd most fractional variables early in the pro cess of solving

an LP subproblem

Our heuristic is based on the complementary slackness conditions If x and

is strictly b etween its upp er y w z are optimal primal and dual solutions and x

and lower b ounds then w z Our heuristic simply lo oks for zeroone

i i

variables with w and z where is a small p ositive constant The choice of

i i

is highly dep endent on the scaling of the dual problem D Wemust cho ose an

which is smaller than any likely nonzero value of w or z Our exp erimental co des

i i

use jj c jj n This is typically small compared to the nonzero elements of

z and w It is also slightly smaller then the centering parameter sowecanbe

sure that z and w will start o with values ab ove the cuto for declaring a variable

i i

fractional

We assume that the optimal solution will have strict complementaritysothat

we will not have x z w This is a consequence of Tsuchiyas work

i i i

on global convergence of the ane scaling algorithm with restricted step length

Tsuchiyas work do es not apply to our algorithm b ecause wetake long steps How

ever wehave not encountered any problems with this assumption in practice

If there are several fractional variables at optimality the heuristic is unlikely

to detect all of them during the same iteration of the dualane metho d In order

to detect as many fractional variables as p ossible wewait until

jv v j

n n

max jv j

b efore searching for any fractional variables This also increases the chance that we

will fathom the current subproblem bylower b ound instead of branching to create

two new subproblems

In the two exp erimental co des a solution is declared integer and optimal when

successivevalues of the dual ob jective dier by less than one part in all

primal variables are within one part in of their upp er and lower b ounds

and the zeroone variables are within of zero or one On the other hand

if successivevalues of the dual ob jective dier by less than x is a zeroone

variable w and z are less than andx is b etween and then x is

i i i i

considered fractional and the co de will branch early

The Branch and Bound Algorithm

Aowchart of our branch and b ound algorithm is shown in gure This

algorithm is similar to the branch and b ound algorithm describ ed in the intro duc tion

Initialize BB

Start

Tree

Tree Empty Pick a New

Subproblem

DualAne

Step

Delete Infeasible or Yes

Output Soln

Subproblem lb ub

End

Yes

Branch Fractional

Var

Integer Yes

ubminubob j

Soln

Figure The branch and b ound algorithm

There are three imp ortantvariables in the owchart The variable lb holds

the b est known lower b ound on the ob jectivevalue of the current subproblem This

b ound comes from The variable ub holds the ob jectivevalue of the b est

known integer solution This is used in step four to fathom the current subproblem

if lb a lower b ound for the current subproblem is larger than ub The upp er b ound

ub is up dated in step nine each time a new integer solution is found The variable

obj holds the optimal ob jectivevalue of the current subproblem This is used in

step nine to up date ub

In step one we initialize the branch and b ound tree by creating a single no de

corresp onding to the LP relaxation of the problem At the same time we nd an

initial dual feasible solution using the pro cedure of section and set ub to

In step two a variety of strategies can b e used to pick the next subproblem

from the branch and b ound tree The two metho ds that wehave implementedin

our exp erimental co de are describ ed in section When wehave solved every

subproblem in the branch and b ound tree the branch and b ound pro cedure is

nished and we can output the solution in step ten

In step three we p erform a dualane step as describ ed in section In

order to do this we need a dual feasible solution The initial dual feasible solution

for the rst subproblem was calculated in step one Eachtimewe start work on a

new subproblem we can get a dual feasible solution from the parent subproblem

This solution is then centered as describ ed in section After the dualane step

we compute a new lower b ound on the ob jectivevalue of the current subproblem

In steps four six and eight wecheck for any of the conditions under which

we will stop work on the current subproblem If the lower b ound lb is larger than

ubthenwe can delete the current subproblem and ignore all of its children in step

ve In step six wecheck for zeroone variables that app ear to b e fractional at

optimalityIfwe detect a fractional variable then we can branch to create twonew

subproblems in step seven In step eight if we are at optimality and the current

solution is integral then we go on to step nine and compute a new upp er b ound

The heuristics for detecting a fractional solution may err in declaring a variable

fractional when it actually has an integer value at optimality or in declaring a

variable fractional when the subproblem is actually infeasible In either case the

algorithm will create two new subproblems instead of immediately fathoming the

current subproblem or nding an integer solution As noted in Chapter the

algorithm will eventually recover from either of these mistakes

Branching Strategies

Twoversions of this basic algorithm were co ded These two exp erimental

co des dier in the manner in whichtheycho ose branching variables and the next

subproblem to work on

The rst version of the co de uses a depth rst searchstrategytocho ose sub

problems until an integer solution has b een found During this early phase of the

algorithm the co de branches on the least fractional variable rst rounding to the

nearest integer value The goal of this phase of the algorithm is to nd an integer

solution The resulting upp er b ound can then b e used to fathom subproblems

Once an integer solution has b een found the algorithm enters a second phase

in whichitbranches on the most fractional variable The goal in this phase of the

algorithm is to prune the branch and b ound tree as quickly as p ossible During this

second phase of the algorithm the co de branches on the most fractional variable

In this phase the co de picks the remaining subproblem with the lowest estimated

ob jectivevalue These estimates are based on pseudo costs similar to those describ ed

in Pseudo costs are heuristic estimates of the lower b ound for a subproblem based on previous exp erience

Each time a subproblem is solved to optimalitywe record the dierence in

optimal ob jectivevalues b etween the subproblem and its parent A pseudo cost is

calculated for eachinteger variable byaveraging the increase in ob jectivevalue that

o ccurs when that variable is xed at zero or one These pseudo costs are then added

to the optimal ob jectivevalue of a parent subproblem to get the estimated optimal

values of its children The estimated optimal values are used in cho osing the next

subproblem in step two of the branch and b ound algorithm As the branch and

b ound tree grows the pseudo costs should b ecome b etter estimates

The second version of the exp erimental co de uses a xed order branch and

b ound strategy In this strategy used by the ZOOM system the branching order

is determined b efore solving any subproblems As a consequence all subproblems

at the same level of the tree havethesamexedvariables Since xing a primal

variable corresp onds to deleting a dual constraint all subproblems at the same level

in the tree have the same dual feasible sets

After every fth dualane iteration we use the current dual solution to eval

uate the dual ob jective for each subproblem that is at the same level as the current

subproblem Sometimes a dual solution for the current subproblem is also a go o d

dual solution for another subproblem with the same xed variables If the solution

is go o d enough it mayeven eliminate the other subproblem from consideration

This is called an indirect hit by the develop ers of the ZOOM system

The xed order branchandboundversion of the co de uses a depth rst search

strategy until it has found an integer solution From then on it picks the subproblem

with the smallest lower b ound This lower b ound is obtained by simply evaluating

the dual ob jective function with the parent subproblems optimal solution If x is

T T

xed at zero then this is simply b y u w Ifx is xed at one then the lower

T T

b ound is b A y u w The xed order branch and b ound co de do es not make

j use of pseudo costs

The xed order branchandboundversion of the program uses twolevel branch

ing instead of single level branching That is instead of branching on one variable

to create two subproblems it branches on twovariables to create four subproblems

This corresp onds to the case K and L in In our exp erimental co de

the branching order is based on the absolute values of the cost co ecients of the

integer variables Those variables with the largest cost co ecients come rst in the

branching order

Computational Details

The algorithm maintains one large data structure the tree of subproblems

Each leaf no de in the tree is a record containing a list of variables that havebeen

xed at zero or one and the b est known dual solution for that subproblem This

dual solution consists of m n numb ers We store only y and w Thevalues of

z can b e easily calculated In contrast a simplex based branchandboundcode

would normally save only a list of the variables that had b een xed at zero or one

and a list of the variables in the current basis Thus our co des use somewhat more

storage than a simplex based co de

A simplex based co de must refactor the basis each time it switches to a new

subproblem As we shall see later this is quite costly The simplex based co de could

avoid this work by storing a factored basis for each subproblem However a factored

basis requires O m data storage which is p otentially far more exp ensive than the

O m n storage required by the exp erimental co des

The currentversions of the co des handle up to primal variables and

constraints so each record takes up over bytes of storage Thus the entire

tree can require many megabytes of storage

The most dicult part of the computation is calculating dy in the dualane

step As long as the constraint matrix A is sparse and has no dense columns the

matrix AD A should also b e sparse The exp erimental co des takeadvantage of this

sparsitybysaving the matrix in sparse form and making use of routines from the Yale

Sparse Matrix Package to factorize the matrix After reading in the problem the

structure of AD A is calculated and the matrix is reordered by the minimum degree

ordering routine of YSMP The reordered matrix is then symb olically factorized

During each iteration of the algorithm information gathered during the symbolic

factorization is used in the numerical factorization of the matrix The resulting

Cholesky factors are then used in the computation of dy

If the matrix A has dense columns whichwould completely ll in AD A they

are handled separately with an up date pro cedure based on the ShermanMorrison

Wo o dbury formula This formula is derived in Its application in an interior

p oint metho d is describ ed in Since none of our sample problems haveany dense

columns this pro cedure has not b een exercised

Execution time proling shows that muchoftheCPUtimeusedby the two

exp erimental co des is sp ent on factoring AD A and solving the resulting triangu

lar systems of equations Most of the remaining CPU time is sp entonmultiplying

AD A and computing dw dz and xFor problems with more complicated struc

ture solving for dy should take a larger share of the CPU time The most likely

source of p erformance improvem ents would b e b etter sparse matrix routines

Note that the matrix A must havefullrowranksothatAD A can b e factored

even after variables have b een removed from the problem This could b ecome a

problem if to o manyvariables were xed at zero or one The diculty could b e

resolved byremoving rows from the matrix A as they b ecome redundant However

the exp erimental co des havenoprovision for removing such redundantrows

One solution to this problem has b een prop osed in Whenever a zero pivot

is detected during the factorization of AD A the zero element is altered so as to

give the matrix full rank while setting the free variable to an arbitrary small value

Wehave implemented this solution in our dualane co de but this has not b een

extensively tested since none of our sample problems requires it

There is a fundamental numerical problem in the dualane scheme in that

the matrix AD A b ecomes badly scaled as the solution approaches optimalityIn

particular if x is a zeroone variable which is fractional at optimalitythen z and

i i

w must go to zero to satisfy complementary slackness Thus D can b ecome very

large Meanwhile if x is at either its upp er or lower b ound then w z remains

j j

relatively large and D will remain small When this happ ens D has some large

elements and some small elements This causes AD A to b e badly scaled This

b ecomes very troublesome if we do not center the dual solution at the start of each

subproblem since if w or z is small when we start work on a subproblem it can b e

i i

driven down to a very small value during the solution of the subproblem In some

alues of w and z as small as computational runs without centering we obtained v

i i

To deal with this problem we mo died the dualane algorithm slightly

At each iteration we compute the square ro ot of the sum of the squares of the

elements of D For eachelementin D that is smaller than times

this norm we x the element at zero This removes the tiny elements of D and

xes the illconditioning of the matrix AD A As a consequence dw is zero and

dz A dy Because of the way dw and dz are calculated this do es not result in

i i

is very small only when a loss of dual feasibility In terms of the primal solution D

x is at its upp er or lower b ound Thus our pro cedure corresp onds to xing primal

variables at their upp er or lower b ounds for an iteration when the complementary

slackness conditions sho w that the variables must b e nonbasic

If a subproblem is primal degenerate then it may happ en that D will have

fewer than m large entries as we approach optimality In this case AD A will

b ecome numerically singular This problem can b e handled by our mo died version

of YSMPwhich arbitrarily restores full rank to the matrix

As we noted earlier there is also a numerical problem in detecting subproblems

that are infeasible When the algorithm works on an infeasible subproblem it will

generate a sequence of dual solutions whose ob jectivevalues grow without b ound

but it is unlikely that the unb oundedness of the dual problem will b e detected

The exp erimental co des deal with this dicultyintwoways First once a feasible

integer solution has b een found infeasible subproblems will b e fathomed bybound

after only a few iterations Second if more than iterations are needed to solve

a subproblem that subproblem is abandoned as b eing primal infeasible and dual

unb ounded

Several changes were made to the basic algorithm to improve its p erformance

Primal solutions are not calculated in the dualane step unless the dual solution

is close to optimality In step ten of the dualane metho d we use the formula

z c A y w

only on every fth iteration and we use the simpler formula

z z dz

for all other iterations The rst formula ensures that the new dual solution satises

the constraints A y w z c but it takes more eort to compute than the

second formula In this way small violations of dual feasibility caused by the simpler

formula are corrected b efore they can b ecome large

Sample Problems

We develop ed four sets of sample problems for computational testing Each

sample problem is a capacitated facility lo cation problem with m facilities and n

customer lo cations Some of our problems are randomly generated while others are

taken from the literature The capacitated facility lo cation problem can b e written

m m n

X X X

min f x c y

i i ij ij

i i j

y Cap x for i m sub ject to

ij i i

y d for j n

ij j

x f g for all i

y d for all ij

ij j

y for all ij

is closed and if facility i is op en y is the ow Here x isiffacility i

ij i

from facility i to customer j Cap is the capacity of facility i f is the xed cost of

i i

op ening facility i c is the cost of shipping from facility i to customer j and d is

ij j

the customer j s total demand

In the randomly generated test problems f is uniformly distributed b etween

and d is uniformly distributed b etween and and c is uniformly

j ij

d m where r is distributed b etween and The Cap are given by r

j i i i

uniformly distributed b etween and

The rst three sets of problems are randomly generated problems Of the

randomly generated problems the problems in problem set one have facilities and

customers the problems in problem set twohave facilities and customers

and the problems in problem set three have facilities and customers

The fourth set of problems are a standard set of capacitated facilitylocation

problems taken from the literature These problems were originally develop ed as

Problem Set rows columns variables density

A AD A L

Table Sample problem statistics

uncapacitated problems by Kuehn and Hamburger in They used a table of

railroad distances b etween fty cities and the p opulations of the cities to generate

uncapacitated facility lo cation problems with up to warehouses and customers

Later the same data was used by Akinc and Khumawala to generate capacitated

problems in

Our fourth set of problems is the problem set IV in the pap er by Akinc and

Khumawala The problems are available through the ORLibrary These prob

lems have p otential warehouse lo cations and customer lo cations The capacity

of eachwarehouse is while the total demand is Thus the capacity con

straints are fairly tight The cost of op ening a warehouse is in problem one

in problem two in problem three and in problem four

These costs are quite small compared to the total shipping costs at optimality

Table gives statistics on the size and density of the problems in these four

problem sets Under densitywe give the density of the original constraint matrix

A the densityof AD A and the density of the Cholesky factor L Notethatin

problem sets one two and three there is only a small increase in densityduring

the factorization of AD A In problem set four the increase in density is more

signicant

Computational Results

In this section we describ e our computational results In addition to seeing

how fast our exp erimental co des are there are a numb er of other questions ab out the

p erformance of the exp erimental co des that wewould like to answer How eective

are the early branching heuristics How eective is our pro cedure for warmstarting

the dualane metho d Howmuch storage is used by the exp erimental co des In

order to test the eectiveness of our exp erimental co des we ran the exp erimental

co des and a program using IBMs Optimization Subroutine Library subroutines

on the four sets of problems describ ed in the previous section

All computations were p erformed on an IBM S under the AIX

op erating system Computations were done in double precision The exp erimen

tal co des were written in Fortran and compiled with the FORTRANVS version

compiler and VAST prepro cessor Full optimization and vectorization was

sp ecied with the ags VEC and OPT Times were measured with the CPUTIME

subroutine of the Fortran runtime library Measured CPU time do es vary byas

much as dep ending on system load and other random factors These CPU times

exclude the time required to read in the problems and to output solutions

For the OSL runs the EKKMSLV routine from release of OSL was used

Default settings were used for the OSL co de The optimal ob jective function val

ues and solutions from OSL were given to seven signicant digits Results for the

exp erimental co des are also given to seven digits although these co des were only

designed to give answers go o d to ab out six signicant digits

For each set of problems we rst solved the LP relaxation of the problems

using the OSL routine EKKSSLV and using the dualane metho d These results

are presented in App endix A We then solved the mixed integer linear programming

problems with the tw o exp erimental co des and with the OSL co de For b oth the

OSL and exp erimental runs wegive the CPU time numb er of iterations number

of subproblems solved and the optimal ob jectivevalue Detail results are shown in

Tables through

In all cases the exp erimental co des pro duced the same integer solutions as

the OSL routines The optimal ob jectivevalues generally matched to at least six

signicant digits and often matched to seven digits Additional accuracy could b e

achieved at the cost of slightly more iterations Recall that only the subproblems

that result in integer solutions need to b e solved to optimality

Figure shows the numb er of subproblems solved by the two exp erimental

co des relativetothenumb er of subproblems solved by OSL The exp erimental

xed order branch and b ound co de typically solved slightly fewer subproblems than

OSL while the exp erimental branch and b ound co de without xed order branching

typically solved more subproblems than OSL The go o d results for the xed order

branch and b ound co de reect the large numb er of subproblems that this co de was

able to fathom without actually solving The exp erimental co des app ear to have

b een eectiveincho osing branching variables and subproblems so as to minimize

the size of the branch and b ound tree

Figure shows the CPU time used bythetwo exp erimental co des to solve the

four sets of problems relative to the CPU time used by OSL The xed order branch

and b ound co de is consistently faster than OSL on the rst three sets of problems

while the exp erimental co de without xed order branching is faster than OSL on

problem sets one and three The relatively p o or p erformance of the exp erimental

ber branch and b ound co de on problem set two can b e explained by the large num

of subproblems that were solved The p o or p erformance of the exp erimental co des

on problem set four can largely b e explained by the diculty that the dualane

co de had in solving the LP relaxations of these problems The data in App endix A

show that OSLs simplex metho d was ab out three times faster than the dualane metho d in solving the LP relaxations of these problems

Branch and Bound.

Relative No. of Subproblems Fixed Order Branch and Bound 2

Set 1 Set 2Set 3 Set 4

Figure Subproblems solved relative to OSL

One interesting asp ect of the p erformance of OSL on these problems is that

OSL slows down in terms of the numb er of iterations p er second in the branch and

b ound algorithm For example OSL p erforms over iterations p er second in

solving the LP relaxations of the problems in problem set one but it p erforms fewer

than iterations p er second in solving the mixed integer programs A p ossible

explanation for this is the need to refactorize the basis each time OSL b egins work

on a new subproblem Another p ossible explanation is the time used in attempts to

simplify the problem by implicit enumeration

Another imp ortant measure of the p erformance of the exp erimental co des is

the amount of storage required to hold the branch and b ound tree The exp erimental

co des were compiled so that up to megabytes of storage was available for the

branch and b ound tree However only ab out megabytes were actually required

Branch and Bound.

Relative CPU time Fixed Order Branch and Bound 5 4.8

1.2 1

Set 1 Set 2Set 3 Set 4

Figure CPU time relative to OSL

by the largest problem For the xed order branch and b ound solution of the rst

problem in set two the branch and b ound tree had leaf no des at its largest

size Since each leaf no de required ab out bytes of storage the entire tree

required ab out megabytes of storage The exp erimental branch and b ound co de

needed a maximum of only megabytes of storage This happ ened on problem one

of problem set one where the branch and b ound co de required a maximum of

leaf no des In contrast the OSL co de used ab out megabytes of storage in direct

access les on most of the problems Since the exp erimental branch and b ound co de

without xed order branching was much more ecient in terms of storage than the

xed order co de wehave concentrated on the version of the co de without xed

order branching in the remainder of this section

Branch and Bound.

B&B without early branching. Relative No. of Subproblems B&B without warm start.

1.5

Set 1 Set 2 Set 3 Set 4

Figure Subproblems solved relative to OSL

To test the eectiveness of the early branching heuristics and our warmstart

pro cedure wedevelop ed twomoreversions of the exp erimental co de The rst of

these co des solves each subproblem to optimality instead of branching earlyWith

this co de we can determine if the early branching heuristics are causing the co de

to solve extra subproblems The second version of the co de uses the pro cedure

of section to nd a new dual solution for each subproblem instead of using

the warmstart pro cedure With this co de we can determine the eectiveness of

the warm start pro cedure Since the pro cedure for nding an initial dual solution

generates solutions with large w and z values we mo died the co de to searchfor

fractional variables only after successive dual ob jectivevalues are within of

each other This gives the co de time to drive the dual variables down to small

values so that fractional variables can b e detected Results for these new co des are

presented in Tables through

Figure shows the numb er of subproblems solved by the exp erimental branch

and b ound co des in problem sets one through four relativetothenumber of sub

problems solved by OSL All three co des generally solved slightly more subproblems

than OSL The exp erimental co de with early branching solved slightly more subprob

lems than the other two co des on most of the problem sets However on problem

set two the exp erimental co de with early branching was signicantly worse than

the other two exp erimental co des It app ears that the early branching heuristics

were p erforming p o orly on problem set two

Figure shows the average numb er of iterations p er subproblem for the

exp erimental branch and b ound co des on the four problem sets The exp erimental

co de with early branching and the warmstart pro cedure used fewer iterations p er

subproblem than the other two co des Of the exp erimental co des the co de without

the warmstart pro cedure had the worst p erformance These results show that the

early branching heuristics and the warmstart pro cedure are eective in reducing

the numb er of iterations p er subproblem

Figure shows the CPU time used by the three exp erimental co des relative

to the CPU time used by OSL Since the three versions of the exp erimental co de

solved roughly the same numb er of subproblems for each problem set it is reasonable

to exp ect that the numb er of iterations p er subproblem will have the largest eect

on the CPU times The exp erimental co de with early branching is consistently faster

than the other exp erimental co des even in cases where it solved more subproblems

This shows that the decreased numb er of iterations p er subproblem is very worth

while when compared to the slightly larger numb er of subproblems solved

Branch and Bound.

B&B without Early Branching.

Iters/Subproblem B&B without warm start.

Set 1 Set 2 Set 3 Set 4

Figure Iterations p er subproblem

Branch and Bound.

B&B without early branching.

CPU Time relative to OSL B&B without warm start.

Set 1 Set 2 Set 3 Set 4

Figure CPU time relative to OSL