Advances in Interior Point Methods and Column Generation

Advances in Interior Point
Methods and Column Generation

Pablo Gonza´lez Brevis

Doctor of Philosophy University of Edinburgh
2013

Declaration

I declare that this thesis was composed by myself and that the work contained therein is my own, except where explicitly stated otherwise in the text.

(Pablo Gonz´alez Brevis)

ii

To my endless inspiration and angels on earth: Paulina, Crist´obal and Esteban

iii

Abstract

In this thesis we study how to efficiently combine the column generation technique (CG) and interior point methods (IPMs) for solving the relaxation of a selection of integer programming problems. In order to obtain an efficient method a change in the column generation technique and a new reoptimization strategy for a primal-dual interior point method are proposed.
It is well-known that the standard column generation technique suffers from unstable behaviour due to the use of optimal dual solutions that are extreme points of the restricted master problem (RMP). This unstable behaviour slows down column generation so variations of the standard technique which rely on interior points of the dual feasible set of the RMP have been proposed in the literature. Among these techniques, there is the primal-dual column generation method (PDCGM) which relies on sub-optimal and well-centred dual solutions. This technique dynamically adjusts the column generation tolerance as the method approaches optimality. Also, it relies on the notion of the symmetric neighbourhood of the central path so sub-optimal and well-centred solutions are obtained. We provide a thorough theoretical analysis that guarantees the convergence of the primal-dual approach even though sub-optimal solutions are used in the course of the algorithm. Additionally, we present a comprehensive computational study of the solution of linear relaxed formulations obtained after applying the Dantzig-Wolfe decomposition principle to the cutting stock problem (CSP), the vehicle routing problem with time windows (VRPTW), and the capacitated lot sizing problem with setup times (CLSPST). We compare the performance of the PDCGM with the standard column generation method (SCGM) and the analytic centre cutting planning method (ACCPM). Overall, the PDCGM achieves the best performance when compared to the SCGM and the ACCPM when solving challenging instances from a column generation perspective. One important characteristic of this column generation strategy is that no specific tuning is necessary and the algorithm poses the same level of difficulty as standard column generation method. The natural stabilization available in the PDCGM due to the use of sub-optimal well-centred interior point solutions is a very attractive feature of this method. Moreover, the larger the instance, the better is the relative performance of the PDCGM in terms of column generation iterations and CPU time.
The second part of this thesis is concerned with the development of a new warmstarting strategy for the PDCGM. It is well known that taking advantage of the previously solved RMP could lead to important savings in solving the modified RMP. However, this is still an open question for applications arising in an integer optimization context and the PDCGM. Despite the current warmstarting strategy in the PDCGM working well in practice, it does not guarantee full feasibility restorations nor considers the quality of the warmstarted iterate after new columns are added. The main motivation of the design of the new warmstarting strategy presented in this thesis is to close this theoretical gap. Under suitable assumptions, the warmstarting procedure proposed in

iv this thesis restores primal and dual feasibilities after the addition of new columns in one step. The direction is determined so that the modiﬁcation of small components at a particular solution is not large. Additionally, the strategy enables control over the new duality gap by considering an expanded symmetric neighbourhood of the central path. As observed from our computational experiments solving CSP and VRPTW, one can conclude that the warmstarting strategies for the PDCGM are useful when dense columns are added to the RMP (CSP), since they consistently reduce the CPU time and also the number of iterations required to solve the RMPs on average. On the other hand, when sparse columns are added (VRPTW), the coldstart used by the interior point solver HOPDM becomes very eﬃcient so warmstarting does not make the task of solving the RMPs any easier.

v

Acknowledgements

I would like to thank Jacek Gondzio for being such a wonderful supervisor. I have learnt a lot from him and for that I am eternally grateful.
I would like to express my gratitude to my examiners, Julian Hall and John Mitchell for their insightful comments and suggestions about this work.
I would like to thank Universidad del Desarrollo, particularly the Research Council and the School of Engineering, for their financial support. Also, I am grateful to MIDEPLAN and CONICYT for granting me the Presidente de la Repu´blica Scholarship. I would like to thank the School of Mathematics at The University of Edinburgh for its financial support during the last six months of my studies.
I am also very grateful to everyone who supported me and my family during this
“adventure”. Specially, I am indebted to Nora Au and Pedro Silva for their constant support from Chile since the beginning.
I would also like to thank all my office mates, Bubacarr, Naiyuan, Kimonas, Robert,
Tim, Hanyi and Luciana. It was great sharing room 5620 with you guys. Special thanks go to Pedro, Robert and Tim for proofreading a first draft of this thesis.
I am also indebted to the staff and administrators at the School of Mathematics at The University of Edinburgh. Particularly, friends and colleges at the Edinburgh Research Group on Optimization (ERGO).
I would like to thank all baristas at the Starbucks store at Forrest Road. You always made my mornings (and some afternoons) with a delicious mocha.
In a more personal note, I am very grateful to friends I had the chance to meet while in Edinburgh. To Cristian, Loreto, Bryan, Lorena, Pedro, Héctor, Valeria and Milan, you are wonderful people!
I would like to thank my parents, Verónica and Enrique; my brothers and sister,
Claudio, Sebastián and Patricia; and my parents-in-law, Carolina and Julio, for their constant support from far away. You have always believed in me!
Last but not least, this thesis could not have been completed without the support of my family. To Cristóbal and Esteban for showing me the beauty on simple things. To Paulina, for her constant encouragement, endless patience and infinite love!

A todos, gracias totales!

vi

Contents

Abstract

v

1 Introduction

1.1 Linear programming problems . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Motivation and contributions . . . . . . . . . . . . . . . . . . . . . . . .

1

12

2 Primal-Dual Interior Point Methods

8

2.1 Neighbourhoods and path-following methods . . . . . . . . . . . . . . . 11 2.2 Convergence and complexity of the N_s(γ) neighbourhood . . . . . . . . 14
2.2.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2.2 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2.3 Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Column Generation and the Decomposition Principle

17

3.1 Dantzig-Wolfe decomposition . . . . . . . . . . . . . . . . . . . . . . . . 17 3.2 Lagrangian relaxation equivalence . . . . . . . . . . . . . . . . . . . . . 20 3.3 A graphical interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.4 Column generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.1 Adding columns to the RMP . . . . . . . . . . . . . . . . . . . . 26 3.4.2 Limitations of the standard version . . . . . . . . . . . . . . . . . 27
3.5 Stabilization strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4 A Selection of Problems: Formulation and Decomposition

32

4.1 Cutting stock problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.1.1 Dantzig-Wolfe decomposition . . . . . . . . . . . . . . . . . . . . 33
4.2 Vehicle routing problem with time windows . . . . . . . . . . . . . . . . 34
4.3 Capacitated lot-sizing problem with setup times . . . . . . . . . . . . . . 36

5 Primal-Dual Column Generation Method

39

5.1 Theoretical developments . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.2 Computational study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.2.1 Cutting stock problem . . . . . . . . . . . . . . . . . . . . . . . . 48 5.2.2 Vehicle routing problem with time windows . . . . . . . . . . . . 50 5.2.3 Capacitated lot-sizing problem with setup times . . . . . . . . . 54 5.2.4 Performance proﬁles for large instances . . . . . . . . . . . . . . 56 5.2.5 Additional comments about stabilized column generation . . . . 60

vii

6 A New Reoptimization Technique for the Primal-Dual Column Gen-

eration Method

62

6.1 Warmstarting strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 6.2 Theoretical developments . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.2.1 Technical conditions . . . . . . . . . . . . . . . . . . . . . . . . . 68 6.2.2 Dual feasibility restoration . . . . . . . . . . . . . . . . . . . . . 70 6.2.3 Primal feasibility restoration . . . . . . . . . . . . . . . . . . . . 73 6.2.4 Centrality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.2.5 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.3 Computational study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6.3.1 Cutting stock problem . . . . . . . . . . . . . . . . . . . . . . . . 82 6.3.2 Vehicle routing problem with time windows . . . . . . . . . . . . 86

7 Conclusions and Further Developments

91

7.1 Summary and concluding remarks . . . . . . . . . . . . . . . . . . . . . 91 7.2 Further studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

A Test problems

96

A.1 Cutting stock problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 A.2 Vehicle routing problem with time windows . . . . . . . . . . . . . . . . 100 A.3 Capacitated lot-sizing problem with setup times . . . . . . . . . . . . . . 102

viii

List of Tables

5.1 Average results on 262 instances of CSP for the SCGM, the PDCGM and the ACCPM adding one column at a time. . . . . . . . . . . . . . . 49
5.2 Average results on 262 instances of CSP for the SCGM, the PDCGM and the ACCPM adding up to k columns at a time. . . . . . . . . . . . 49
5.3 Results on 14 large instances of CSP for the SCGM, the PDCGM and the ACCPM adding up to 100 columns at a time. . . . . . . . . . . . . . 50
5.4 Average results on 87 instances of VRPTW for the SCGM, the PDCGM and the ACCPM adding one column at a time. . . . . . . . . . . . . . . 52
5.5 Average results on 87 instances of VRPTW for the SCGM, the PDCGM and the ACCPM adding up to k columns at a time. . . . . . . . . . . . 53
5.6 Results on 9 large instances of VRPTW for the SCGM, the PDCGM and the ACCPM adding 300 columns at a time. . . . . . . . . . . . . . . 54
5.7 Average results on 751 instances of CLSPST for the SCGM, the PDCGM and the ACCPM adding one column per subproblem at a time. . . . . . 55
5.8 Average results on 11 modiﬁed instances of CLSPST for the SCGM, the
PDCGM and the ACCPM adding one column per subproblem at a time. 55

6.1 Average results on 262 instances of CSP for PFR, CS and 1SPDR strategies adding k columns at a time: RMP iterations and times (RMP and total). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.2 Average results on 262 instances of CSP for PFR, CS and 1SPDR adding k columns at a time: column generation calls, time and inner iterations per RMP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.3 Average results on 87 instances of VRPTW for PFR, CS and 1SPDR adding k columns at a time: RMP iterations and times (RMP and total). 87
6.4 Average results on 87 instances of VRPTW for PFR, CS and 1SPDR adding k columns at a time: columns added, column generation calls and time per RMP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

A.1 CSP instance statistics - Set A . . . . . . . . . . . . . . . . . . . . . . . 96 A.2 CSP instance statistics - Set B . . . . . . . . . . . . . . . . . . . . . . . 100 A.3 VRPTW instance statistics - Set A . . . . . . . . . . . . . . . . . . . . . 101 A.4 VRPTW instance statistics - Set B . . . . . . . . . . . . . . . . . . . . . 101 A.5 CLSPST instance statistics - Set A . . . . . . . . . . . . . . . . . . . . . 102 A.6 CLSPST instance statistics - Set B . . . . . . . . . . . . . . . . . . . . . 107

ix

List of Figures

1.1 Solving the MP - Column generation ﬂow chart considering a warmstart-

ing procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4

2.1 Neighbourhoods of the central path . . . . . . . . . . . . . . . . . . . . . 13 3.1 Block diagonal structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2 Feasible solutions for the IP, DWD/LR and LP formulations . . . . . . 22 3.3 The standard column generation method behaviour . . . . . . . . . . . . 28

5.1 Schematic illustration of solutions provided by the SCGM (◦), the PDCGM
(ꢀ) and the ACCPM (ꢀ) strategies in the dual space . . . . . . . . . . . 40
5.2 Comparison between the standard column generation method and the primal-dual column generation method . . . . . . . . . . . . . . . . . . . 43
5.3 Performance profiles for CSP with the the SCGM, the PDCGM and the
ACCPM (large instances) . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.4 Performance profiles for VRPTW with the SCGM, the PDCGM and the
5.5 Performance profiles for CLSPST with the SCGM, the PDCGM and the

6.1 Differences between two consecutive RMPs - Changes in the central path and its neighbourhood . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
6.2 The one step primal-dual restoration strategy . . . . . . . . . . . . . . . 66 6.3 Performance profiles for CSP with PFR, CS and 1SPDR . . . . . . . . . 86 6.4 Performance profiles for VRPTW with PFR, CS and 1SPDR . . . . . . 89

x

Chapter 1

Introduction

In this chapter we give a brief introduction to linear programming and some solution techniques to solve this class of problems. We also state the scope and the main contributions of this thesis. At the end of the chapter, we present the outline of this document.

1.1 Linear programming problems

Linear programming or linear optimization is regarded as one of the most well-established ﬁelds in optimization due to its long history, understanding and wide range of solvers and methodologies available to tackle it. The linear programming problem is a special case of the general mathematical optimization problem in which an objective function f is minimized over a feasible set G ⊆ Rⁿwhich is described as

min{f(x) : x ∈ G},

(1.1)

where f : G → R [93, 101]. In its linear case, f(x) is a linear function and G is a convex feasible set deﬁned by linear (in)equalities. Linear programming problems can be found in a wide spectrum of real-life applications such as transportation, economics, energy and logistics among several others. More sophisticated applications are in the context of non-linear programming problems where in many cases the non-linearities are approximated by linear functions and therefore one ends up solving a sequence of linear programming problems. Also, very interesting applications are integer and combinatorial optimization problems in which linear continuous relaxations are commonly used to solve these problems [100].
Among the preferred methods to solve linear programming problems are the simplex

method and interior point methods.

Simplex method. This method was introduced by Dantzig [23] and aims to ﬁnd an optimal solution by using partitions of the variables into basic and nonbasic sets, where the variables in the former set may take non-negative values while the variables included in the latter set are set to zero. The simplex method starts from a vertex of the polytope describing the feasible set G and seeks a direction that improves the value of the objective function by checking the rate of change of the objective in the directions of the neighbouring vertices. Since only neighbouring vertices are considered, directions along the edges of this polytope are obtained. In this sense, the simplex method has a very limited vision of the complete polytope and may prove to be ineﬃcient for some

1problems, as shown by Klee and Minty in [81]. Although the theoretical worst case complexity bound on the possible number iterations is exponential in the dimension of the vector x, contemporary codes are stunningly eﬃcient and simplex is regarded as one of the major breakthrough in optimization [67, 68] and is the default choice of commercial/academic solvers such us CPLEX [71] and XPRESS [38].

Interior point methods A completely diﬀerent approach was proposed by Dikin [30] and then developed by Karmarkar [77]. In contrast to the simplex method in which every iterate is an extreme point of the feasible polytope G, these algorithms generate a sequence of iterates which traverse the relative interior of the feasible region. Several variants of interior point methods (IPMs) oﬀer polynomial complexity such as

the projective methods, path-following methods and aﬃne potential reduction methods

[66]. Among these variants, a family of path-following primal-dual methods has proven to be the most important and widely used [121]. Additionally, interior point methods also known as barrier methods due to the use of logarithmic barriers replacing nonnegative constraints, have been shown to be frequently more eﬃcient in general than simplex methods for solving large scale problems [87, 92].

Another powerful tool used in solving large scale linear programming problems, known as column generation, has become an important technique in past decades [28, 118]. Although this technique is not a solver itself as simplex or interior point methods are, column generation is an iterative algorithm that is commonly used in the context of integer programming to solve problems with a large number of variables. This technique was introduced by Dantzig and Wolfe in [26] when solving a linear programming problem with a large number of variables in a column-wise manner. This was shortly after Gilmore and Gomory [46, 47] proposed this technique to solve the cutting stock problem. This technique is an iterative procedure applied to solve a linear programming problem with a possibly huge number of variables, called the master problem (MP), such that the columns in the coeﬃcient matrix of this problem can be generated by following a known rule. By exploiting this characteristic, the column generation technique starts with a reduced version of the problem, called the restricted master problem (RMP), in which only few columns of the MP are considered at ﬁrst. Iteratively, new columns with negative reduced costs are generated and added to the RMP. These columns are given by one or more pricing subproblems. In general the method converges to an optimal solution by generating a relatively small subset of columns. Note that at termination the RMP is equivalent to the original problem (MP), and therefore, the optimal solution of the RMP is the optimal solution of the MP.
From a dual point of view, the dual of the RMP is a relaxation of the dual MP
(i.e., problem with fewer constraints). Hence, the process which generates the columns in the primal space (pricing subproblems) can be also interpreted as the process which generates cuts in the dual space (separation routine). This cut generation method applied to the dual is known as Kelley’s cutting plane method [78]. Due to this primaldual correspondence, column generation and cutting plane methods are equivalent.

1.2 Motivation and contributions

Although interior point methods, column generation and cutting plane methods have proven to be powerful techniques for solving large linear programming problems, few

2attempts have been made to combine them efficiently. Among these attempts one could mention [34, 49, 62, 89, 93, 96].
Despite the benefits showed by these methods for particular applications, somehow these efforts still have not been widely taken up by the column generation community and simplex-based methods are still the preferred techniques in this context.
In [86], one of the key papers in column generation, the following statement can be found:

Besides subgradients and simplex-based methods, the Lagrangian dual can be solved with more advanced (nonlinear) alternatives with stronger convergence properties. Among them are the bundle method [69] based on quadratic programming, and the analytic center cutting-plane method [53], an interior point solution approach. However, the performance of these alternatives is still to be evaluated in the context of integer programming.