Column generation algorithms for nonlinear optimization, II: Numerical investigations Ricardo Garcia-Rodenas a, Angel Marinb'*, Michael Patriksson c,d a Universidad de Castilla La Mancha, Escuela Superior de Informatica, Paseo de la Universidad, 4, 13071 Ciudad Real, Spain b Universidad Politecnica de Madrid, Escuela Tecnica Superior de Ingenieros Aeronauticos, Plaza Cardenal Cisneros, 3, 28040 Madrid, Spain c Department of Mathematical Sciences, Chalmers University of Technology, SE-412 96 Goteborg, Sweden d Department of Mathematical Sciences, University of Gothenburg, SE-412 96 Goteborg, Sweden ABSTRACT Garcia et al. [1] present a class of column generation (CG) algorithms for nonlinear programs. Its main motivation from a theoretical viewpoint is that under some circumstances, finite convergence can be achieved, in much the same way as for the classic simplicial decomposition method; the main practical motivation is that within the class there are certain nonlinear column generation problems that can accelerate the convergence of a solution approach which generates a sequence of feasible points. This algorithm can, for example, accelerate simplicial decomposition schemes by making the subproblems nonlinear. This paper complements the theoretical study on the asymptotic and finite convergence of these methods given in [1] with an experimental study focused on their computational efficiency. Three types of numerical experiments are conducted. The first group of test problems has been designed to study the parameters involved in these methods. The second group has been designed to investigate the role and the computation of the prolongation of the generated columns to the relative boundary. The last one has been designed to carry out a more complete investigation of the difference in computational efficiency between linear and nonlinear column generation approaches. In order to carry out this investigation, we consider two types of test problems: the first one is the nonlinear, capacitated single-commodity network flow problem of which several large-scale instances with varied degrees of nonlinearity and total capacity are constructed and investigated, and the second one is a combined traffic assignment model. 1. Introduction problem and the origin-destination matrix problem have led to the appearance of methods based on sensitivity analysis [2]. These Analyzing and solving combined traffic models for large-scale new methods should solve many of these CDP(pC) with great networks of practical interest requires computationally efficient accuracy. This motivates an interest in accelerating classical solution methods for the following constrained optimization simplicial decomposition methods, which is the objective of this problem: work. For example, [3] analyses new ways of generating columns by exploiting the acyclicity of user equilibrium flows. minimize/(x), (CDP(f,X)) XEX We consider here the class of simplicial decomposition methods for this problem, as represented by the work in Holloway [4], von where XsB" is non-empty and convex, and f:Xt-^M is Hohenbalken [5], Hearn et al. [6,7], Ventura and Hearn [8], Larsson continuously differentiable on X. et al. [9], Patriksson [10], Rosas et al. [11], Bertsekas and Yu et al. The class of simplicial decomposition methods successfully [12]. There are two main characteristics of the methods in this class: solves network equilibrium models formulated as CDP(/,X). At (i) an approximation of the original problem is constructed and present, certain mathematical programming problems with solved, wherein the original feasible set is replaced by a compact equilibrium constraints, such as the capacity enhancement subset, which is an inner approximation, and is the so-called restricted problem, the congestion toll pricing problem, the signal setting master problem (RMP); and (ii) this inner approximation is improved by generating a new vector (column) in the feasible set through the solution of another approximation of the original problem wherein * Corresponding author. Tel.: +3491 3366323; fax:+3491 3366324. E-mail addresses: [email protected] (R. Garcia-Rodenas), the cost function is approximated. This stage is called the column [email protected] (A. Marin), [email protected] (M. Patriksson). generation problem (CGP). Concretely, we consider: algorithm. The RSDCC method of Ventura and Hearn [8] follows from letting Ac describe the Topkis-Veinott algorithm, of which only (1) The column generation problem is characterized by an one iteration is performed. Daneva et al. [17] provide a Frank-Wolfe iterative procedure. This iterative procedure is denoted by type method for the stochastic transportation problem, which k A and belongs to a finite collection Kc. The assumptions on includes a multidimensional search. Ak are associated with an algorithm that operates as a descent The set augmentation rules applied in the original work algorithm on the original problem. In order to rule out the on simplicial decomposition [4,5], as well as in the later uninteresting case that the original problem is solved by developments in Hearn et al. [7] can be used in this framework means of only using the column generation problem an as well. The role played by the extreme points is now assumed by infinite number of times starting from a given iterate x eX, we the column generated. Since the columns generated by the CG presume that the number of iterations performed from x is algorithm are not necessarily extreme points or even boundary finite. points, Larsson et al. [18] propose extending the column (2) The restricted master problem is assumed to be solved by the generated to the (relative) boundary. Under the realization of use of an iterative procedure, denoted A^ and belonging to a this operation the CG algorithm can be interpreted as a finite collection Kr. It operates on X cX being equal to the (restricted) simplicial decomposition method where the choice current inner approximation of X Also this algorithm will be of columns belonging to the set of extreme points of the feasible presumed to be applied a finite number of times; we can still region is enriched by the inclusion also of other points on the solve any given RMP exactly in this way, by making the proper boundary. choice of the procedure A$. While the CG method appears to have traditional, in particular primal, sub and master problems, one may derive through the CG In Table 1, we summarize the different steps of this CG framework also primal-dual types of methods. Among other algorithm. The algorithm described is conceptual, but important methods, Larsson et al. [13] derive the Dantzig-Wolfe method in algorithms are readily placed in its framework. The concept of the linear programming as a CG method, as well as a sequential CG method was first outlined and established convergent in quadratic programming algorithm with a side constrained master Larsson et al. [13]. Garcia et al. [1] establish the global problem. Daneva et al. [19] similarly derive a sequential linear (asymptotic) convergence for the CG algorithm under the above programming type method. k k assumptions on A c with k belonging to Kc and A with k Table 2 summarizes these rules, which constitute realizations belonging to Kr similar to those utilized in the convergence of Step 3 in the conceptual algorithm of Table 1. analysis in Zangwill [14]. Finite convergence results with respect (The corresponding necessary initialization step is not included.) to the optimal face and the optimal solution are established for weak sharp minima. The CG method offers a different viewpoint of the CGP from 2.2. Motivations previous simplicial decomposition methods. There, the CGP is seen as an approximation to the original problem which provides Garcia et al. [1 ] have theoretically analyzed the asymptotic and a new column by means of its (truncated) solution. The approach finite convergence of the CG algorithm, but this study is not taken in the CG algorithm is to construct profitable columns by sufficient from the viewpoint of applications. This paper is a roughly solving the original problem through the use of a limited complementary study which computationally addresses impor­ number of iterations of a convergent algorithm. Now, the tant questions about the performance of the CG algorithms in emphasis is placed on the algorithm used to generate a new terms of the elements that define them. column, and not on the definition of the approximation The following three main aspects of the CG algorithms are subproblem. studied in this paper. Previous simplicial decomposition methods, however, constitute Performance of the CG algorithms: The SD algorithm alternates instances of the CG algorithm. The classic simplicial decomposition between the solution of two subproblems, the CGP and the RMP, method discussed earlier can be described by considering problems in each iteration. The number of variables of the RMPs grows k where X is a polyhedral set, A c describes the Frank-Wolfe algorithm, when this algorithm progresses, because in each iteration one and only one iteration is performed. The nonlinear simplicial new extreme point (a new variable) is stored and is usually not decomposition (NSD) method of Larsson et al. [9], which is applied to the traffic assignment problem (see Sheffi [15], Patriksson [16]) Table 2 k can be described as an instance of the CG algorithm where A The set augmentation phase. realizes one iteration of a truncated Newton algorithm, and in 0 which the quadratic subproblems which appear in the Newton 0. (Initialization): choose an initial point x° eX, let t: = 0, T°5 = 0, Pj = {x }, type algorithm are approximately solved using the Frank-Wolfe P° = T°s u Pj and X° = conv(P°). Further, let r be a positive integer, and let {£',(^0 be a sequence of positive real numbers f 3.1 Column dropping rules): let x' — YT= I ftP>. where m — \T \ and pt e V' Table 1 3.1.a (Exact solution of RMP). Discard all elements pf with weight ft — 0 The conceptual algorithm.