A Primal-Dual Augmented Lagrangian Penalty-Interior-Point Filter Line Search Algorithm

Journal name manuscript No. (will be inserted by the editor) A Primal-Dual Augmented Lagrangian Penalty-Interior-Point Filter Line Search Algorithm Renke Kuhlmann · Christof Büskens Received: date / Accepted: date Abstract Interior-point methods have been shown to be very efficient for large-scale nonlinear programming. The combination with penalty methods increases their robustness due to the regularization of the constraints caused by the penalty term. In this paper a primal-dual penalty-interior-point algorithm is proposed, that is based on an augmented Lagrangian approach with an `2-exact penalty function. Global convergence is maintained by a combination of a merit function and a filter approach. Unlike other filter methods no separate feasibility restoration phase is required. The algorithm has been implemented within the solver WORHP to study different penalty and line search options and to compare its numerical performance to two other state-of-the- art nonlinear programming algorithms, the interior-point method IPOPT and the sequential quadratic programming method of WORHP. Keywords Nonlinear Programming · Constrained Optimization · Augmented Lagrangian · Penalty-Interior-Point Algorithm · Primal-Dual Method Mathematics Subject Classification (2000) 49M05 · 49M15 · 49M29 · 49M37 · 90C06 · 90C26 · 90C30 · 90C51 Renke Kuhlmann Optimization and Optimal Control, Center for Industrial Mathematics (ZeTeM), Biblio- thekstr. 5, 28359 Bremen, Germany E-mail: [email protected] Christof Büskens E-mail: [email protected] 2 Renke Kuhlmann, Christof Büskens 1 Introduction In this paper we consider the nonlinear optimization problem min f(x) n x2R s.t. c(x) = 0 (1.1) x ≥ 0 with twice continuously differentiable functions f : Rn ! R and c : Rn ! Rm, but the methods can easily be extended to the general case with l ≤ x ≤ g and c(x) ≤ 0 (cf. [45]). The widely used and very efficient interior-point strategy (cf. [6,21,34]) handles the inequality constraints by adding a barrier term to the objective function f(x) and solving a sequence of barrier problems n X (i) min 'µ(x) := f(x) − µ ln x x2 n R i=1 (1.2) s.t. c(x) = 0 with a decreasing barrier parameter µ > 0. In this paper, we consider an algorithm that penalizes both, the inequality box constraints and the nonlinear equality constraints c(x), by a log-barrier term and an augmented Lagrangian term, respectively. However, unlike other augmented Lagrangian methods we do not use a quadratic `2-norm as measure for the constraint violation, but an exact `2-penalty-interior-point algorithm (see Chen and Goldfarb [10,11,12]). The resulting unconstrained reformulation is n ! X (i) > min Φµ,λ,ρ,τ (x) := ρ f(x) − µ ln x + λ c(x) + τ kc(x)k (1.3) x 2 i=1 with penalty parameters ρ ≥ 0 and τ > 0, a barrier parameter µ ≥ 0 and Lagrangian multipliers λ 2 Rm. For improved readability the depen- dences of ρ, τ and λ are neglected when clear from the context and we write Φµ(x) := Φµ,λ,ρ,τ (x). The penalty parameter τ controls the size of the multipliers and will be updated until a certain threshold value is reached. The penalty parameter ρ balances the optimization of the Lagrangian function and the constraint violation of problem (1.2). In particular the algorithm solves a sequence of (1.3) with a decreasing penalty parameter ρ until finding a first-order optimal point of (1.2). However, unlike penalty-interior-point algorithms with a quadratic penalty function (e.g. Armand et al. [1], Armand and Omheni [2,3] or Yamashita and Yabe [47]) the penalty parameter ρ does not have to converge to zero. A first-order optimal point of (1.2) satisfying the Mangasarian- Fromovitz constraint qualification (MFCQ) is a stationary point of the merit function Φµ(x) if ρ is smaller than a certain threshold value or the duals of (1.3) equal ρλ. Using two penalty parameters is mainly motivated by a better accuracy of the implemented algorithm. Primal-Dual Augmented Lagrangian Penalty-Interior-Point Algorithm 3 It is an important feature of optimization algorithms to detect infeasibility of the given problem. In such a case a first-order optimal point of (1.2) does not exist and the penalty parameter ρ will converge to zero resulting in the optimization of min kc(x)k : (1.4) x≥0 2 The solution of (1.4) that is infeasible for (1.1) serves as a certificate of infeasibility. The presented algorithm follows the idea of Fletcher [18] and Byrd et al. [8] to place the penalty parameter in front of the objective function or the Lagrangian function, respectively, instead of in front of the measure of constraint violation for better solver performance for infeasible problems. The proposed algorithm shares the following properties with other primal- dual penalty-interior-point algorithms (e.g. [1,10,15]): The step is a guaranteed descent direction for the merit function Φµ(x) and a rank-deficient Jacobian of the constraints at infeasible non-stationary points can be handled without modification of the Newton system. The latter avoids failure of global convergence for example for the optimization problem in Wächter and Biegler [44]. An extension to the pure (quadratic) `2-penalty function are the augmented Lagrangian methods (e.g. [14,35]). Recently, primal-dual augmented Lagrangian methods have enjoyed an increased popularity. They have been studied by Armand and Omheni [2,3], Forsgren and Gill [20], Gertz and Gill [22], Gill and Robinson [23] and Goldfarb et al. [25]. These methods can re- move the perturbation of the KKT system caused by the penalty term by an appropriate update of the Lagrangian multipliers λ. This makes it unnecessary to calculate a further unperturbed step per iteration like in Chen and Goldfarb [11,12], and naturally leads to a quadratic rate of convergence to first-order optimal points of (1.2) and a superlinear rate in case of the nonlinear program (1.1). Our update of the Lagrangian multipliers λ differs from other augmented Lagrangian based algorithms (e.g. [2,3,13]), as it does not rely on a criterion that measures the reduction of the constraint violation. Instead, it is based on the dual information and is designed to be applied as often as possible when approaching the optimal solution. For step acceptance, instead of following recent research trends to avoid penalties and a filter like in Liu and Yuan [33] or Gould and Toint [30], we combine the two { the merit function and the filter mechanism { as line search criteria, of which at least one has to indicate progress for a trial iterate. Com- parable combinations have been proposed by Chen and Goldfarb [12] and Gould et al. [26,27]. The filter, originally introduced by Fletcher and Leyf- fer [19] significantly increases the flexibility of the step acceptance and, thus, is widely used by nonlinear programming solvers (e.g. [4,9,19,40,45]). Global convergence has been proved for several filter methods and usually depends on a further algorithm phase: the feasibility restoration. Due to the combination with the merit function, a feasibility restoration phase { which we believe to be a drawback of the filter approach { is not necessary for global convergence. 4 Renke Kuhlmann, Christof Büskens A further advantage is that our filter entries do not depend on parameter choices, e.g. the barrier parameter µ. Other penalty-interior-point algorithms consider an `1-penalty, see e.g., Benson et al. [5], Boman [7], Curtis [15], Fletcher [18], Tits et al. [39], Gould et al. [29], Yamashita [46]. Many `1-penalty-interior-point algorithms reformulate the problem into a smooth one using additional elastic variables. However, for large-scale nonlinear programming this can be a disadvantage. Closely related are also the stabilized sequential quadratic programming methods like the works of Gill and Robinson [24] or Shen et al. [38]. The aim of this paper is to study the convergence properties of the proposed algorithm and its numerical performance. Therefore, we implemented the algorithm within the large-scale nonlinear programming solver WORHP. The paper is organized as follows. In Section2 we describe the algorithm in- cluding the general approach of primal-dual penalty-interior-point algorithms, the step calculation and the line search. The global and local convergence of the presented algorithm are shown in Section3 and Section4, respectively. Finally, in Section5 we perform numerical experiments using the CUTEst test set [28] to show the efficiency of the proposed algorithm and compare it to other solvers, in particular the interior-point method IPOPT [45] and the sequential quadratic programming algorithm of WORHP [9]. Notation Matrices are written in uppercase and vectors in lowercase. The i-th component of a vector x is denoted by x(i). A diagonal matrix with the entries of a vector x on its diagonal has the same name in uppercase, i.e X := diag(x). The vector e stands for a vector of all ones with appropriate dimension. The norm k·k is the Euclidean norm k·k2 unless stated differently, e.g. k·k1 is the maximum norm. The notation In(X) = (λ+; λ−; λ0) stands for the inertia of a matrix X, in particular (λ+; λ−; λ0) are the numbers of positive, negative and zero eigenvalues, respectively. We will denote the gradient of a function n n h1 : R ! R at the point x0 as rh1(x0) 2 R , the Jacobian of a function n m n×m h2 : R ! R as rh2(x0) 2 R and the subdifferential of h1(x) at x0 as @h1(x0).

Load more