An Efficient Global Optimization Algorithm for Nonlinear Sum-of-Ratios Problem ∗y Yun-Chol Jonga a Center of Natural Sciences, University of Science, Pyongyang, DPR Korea May 3, 2012 Abstract This paper presents a practical method for finding the globally optimal solution to nonlinear sum-of-ratios problem arising in image processing, engineering and man- agement. Unlike traditional methods which may get trapped in local minima due to the non-convex nature of this problem, our approach provides a theoretical guar- antee of global optimality. Our algorithm is based on solving a sequence of convex programming problems and has global linear and local superlinear/quadratic rate of convergence. The practical efficiency of the algorithm is demonstrated by numerical experiments for synthetic data. 2010 Mathematics subject Classification: 90C26, 90C32, 65K05 Keywords: Fractional programming, non-convex optimization, global optimization algo- rithm, sum-of ratios problem, guaranteed global optimality. 1 Introduction The sum-of-ratios problem, which is to minimize (maximize) a sum of several fractional func- tions subject to convex constraints, is a non-convex optimization problem that is difficult to solve by traditional optimization methods. The problem arises in many applications such as optimization of the average element shape quality in the finite element method, computer ∗E-mail: [email protected] yAddress: Gwahak-1 dong, Unjong distrct, Pyongyang, DPR Korea 1 graphics and management ([2],[3],[4]). In [4], many problems of projective geometry in- cluding multiview triangulation, camera resectioning and homography estimation have been formulated as the sum-of-ratios problem and a branch-and-bound method has been proposed to find its global solution which relies on recent developments in fractional programming and the theory of convex underestimators ([6],[7],[8],[9],[14],[15]). In the method of [4], number of variables increases as twice as the number of fractional functions involving in the sum and a second-order cone programming problem is needed to solve for obtaining a lower bound of the optimal value in each iteration. Their algorithm is provably optimal, that is, given any tolerance , if the optimization problem is feasible, the algorithm returns a solution which is at most far from the global optimum. The branch-and-bound method requires a lot of computations, has low convergence and it is not easy to find a reasonable branching strategy. Recently there has been some progress made towards finding the global solution to a few of these optimization problems ([16],[17]). However, the resulting algorithm is numerically unstable, computationally expensive and does not generalize for more views or harder prob- lems like resectioning. In [18], linear matrix inequalities were used to approximate the global optimum, but no guarantee of actually obtaining the global optimum is given. Also, there are unsolved problems concerning numerical stability. Robustification using L1-norm was presented in [19], but the approach is restricted to the affine camera model. In this paper, an efficient global optimization algorithm is presented which transforms the sum-of-ratios problem into parametric convex programming problem and finds the global solution successfully. 2 Equivalent parametric convex programming The sum-of-ratios problem seeks to minimize the sum of fractional functions subject to convex constraints, which is formulated as follows. N X min F (x) = Fi(x); i=1 (2.1) subject to gi(x) ≤ 0; i = 1; ··· ; m; x 2 Rn fi(x) where Fi(x) = ; i = 1; ··· ; N; and fi(x); gi(x) and −hi(x) are twice continuously differ- hi(x) entiable convex functions. n Let X = fx 2 R jgi(x) ≤ 0; i = 1; ··· ; mg. It is assumed that fi(x) ≥ 0 and hi(x) > 0 n for every x 2 X, and that intX = fx 2 R jgi(x) < 0; i = 1; ··· ; mg 6= ;. Even with these restrictions the above problem is NP-complete [5]. 2 It is easy to see that the problem (2.1) is equivalent to the following problem. N X min F (x) = βi; i=1 subject to Fi(x) ≤ βi; i = 1; :::; N; (2.2) gi(x) ≤ 0; i = 1; :::; m; x 2 Rn ¯ Lemma 2.1. If (¯x; β) is the solution of the problem (2.2), then there exist u¯i; i = 1; ··· ;N such that x¯ is a solution of the following problem for u =u ¯ and β = β¯. N X min ui(fi(x) − βihi(x)); i=1 (2.3) subject to gi(x) ≤ 0; i = 1; :::; m; x 2 Rn And x¯ also satisfies the following system of equations for u =u ¯ and β = β¯: 1 ui = ; i = 1; :::; N (2.4) hi(x) fi(x) − βihi(x) = 0; i = 1; :::; N (2.5) Proof. The constraint Fi(x) ≤ βi is equivalent to fi(x) − βihi(x) ≤ 0. Let's define the following function for the problem (2.2). N N m X X X L(x; β; w; u; v) = w βi + ui(fi(x) − βihi(x)) + vigi(x): i=1 i=1 i=1 By Fritz-John optimality condition (Theorem 4.2.8 of [1]), there existw; ¯ u¯ = (u ¯1; ··· ; u¯N ) andv ¯ = (v ¯1; ··· ; v¯m) such that N m @L X X = u¯ (rf (¯x) − β¯ rh (¯x)) + v¯ rg (¯x) = 0 (2.6) @x i i i i i i i=1 i=1 @L =w ¯ − u¯ihi(¯x) = 0; i = 1; ··· ;N (2.7) @βi @L ¯ u¯i =u ¯i(fi(¯x) − βihi(¯x)) = 0; i = 1; ··· ;N (2.8) @ui @L vi =v ¯igi(¯x) = 0; i = 1; ··· ; m (2.9) @vi 3 gi(¯x) ≤ 0; v¯i ≥ 0; i = 1; ··· ; m (2.10) ¯ fi(¯x) − βihi(¯x) ≤ 0; u¯i ≥ 0; i = 1; ··· ;N (2.11) w¯ ≥ 0; (w; ¯ u;¯ v¯) 6= (0; 0; 0) (2.12) Suppose thatw ¯ = 0. Then, by (2.7), we haveu ¯ = 0 because hi(x) > 0; i = 1; ··· ;N for all x 2 X. Hence, it follows from (2.6), (2.9), (2.10) and (2.12) that X v¯irgi(¯x) = 0; (2.13) i2I(¯x) X v¯i > 0; v¯i ≥ 0; i 2 I(¯x); (2.14) i2I(¯x) 0 where I(¯x) = fijgi(¯x) = 0; 1 ≤ i ≤ mg. By Slater condition, there exists a point x such that 0 gi(x ) < 0; i = 1; ··· ; m: (2.15) Since gi(x); i = 1; ··· ; m are convex, it follows from (2.15) that T 0 0 rgi(¯x) (x − x¯) ≤ gi(x ) − gi(¯x) < 0; i 2 I(¯x) (2.16) !T 0 P Letting d = x − x¯, from (2.16) and (2.14), we have v¯irgi(¯x) d < 0, which contra- i2I(¯x) dicts (2.13). Thus, we havew ¯ > 0. u¯ v¯ Denoting w¯ and w¯ byu ¯ andv ¯ again, respectively, we see that (2.7) is equivalent to (2.4) and so (2.8) is equivalent to (2.5) becauseu ¯i > 0; i = 1; ··· ;N by (2.4). Given u =u ¯ and β = β¯, (2.6), (2.9) and (2.10) is just the KKT condition for the problem (2.3). Since the problem (2.3) is convex programming for parameters u > 0 and β ≥ 0, the KKT condition is also sufficient optimality condition and thenx ¯ is the solution of (2.3) for u =u ¯ and β = β¯. Remark 2.1. Consider the maximization problem N X max F (x) = Fi(x); i=1 subject to gi(x) ≤ 0; i = 1; ··· ; m; x 2 Rn fi(x) where Fi(x) = ; i = 1; ··· ;N and fi(x); −gi(x) and −hi(x) are concave functions, and hi(x) fi(x) ≥ 0; hi(x) > 0; i = 1; ··· ;N in the feasible set X 4 The above problem is equivalent to the following problem. N X max F (x) = βi; i=1 subject to Fi(x) ≥ βi; i = 1; :::; N; gi(x) ≤ 0; i = 1; :::; m; x 2 Rn Then we can obtain the same result as the Lemma 2.1: If (¯x; β¯) is the solution of the above maximization problem , then there existu ¯i; i = 1; ··· ;N such thatx ¯ is a solution of the following problem for u =u ¯ and β = β¯. N X max ui(fi(x) − βihi(x)); i=1 subject to gi(x) ≤ 0; i = 1; :::; m; x 2 Rn Andx ¯ also satisfies the system of equations (2.4) and (2.5) for u =u ¯ and β = β:¯ The above problem is convex programming for given parameters β ≥ 0 and u > 0. In what follows, all the results for the minimization problem hold true for the maximization problem. Let α = (β; u) denote parameter vector and let Ω = fα = (β; u) 2 R2N j0 ≤ β ≤ βu; 0 < ul ≤ u ≤ uug; where u u u l l l u u u u fi(x) u 1 β = (β1 ; ··· ; βN ); u = (u1; ··· ; uN ); u = (u1 ; ··· ; uN ); βi = max ; ui = max ; x2X hi(x) x2X hi(x) and l 1 ui = min ; i = 1; ··· ; N: x2X hi(x) Remark 2.2. The βu; ul and uu involved in the definition of Ω are needed only for theoretical consideration and they are not needed in solving the problem (2.1) at all. 5 Let x(α) be the solution of the problem (2.3) for fixed α 2 Ω and let N X '(α) = ui(fi(x(α)) − βihi(x(α))) (2.17) i=1 1 i (α) = −fi(x(α)) + βihi(x(α)); i = 1; ··· ; N; (2.18) 2 i (α) = −1 + uihi(x(α)); i = 1; ··· ;N (2.19) Let ^ 1 2 Ω = fα 2 Ωj i (α) = 0; i (α) = 0; i = 1; ··· ;Ng: and 1 i (α) = 0; i = 1; ··· ; N; (2.20) 2 i (α) = 0; i = 1; ··· ;N (2.21) Corollary 2.1.