arXiv:1707.04268v5 [math.OC] 19 Jan 2018 rbe,wihisie h eerhi hsfil se[4–8]). (see field this in research proble the game inspired bimatrix restated the which [3] and problem, Dantzig problem programming and te quadratic a Cottle the in 1968, lem, Cottle In t and seems Dantzig orthant. It by non-negative [1]. considered 1939 first conc in was underlying Karush problems the by mentarity only, introduced defin firstly decades being the few older, a (see much back problems goes complementarity Preliminaries) cone the in research Although Introduction 1 neoois niern,pyis nneadtac xmlsi e in Examples a traffic. These pract and 9]). the [8, finance and books physics, in the cone engineering, works to general economics, (similarly Earlier a in only for orthant fields. theory non-negative other the the and present finance problems economics, ity in applications of h opeetrt rbe sacosctigae frsac w research of area cross-cutting a is problem complementarity The iercmlmnaiypolm netne second extended on problems complementarity Linear aqad loih.Fnly epeetsm ueia ex con numerical Ne some the the present of as we solution such Finally, a problem, algorithm. this be Marquardt for to strategies point solution a or present non-negative for ext the conditions an on sufficient on problem and complementarity problem complementarity mixed a linear into a convert We cones. mization Keywords: problems complementarity linear the study we paper, this In 00ASSbetClassification: Subject AMS 2010 colo ahmtc,Uiest fBirmingham of University Mathematics, of School colo ahmtc,Uiest fBirmingham of University Mathematics, of School opeetrt rbe,Etne eodOdrCn,Coni Cone, Order Second Extended Problem, Complementarity imnhmB52T ntdKingdom United 2TT, B15 Birmingham imnhmB52T ntdKingdom United 2TT, B15 Birmingham mi:[email protected] email: asnBidn,Edgbaston Building, Watson asnBidn,Edgbaston Building, Watson mi:[email protected] email: oebr9 2018 9, November re cones order .Z N´emeth Z. S. Abstract .Xiao L. 03,90C25 90C33, 1 tnmto n Levenberg- and method wton amples. hn.W tt necessary state We thant. h ierpormigprob- programming linear the netne eodorder second extended on etdpolm ealso We problem. verted ne eododrcone order second ended clapiain eeyfor merely applications ical a h ocp fcomple- of concept the hat p fcmlmnaiyis complementarity of ept to ntebgnigof beginning the in ition hia eot[] o the for [2], report chnical erltdt equilibrium to related re ooisaeWalrasian are conomics sacomplementarity a as m ihhsawd range wide a has hich oecomplementar- cone Opti- c price equilibrium models, price oligopoly models, Nash-Cournot production/distribution mod- els, models of invariant capital stock, Markov perfect equilibria, models of decentralised econ- omy and perfect competition equilibrium, models with individual markets of production factors. Engineering and physics applications are frictional contact problems, elastoplastic structural analysis and nonlinear obstacle problems. An example in finance is the discretisation of the differential complementarity formulation of the Black-Scholes models for the American op- tions [10]. An application to congested traffic networks is the prediction of steady-state traffic flows. In the recent years several applications have emerged where the complementarity prob- lems are defined by cones essentially different from the non-negative orthant such as positive semidefinite cones, second order cones and direct product of these cones (for mixed comple- mentarity problems containing linear subspaces as well). Recent applications of second order cone complementarity problems are in elastoplasticity [11,12], robust game theory [13,14] and robotics [15]. All these applications come from the Karush-Kuhn-Tucker conditions of second order conic optimization problems. N´emeth and Zhang extended the concept of second order cone in [16] to the extended second order cone. Their extension seems the most natural extension of second order cones. Sznajder showed that the extended second order cones in [16] are irreducible cones (i.e., they cannot be written as a direct product of simpler cones) and calculated the Lyapunov rank of these cones [17]. The applications of second order cones and the elegant way of extending them suggest that the extended second order cones will be important from both theoretical and practical point of view. Although conic optimization problems with respect to extended second order cones can be reformulated as conic optimization problems with respect to second order cones, we expect that for several such problems using the particular inner structure of the second order cones provides a more efficient way of solving them than solving the transformed conic optimization problem with respect to second order cones. Indeed, such a particular problem is the projection onto an extended second order cone which is much easier to solve directly than solving the reformulated second order conic optimization problem [18]. Until now the extended second order cones of N´emeth and Zhang were used as a working tool only for finding the solutions of mixed complementarity problems on general cones [16] and variational inequalities for cylinders whose base is a general convex set [19]. The applications above for second order cones show the importance of these cones and motivates considering conic optimization and complementarity problems on extended second order cones. As another motivation we suggest the application to mean variance portfolio optimization problems [20,21] described in Section 3. The paper is structured as follows: In Section 2, we illustrate the main terminology and definitions used in this paper. In Section 3 we present an application of extended second order cones to portfolio optimization problems. In Section 4, we introduce the notion of mixed implicit complementarity problem as an implicit complementarity problem on the direct product of a cone and a Euclidean space. In Section 5, we reformulate the linear complementarity problem as a mixed (implicit, mixed implicit) complementarity problem on the non-negative orthant (MixCP). Our main result is Theorem 1, which discusses the connections between an ESOCLCP and mixed (implicit, mixed implicit) complementarity problems. In particular, under some mild conditions, given the definition of Fischer-Burmeister (FB) regularity and of the stationarity of a point, we prove in Theorem 2 that a point can be the solution of a mixed complementarity problem if it satisfies specific conditions related to FB regularity and stationarity (Theorem 2). This theorem can be used to determine whether a point is a solution of a mixed com- plementarity problem converted from ESOCLCP. In Section 6, we use Newton’s method and Levenberg-Marquardt algorithm to find the solution for the aforementioned MixCP. In Sec- tion 7, we provide an example of a linear complementarity problem on an extended second
2 order cone. Based on the above, we convert this linear complementarity problem into a mixed complementarity problem on the non-negative orthant, and use the aforementioned algorithms to solve it. A solution of this mixed complementarity problem will provide a solution of the corresponding ESOCLCP. As a first step, in this paper, we study the linear complementarity problems on extended second order cones (ESOCLCP). We find that an ESOCLCP can be transformed to a mixed (implicit, mixed implicit) complementarity problem on the non-negative orthant. We will give the conditions for which a point is a solution of the reformulated MixCP problem, and in this way we provide conditions for a point to be a solution of ESOCLCP.
2 Preliminaries
Let m be a positive integer and F : Rm Rm be a mapping and y = F (x). The definition of the classical complementary problem [22]→
x 0, y 0, and x, y =0, ≥ ≥ h i where denotes the componentwise order induced by the non-negative orthant and , is the ≥ h· ·i canonical scalar product in Rm, was later extended to more general cones K, as follows:
x K, y K∗, and x, y =0, ∈ ∈ h i where K∗ is the dual of K [23]. Let k, ℓ, ℓˆ be non-negative integers such that m = k + ℓ. Recall the definitions of the mutually dual extended second order cone L(k,ℓ) and M(k,ℓ) in Rm Rk Rℓ: ≡ × L(k,ℓ)= (x, u) Rk Rℓ : x u e , (1) { ∈ × ≥k k } M(k,ℓ)= (x, u) Rk Rℓ : e⊤x u , x 0 , (2) { ∈ × ≥k k ≥ } where e = (1,..., 1)⊤ Rk. If there is no ambiguity about the dimensions, then we simply denote L(k,ℓ) and M(k,ℓ∈ ) by L and M, respectively. Denote by , the canonical scalar product in Rm and by the corresponding Euclidean norm. The notationh· ·i x y means that x, y = 0, where x, y k·kRm. Let K Rm be a nonempty⊥ closedh convexi cone and K∗ its∈ dual. ⊂ Definition 1 The set (K) := (x, y) K K∗ : x y C { ∈ × ⊥ } is called the complementarity set of K.
Definition 2 Let F : Rm Rm. Then, the complementarity problem CP(F,K) is defined by: → CP(F,K): (x, F (x)) (K). (3) ∈ C The solution set of CP(F,K) is denoted by SOL-CP(F,K):
SOL-CP(F,K)= x Rm :(x, F (x)) (K) . { ∈ ∈ C } If T is a matrix, r Rm and F is defined by F (x) = T x + r, then CP(F,K) is denoted by LCP(T,r,K) and is∈ called linear complementarity problem. The solution set of LCP(T,r,K) is denoted by SOL-LCP(T,r,K).
3 Definition 3 Let G, F : Rm Rm. Then, the implicit complementarity problem ICP(F,G,K) is defined by → ICP(F,G,K): (G(x), F (x)) (K). (4) ∈ C The solution set of ICP(F,G,K) is denoted by SOL-ICP(F,G,K): SOL-ICP(F,G,K)= x Rm :(G(x), F (x)) (K) . { ∈ ∈ C } Let m,k,ℓ be non-negative integers such that m = k + ℓ,Λ Rk be a nonempty closed convex cone and K =Λ Rℓ. Denote by Λ∗ the dual of Λ in Rk ∈and by K∗ the dual of K in × Rk Rℓ. It is easy to check that K∗ =Λ∗ 0 . × ×{ } Definition 4 Consider the mappings F : Rk Rℓ Rk and F : Rk Rℓ Rℓˆ. The mixed 1 × → 2 × → complementarity problem MixCP(F1, F2, Λ) is defined by
F2(x, u)=0 MixCP(F , F , Λ): (5) 1 2 (x, F (x, u)) (Λ). 1 ∈ C The solution set of MixCP(F1, F2, Λ) is denoted by SOL-MixCP(F1, F2, Λ): SOL-MixCP(F , F , Λ) = x Rm : F (x, u)=0, (x, F (x, u)) (Λ) . 1 2 { ∈ 2 1 ∈ C } Definition 5 [8, Definition 3.7.29] A matrix Π Rn×n is said to be an S matrix if the system ∈ 0 of linear inequalities Πx 0, 0 = x 0 ≥ 6 ≥ has a solution.
The proof of our next result follows immediately from K∗ = Λ∗ 0 and the definitions ×{ } of CP(F,K) and MixCP(F1, F2, Λ). Proposition 1 Consider the mappings F : Rk Rℓ Rk, F : Rk Rℓ Rℓ. 1 × → 2 × → Define the mapping F : Rk Rℓ Rk Rℓ × → × by F (x, u)=(F1(x, u), F2(x, u)). Then, (x, u) SOL-CP(F,K) (x, u) SOL-MixCP(F , F , Λ). ∈ ⇐⇒ ∈ 1 2 Definition 6 [24, Schur complement] The notation of the Schur complement for a matrix PQ Π= R S , with P nonsingular, is (Π/P )= S RP −1Q. − Definition 7 [25, Definition 4.6.2] (i) Let I be an open subset with I Rm and f : I Rm. We say that f is Lipschitz function, if there is a constant λ> 0 such that⊂ → f(x) f(x′) λ x x′ x, x′ I. (6) k − k ≤ k − k ∀ ∈ (ii) We say that f is locally Lipschitz if for every x I, there exists ε > 0 such that f is m ∈ Lipschitz on I Bε(x), where Bε(x)= y R : y x ε . ∩ { ∈ k − k ≤ } 4 3 An Application of Extended Second Order Cones to Portfolio Optimisation Problems
Consider the following Portfolio Optimisation Problem:
min w⊤Σw : r⊤w R, e⊤w =1 , w ≥ where Σ Rn×n is the covariance matrix, e = (1,..., 1)⊤ Rn, w Rn is the weight of ∈ ∈ ∈ asset allocation for the portfolio and R is the required return of the portfolio. In order to guarantee the diversified allocation of the fund into different assets in the market, a new constraint can be reasonably introduced: w ξ, where ξ is the limitation of the k k ≤ concentration of the fund allocation. If short selling is allowed, then w can be less than zero. The introduction of this constraint can guarantee that the fund will be allocated into few assets only. Since the covariance matrix Σ can be decomposed into Σ = U ⊤U, the problem can be rewritten as min y : r⊤w R, Uw y, w ξ, e⊤w =1 . w,ξ,y ≥ k k ≤ k k ≤ The constraint Uw y is a relaxation of the constraint U w y, where U = k k ≤ k kk k ≤ k k max x Ux . The strengthened problem will become: k k≤1 k k y ⊤ min y : r⊤w R, w e ξ, , e⊤w =1 . w,ξ,y ≥ k k ≤ U ( k k ) The minimal value of the objective of the original problem is at most as large as the minimal value of the objective for this latter problem. The second constraint of the latter portfolio optimisation problem means that the point (ξ,y/ U ,w)⊤ belongs to the extended second k k order cone L(2, n). Hence, the strenghtened problem is a conic optimisation problem with respect to an extended second order cone.
4 Mixed Implicit Complementarity Problems
Let m, k, ℓ, ℓˆ be non-negative integers such that m = k + ℓ,Λ Rk be a nonempty, closed, ∈ convex cone and K =Λ Rℓ. Denote by Λ∗ the dual of Λ in Rk and by K∗ the dual of K in Rk Rℓ. × × Definition 8 Consider the mappings
F ,G : Rk Rℓ Rk, F : Rk Rℓ Rℓˆ. 1 1 × → 2 × →
The mixed implicit complementarity problem MixICP(F1, F2,G1, Λ) is defined by
F2(x, u)=0 MixICP(F , F ,G , Λ): (7) 1 2 1 (G (x, u), F (x, u)) (Λ). 1 1 ∈ C
The solution set of the mixed complementarity problem MixICP(F1, F2,G1, Λ) is denoted by SOL-MixICP(F1, F2,G1, Λ):
SOL-MixICP(F1, F2,G1, Λ) = x Rm : F (x, u)=0, (G (x, u), F (x, u)) (Λ) . { ∈ 2 1 1 ∈ C } 5 The proof of our next result follows immediately from K∗ = Λ∗ 0 and the definitions ×{ } of ICP(F,G,K) and MixICP(F1, F2,G1, Λ).
k ℓ k k ℓ ℓ Proposition 2 Consider the mappings F1,G1 : R R R , F2,G2 : R R R . Define the mappings F,G : Rk Rℓ Rk Rℓ by F×(x, u)=(→ F (x, u), F (x, u))×, G(x,→ u) = × → × 1 2 (G1(x, u),G2(x, u)), respectively. Then,
(x, u) SOL-ICP(F,G,K) (x, u) SOL-MixICP(F , F ,G , Λ). ∈ ⇐⇒ ∈ 1 2 1 5 Main Results
The linear complementarity problem is the dual problem of a quadratic optimisation problem, which has a wide range of applications in various areas. One of the most famous application is the portfolio optimisation problem first introduced by Markowitz [20]; see the application of the extended second order cone to this problem presented in the Introduction.
Proposition 3 Let x, y Rk and u, v Rℓ 0 . ∈ ∈ \{ } (i) (x, 0,y,v) (L) if and only if e⊤y v and (x, y) (Rk ). ∈ C ≥k k ∈ C + (ii) (x, u, y, 0) (L) if and only if x u and (x, y) (Rk ). ∈ C ≥k k ∈ C + (iii) (x,u,y,v) := ((x, u), (y, v)) (L) if and only if there exists a λ> 0 such that v = λu, e⊤y = v and (x u e, y)∈ CC(Rk ). − k k −k k ∈ + Proof. Items (i) and (ii) are easy consequence of the definitions of L, M and the complemen- tarity set of a nonempty closed convex cone. Item (iii) follows from Proposition 1 of [18]. For the sake of completeness, we will reproduce its proof here. First assume that there exists λ > 0 such that v = λu, e⊤y = v and (x u e, y) C(Rp ). Thus, (x, u) L and (y, v) M. On the other hand,− k k −k k ∈ + ∈ ∈ (x, u), (y, v) = x⊤y + u⊤v = u e⊤y λ u 2 = u v λ u 2 =0. h i k k − k k k kk k − k k Thus, (x,u,y,v) C(L). Conversely, if∈ (x,u,y,v) C(L), then (x, u) L, (y, v) M and ∈ ∈ ∈ 0= (x, u), (y, v) = x⊤y + u⊤v u e⊤y + u⊤v u v + u⊤v 0. h i ≥k k ≥k kk k ≥ This implies the existence of a λ> 0 such that v = λu, e⊤y = v and (x u e)⊤y =0. It follows that (x u e, y) C(Rp ). − k k −k k −k k ∈ +
Theorem 1 Denote z =(x, u), zˆ =(x u ,u), z˜ =(x t, u, t) and r =(p, q) with x, p Rk, u, q Rℓ and t R. Let T =( A B ) with−kAk Rk×k, B −Rk×ℓ, C Rℓ×k and D Rℓ×ℓ.∈ The ∈ ∈ C D ∈ ∈ ∈ ∈ square matrices T , A and D are assumed to be nonsingular.
(i) Suppose u =0. We have
z SOL-LCP(T, r, L) ∈ x SOL-LCP(A, p, Rk ) and e⊤(Ax + p) Cx + q . ⇐⇒ ∈ + ≥k k
6 (ii) Suppose Cx + Du + q =0. Then,
z SOL-LCP(T, r, L) z SOL-MixCP(F , F , Rk ) and x u , ∈ ⇐⇒ ∈ 1 2 + ≥k k
where F1(x, u)= Ax + Bu + p and F2(x, u)=0. (iii) Suppose u =0 and Cx + Du + q =0. We have 6 6 z SOL-LCP(T, r, L) z SOL-MixICP(F , F ,G , Rk ), ∈ ⇐⇒ ∈ 1 2 1 + where F (x, u)= u C + ue⊤A x + ue⊤(Bu + p)+ u (Du + q), 2 k k k k G1(x, u)= x u e and F1(x, u)= Ax + Bu + p. −k k (iv) Suppose u =0 and Cx + Du + q =0. We have 6 6 z SOL-LCP(T, r, L) zˆ SOL-MixCP(F , F , Rk ), ∈ ⇐⇒ ∈ 1 2 + where
F (x, u)= u C + ue⊤A (x + u e)+ ue⊤(Bu + p)+ u (Du + q) 2 k k k k k k and F (x, u)= A(x + u e)+ Bu +p. 1 k k (v) Suppose u =0, Cx + Du + q =0 and u C + u⊤eA is a nonsingular matrix. We have 6 6 k k z SOL-LCP(T, r, L) zˆ SOL-ICP(F , F , Rk ), ∈ ⇐⇒ ∈ 1 2 + where
−1 F (u)= A u C + ue⊤A ue⊤(Bu + p)+ u (Du + q) + Bu + p 1 k k k k and −1 F (u)= u C + ue⊤A ue⊤(Bu + p)+ u (Du + q) . 2 k k k k (vi) Suppose u =0, Cx + Du + q =0. We have 6 6 z SOL-LCP(T, r, L) t> 0 ∈ ⇐⇒ ∃ such that z˜ MixCP(F , F , Rk ), ∈ 1 2 + where e e F1(x, u, t)= A(x + te)+ Bu + p and tCe + ue⊤A (x + te)+ ue⊤(Bu + p)+ t(Du + q) F (x, u, t)= . (8) 2 t2 u 2 −k k Proof. e
(i) We have that z SOL-LCP(T, r, L) is equivalent to (x, 0, Ax + p,Cx + q) (L) or, by item (i) of Proposition∈ 3, to (x, Ax + p) (Rk ) and e⊤(Ax + p) Cx +∈q C. ∈ C + ≥k k
7 (ii) We have that z SOL-LCP(T, r, L) is equivalent to (x, u, Ax + Bu + p, 0) (L) or, by item (ii) of Proposition∈ 3, to (x, Ax + Bu + p) (Rk ) and x u , or to∈ C ∈ C + ≥k k z SOL-MixCP(F , F , Rk ) and x u , ∈ 1 2 + ≥k k
where F1(x, u)= Ax + Bu + p and F2(x, u) = 0. (iii) Suppose that z SOL-LCP(T, r, L). Then, (x,u,y,v) (L), where y = Ax + Bu + p and v = Cx + Du∈ + q. Then, by item (iii) of Proposition∈ C 3, we have that λ > 0 such ∃ that Cx + Du + q = v = λu, (9) − e⊤(Ax + Bu + p)= e⊤y = v = Cx + Du + q = λ u , (10) k k k k k k (G (x, u), F (x, u))=(x u e, Ax + Bu + p)=(x u e, y) (Rk ). (11) 1 1 −k k −k k ∈ C + From equation (9) we obtain u (Cx+Du+q)= λ u u, which by equation (10) implies u (Cx + Du + q)= ue⊤(Axk k+ Bu + p), which− afterk k some algebra gives k k −
F2(x, u)=0. (12)
From equations (11) and (12) we obtain that z SOL-MixICP(F , F ,G ). ∈ 1 2 1 Conversely, suppose that z SOL-MixICP(F , F ,G ). Then, ∈ 1 2 1 u v + ue⊤y = u (Cx + Du + q)+ ue⊤(Ax + Bu + p)= F (x, u)=0 (13) k k k k 2 and
(x u e, y)=(x u e, Ax + Bu + p)=(G (x, u), F (x, u)) (Rk ), (14) −k k −k k 1 1 ∈ C + where v = Cx + Du + q and y = Ax + Bu + p. Equations (14) and (13) imply
v = λu, (15) − where λ =(e⊤y)/ u > 0. (16) k k Equations (15) and (16) imply e⊤y = v . (17) k k By item (iii) of Proposition 3, equations (15), (17) and (14) imply
(x,y,u,v) C(L) ∈ and therefore z SOL-LCP(T, r, L). ∈ (iv) It is a simple reformulation of item (iii) by using the change of variables
(x, u) (x u e, u). 7→ −k k (v) Again it is a simple reformulation of item (iv) by using that u C +u⊤eA is a nonsingular matrix. k k
8 (vi) Suppose that z SOL-LCP(T, r, L). Then, (x,u,y,v) (L), where y = Ax + Bu + p and v = Cx + Du∈ + q. Let t = u , Then, by item (iii)∈ ofC Proposition 3, we have that k k λ> 0 such that ∃ Cx + Du + q = v = λu, (18) − e⊤(Ax + Bu + p)= e⊤y = v = Cx + Du + q = λt, (19) k k k k (˜z, F (x, u, t))=(x te, Ax + Bu + p)=(x te, y) (Rk ), (20) 1 − − ∈ C + wherez ˜ = (x t, u, t). From equation (18), we get t(Cx + Du + q) = tλu, which, by − e − equation (19), implies t(Cx + Du + q)= ue⊤(Ax + Bu + p), which after some algebra gives − F2(x, u, t)=0. (21) From equations (20) and (21) we obtain that z SOL-MixCP(F , F , Rk ). e ∈ 1 2 + e e
Note that the item(vi) makes F1(x, u, t) and F2(x, u, t) become smooth functions by adding the variable t. The smooth functions therefore make the smooth Newton’s method applicable to the mixed complementarity problem.e e The conversion of LCP on extended second order cones to a MixCP problem defined on the non-negative orthant is useful, because it can be studied by using the Fischer-Burmeister function. In order to ensure the existence of the solution of MixCP, we introduce the scalar Fischer-Burmeister C-function (see [26, 27]).
2 2 2 ψF B(a, b)= √a + b (a + b) (a, b) R . − ∀ ∈ 2 R2 Obviously, ψF B(a, b) is a continuously differentiable function on . The equivalent FB- based equation formulation for the MixCP problem is:
1 ψ(x1, F1 (x, u, t)) . FMixCP . 0= F B (x, u, t)= e , (22) ψ(x , F k(x, u, t)) k 1 F (x, u, t) 2 e with the associated merit function: e 1 θMixCP(x, u, t)= FMixCP(x, u, t)T FMixCP(x, u, t). F B 2 F B F B We continue by calculating the Jacobian matrix for the associated merit function. If i (1, ..., k) i m+1∈ is such that (zi, F ) = (0, 0), then the differential with respect to z =(x, u, t) R is 1 6 ∈ e FMixCP ∂ F B xi i = 1 ei ∂z 2 − 2 i xi + F1(x, u, t) r e F i(x, u, t) ∂F i(x, u, t) + 1 1 1 , 2 − ∂z 2 i xi +e F1(x, u, t) e r e
9 i where e denotes the i-th canonical unit vector. The differential with respect to zj with j = i is 6
FMixCP i i ∂ F B F (x, u, t) ∂F (x, u, t) i = 1 1 1 , ∂zj 2 − ∂zj 2 i xi +e F1(x, u, t) e r e i Obviously, the differential with respect to zj with j>k, is equal to zero. Note that if (zi, F1)= FMixCP ∂( F B ) (0, 0), then i will be a generalised gradient of a composite function, i.e., a closed unit ∂z e ball B(0, 1). However, this case will not occur in our paper. As for the term F2(x, u, t) with i (k +1, ...m + 1), the Jacobian matrix is much more simple, since ∈ e ∂ FMixCP ∂F i(x, u, t) F B i = 2 . ∂z ∂z e Therefore, the Jacobian matrix for the associated merit function is:
D + D J F (x, u, t) D J F (x, u, t) = a b x 1 b (u,t) 1 , A JxF2(x, u, t) J(u,t)F2(x, u, t) ! e e where e e
ei xi F1(x,u,t) 2 ei 2 1 1 Da = diag x +F (x,u,t) ,Db = diag 2 ei 2 , √ i 1 − √xi +F1(x,u,t) − i =1, . . . , k.
Define the following index sets:
i i i : xi 0, F 0, xiF (x, u, t)=0 complementarity index C ≡ ≥ 1 ≥ 1 n1,...,k o residual index R≡{ } \e C i e i R : xi > 0, F (x, u, t) > 0 positive index P ≡ ∈ 1 n o negative index N ≡R\P e Definition 9 A point (x, u, t) Rm+1 is called FB-regular for the merit function θMixCP (or ∈ F B Rk FMixCP for the MixCP F1, F2, + ) if its partial Jacobian matrix of F B (x, u, t) with respect to x, k JxF1(x, u, t) is nonsingular and if for w R ,w =0 with e e ∀ ∈ 6 e wC =0, wP > 0, wN < 0, there exists a nonzero vector v Rk such that ∈ v =0, v 0, v 0, (23) C P ≥ N ≤ and T w Π(x, u, t)/JxF (x, u, t) v 0, (24) 1 ≥ where e J F (x, u, t) J F (x, u, t) Π(x, u, t) x 1 (u,t) 1 R(m+1)×(m+1), ≡ JxF2(x, u, t) J(u,t)F2(x, u, t)! ∈ e e and Π(x, u, t)/JxF1(x, u, t) ise the Schur complement ofeJxF1(x, u, t) in Π(x, u, t).
e 10 e Rk In our case, for the MixCP F1, F2, + , the Jacobian matrices are: e e JF (x, u, t) A B 1 ≡ and e e e
JF (x, u, t) C D 2 ≡ where e e e tC + ue⊤A A = A, B =(B Ae) C = , 0 e e e e⊤ (A(x + te)+ Bu + p) I + diag(e⊤Bu)+ tD Cx + 2tCe + ue⊤Ae + Du D = . 2u⊤ 2t − e In our case, if the Jacobian matrix block JxF1(x, u, t) = A is nonsingular, then the Schur complement Π(x, u, t)/JxF1(x, u, t) is e −1 e Π(x, u, t)/JxF (x, u, t) = D CA B. (25) 1 − Proposition 4 If the matrices A and Deare nonsingulare fore anye ez Rm+1, then the Jacobian matrix for the associated merit function is nonsingular. ∈ A e e Proof. It is easy to check that
D + D AD B = a b b . A C D ! e e
is a nonsingular matrix if and only if thee sub-matrix Dea + DbA and its Schur complement Aare nonsingular, and they are nonsingular if and only if the matrices A and D are nonsingular. e The following theorem is [8, Theorem 9.4.4]. For the sake of completenee e ss, we provide a proof here.
m+1 k Theorem 2 A point (x, u, t) R is a solution of the MixCP(F1, F2, R ) if and only if ∈ MixCP FMixCP (x, u, t) is an FB regular point of θF B and (x, u, t) is a stationary point of F B . e e Proof. ∗ ∗ ∗ ∗ k ∗ Suppose that z =(x ,u , t ) SOL-MixCP(F1, F2, R ). Then, it follows that z is a global ∈ MixCP ∗ ∗ Rk minimum and hence a stationary point of θF B . Thus, (x , F1(z )) ( +), and we have = = . Therefore, the FB regularity of xe∗ holdse since x∗ = x∈, C because there is no P N ∅ C nonzero vector x satisfying conditions (23). Conversely, supposee that x∗ is FB regular and z∗ =(x∗,u∗, t∗) is a stationary point of θMixCP. It follows that θMixCP = 0, i.e.: F B ∇ F B ∗ ∗ ⊤FMixCP Da + DbJxF1(z ) JxF2(z ) FMixCP F B = ∗ ∗ F B =0, A DbJ(u,t)F1(z ) J(u,t)F2(z )! e e e e
11 where
x∗ ei ∗ i 1 F1(z ) Da = diag ∗ 2 ei ∗ 2 ,Db = diag ∗ 2 ei ∗ 2 1 , √(xi ) +F1(z ) − √(xi ) +F1(z ) − i =1, . . . , k. Hence, for any w Rm+1, we have ∈ ∗ ∗ ⊤ Da + DbJxF1(z ) JxF2(z ) FMixCP w ∗ ∗ F B =0. (26) DbJ(u,t)F1(z ) J(u,t)F2(z )! e e Assume that z∗ is not a solution of MixCP. Then, we have that the index set is not empty. MixCP e e R Define v DbF . We have ≡ F B
vC =0, vP > 0, vN < 0. Take w with
wC =0, wP > 0, wN < 0. FMixCP FMixCP From the definition of Da and Db, we know that Da F B and Db F B have the same sign. Therefore,
⊤ FMixCP ⊤ FMixCP ⊤ FMixCP ⊤ FMixCP w (Da F B )= wC (Da F B )C + wP (Da F B )P + wN (Da F B )N > 0. (27) ⊤ By the regularity of JF1(z) , we have
⊤ ⊤ MixCP ⊤ ⊤ e w JF (z) (DaF )= w JF (z) w 0. (28) 1 F B 1 ≥ The inequalities (27) and (28) together contradict condition (26). Hence = . It means that ∗ e k e R ∅ z is a solution of MixCP(F1, F2, R ). e e
6 Algorithms
For solving a complementarity problem, there are many different algorithms available. The common algorithms include numerical methods for systems of nonlinear equations (such as Newton’s method [28]), the interior point method (Karmarkar’s Algorithm [29]), the projection iterative method [30], and the multi-splitting method [31]. In the previous sections, we have already provided sufficient conditions for using FB regularity and stationarity to identify a solution of the MixCP problem. In this section, we are trying to find a solution of LCP by finding the solution of MixCP which is converted from LCP. One convenient way to do this is using the Newton’s Method as follows:
Algorithm (Newton’s method):
Given initial data z0 Rm+1, and r = 10−7. ∈ Step 1: Set k = 0. Step 2: If FMixCP(zk) r, then STOP. F B ≤ Step 3: Find a direction dk Rm+1 such that ∈ FMixCP(zk)+ ⊤(zk)dk =0. F B A Step 4: Set zk+1 := zk + dk and k := k +1, go to Step 2.
12 If the Jacobian matrix ⊤ is nonsingular, then the direction dk Rm+1 for each step can be found. The following theorem,A which is based on an idea similar to∈ the one used in [32], proves that such a Newton’s Method can efficiently solve the LCP on extended second order cone (i.e. solve the problem within polynomial time), by finding the solution of the MixCP:
Theorem 3 Suppose that the Jacobian matrix is nonsingular. Then, Newton’s method for Rk A MixCP(F1, F2, +) converges at least quadratically to
∗ Rk e e z SOL-MixCP(F1, F2, ), ∈ + 0 ∗ if it starts with initial data z sufficiently close to ze . e Proof. Suppose that the starting point z0 is close to the solution z∗, and suppose that is a ∗ A Lipschitz function. There are ρ> 0, β1 > 0, β2 > 0, such that for all z with z z < ρ, there holds −1(z) < β , and (zk) (z∗)) β zk z∗ . By the definitionk − of thek Newton’s kA k 1 kA −A k ≤ 2k − k method, we have
zk+1 z∗ = zk z∗ −1(zk)FMixCP(zk) k − k k − −A F B k = −1(zk) (zk)(zk z∗) FMixCP(zk) FMixCP(z∗) , A A − − F B − F B because FMixCP(z∗) = 0 when z∗ SOL-MixCP. By Taylor’s theorem, we have F B ∈ 1 FMixCP(zk) FMixCP(z∗)= zk + s(z∗ zk) (xk z∗)ds, F B − F B A − − Z0 so
k k ∗ FMixCP k FMixCP ∗ (z )(z z ) F B (z ) F B (z ) kA − − 1 − k = (zk) zk + s(z∗ zk) ds(zk z∗) A −A − − Z0 1 (zk) zk + s(z∗ zk) ds zk z∗ ≤ kA −A − k k − k Z0 1 1 zk z∗ 2 β sds = β zk z∗ 2. ≤k − k 2 2 2k − k Z0 Also, we have z z∗ < ρ, that is, k − k 1 zk+1 z∗ β β zk z∗ 2. k − k ≤ 2 1 2k − k
Another widely-used algorithm is presented by Levenberg and Marquardt in [33]. Levenberg- Marquardt algorithm can approach second-order convergence speed without requiring the Ja- cobian matrix to be nonsingular. We can approximate the Hessian matrix by:
(z)= ⊤(z) (z), H A A and the gradient by: (z)= ⊤(z)FMixCP(z). G A F B Hence, the upgrade step will be
−1 zk+1 = zk ⊤(zk) (zk)+ µI ⊤(zk)FMixCP(zk). − A A A F B 13 As we can see, Levenberg-Marquardt algorithm is a quasi-Newton’s method for an uncon- strained problem. When µ equals to zero, the step upgrade is just the Newton’s method using approximated Hessian matrix. The number of iterations of Levenberg-Marquardt algorithm to find a solution is higher than that of Newton’s method, but it works for singular Jacobian as well. The greater the parameter µ, the slower the calculation speed becomes. Levenberg- Marquardt algorithm is provided as follows:
Algorithm (Levenberg-Marquardt):
Given initial data z0 Rm+1, µ =0.005, and r = 10−7. Step 1: Set k = 0. ∈ Step 2: If FMixCP(zk) r, stop. F B ≤ Step 3: Find a direction dk Rm+1 such that ∈ (zk)⊤FMixCP(zk)+ ⊤(zk) (zk)+ µI dk =0. A F B A A Step 4: Set zk+1 := zk + dk and k := k +1, go to Step 2.
Theorem 4 [34] Without the nonsingularity assumption on the Jacobian matrix , Levenberg- Rk A Marquardt Algorithm for MixCP(F1, F2, +) converges at least quadratically to
∗ k z e SOL-MixCP(e F1, F2, R ), ∈ + 0 ∗ if it starts with initial data z sufficiently close to ze . e The proof is omitted.
7 A Numerical Example
In this section, we will provide a numerical example for LCP on extended second order cones. Let L(3, 2) be an extended second order cone defined by (1). Following the notation in Theorem 1, let z =(x, u),z ˆ =(x u ,u),z ˜ =(x t, u, t) and r =(p, q)= ( 55, 26, 50)⊤, ( 19, 26)⊤ with x, p R3 , u, q R−k2, andk t R. Consider− − − − − ∈ ∈ ∈ 26 15 3 51 42 7 39 16 17− 18 − − − − T =( A B )= 32 23 40 38 46 , C D − 6 22 28 17 27 − − − 38 25 24 47 16 − − − with A R3×3, B R3×2, C R2×3 and D R2×2. It is easy to show that square matrices T, ∈ ∈ ∈ ∈ A and D are nonsingular. By item (vi) of Theorem 1, we can reformulate this LCP problem as a smooth MixCP problem. We will use the Levenberg-Marquardt algorithm to find the solution of the FB-based equation formulation (22) of MixCP problem. The convergence point is:
z˜∗ =(x t, u, t) − 439 ⊤ 341 724 ⊤ 1271 = 0, , 0 , , , . 660 1460 2683 3582 !
14 We need to check the FB regularity ofz ˜∗. It is easy to show that the partial Jacobian matrix ∗ of F1(˜z ) 26 15 3 e J F (˜z∗)= A = 7 39 16 x 1 −32− 23− 40 e e is nonsingular. Moreover, we have that 439 ⊤ 3626 12148 ⊤ x t = 0, , 0 0, F (˜z∗)= , 0, 0, − 660 ≥ 1 145 185 ≥ and therefore e x t, F (˜z∗) =0. − 1 ∗ R3 D E That is, (x, F1(˜z )) ( +), so the index setse = = . The matrix A is invertible. In ∈ C P N ∅∗ ∗ addition, we can calculate that the Schur complement of Π(z ) with respect to JxF1(˜z ):
e 3991 11387 7203 e 58 95 268 ∗ ∗ −1 15910 e 5185 − 5941 e Π(˜z )/JxF1(˜z ) = D CA B = 93 163 248 . − 341 741 − 1271 − 740 − 1373 1791 e e e e e The FB regularity of x∗ holds as there is no nonzero vector x satisfying conditions (23). Then, we compute the gradient of the merit function, which is
∗ ∗ ⊤FMixCP Da + DbJxF1(˜z ) JxF2(˜z ) FMixCP F B = ∗ ∗ F B A DbJ(u,t)F1(˜z ) J(u,t)F2(˜z )! 598 e 4844e 345 605 7 0 349 1238 0 0 − 32 e39 0 3946e 4031 0 0 − 21195 − 491 − 441 0 16 413 26 1754 0 0 = − 415 − 7 111 =0. 33 7 0 12462 78767 341 0 − 12610 139 701 − 740 32 39 0 13790 9451 741 0 − 21195 131 105 − 1373 0 16 0 3341 3233 1271 0 − 135 − 190 1791 ∗ MixCP ∗ Hence, z is a stationary point of FF B . By Theorem 2, we conclude that z is the solution of the MixCP problem. By the item (vi) of Theorem 1, we have that z =(x, u) 1271 1072 1271 ⊤ 341 724 ⊤ = , , , , , 3582 1051 3582 1480 2683 ! is the solution of LCP(T, r, L) problem.
8 Conclusions
In this paper, we studied the method of solving a linear complementarity problem on an ex- tended second order cone. By checking the stationarity and FB regularity of a point, we can verify whether it is a solution of the mixed complementarity problem. Such conversion of a linear complementarity problem to a mixed complementarity problem reduces the complexity of the original problem. The connection between a linear complementarity problem on an ex- tended second order cone and a mixed complementarity problem on a non-negative orthant will be useful for our further research about applications to practical problems, such us portfolio selection and signal processing problems.
15 References
[1] W. Karush. Minima of functions of several variables with inequalities as side conditions. Master thesis, University of Chicago, 1939.
[2] G. B. Dantzig and R. W. Cottle. Positive (semi-) definite matrices and mathematical programming. Technical report, California Univ Berkeley Operations Research Center, 1963.
[3] R. W. Cottle and G. B. Dantzig. Complementary pivot theory of mathematical program- ming. Linear algebra and its applications, 1(1):103–125, 1968.
[4] O. L. Mangasarian. Linear complementarity problems solvable by a single linear program. Mathematical Programming, 10(1):263–270, 1976.
[5] C. B. Garcia. Some classes of matrices in linear complementarity theory. Mathematical Programming, 5(1):299–310, 1973.
[6] J. M. Borwein and M. A. H. Dempster. The linear order complementarity problem. Math- ematics of Operations Research, 14(3):534–558, 1989.
[7] F. Alizadeh and D. Goldfarb. Second-order cone programming. Mathematical program- ming, 95(1):3–51, 2003.
[8] F. Facchinei and J.-S. Pang. Finite-dimensional variational inequalities and complemen- tarity problems. Vol. II. Springer-Verlag, New York, 2003.
[9] Igor Konnov. Equilibrium models and variational inequalities, volume 210. Elsevier, 2007.
[10] P. Jaillet, D. Lamberton, and B. Lapeyre. Variational inequalities and the pricing of american options. Acta Applicandae Mathematicae, 21(3):263–289, 1990.
[11] K. Yonekura and Y. Kanno. Second-order cone programming with warm start for elasto- plastic analysis with von Mises yield criterion. Optim. Eng., 13(2):181–218, 2012.
[12] L. L. Zhang, J. Y. Li, H. W. Zhang, and S. H. Pan. A second order cone complementarity approach for the numerical solution of elastoplasticity problems. Comput. Mech., 51(1):1– 18, 2013.
[13] G. Luo, X. An, and J. Xia. Robust optimization with applications to game theory. Appl. Anal., 88(8):1183–1195, 2009.
[14] R. Nishimura, S. Hayashi, and M. Fukushima. Robust Nash equilibria in N-person non- cooperative games: uniqueness and reformulation. Pac. J. Optim., 5(2):237–259, 2009.
[15] R. Andreani, A. Friedlander, M. P. Mello, and S. A. Santos. Box-constrained minimization reformulations of complementarity problems in second-order cones. J. Global Optim., 40(4):505–527, 2008.
[16] S. Z. N´emeth and G. Zhang. Extended Lorentz cones and mixed complementarity prob- lems. Journal of Global Optimization, 62(3):443–457, 2015.
[17] R. Sznajder. The Lyapunov rank of extended second order cones. Journal of Global Optimization, 66(3):585–593, 2016.
16 [18] O. P. Ferreira and S. Z. N´emeth. How to project onto extended second order cones. arXiv:1610.08887v2, 2016.
[19] S. Z. N´emeth and G. Zhang. Extended Lorentz cones and variational inequalities on cylinders. J. Optim. Theory Appl., 168(3):756–768, 2016.
[20] Harry Markowitz. Portfolio selection. The journal of finance, 7(1):77–91, 1952.
[21] Andrew Donald Roy. Safety first and the holding of assets. Econometrica: Journal of the Econometric Society, 20(3):431–449, 1952.
[22] J.-S. Facchinei, F.and Pang. Finite-dimensional variational inequalities and complemen- tarity problems. Vol. I. Springer-Verlag, New York, 2003.
[23] S. Karamardian. Generalized complementarity problem. J. Optimization Theory Appl., 8:161–168, 1971.
[24] F. Zhang. The Schur complement and its applications, volume 4. Springer Science & Business Media, 2006.
[25] H. H. Sohrab. Basic real analysis, volume 231. Springer, 2003.
[26] A. Fischer. A special newton-type optimization method. Optimization, 24(3-4):269–284, 1992.
[27] A. Fischer. A newton-type method for positive-semidefinite linear complementarity prob- lems. Journal of Optimization Theory and Applications, 86(3):585–608, 1995.
[28] K. E. Atkinson. An introduction to numerical analysis. John Wiley & Sons, 2008.
[29] Narendra Karmarkar. A new polynomial-time algorithm for linear programming. In Pro- ceedings of the sixteenth annual ACM symposium on Theory of computing, pages 302–311. ACM, 1984.
[30] O. L. Mangasarian. Solution of symmetric linear complementarity problems by iterative methods. Journal of Optimization Theory and Applications, 22(4):465–485, 1977.
[31] D. P. O’Leary and R. E. White. Multi-splittings of matrices and parallel solution of linear systems. SIAM Journal on algebraic discrete methods, 6(4):630–640, 1985.
[32] D. G. Luenberger and Y. Ye. Linear and nonlinear programming, volume 228. Springer, 2015.
[33] D. W. Marquardt. An algorithm for least-squares estimation of nonlinear parameters. Journal of the society for Industrial and Applied Mathematics, 11(2):431–441, 1963.
[34] N. Yamashita and M. Fukushima. On the rate of convergence of the Levenberg-Marquardt method. In Alefeld, G., Chen, X. (eds.) Topics in numerical analysis, pages 239–249. Springer, 2001.
17