Universite¶ catholique de Louvain Faculte¶ des Sciences appliquees¶ Departement¶ d'Ingenierie¶ mathematique¶ Center for Operation Research and Econometrics Center for Systems Engineering and Applied Mechanics Spectral Functions and Smoothing Techniques on Jordan Algebras Michel Baes Thesis submitted in partial ful¯llment of the requirements for the degree of Docteur en Sciences Appliqu¶ees Dissertation Committee: Fran»coisGlineur Universit¶ecatholique de Louvain Yurii Nesterov (advisor) Universit¶ecatholique de Louvain Cornelius Roos Technische Universiteit Delft Jean-Pierre Tignol Universit¶ecatholique de Louvain Paul Van Dooren (advisor) Universit¶ecatholique de Louvain Jean-Philippe Vial Universit¶ede Gen`eve Vincent Wertz (chair) Universit¶ecatholique de Louvain ii Contents List of notation vii 1 Introduction and preliminaries 1 1.1 Comparing algorithms .............................. 2 1.2 Linear Programming ............................... 3 1.3 Convex Programming .............................. 5 1.4 Self-scaled Optimization, and formally real Jordan algebras ......... 10 1.5 A closer look at interior-point methods ..................... 12 1.5.1 Newton's Algorithm: solving unconstrained problems ........ 12 1.5.2 Barrier methods: dealing with constraints ............... 13 1.5.3 Choosing an appropriate barrier .................... 13 1.5.4 Path-following interior-point methods for Linear Programming ... 17 1.5.5 Path-following interior-point methods for Self-Scaled Programming . 18 1.6 Smoothing techniques .............................. 20 1.7 Eigenvalues in Jordan algebra make it work: more applications ....... 22 1.7.1 A concavity result ............................ 22 1.7.2 Augmented barriers in Jordan algebras ................. 23 1.8 Overview of the thesis and research summary ................. 25 2 Jordan algebras 27 2.1 The birth of Jordan algebras .......................... 28 2.2 Algebras and Jordan algebras .......................... 30 2.2.1 Extensions of vector spaces ....................... 30 2.2.2 Jordan algebras .............................. 32 2.2.3 Strictly power-associative algebras ................... 34 iii iv Contents 2.2.4 Examples ................................. 36 2.3 Characteristic polynomial ............................ 38 2.3.1 Minimal polynomial over associative and commutative algebras ... 38 2.3.2 Characteristic polynomial over strictly power-associative algebras .. 42 2.3.3 Examples ................................. 54 2.4 Di®erential calculus ............................... 55 2.5 The quadratic operator ............................. 57 2.5.1 De¯nition and ¯rst properties ...................... 58 2.5.2 Quadratic operator and determinant .................. 59 2.5.3 Polarization of the quadratic operator ................. 60 2.5.4 Examples ................................. 61 2.6 Pierce decompositions .............................. 61 2.6.1 An illustrative example ......................... 61 2.6.2 Pierce decomposition theorems and ¯rst consequences ........ 63 2.6.3 Further examples ............................. 67 2.7 Spectral decomposition .............................. 68 2.7.1 Spectral decomposition in power-associative algebras ......... 68 2.7.2 More properties of the determinant ................... 71 2.7.3 Spectral decomposition in formally real Jordan algebras ....... 72 2.7.4 Minimal idempotents ........................... 74 2.7.5 A second spectral decomposition theorem for formally real Jordan algebras .................................. 77 2.7.6 A Euclidean topology in J ....................... 79 2.7.7 Operator commutativity ......................... 80 2.7.8 Eigenvalues of operators ......................... 82 2.7.9 Examples ................................. 84 2.8 Cone of squares .................................. 86 2.8.1 Examples ................................. 89 2.9 Simple Jordan algebras .............................. 89 2.10 Automorphisms .................................. 90 2.10.1 The structure group ........................... 91 2.10.2 Automorphisms of Jordan algebras ................... 93 2.11 Jordan algebras make it work: proofs for Section 1.7 ............. 96 2.11.1 A concavity result ............................ 96 2.11.2 Augmented barriers in Jordan algebras ................. 100 2.12 Conclusion .................................... 102 Contents v 3 Variational characterizations of eigenvalues 103 3.1 Introduction .................................... 104 3.2 Ky Fan's inequalities ............................... 105 3.3 Subalgebras J1(c) ................................ 109 3.4 Courant-Fischer's Theorem ........................... 112 3.5 Wielandt's Theorem ............................... 118 3.6 Applications of Wielandt's Theorem ...................... 125 4 Spectral functions 129 4.1 Introduction .................................... 130 4.1.1 Functions and di®erentials ........................ 131 4.1.2 Symmetric functions ........................... 133 4.2 Further results on Jordan algebras ....................... 133 4.3 Properties of spectral domains ......................... 138 4.4 Inherited properties of spectral functions .................... 141 4.4.1 The conjugate and the subdi®erential of a spectral function ..... 141 4.4.2 Directional derivative of eigenvalue functions ............. 142 4.4.3 First derivatives of spectral functions .................. 146 4.4.4 Convex properties of spectral functions ................ 149 4.5 Clarke subdi®erentiability ............................ 151 5 Spectral mappings 157 5.1 Introduction .................................... 158 5.2 De¯ning the problem ............................... 159 5.3 Fixing a converging sequence .......................... 160 5.4 Limiting behavior of a sequence of Jordan frames ............... 161 5.5 Jacobian of spectral mapping .......................... 167 5.6 Continuous di®erentiability of spectral mappings ............... 171 5.7 Application: complementarity problems .................... 174 5.7.1 Chen-Mangasarian smoothing functions ................ 176 5.7.2 Fischer-Burmeister smoothing functions ................ 179 6 Smoothing techniques 183 6.1 Introduction .................................... 184 6.2 Smoothing techniques in non-smooth convex optimization .......... 184 6.3 Smoothing for piecewise linear optimization .................. 187 6.4 An upper bound on the Hessian of the power function ............ 189 vi Contents 6.5 Sum-of-norms problem .............................. 194 6.6 Computational experiments ........................... 198 7 Conclusions and perspectives 203 Bibliography 207 Index 215 List of notation Basic sets N Set of nonnegative integers R Set of real numbers R+ Set of nonnegative real numbers R++ Set of positive real numbers Ln Second-order cone (see p. 11) Sn Set of n £ n real symmetric matrices n S+ Set of n £ n real symmetric positive semide¯nite matrices n R# Set of n-real dimensional vectors with decreasingly ordered components (see p. 105) ¢n n-dimensional simplex (see p. 187) P Set of r £ r permutation matrices (see p. 105) K¤ Dual cone of the cone K (see p. 9) conv(A) Convex hull of the set A SC(¸) Permutahedron generated by ¸ (see p. 106) Basic elements and functions T 1p r-dimensional vector (1; ¢ ¢ ¢ ; 1; 0 ¢ ¢ ¢ ; 0) , with p "1" and r ¡ p "0" (see p. 105) 1 All-one r-dimensional vector sp(¸) (for 1 · p · r) Sum of the p largest components of the r-dimensional vector ¸ (see p. 106) Det(A) Determinant of the matrix A Tr(A) Trace of the matrix A O Big-Oh asymptotic notation (see p. 3) £ Theta asymptotic notation (see p. 3) vii viii List of notation Functional analysis epi (f) Epigraph of the function f h rxf(x) Directional derivative of f in x, in the direction h rxf(x) Di®erential of f in x (see p. 132) f 0(x) Gradient of f in x (see p. 132) @f(x) Subdi®erential of f in x (see p. 132) @Bf(x) Bouligand subdi®erential of f in x (see p. 152) @C f(x) Clarke subdi®erential of f in x (see p. 133) Algebra N J F R Extension of the base ¯eld F of the algebra J with the ring R (see p. 31) [¢; ¢] Commutator (see p. 33) h¢; ¢iJ Jordan scalar product (see p. 79) L(x) Operator of multiplication by x (see p. 32) H(J ) Subalgebra of self-adjoint elements of J (see p. 37) Sn nth Jordan spin algebra (see p. 37) F" Ring of dual numbers built from F (see p. 51) " Nonzero element of F", whose square is null (see p. 51) ¹u Minimal polynomial of u (see p. 39) gu Reduced minimal polynomial of u (see p. 39) f(¿; x) Characteristic polynomial (see p. 48) ¸(u) Ordered vector of eigenvalues of u (see p. 49 and p. 79) ¸(u; J 0) Ordered vector of eigenvalues of u, considered in the subalgebra J 0 (see p. 79) det(u) Determinant of u (see p. 50) tr(u) Trace of u (see p. 50) detrj(u) jth dettrace of u (see p. 49) 0 ¸i(u; h) Short writing for the directional derivative of ¸i in u, in the direction h (see p. 162) Qu Quadratic operator (see p. 58) Qu;v Polarized quadratic operator (see p. 60) J1(c); J1=2(c); J0(c) Pierce subspace of J with respect to c (see p. 65) Ei Set of subalgebras J1(c) of J , where the trace of c equals i (see p. 113) 0 0 0 Ei(J ) Set of subalgebras J1(c) of J , where the trace of c equals i (see p. 113) KJ Cone of squares (see p. 86) lp(u); up(u) Index numbers related to the spectral decomposition of u (see p. 143) 0 00 fp(u); fp(u); fp (u) Idempotents related to the spectral decomposition of u (see p. 144) G(J ) Set of invertible linear operators from J to J (see p. 90) A(J ) Set of automorphisms of J (see p. 90) ¡(J ) Structure group of J